Your decisions
In this chapter
- Answer the open decisions, one at a time or from a file.
- Know the guideline behind the hard ones: stratification, sidedness, multiplicity, missing data.
- Read the history of the loop, round by round.
- Time
- 40 minutes: a real study takes longer, with your team
- At the end you have
- 0 open decisions, each answer recorded with your name and the date; Part I is done.
10.1 The idea: what the protocol leaves open is yours
Section titled “10.1 The idea: what the protocol leaves open is yours”Chapter 4 listed 22 open decisions. Each is a place where the protocol does not say what the SAP must say. Some are small: the date of the final analysis. Some change the result: a stratification factor becomes a term of the model, and the sidedness of a test changes its level. An agent can propose a default; only you decide.
You answer one at a time (sapling decide 3 "..." --by NAME), or all at once from a file: one line per decision, its
number, a tab, your answer.
10.2 Four decisions that change the analysis
Section titled “10.2 Four decisions that change the analysis”Decision 3: randomization and stratification. If the randomization was stratified, the analysis model should generally include the stratification factors ICH E9 5.7 EMA covariates 5.2. The protocol of the pilot does not state the method. The answer takes no stratification factor, and keeps the pooled site in the model as the protocol’s investigator effect.
Decision 8: one-sided tests. The protocol asks for one-sided tests of efficacy. ICH E9 accepts a one-sided test when it is justified in advance, at half the two-sided level: 0.025 for 0.05 ICH E9 5.5. The book’s answer says one-sided and which p-values the tables show. A complete answer also states the level, 0.025.
Decision 9: the family of hypotheses. The two co-primary hypotheses of chapter 7, with the multiplicity block MULT.
Decisions 18 and 19: missing data. LOCF with its stated assumption (chapter 7), and the tipping-point analysis as an addition (chapter 8).
10.3 Answer them
Section titled “10.3 Answer them”The answers of the book, in a file:
1 Written by the study statistician, reviewed by a senior statistician, approved by the sponsor's head of biostatistics; their names are on the title page at the approval.2 Equal allocation to the three groups (about 100 patients each, protocol section 4.2); the comparison is superiority, tested as a linear dose response.3 The protocol does not state the method or the block size; the SAP takes no stratification factor, and the primary model adjusts for the pooled site (SITEGR1), the protocol's investigator effect.4 Use the protocol's numbers (section 4.2): about 100 patients per group and 90% power; the sample size block states them.5 As the protocol (section 4.6): two looks for internal decisions only, at about 50% of the patients at week 8 and at week 24, under the data monitoring board; no stopping for efficacy and no alpha spent.6 As the protocol (section 4.7): reviews of cardiovascular safety by the data monitoring board; no efficacy decision and no alpha spent.7 After the last patient completes the study, the database is locked and the treatment codes are released.8 Efficacy tests are one-sided, as the protocol asks (section 4.3.1); the tables show the one-sided and the two-sided p-values, as the protocol says. Safety tests are two-sided at 0.05.9 The two primary hypotheses (ADAS-Cog (11) and CIBIC+), co-primary as the protocol says in section 4.3.1: the multiplicity block MULT.10 95% two-sided confidence intervals for the differences from placebo of the efficacy models.11 Adherence is the percent of the planned patches that a patient applied (exposure); the protocol deviations are listed, not tested.12 Kept after review: the populations, the visit windows, the baseline (the last value before the first dose), the derivations of the analysis data sets and the standard tables.13 The data have the DAD; this SAP does not analyse it (the book keeps the plan short). A real SAP defines its endpoint and analysis.14 The ADAS-Cog (14) is not analysed in this SAP (the book keeps the plan short).15 The Apo E genotype is not analysed in this SAP.16 Confirmed as written: the efficacy population, the treatment policy strategy for both intercurrent events, and the summary from the ANCOVA of T14_3_1_CHG.17 The residuals of the ANCOVA are checked with a normal quantile plot by group; if they fail, a rank ANCOVA is a supportive analysis.18 LOCF, as the protocol's 24-week endpoint: the block MD_T14_3_1_CHG states the assumption and why.19 A tipping point analysis under MNAR for EST1 (SENS1), an addition of the SAP (Changes from the protocol).20 Descriptive summaries by age group, sex and race, as the protocol allows if the sample sizes are sufficient; no test of interaction.21 R 4.4 and its packages (admiral, cards, cardx, mmrm), in the versions the run records; MedDRA as coded in the SDTM data. The protocol names SAS: the SAP uses R.22 ICH E9(R1) (2019); EMA Guideline on Missing Data in Confirmatory Clinical Trials (2010); Gamble et al., JAMA 2017.$ sapling decide --file answers.tsv --by "A. Statistician"decide 22 decision(s) recorded by A. Statistician on 2026-11-20; 0 open; SAPLING CHECK PASS$ sapling eval... AGENT SCORE 95/100 (+0 since round 5)... READINESS NOT READY (the statistician's) Decisions 0 open decision(s) Agreed 4 of 84 block(s) agreed Approved not approved... AGAINST ROUND 5: IMPROVED (decisions 0->100, agreed 0->5); 26 item(s) answered... NEXT (statistician): alphalevel alphalevel#0: not judged yet (line 16)...The whole output (24 lines)
$ sapling evalsap.qmd (plan 3923e2240db9)
AGENT SCORE 95/100 (+0 since round 5) Sound 91 #########. 0 error(s), 1 warning(s) of the compiler Grounded 100 ########## 22 of 22 deciding block(s) cite the protocol or are declared additions Estimands 100 ########## 8 of 8 estimand check(s) hold Rigour 100 ########## 0 error(s), 0 warning(s) of the knowledge base's rules open Complete 80 ########.. 45 of 56 checklist item(s) shown in the plan Feedback 100 ########## 1 of 1 note(s) of the statistician answered Executable - not run yet (sapling run): not scored
READINESS NOT READY (the statistician's) Decisions 0 open decision(s) Agreed 4 of 84 block(s) agreed Approved not approved
AGAINST ROUND 5: IMPROVED (decisions 0->100, agreed 0->5); 26 item(s) answered
NEXT (agent): Gamble 5: 5. Roles and responsibilities (no setting `author:` (roles and responsibilities)) how: write the block that states it (sapling generate ..., or a block of docs/markdown.md), cited; when only the statistician can say it: a ::: {.decision} NEXT (statistician): alphalevel alphalevel#0: not judged yet (line 16)
SAPLING EVAL: agent score 95/100; NOT READY (80 of 84 block(s) not agreed)The plan records each answer under its decision, with your name and the date. 26 items are answered: the 22 decisions, and the agreements of chapter 9. The next item for you is the first block you have not judged. Readiness now waits for your agreement to the other 80 blocks, then for your approval (chapter 20).
10.4 Part I in one picture
Section titled “10.4 Part I in one picture”Five rounds of the loop, from the first draft to the answered note:
The agent score, round by round
- Round 184the first draft
- Round 290+6the missing data of the primary analysisestimands 75→88 · rigour 77→91 · complete 70→74
- Round 392+2the multiplicity blockrigour 91→100 · complete 74→76
- Round 495+3the tipping point analysis: an addition of the SAPestimands 88→100 · complete 76→80
- Round 595±0the statistician's note on SENS1
The bars start at 60, so that small steps show.
$ sapling eval historyround when score sound groun estim rigou compl feedb execu failing readiness 1 2026-11-20 14:07 84 91 100 75 77 70 100 - 125 NOT READY the first draft 2 2026-11-20 14:07 90 91 100 88 91 74 100 - 121 NOT READY the missing data of the primary analysis 3 2026-11-20 14:07 92 91 100 88 100 76 100 - 120 NOT READY the multiplicity block 4 2026-11-20 14:07 95 91 100 100 100 80 100 - 119 NOT READY the tipping point analysis: an addition of the SAP 5 2026-11-20 14:07 95 91 100 100 100 80 100 - 119 NOT READY the statistician's note on SENS1Your turn 10.1 Agree in the name of the agent
Run sapling agree T14_1_1 --by "Claude" in your own terminal (not in the agent’s process). Is it accepted?
Show the answer
$ sapling agree T14_1_1 --by "Claude"refused: "Claude" is not a person's name: only a person agrees to a block of the planNo. The command runs in a person’s terminal, but Claude is the name of an agent, so it refuses and exits with code 2.
An agreement names the person who read the block.
What you learned
- Each open decision is answered by a person, with a name and a date: one at a time, or from a file.
- Stratification factors go into the model; a one-sided test uses half the two-sided level (ICH E9 5.5, 5.7).
- Part I took the agent score from 84 to 95 in five rounds; your readiness moved only by your decisions and agreements.
Found an error or have a question? Write to datastep.studio@gmail.com· The plan is written in Sapling; every R program runs in DataStep Studio for R.
Data: the CDISC pilot study (cdiscpilot01). The protocol as CDISC USDM from the CDISC 360i project (MIT licence); the SDTM data from the PhUSE scripts repository (MIT licence). Public example data: no real patient.
R is a project of the R Foundation. SAS is a registered trademark of SAS Institute Inc. CDISC, SDTM, ADaM and USDM are trademarks of the Clinical Data Interchange Standards Consortium. Claude is a trademark of Anthropic. This book is not affiliated with or endorsed by any of them.
© 2026 DataStep Labs. All rights reserved.