Your judgment, block by block
In this chapter
- Open the dossier and read a block with its protocol sentence and its items.
- Ask for a change with
sapling revise, and see the agent answer it first. - Agree to blocks, and know what an agreement covers.
- See what happens when a block changes after you agreed.
- Time
- 30 minutes, longer in a real study: this is where you read
- At the end you have
- four key blocks agreed (the estimand, the missing data, the multiplicity, the sensitivity analysis), one of them after a change you asked for.
9.1 The idea: a score is not a judgment
Section titled “9.1 The idea: a score is not a judgment”The agent score of chapter 8 is 95. That says the plan compiles, cites the protocol, and follows the rules Sapling knows. It does not say that the plan is right for this trial. You read each block that decides something, and you say yes, or you say what to change.
Two rules make this safe:
- A note comes first. When you ask for a change, the agent answers that note before anything else, and you cannot agree to the block until it is answered.
- An agreement is pinned to the text. Sapling records a hash of the block’s text with your agreement. If the agent changes one word later, your agreement no longer counts for that block. An agent’s edit can never ride on an earlier yes.
9.2 The dossier
Section titled “9.2 The dossier”$ sapling judgeSAPLING EVAL: agent score 95/100; NOT READY (22 open decision(s) for the statistician; 84 of 84 block(s) not agreed)
not judged yet (84): 16 alphalevel alphalevel#0 22 treatments treatments#0 27 visits visits#0 91 design design#0 113 samplesize SS1 194 population ITT 199 population SAF 205 population EFF 211 population COMP24 216 grouping PLANNED 231 tlf T14_1_1 251 tlf T14_1_2 264 test test#0...sapling judge lists the blocks to read and writes the dossier out/eval/judge.html: open it in the browser of the lab.
It starts with the two scores. Then each block shows its text, the protocol sentence it cites, the items of the
scorecard on it, and the two commands to agree or to ask for a change.

9.3 Ask for a change
Section titled “9.3 Ask for a change”The tipping-point analysis of chapter 8 shifts the missing values by delta from 0 to 6 in steps of 0.5. The ADAS-Cog (11) is scored in whole points, so a step of half a point has no clinical meaning. You ask for whole points, up to 8:
$ sapling revise SENS1 "Step delta by 1 point from 0 to 8: the ADAS-Cog (11) is scored in whole points." --by "A. Statistician"NOTE on SENS1 (line 468) by A. Statistician: the agent answers it first (sapling next)The note is now the agent’s first item:
$ sapling next --n 2For the agent (13 open; agent score 84/100): 1. [feedback] A. Statistician on SENS1 (2026-11-20): Step delta by 1 point from 0 to 8: the ADAS-Cog (11) is scored in whole points. (line 469) how: change SENS1 as the note asks (or say why not), check it, then: sapling reply SENS1 "what you changed"...The agent score falls to 84 while your note is open: the Feedback part counts your notes that have no answer.
$ sapling agree SENS1 --by "A. Statistician"sensitivity:SENS1 has an open note (A. Statistician: Step delta by 1 point from 0 to 8: the ADAS-Cog (11) is scored in whole points.): the agent answers it first, or you resolve it in the editor9.4 The agent answers
Section titled “9.4 The agent answers”The agent changes the block as you asked, checks it in its place with --replace, replies to your note, and measures:
::: {.sensitivity #SENS1 estimand="EST1" method="tipping-point" assumption="MNAR" delta="0 to 8 by 1" imputations="100" seed="20261110"}**Tipping point analysis**
The missing values of the active groups are imputed under MAR, then shifted by delta points of the ADAS-Cog (11);the smallest delta that makes the primary test not significant is reported.:::$ sapling block check sens.qmd --replace SENS1 --writemakes sensitivity SENS1 (line 470)BLOCK CHECK PASS: 0 error(s); written to sap.qmd
$ sapling reply SENS1 "Delta now goes from 0 to 8 in steps of 1 point."REPLY on SENS1 (line 470): the statistician reads it, then agrees or writes again$ sapling round --note "the statistician's note on SENS1"... AGENT SCORE 95/100 (+0 since round 4)... AGAINST ROUND 4: SAME...The score is back to 95. The round is SAME: no item of the agent changed, and your note has its answer.
9.5 Agree
Section titled “9.5 Agree”You read the four blocks that carry the statistics of the primary analysis, and you agree to them in one command:
$ sapling agree EST1 MD_T14_3_1_CHG MULT SENS1 --by "A. Statistician" --note "read in the dossier"AGREED estimand EST1 (line 332, text 4e7b5446fc6b) by A. StatisticianAGREED missingdata MD_T14_3_1_CHG (line 398, text 5346ec7fda04) by A. StatisticianAGREED multiplicity MULT (line 431, text bbff2d0ced00) by A. StatisticianAGREED sensitivity SENS1 (line 471, text a8b3fc378e0f) by A. Statistician
$ sapling blocks --state agreed 332 estimand EST1 agreed 398 missingdata MD_T14_3_1_CHG agreed agent 431 multiplicity MULT agreed agent 471 sensitivity SENS1 agreed agent 1 note(s)4 block(s): 4 agreed, 0 changed since agreed, 0 not judgedEach agreement records the block’s line, a hash of its text (text 4e7b5446fc6b for EST1), your name and the date. The
list marks the three blocks that the agent drafted (agent). An agent’s process cannot agree, even with your name:
# in the agent's terminal$ sapling agree T14_3_1 --by "A. Statistician"refused: this is a process of an agent (SAPLING_AGENT is set). Only a person agrees to a block of the plan: run it yourself in your terminalYour turn 9.1 One word after your yes
In a copy of the study, change one sentence of SENS1 after you agreed to it, as an agent might: the smallest delta
that makes the primary test not significant becomes the largest delta that keeps the primary test significant. What
does sapling blocks --state changed say?
Show the answer
the largest delta that keeps the primary test significant is reported.$ sapling blocks --state changed 471 sensitivity SENS1 changed agent 1 note(s)1 block(s): 0 agreed, 1 changed since agreed, 0 not judged
$ sapling show SENS1sensitivity SENS1 sap.qmd:471 changed (A. Statistician, 2026-11-20) drafted by an agent...SENS1 is changed since agreed. Your agreement was to the old text, so it no longer counts. The block counts again only when you read it and agree again. (Here the two sentences mean nearly the same; Sapling cannot know that, and it does not try. That is your judgment.)
What you learned
- The dossier (
sapling judge,out/eval/judge.html) shows each block with its protocol sentence, its items and your two commands. sapling revisewrites a note that the agent answers first; a block with an open note cannot be agreed.sapling agreepins your yes to a hash of the block’s text; any later change makes the block changed since agreed.- An agent’s process and an agent’s name cannot agree.
Next: 10 · Your decisions →
Found an error or have a question? Write to datastep.studio@gmail.com· The plan is written in Sapling; every R program runs in DataStep Studio for R.
Data: the CDISC pilot study (cdiscpilot01). The protocol as CDISC USDM from the CDISC 360i project (MIT licence); the SDTM data from the PhUSE scripts repository (MIT licence). Public example data: no real patient.
R is a project of the R Foundation. SAS is a registered trademark of SAS Institute Inc. CDISC, SDTM, ADaM and USDM are trademarks of the Clinical Data Interchange Standards Consortium. Claude is a trademark of Anthropic. This book is not affiliated with or endorsed by any of them.
© 2026 DataStep Labs. All rights reserved.