What only you can add: a tipping-point analysis
In this chapter
- Say what a sensitivity analysis is for, and which assumption it must change (ICH E9(R1) A.5.2).
- Read a tipping-point analysis: the shift delta, and what the tipping point means.
- Tell what the protocol says from what only the SAP adds, and declare an addition.
- Time
- 25 minutes
- At the end you have
- round 4: agent score 95, Estimands 100; the plan has a tipping-point analysis that you declared as an addition to the protocol.
8.1 The idea: test the assumption, not the method
Section titled “8.1 The idea: test the assumption, not the method”Chapter 7 kept LOCF as the primary handling of the missing values, and said that its assumption is weak. ICH E9(R1) asks for sensitivity analyses: analyses of the same estimand under other assumptions, to see whether the conclusion holds ICH E9(R1) A.5.2.1. A sensitivity analysis changes the assumptions in a structured way, one at a time ICH E9(R1) A.5.2.2. Analyses that make similar assumptions to the primary one teach little EMA 2010 7.
The protocol of the pilot plans LOCF and an observed-case analysis. Both assume, in different ways, that the patients who left are like the ones who stayed. Neither tests what happens if they were worse.
8.2 The tipping-point analysis
Section titled “8.2 The tipping-point analysis”A tipping-point analysis asks: how much worse must the missing values be before the conclusion changes?
- Impute the missing values of the active groups under MAR (missing at random: like similar patients who stayed), many times (here 100 imputations).
- Make them worse by a shift delta: add delta points to each imputed ADAS-Cog (11) value of the active groups. A higher score is worse.
- Analyse each data set and combine the results. Increase delta step by step.
- The tipping point is the smallest delta at which the primary test is no longer significant.
You then judge the tipping point clinically. If the conclusion changes only when the patients who left are, say, 8 points worse than similar patients who stayed, and such a decline is not plausible, the conclusion is robust. If 1 point is enough, it is fragile. You plan this before the unblinding, when you do not know the result yet.
8.3 The agent drafts it
Section titled “8.3 The agent drafts it”The next item of the loop asks for a sensitivity analysis under MNAR. The agent writes it with the conventional block of Sapling:
$ sapling generate sensitivityinsert SENS1 in "Additional analyses" (sap.qmd)check 0 error(s), 2 warning(s): SAPLING CHECK PASS
$ sapling show SENS1sensitivity SENS1 sap.qmd:468 not judged
| ::: {.sensitivity #SENS1 estimand="EST1" method="tipping-point" assumption="MNAR" delta="0 to 6 by 0.5" | imputations="100" seed="20261110"} | **Tipping point analysis** | | The missing values of the active groups are imputed under MAR, then shifted by delta; the smallest delta that makes | the primary test not significant is reported. | :::
OPEN [grounded] sensitivity SENS1 cites no protocol sentence and is no declared addition how: cite the protocol on the line after the block: []{.cite source="protocol:<section>" words="words of the sentence"} (sapling find, or the protocol search of the editor); if the protocol does not say it, write a ::: {.decision} that leaves it open for the statistician OPEN [agreed] sensitivity SENS1: not judged yet how: the statistician reads it (sapling show SENS1), then: sapling agree SENS1 --by NAME, or sapling revise SENS1 "..." --by NAMEThe block plans 100 imputations, a fixed seed, and delta from 0 to 6 in steps of 0.5. But Sapling scores it not grounded: it cites no protocol sentence, and nobody has declared it as an addition. The agent cannot cite a sentence that is not there (chapter 7). It must not pretend that the protocol plans it.

8.4 You declare the addition
Section titled “8.4 You declare the addition”What the SAP adds to the protocol is your decision, with your reason. You write it in the section Changes from the
protocol, as a change block that names the block it adds:
# Changes from the protocol
::: {.change adds="SENS1"}**The plan adds a tipping point analysis of the primary estimand EST1 under MNAR.**
The protocol's LOCF and its observed-case analyses make similar assumptions about the patients who stop. The tippingpoint analysis shows how far the missing values of the active groups must be worse than under MAR before theconclusion changes (ICH E9(R1) A.5.2; EMA 2010, section 7).:::[]{.cite source="protocol:4.3.2" words="Additionally, each of the efficacy variables will be analyzed"}$ sapling checkpass 1 syntax 0 error(s) 0 warning(s) PASSpass 2 structure 0 error(s) 1 warning(s) WARN sap.qmd:618:22: warning[software]: SAS has no version. 618 | ::: {.software name="SAS"} | ^^^ fix: write `::: {.software name="SAS" version="..."}`: the results depend on itpass 3 data 0 error(s) 0 warning(s) PASSpass 4 knowledge 0 error(s) 0 warning(s) PASSpass 5 chain 0 error(s) 0 warning(s) PASS0 error(s), 1 warning(s)SAPLING CHECK PASS$ sapling round --note "the tipping point analysis: an addition of the SAP"... AGENT SCORE 95/100 (+3 since round 3) Sound 91 #########. 0 error(s), 1 warning(s) of the compiler Grounded 100 ########## 22 of 22 deciding block(s) cite the protocol or are declared additions Estimands 100 ########## 8 of 8 estimand check(s) hold Rigour 100 ########## 0 error(s), 0 warning(s) of the knowledge base's rules open Complete 80 ########.. 45 of 56 checklist item(s) shown in the plan... AGAINST ROUND 3: IMPROVED (estimands 88->100, complete 76->80); 3 item(s) answered...The whole output (24 lines)
$ sapling round --note "the tipping point analysis: an addition of the SAP"sap.qmd (plan 53cdf1274a8e)
AGENT SCORE 95/100 (+3 since round 3) Sound 91 #########. 0 error(s), 1 warning(s) of the compiler Grounded 100 ########## 22 of 22 deciding block(s) cite the protocol or are declared additions Estimands 100 ########## 8 of 8 estimand check(s) hold Rigour 100 ########## 0 error(s), 0 warning(s) of the knowledge base's rules open Complete 80 ########.. 45 of 56 checklist item(s) shown in the plan Feedback 100 ########## no note of the statistician Executable - not run yet (sapling run): not scored
READINESS NOT READY (the statistician's) Decisions 22 open decision(s) Agreed 0 of 84 block(s) agreed Approved not approved
AGAINST ROUND 3: IMPROVED (estimands 88->100, complete 76->80); 3 item(s) answered
NEXT (agent): Gamble 5: 5. Roles and responsibilities (no setting `author:` (roles and responsibilities)) how: write the block that states it (sapling generate ..., or a block of docs/markdown.md), cited; when only the statistician can say it: a ::: {.decision} NEXT (statistician): open decision (section 1): Name the roles and the persons: who writes this SAP, the senior statistician who reviews it, and the chief investigator or sponsor who approves it (Gamble et al. 2017, items 5 and 6). (line 47)
SAPLING EVAL: agent score 95/100; NOT READY (22 open decision(s) for the statistician; 84 of 84 block(s) not agreed); saved as round 4Grounded is 22 of 22 again, Estimands is 100, and the agent score is 95.
Agent score measured by the compiler and the rules
95/100
Readiness yours: no agent can move it
NOT READY
- Decisions
- 22 open decision(s)
- Agreed
- 0 of 84 block(s) agreed
- Approved
- not approved
Next for the agent Gamble 5: 5. Roles and responsibilities (no setting `author:` (roles and responsibilities))
Your turn 8.1 Grounded again
Run sapling show SENS1 after the change block. What does the grounded item say now? What is still open?
Show the answer
$ sapling show SENS1sensitivity SENS1 sap.qmd:468 not judged
| ::: {.sensitivity #SENS1 estimand="EST1" method="tipping-point" assumption="MNAR" delta="0 to 6 by 0.5" | imputations="100" seed="20261110"} | **Tipping point analysis** | | The missing values of the active groups are imputed under MAR, then shifted by delta; the smallest delta that makes | the primary test not significant is reported. | :::
ok [grounded] sensitivity SENS1 rests on a declared addition of the SAP OPEN [agreed] sensitivity SENS1: not judged yet how: the statistician reads it (sapling show SENS1), then: sapling agree SENS1 --by NAME, or sapling revise SENS1 "..." --by NAMEThe block now rests on a declared addition of the SAP. One item is open: you have not judged the block. Chapter 9 judges it, and asks for a change first.
What you learned
- A sensitivity analysis tests the assumption of the primary analysis on the same estimand (ICH E9(R1) A.5.2).
- The tipping point is the smallest shift of the missing values that changes the conclusion; you judge it clinically.
- What the protocol does not have is an addition of the SAP: the agent proposes it, you declare it with
adds=and a reason.
Found an error or have a question? Write to datastep.studio@gmail.com· The plan is written in Sapling; every R program runs in DataStep Studio for R.
Data: the CDISC pilot study (cdiscpilot01). The protocol as CDISC USDM from the CDISC 360i project (MIT licence); the SDTM data from the PhUSE scripts repository (MIT licence). Public example data: no real patient.
R is a project of the R Foundation. SAS is a registered trademark of SAS Institute Inc. CDISC, SDTM, ADaM and USDM are trademarks of the Clinical Data Interchange Standards Consortium. Claude is a trademark of Anthropic. This book is not affiliated with or endorsed by any of them.
© 2026 DataStep Labs. All rights reserved.