Skip to content

Working with an AI agent

In this chapter

  • Read the rules an agent follows in the study folder (AGENTS.md).
  • Start your agent with sapling agent, or give any agent the same request.
  • See the first item as the agent sees it: as data.
  • Name the limits Sapling enforces, and see one refusal.
Time
15 minutes
At the end you have
an agent started in the study folder, with its rules and its first item; you know the four things it may never do.

6.1 The idea: an agent is a program that works a loop

Section titled “6.1 The idea: an agent is a program that works a loop”

An AI agent, such as Claude Code, Codex or Gemini CLI, is a language model that runs commands in a terminal. It reads the output, decides on the next command, and runs it. It is fast and it writes well. It can also be wrong with confidence, and it does not carry the responsibility for the analysis. You do.

So the work is divided in a way that does not depend on the agent’s good will:

  • The agent gets small commands with clear verdicts: sapling next (what to do), sapling block check (is this block correct in its place), sapling round (is the plan better or worse than before).
  • The agent gets written rules in the study folder: AGENTS.md.
  • Sapling refuses the four acts that are yours: decide, agree, approve, unblind. The refusal is in the program, not in a prompt.

Every study folder has an AGENTS.md. Agents read it first: Claude Code through CLAUDE.md, which points to it; Codex, Gemini CLI and GitHub Copilot read AGENTS.md itself.

AGENTS.md
# AGENTS.md: how an AI agent works in this study
You help a statistician write the statistical analysis plan (SAP) of this study in Sapling, and make the ADaM data,
the tables (TLFs) and the dashboard from it in R. The statistician decides; you write blocks and measure them.
The whole workflow: https://sapling-sap.pages.dev (the agentic workflow). Your toolbox is every command of
`sapling help --all` (the statistician sees only a few); every command takes --json.
## The loop (one block at a time)
1. `sapling next` - the first open item: what, where, and how to answer it. A note of the statistician comes first.
2. `sapling show <block>` - the block, the protocol sentence it cites, its items. `sapling find <words>` - what the
SDTM and ADaM data have; ask it before you write a derivation or an analysis.
3. Write ONE block in a file, then `sapling block check <file> --after <block> --write`: it is written only when it
compiles in its place with no new error, and it is marked "drafted by: agent". Fix each error with its `fix:` line.
4. `sapling round --note "what you did"` - the scorecard, saved, against the last round. REGRESSED (exit 1): an item
that held before fails now; correct or undo your change before anything else.
5. A note of the statistician on a block: change the block as it asks (or say why not), then
`sapling reply <block> "what you changed"`.
6. When `sapling next` has nothing left for you: `sapling final --make` makes the ADaM, the TLFs and the dashboard
in R (DataStep Studio for R) and says which of the four products are FINAL and what is left. A failed step:
`sapling diagnose`, then ONE correction (`sapling fix`); a person accepts it.
...
CLAUDE.md
@AGENTS.md

The loop has six steps. The agent looks (next, show, find), writes one block, checks it in its place, measures the round, answers your notes first, and at the end makes the products in R.

Every command has --json. The agent reads the next item as data, with an id, a part of the scorecard, an owner and a fix:

Terminal
$ sapling next --json --n 1
{
"agent_score": 84,
"readiness": {
"verdict": "NOT READY",
"reasons": [
"22 open decision(s) for the statistician",
"80 of 80 block(s) not agreed"
]
},
"agent": [
{
"line": null,
"block": "EST1",
"id": "estimands:EST1:missing-data",
"dim": "estimands",
"owner": "agent",
"ok": false,
"what": "EST1 (primary): no missingdata block states how missing values of its analysis are handled, and under which assumption",
"fix": "write ::: {.missingdata #MD_EST1 analysis=\"T14_3_1_CHG\" assumption=\"MAR\" handling=\"...\"} with the justification (EMA 2010; ICH E9(R1) A.5)",
"severity": "gap"
}
],
...
The whole output (37 lines)
Terminal
$ sapling next --json --n 1
{
"agent_score": 84,
"readiness": {
"verdict": "NOT READY",
"reasons": [
"22 open decision(s) for the statistician",
"80 of 80 block(s) not agreed"
]
},
"agent": [
{
"line": null,
"block": "EST1",
"id": "estimands:EST1:missing-data",
"dim": "estimands",
"owner": "agent",
"ok": false,
"what": "EST1 (primary): no missingdata block states how missing values of its analysis are handled, and under which assumption",
"fix": "write ::: {.missingdata #MD_EST1 analysis=\"T14_3_1_CHG\" assumption=\"MAR\" handling=\"...\"} with the justification (EMA 2010; ICH E9(R1) A.5)",
"severity": "gap"
}
],
"statistician": [
{
"line": 47,
"block": null,
"id": "decisions:99e58839",
"dim": "decisions",
"owner": "statistician",
"ok": false,
"what": "open decision (section 1): Name the roles and the persons: who writes this SAP, the senior statistician who reviews it, and the chief investigator or sponsor who approves it (Gamble et al. 2017, items 5 and 6).",
"fix": "ask the statistician, one question at a time, with the default you expect (AGENTS.md rule 5); the statistician writes the answer: sapling decide",
"severity": "gap"
}
]
}

Look at "owner". The agent’s item is "agent": the missing-data handling of the primary estimand. Your item is "statistician", and its fix says the statistician writes the answer. The agent asks you; it does not answer for you.

An agent such as Claude Code works in a terminal. It uses the whole shell: grep, jq, diff, a loop that tests an exit code. Sapling is made for that way of working:

  • Plain text. The plan (sap.qmd) and the specifications are Markdown files. An agent can read, search and compare them like code, and git diff shows every change.
  • --json on every command. The agent reads data, not prose: the next item, the scorecard, a block.
  • Exit codes a loop can test. sapling round exits 1 on a regression, sapling block check exits 1 on an error, and a command that is yours exits 2 when an agent runs it.
  • The rules in the folder. AGENTS.md says what the loop is and what the agent may not do.
  • The person-only commands refuse the agent’s process. The agent may use any tool; it still cannot decide for you.

Three real commands of an agent at the start of the pilot. It asks for its next three items, one line each, with jq; it reads the two scores from the scorecard; and it counts the decision blocks in the plain text of the plan, all of them open at this moment:

The agent's terminal
# in the agent's terminal
$ sapling next --json --n 3 | jq -r '.agent[] | "\(.dim) \(.block // "-") \(.what)"'
estimands EST1 EST1 (primary): no missingdata block states how missing values of its analysis are handled, and under which assumption
estimands EST1 EST1 (primary): no sensitivity analysis of the assumptions about the missing data (ICH E9(R1) A.5.2)
rigour - 17. Multiplicity: the protocol names 2 primary endpoints (ADAS-Cog (11); CIBIC+), and the plan has no multiplicity strategy.
# in the agent's terminal
$ sapling eval --json | jq '{agent_score, readiness: .readiness.verdict, failing: [.dims[] | select(.failing > 0) | "\(.id) \(.failing)"]}'
{
"agent_score": 84,
"readiness": "NOT READY",
"failing": [
"sound 1",
"estimands 2",
"rigour 3",
"complete 16",
"decisions 22",
"agreed 80",
"approved 1"
]
}
# in the agent's terminal
$ grep -c '{\.decision' sap.qmd
22

The pages of this book mark such commands with in the agent’s terminal. You never need them: your steps are sapling commands. They show why an agent can work fast and stay inside its limits.

In the terminal of the study folder:

Terminal window
sapling agent

sapling agent starts your agent in the study folder (Claude Code by default; --provider codex or --provider gemini for the others) with this request:

Terminal
$ sapling agent --print
You work in this study folder with me, the statistician. Read AGENTS.md and follow it. Start with `sapling status` and `sapling next`. Work the loop one block at a time: before each block, tell me in two lines what you will write and why; then check it in its place and run `sapling round`. When a choice is mine, ask me one question at a time, with the default you expect. Answer my notes first. Stop when `sapling next` has nothing left for you, and tell me what is left for me.

Any other agent works the same: give it this request. sapling agent --print shows it.

An agent that speaks MCP (the Model Context Protocol) gets the same loop as tools: sapling_next, sapling_insert_block, sapling_eval, sapling_blocks and sapling_reply. None of them decides, agrees or approves.

Your turn 6.1 Try to decide as the agent

Act as the agent: set SAPLING_AGENT=1 and answer decision 1 with sapling decide. What happens? Does the plan change?

Show the answer
The agent's terminal
# in the agent's terminal
$ sapling decide 1 "Written by the study statistician" --by "A. Statistician"
sapling decide: refused: this is a process of an agent (SAPLING_AGENT is set); a person decides, in their own terminal

The command refuses and exits with code 2. The plan does not change: decision 1 stays open. A person answers it in their own terminal (chapter 10).

What you learned

  • AGENTS.md holds the agent’s loop and its limits; CLAUDE.md points to it.
  • The agent reads every item as data (--json), each with its owner.
  • sapling agent starts your agent with one request; any agent can work the same loop.
  • Decide, agree, revise, approve and unblind refuse an agent’s process and an agent’s name.
  • Sapling suits an agent’s shell: plain-text files, --json everywhere, exit codes a loop can test.

Next: 7 · The agent's loop →

Found an error or have a question? Write to datastep.studio@gmail.com· The plan is written in Sapling; every R program runs in DataStep Studio for R.

Data: the CDISC pilot study (cdiscpilot01). The protocol as CDISC USDM from the CDISC 360i project (MIT licence); the SDTM data from the PhUSE scripts repository (MIT licence). Public example data: no real patient.

R is a project of the R Foundation. SAS is a registered trademark of SAS Institute Inc. CDISC, SDTM, ADaM and USDM are trademarks of the Clinical Data Interchange Standards Consortium. Claude is a trademark of Anthropic. This book is not affiliated with or endorsed by any of them.

© 2026 DataStep Labs. All rights reserved.