vs Spellbook Benchmarks. Spellbook の State of Contracts 2026 は、30 か国の数十万件の契約から抽出した 13 種類の契約タイプにわたる 270 超の条項ベンチマークを、相互提供型のデータモデルで公開しています。これは別の問いに答えるもので、市場がどこにあるかを示します。相手方の要求が市場水準を外れていると論じるには、こちらのほうが優れた道具です。ただし あなた がどこにいるかは教えてくれません。そして両者のギャップこそが、しばしば得られる最も有用な発見です。両方使ってください。一方を他方で代替しないでください。
vs 自社 CLM の条項ライブラリ. Ironclad も Juro もリポジトリ全体の条項出現頻度を出してくれますし、すでにリポジトリがあるならそれが最も安い出発点です。通常与えてくれないのは、ペーパー由来によるセグメンテーション、ポジションと偶然を分ける信頼区間の下限、そして決裂ラインを述べることの明示的な拒否です。この 3 つが、このバンドルが上乗せするものです。
vs 記憶を頼りに Word 文書へ書くこと. ほとんどのチームがやっていることなので、これが誠実な比較対象です。1 人が全部読んでいる 10 件規模なら問題ありません。それを超えると、失敗は記憶が間違っていることではなく、しっかり裏付けられたポジションと思い出しただけのポジションを何も区別しないことです。結果として文書全体が同じ権威を帯びます。カバレッジレポートの価値は、自分自身に成績を付ける点にあります。
---
name: playbook-from-precedent
description: Derives a contract negotiation playbook from a folder of executed agreements — standard position and observed landing range per clause, a fallback ladder only where redline history supports one, and a coverage report naming every field the corpus cannot support. Never invents a walk-away.
---
# Playbook from precedent
Reads a corpus of executed agreements and produces a per-clause playbook: the standard position you actually hold, the range of places you actually land, the worst term you have ever signed, and an explicit list of what the corpus cannot tell you. It is read-only. It does not edit agreements, does not write to a CLM, and does not approve positions — a playbook position is a delegation of authority and belongs to a named person.
## When to invoke
- A playbook is being written or rewritten and the drafters are working from memory rather than from the book of business.
- An existing playbook is being audited against actual practice, and the question is where the written standard and the signed reality have diverged.
- Signature authority is being delegated to a business team and the escalation thresholds have to be defensible.
- A CLM rollout needs seed positions for its clause library and the vendor's extraction has produced frequencies with no segmentation.
## When NOT to invoke
- **Fewer than roughly 25 executed agreements on your own paper for the agreement type in question.** Below that the arithmetic in Phase 4 cannot separate a standard position from a coincidence, and the run produces a coverage report with almost nothing in it. Read the agreements.
- **You need the walk-away line.** Phase 5 explains why executed agreements cannot produce one. If that is the deliverable, this is the wrong tool.
- **The corpus is mostly counterparty paper and you cannot label which is which.** Phase 1 refuses. Unsegmented clause frequency across mixed paper measures your counterparties' drafting preferences and reports them as yours.
- **You want the playbook approved, not drafted.** The output is a proposal with evidence attached. Approval is a legal decision.
## Inputs
**Required:**
| Input | Type | Notes |
|---|---|---|
| `corpus_dir` | path | Directory of executed agreements. PDF, DOCX, or text. Subdirectories are walked. |
| `manifest` | path | `references/1-corpus-manifest.md`, filled in. One row per agreement. The run refuses without it. |
| `agreement_type` | string | One type per run — `msa`, `nda`, `dpa`, `saas_subscription`, `reseller`. Mixing types blends unrelated negotiations. |
**Optional:**
| Input | Type | Notes |
|---|---|---|
| `redline_dir` | path | Redline or version history keyed to the manifest's `agreement_id`. Without it, Phase 6 emits an observed range instead of a fallback ladder. |
| `clause_set` | list | Subset of the taxonomy in `references/2-clause-position-schema.md`. Defaults to the full set. |
| `recency_window_months` | integer | Split point for the drift test. Default 24. |
| `proposed_walk_away` | path | A human-supplied walk-away line per clause. Triggers the contradiction check in Phase 7. |
## Reference files
- `references/1-corpus-manifest.md` — the per-agreement manifest. Paper of origin, counterparty tier, deal-value band, execution date, redline availability. Fill this in before the first run; it is what makes every later segmentation possible.
- `references/2-clause-position-schema.md` — the clause taxonomy and the output record shape, including which fields the skill is forbidden to populate.
- `references/3-coverage-report-template.md` — the report that leads with what could not be inferred.
## Method
Seven phases, fixed order, with hard refusals in two of them.
**Phase 1 — intake and segmentation.** Every agreement is joined to its manifest row. The load-bearing field is `paper_of_origin`: `own`, `counterparty`, or `hybrid`. The skill refuses to proceed if more than 10% of the corpus lacks it, and it does not guess from the document. This refusal exists because the most common way a precedent-derived playbook goes wrong is silent: a term appears in 78% of executed agreements, gets written down as the company's standard position, and is in fact an artifact of 78% of those agreements having been signed on the other side's form. Frequency across mixed paper is a measurement of your counterparties.
**Phase 2 — verbatim extraction.** Each clause slot in the taxonomy is extracted as quoted text with a document and page pointer. Nothing is normalized in this pass. A clause slot with no match renders as `absent`, which is a finding — an MSA with no limitation-of-liability clause is a data point, not a gap in the extraction.
**Phase 3 — normalization.** The quoted text is converted to a comparable value in a stated unit: a liability cap becomes a multiple of trailing-12-month fees or `uncapped`; a payment term becomes days; an indemnity becomes a scope enum. This is a separate pass on purpose. A single pass conflates "the clause says X" with "X equals Y in our units," and once those are merged the normalization errors hide inside quotes nobody re-reads. Every normalized value keeps its source quote, so the second pass is auditable against the first.
**Phase 4 — position statistics, per cell.** Cells are `(clause, paper_of_origin, agreement_type, counterparty_tier)`. Within a cell, the modal normalized value is a candidate standard position, and it is reported with the Wilson score interval on its proportion — not the raw percentage. The threshold is a stated engineering choice: a candidate is labelled `standard` only when the 95% lower bound clears 0.60. That bound is what makes small samples behave. A term appearing in 8 of 10 agreements is 80% by naive count and has a 95% interval of roughly 49% to 94% — it cannot be called a standard position, because the interval spans "most of the time" and "barely more than half." The same 80% at 20 of 25 has a lower bound of about 61% and clears. So roughly 25 same-paper instances per clause is the practical floor, and cells below it render as `insufficient-evidence` with the n they would need, never as a weak position.
**Phase 5 — the censoring audit.** This is the phase that separates the output from a clause-frequency report, and it turns on a property of the corpus: **a folder of executed agreements is a censored sample.** It contains every negotiation that ended in signature and no negotiation that ended in a walk. So per clause the skill emits three fields and refuses to emit a fourth:
- `standard_position` — from Phase 4, with its interval.
- `observed_range` — the full distribution of landing spots, minimum to maximum, with counts.
- `worst_signed` — the single most adverse instance in the corpus, with agreement, date, and counterparty tier. This is an **upper bound on your demonstrated tolerance**: you did not walk at that term, because there is a signature on it.
- `walk_away` — rendered as `not-inferable-from-executed-agreements`, always. The walk-away is definitionally the position that produces no executed agreement, so it is the one field the corpus is censored on. A tool that emits a confident walk-away from signed contracts has invented it.
**Phase 6 — the fallback ladder, conditionally.** Executed text shows where you landed, not what you asked for first, so a `fallback_1` / `fallback_2` ladder is not recoverable from final PDFs. Two conditions gate it: `redline_dir` is supplied, and the cell's landing-spot distribution is multi-modal rather than a single cluster with noise. When both hold, the ladder is derived from opening-to-final movement across the version history. When either fails, the output carries `observed_range` under that name and says why the ladder is absent. Labelling a landing-spot spread as a fallback ladder is the same error as labelling a correlation a cause — it asserts an intent the evidence does not contain.
**Phase 7 — drift, contradiction, and report.** Cells are recomputed on the trailing `recency_window_months` and on everything older. Divergent cells are reported as two positions with dates, never averaged, because a 2022 liability cap averaged with a 2026 one describes neither. Then, if `proposed_walk_away` was supplied, the run tests it backwards against `worst_signed`: any asserted walk-away stricter than a term already in the corpus is flagged with the agreement and date. That flag matters commercially, not just editorially — a counterparty holding your prior agreement can produce it, and a red line you have already crossed is not a red line. Output goes to `references/3-coverage-report-template.md`, leading with the `insufficient-evidence` and `not-inferable` rows because those are the ones that need a human decision this week.
## Output format
One record per clause per cell, plus the coverage report. A literal record:
```yaml
clause: limitation_of_liability
cell:
paper_of_origin: own
agreement_type: msa
counterparty_tier: mid_market
n: 31
standard_position:
value: "12mo_fees"
proportion: 0.806
wilson_95_lower: 0.634
label: standard
observed_range:
- {value: "12mo_fees", n: 25}
- {value: "24mo_fees", n: 4}
- {value: "uncapped_for_ip_indemnity", n: 2}
worst_signed:
value: "uncapped_for_ip_indemnity"
agreement_id: MSA-2025-0184
executed: "2025-11-03"
counterparty_tier: enterprise
fallback_ladder: absent
fallback_ladder_reason: redline_dir_not_supplied
walk_away: not-inferable-from-executed-agreements
drift:
trailing_24mo: {value: "12mo_fees", n: 19}
older: {value: "12mo_fees", n: 12}
divergent: false
evidence:
- {agreement_id: MSA-2025-0184, page: 14, quote: "…shall not exceed the fees paid…"}
```
The coverage report opens with counts: clauses at `standard`, clauses at `insufficient-evidence` with the n required, cells suppressed for segment disagreement, and drift-divergent clauses. `walk_away` is a section, not a row — one line per clause naming who has to decide it.
## Watch-outs
- **The paper-of-origin confound.** A frequency built across mixed paper reads as your standard and is your counterparties' standard. **Guard:** Phase 1 refuses above a 10% missing rate, and `paper_of_origin` is a cell dimension rather than a filter, so a position derived on counterparty paper can never silently merge into the own-paper position.
- **Small-n false confidence.** A clause seen 8 times out of 10 looks decisive and is not. **Guard:** the `standard` label requires a Wilson 95% lower bound above 0.60, and short cells render as `insufficient-evidence` with the sample size needed rather than as a hedged position.
- **Inventing the walk-away.** The most valuable field in a playbook is the one the corpus cannot supply, which is exactly the pressure that produces a fabricated one. **Guard:** the schema in `references/2-clause-position-schema.md` gives `walk_away` a fixed enum with one permitted value, so there is no code path that emits a derived one.
- **A walk-away that contradicts your own file.** A drafter sets the red line at 12-month fees while the corpus contains a signed uncapped IP indemnity. **Guard:** Phase 7's backwards test flags it with the agreement ID and date, so the contradiction surfaces before a counterparty finds it.
- **Bargaining position read as preference.** Positions look weaker in the enterprise segment because that is where you conceded, not because your standard is different there. **Guard:** `counterparty_tier` is a cell dimension; when segments disagree materially the blended position is suppressed rather than reported, and the report names the suppression.
- **The playbook is stale on delivery.** DocJuris's survey of roughly 300 legal departments, published in October 2024, found that even at the most mature stage the recurring failure is ownership — the first version does not improve. **Guard:** the manifest persists between runs, so the second run is a diff against the first rather than a rebuild, and re-running per quarter costs a fraction of the initial pass.
- **Absence read as extraction failure.** A missing limitation-of-liability clause gets treated as a parse error and quietly dropped. **Guard:** Phase 2 emits `absent` as a value, and `absent` participates in the Phase 4 statistics like any other landing spot.
# Corpus manifest
One row per executed agreement. Fill this in before the first run — the skill refuses to proceed if more than 10% of the corpus lacks `paper_of_origin`, and it will not infer that field from the document. Everything the later phases can segment on comes from here.
The manifest persists between runs. The second run is a diff against this file, not a rebuild.
## Part A — how to fill each column
| Column | Values | How to decide |
|---|---|---|
| `agreement_id` | your identifier | Must match the filename stem in `corpus_dir` and, if you supply one, the folder name in `redline_dir`. |
| `filename` | relative path | Relative to `corpus_dir`. |
| `agreement_type` | `msa` / `nda` / `dpa` / `saas_subscription` / `reseller` | One type per run. If a document is a hybrid, file it under its governing form. |
| `paper_of_origin` | `own` / `counterparty` / `hybrid` | **The load-bearing column.** `own` means the negotiation started from your template. `counterparty` means theirs. `hybrid` means a materially rewritten form where neither side's template survived — use it sparingly, because hybrid rows are reported as their own cell and a large hybrid population usually means the column was filled in by guessing. |
| `counterparty_tier` | `smb` / `mid_market` / `enterprise` / `public_sector` | Use your own segmentation if you have one and record the definition in Part C. Consistency matters more than the labels. |
| `deal_value_band` | `under_50k` / `50k_250k` / `250k_1m` / `over_1m` / `unknown` | Annual contract value at execution, in USD. `unknown` is permitted and is reported as its own band rather than imputed. |
| `executed_date` | `YYYY-MM-DD` | Execution date, not effective date, and not the date the file was scanned. The drift test in Phase 7 keys on this. |
| `redline_available` | `yes` / `no` | `yes` requires a matching folder in `redline_dir`. A `yes` with no folder fails intake for that row. |
| `governing_law` | jurisdiction | Optional but recommended. A liability cap under one governing law is not always the same instrument as the same number under another. |
| `notes` | free text | Anything that would change how a reader interprets this row — a distressed renewal, a settlement-adjacent signature, an acquisition-driven novation. |
## Part B — the rows
Replace these examples. Keep the header.
```csv
agreement_id,filename,agreement_type,paper_of_origin,counterparty_tier,deal_value_band,executed_date,redline_available,governing_law,notes
MSA-2024-0031,msa/acme-msa-executed.pdf,msa,own,mid_market,50k_250k,2024-03-11,yes,Delaware,
MSA-2024-0047,msa/globex-msa-executed.pdf,msa,counterparty,enterprise,over_1m,2024-06-28,no,New York,their paper, procurement-led
MSA-2025-0184,msa/initech-msa-executed.pdf,msa,own,enterprise,250k_1m,2025-11-03,yes,Delaware,uncapped IP indemnity conceded at close
MSA-2026-0009,msa/soylent-msa-executed.pdf,msa,hybrid,mid_market,50k_250k,2026-01-22,yes,England and Wales,
```
## Part C — segmentation definitions
Write down what your tier labels mean. Two people filling this manifest with different definitions of `enterprise` produces a segment that reports a blended position for two different negotiations, and nothing downstream can detect it.
- `smb` — replace with your definition (headcount, ACV, or account-team ownership).
- `mid_market` — replace.
- `enterprise` — replace.
- `public_sector` — replace. Keep this separate from `enterprise` even at similar deal sizes; procurement constraints make the positions non-comparable.
## Part D — known exclusions
List agreements deliberately kept out of the corpus and why. This is the counterpart to the censoring audit in Phase 5: the skill can report what the corpus does not contain by construction, but it cannot report what you removed by hand.
| `agreement_id` | Reason for exclusion |
|---|---|
| MSA-2023-0112 | Settlement-driven terms; not representative of a negotiated position. |
| *(add rows)* | |
# Clause taxonomy and position schema
Defines the clause slots the skill extracts and the shape of each output record. Edit the taxonomy to match your forms; do not edit the field constraints, because two of them are what stop the run from asserting things the corpus cannot support.
## Part A — clause taxonomy
One slot per row. `unit` is the unit normalization must produce in Phase 3; a slot with no stated unit cannot be normalized and will render as verbatim-only.
| Slot | Unit | Normalization notes |
|---|---|---|
| `limitation_of_liability` | multiple of trailing-12-month fees, or `uncapped` | Record super-caps and carve-outs as distinct values, not as footnotes to the general cap. |
| `ip_indemnity_scope` | enum: `none` / `defend_only` / `defend_and_indemnify` / `defend_indemnify_uncapped` | |
| `data_breach_liability` | multiple of fees, `uncapped`, or `follows_general_cap` | Frequently carved out of the general cap; if so it is its own slot value, not a variant of `limitation_of_liability`. |
| `payment_terms` | days | Net days from invoice. |
| `termination_for_convenience` | enum: `none` / `customer_only` / `mutual`, plus notice days | Two values; record both. |
| `auto_renewal` | enum: `none` / `annual` / `monthly`, plus notice-window days | |
| `governing_law` | jurisdiction string | Not normalized further. |
| `dispute_resolution` | enum: `courts` / `arbitration` / `arbitration_with_carve_outs` | |
| `warranty_period` | days | |
| `assignment_on_change_of_control` | enum: `free` / `consent_required` / `consent_not_unreasonably_withheld` | |
| `insurance_minimum` | USD per occurrence | |
| `audit_rights` | enum: `none` / `annual` / `on_notice` / `unlimited` | |
Add slots for the terms your negotiations actually turn on. A slot nobody argues about produces a clean, high-confidence, useless position.
## Part B — the position record
```yaml
clause: <slot>
cell:
paper_of_origin: own | counterparty | hybrid
agreement_type: <type>
counterparty_tier: <tier>
n: <integer> # instances in this cell
standard_position:
value: <normalized value>
proportion: <float>
wilson_95_lower: <float>
label: standard | insufficient-evidence
n_required: <integer> # present only when label is insufficient-evidence
observed_range:
- {value: <normalized value>, n: <integer>} # every landing spot, including `absent`
worst_signed:
value: <normalized value>
agreement_id: <id>
executed: <YYYY-MM-DD>
counterparty_tier: <tier>
fallback_ladder: absent | [<value>, <value>, ...]
fallback_ladder_reason: <string> # required whenever fallback_ladder is absent
walk_away: not-inferable-from-executed-agreements
drift:
trailing_24mo: {value: <normalized value>, n: <integer>}
older: {value: <normalized value>, n: <integer>}
divergent: true | false
evidence:
- {agreement_id: <id>, page: <integer>, quote: <verbatim text>}
```
## Part C — field constraints that are not editable
Three constraints carry the design. Relaxing any of them turns the output back into a clause-frequency report with a playbook's title.
**1. `walk_away` is a single-value enum.** The only permitted value is `not-inferable-from-executed-agreements`. There is no code path that derives one, because the corpus is censored on exactly that variable: a walk-away is the position that produces no executed agreement, so every instance of it is absent from the folder by construction. `worst_signed` is what the corpus does support — an upper bound on demonstrated tolerance, evidenced by a signature.
**2. `label: standard` requires `wilson_95_lower` above 0.60.** The Wilson score interval on the proportion, not the raw percentage. A term at 8 of 10 is 80% by count with a 95% interval of roughly 49% to 94%; the same 80% at 20 of 25 has a lower bound near 61%. The first is a coincidence you can describe, the second is a position you can defend, and only the interval tells them apart. Cells that fail render as `insufficient-evidence` with `n_required` — never as a softened standard.
**3. `fallback_ladder_reason` is required whenever the ladder is absent.** Final executed text records where you landed, not what you opened with, so a ladder needs the redline history plus a genuinely multi-modal distribution. Absent either, the spread is reported as `observed_range` under that name. The required reason field is what stops "we had no version history" from being silently indistinguishable from "there is no ladder here."
## Part D — your position overrides
Positions the business has decided regardless of what the corpus says. Record them here rather than editing derived output, so a later run can diff intent against practice — the divergence between the two is usually the most useful thing the exercise produces.
| Slot | Asserted position | Owner | Decided on | Rationale |
|---|---|---|---|---|
| `limitation_of_liability` | 12mo_fees | *(name)* | *(date)* | *(why)* |
| *(add rows)* | | | | |
Asserted walk-aways go in the same table with an `asserted_walk_away` column. Supplying that file as `proposed_walk_away` triggers the Phase 7 contradiction check against `worst_signed`.
# Coverage report
The run's primary deliverable. It opens with what the corpus could not support, because those rows need a human decision and the derived positions do not.
Replace the bracketed values. Keep the section order — it is the order a reviewer should read them in.
---
## Header
- **Agreement type:** `[msa]`
- **Corpus:** `[N]` agreements, executed `[earliest]` to `[latest]`
- **Own paper / counterparty / hybrid:** `[a]` / `[b]` / `[c]`
- **Redline history supplied:** `[yes | no]` — `[k]` of `[N]` agreements
- **Excluded by hand:** `[j]` agreements (see manifest Part D)
- **Run date:** `[YYYY-MM-DD]`
## 1. Not inferable — decide these
One row per clause. Every clause appears here, because `walk_away` is never derived from executed agreements. `worst_signed` is the bound the corpus does supply: you did not walk at that term.
| Clause | `worst_signed` | Agreement | Executed | Walk-away owner | Decided? |
|---|---|---|---|---|---|
| `limitation_of_liability` | `[uncapped_for_ip_indemnity]` | `[MSA-2025-0184]` | `[2025-11-03]` | `[name]` | `[ ]` |
| *(one row per clause)* | | | | | |
## 2. Contradictions — asserted walk-away already crossed
Populated only when `proposed_walk_away` was supplied. Each row is a red line that appears in a signed agreement in your own file. A counterparty holding that agreement can produce it.
| Clause | Asserted walk-away | Contradicting term | Agreement | Executed |
|---|---|---|---|---|
| `[limitation_of_liability]` | `[12mo_fees, hard floor]` | `[uncapped_for_ip_indemnity]` | `[MSA-2025-0184]` | `[2025-11-03]` |
| *(add rows, or state "none")* | | | | |
## 3. Insufficient evidence — sample too small to call
These clauses have a modal value and not enough instances to defend it. `n_required` is what the cell needs to clear a 95% lower bound of 0.60 at the observed proportion.
| Clause | Cell | `n` | Modal value | Proportion | Wilson 95% lower | `n_required` |
|---|---|---|---|---|---|---|
| `[audit_rights]` | `[own / mid_market]` | `[9]` | `[annual]` | `[0.78]` | `[0.45]` | `[24]` |
| *(add rows)* | | | | | | |
## 4. Suppressed — segments disagree
Cells where the position differs materially by counterparty tier or deal-value band, so no blended position is reported. This is usually bargaining position showing up as if it were preference.
| Clause | Segment A | Segment B | Suppressed blend |
|---|---|---|---|
| `[limitation_of_liability]` | `[mid_market: 12mo_fees, n=25]` | `[enterprise: 24mo_fees, n=11]` | `[yes]` |
| *(add rows)* | | | |
## 5. Drift — position moved
Trailing window against everything older. Reported as two positions, never averaged.
| Clause | Trailing `[24]`mo | Older | Divergent |
|---|---|---|---|
| `[payment_terms]` | `[45 days, n=19]` | `[30 days, n=22]` | `[yes]` |
| *(add rows)* | | | |
## 6. Fallback ladders absent, and why
| Clause | Reason |
|---|---|
| `[limitation_of_liability]` | `[redline_dir_not_supplied]` |
| `[payment_terms]` | `[distribution_unimodal]` |
| *(add rows)* | |
## 7. Derived positions
Everything that cleared. This section is last on purpose — it is the part that needs the least review.
| Clause | Cell | `n` | Standard position | Wilson 95% lower | Observed range |
|---|---|---|---|---|---|
| `[limitation_of_liability]` | `[own / msa / mid_market]` | `[31]` | `[12mo_fees]` | `[0.634]` | `[12mo (25), 24mo (4), uncapped-IP (2)]` |
| *(add rows)* | | | | | |
## 8. Next run
- Manifest rows added since last run: `[n]`
- Clauses that moved from `insufficient-evidence` to `standard`: `[list]`
- Clauses still short of `n_required`: `[list]`
- Owner and date for the next pass: `[name]`, `[YYYY-MM-DD]`