Mizan System — Usage Guide (v1.0)

Evidence-tiered research & engineering discipline: install, workflows, rules

Turkish original: docs/tr/kullanim-kilavuzu.md

This guide is the permanent reference. The system has three layers, based on the architecture that survived the audit of the original proposal — the rest was discarded:

Layer	Component	Responsibility	Where it lives
Behavior	userPreferences + Mizan Project Instructions	How the AI behaves	Claude settings + Project instructions
Audit	Mizan skill (v3.0)	How a claim is evaluated	Installed as a skill
Data	mizan-registry.yaml	How research knowledge is stored	Project knowledge / repo

The three layers work independently: the schema is readable without the skill, and the skill audits without the schema. Together they are the full system.

---

1. Install (once)

1. Skill: upload mizan.skill to Claude (Settings → Capabilities → Skills). On updates, upload the new version over the old one — it does not auto-update. Current version: v3.0 (7 modes + schema).
2. Project instructions: paste the block from project-instructions.md into the “Project instructions” field of every Claude Project you run with Mizan.
3. Schema: copy the mizan-registry.yaml template, fill in your project name, and add it as a file to Project knowledge OR keep it in your repo (recommended: in the repo — the git diff IS the schema’s design rationale).
4. For other AIs: the schema and documentation are model-independent. If you give ChatGPT/Gemini the schema file + this documentation, they work with the same discipline (with a weaker triggering guarantee).

2. Daily workflows

2.1 New hypothesis (preregistration)

When a testable claim arises: say “preregister this in the registry” or approve Claude’s suggestion. Claude produces an H-entry in schema format — the threshold and refutation condition are LOCKED BEFORE ANY RESULT IS SEEN. Add the entry to your registry file (Claude cannot directly edit the file in project knowledge; it produces the up-to-date block and you paste it — if you work in a repo, Claude Code can write it directly).

2.2 Experiment and result

Two fields are non-negotiable in an experiment entry: baseline (if none, a written justification; a baseline-less experiment can never promote a hypothesis to [K]) and confound_controls (each item in the hypothesis’s confound list is either controlled or explicitly accepted as a risk). The result block is append-only; honesty_annexes cannot be empty.

2.3 Surprising-positive protocol

If a result is better than expected, mark surprising_positive: true and the [K]-promotion waits for the symmetric control (“is it this specific mechanism, or a generic alternative?”). The headline is published after the control.

2.4 Audit requests (Mode 1)

Hand over any claim set — an AI summary, report, or self-assessment — by saying “audit this with Mizan”. Output: a coverage declaration, a tiered claim table, a counter-example sweep, a hit rate, the missing card, and next steps.

2.5 Code audit (Mode 3)

“Audit this repo/module with Mizan.” Claude extracts a claim inventory (tests → comments → names → docs), verifies each hop separately (a comment’s existence ≠ a behavior’s existence), and produces an evidence-tiered behavior report + a Gap Map. In an undocumented project, that report IS the documentation. A coverage declaration is mandatory on a large codebase — a sampled audit is never presented as a full one. When you want full coverage on a repo too big for one pass, ask Claude to phase it: sequential slices sharing one append-only registry plus a Coverage Ledger (procedure in code-audit.md §A5.1) — no need to hold the whole repo in one session.

2.6 Bug hunt (Mode 4)

When starting to debug: “register this as a bug hypothesis.” The symptom (no interpretation), the mechanism hypothesis (specific enough to be wrong), a rival hypothesis, and a refutation test are recorded. “The fix worked” is a surprising positive: no mechanism becomes [K] without a revert-check. After 10–15 entries you get your real bug-instinct hit rate.

2.7 Feature gate (Mode 5)

For a new feature/PRD: “gate this.” The PRD is atomized (problem / value / cost / dependency / scope claims are tiered — dependency claims are verified NOW, not mid-sprint), the success threshold AND the kill condition are preregistered, and alternative-forcing is applied (≥1 cheaper alternative + the null alternative, on the same metric). Acceptance criteria are written as refutation conditions. Candidates from the Gap Map are an evidence-carrying, ready-made backlog.

2.8b Security probe (Mode 7)

“Run the security probe.” Modes 3-5 start from a sentence someone wrote; a vulnerability is the sentence nobody wrote, so they cannot reach it. Mode 7 maps the trust boundaries first – who is on the far side, what they control, and what the near side ASSUMES about them – and derives the scenarios from a threat model rather than from the builders, who cannot name an assumption they never knew they made. The tier rule is inverted: not exploited is not a pass. A failed attempt and a clean scanner are silence and cap the entry at [KKE] (R26); only a named control, shown on every path to the asset, promotes. A Mode 7 pass that finds nothing produces a [KKE] coverage statement, never a clean bill of health.

2.8 Meta-review (Mode 6)

Every ~10 entries or on request: which hypothesis types hit, which instruments are reliable, where bias is accumulating. The output feeds the methodology — the instructions and schema are themselves subject to revision (following their own discipline: changes are justified, history is never deleted).

3. Hard rules (summary — R1–R28 in the schema)

1. Threshold + refutation condition before any result (HARKing structurally closed).
2. Baseline mandatory; a baseline-less result cannot produce [K].
3. Every confound is either controlled or an explicitly accepted risk.
4. History is append-only; [R] is never deleted.
5. Honesty annexes cannot be empty.
6. Surprising positive → symmetric control before promotion.
7. Producer ≠ sole auditor: a tier change is proposed, then the owner / a separate audit confirms it. (RSI safety: the agent that writes its own result cannot promote its own hypothesis.)
8. Every threshold names its arbiter — the judge that returns the verdict (runtime / instrument / third_party / author / none). A self-judged claim (author) carries a permanent [KKE] and can never reach [K]; with no arbiter (none) the threshold is decorative and the entry stays [S]. This is what keeps the discipline honest outside software, where the runtime no longer decides for you.

Four more come in at schema 1.4, and each one closes a place where the schema permitted what the prose already forbade:

1. A [K] entry is backed by a result that met its threshold, or by a cited external source. Writing the tier is not the same as earning it. Prior art does not count: relatives are context for originality, not evidence for this claim.
2. Thresholds name a quantity. “Improves things” is the guide’s own counter-example; there is an explicit escape hatch (non_numeric_justification) for the rare categorical verdict.
3. Every entry carries a tier. The old check only validated the spelling of a tier that happened to be there.
4. The mandatory entry fields exist: formal, metric with a named instrument, cost, status, and prior_art present — “no known relatives” is an answer, an absent field is silence.

Three more apply to the specialised blocks, from schema 1.3 onward. A bug entry and a feature gate ARE hypotheses — same fields, same rules 1–8 — plus:

1. A feature names its kill condition: the post-ship measurement that would justify removing it. Without one it can only accumulate cost, because nothing is defined that would end it.
2. A feature lists its alternatives, including at least one cheaper option and the null alternative (“do nothing”), priced on the same value metric. This is where a feature nobody needs gets killed before it is built.
3. A bug names at least one rival hypothesis — another mechanism that produces the same symptom. A mechanism with no rival is a story: nothing distinguishes it from the first explanation that came to mind.

Registries declaring 1.0–1.4 keep working unenforced; bump the version when the entries are ready. The shipped schema version is the one in the banner of skill/mizan/schemas/mizan-registry.yaml.

1. A coverage claim waits for reconciliation. For a phased audit, the coverage block’s claim_tier may only be [K] once every phase row and the MERGE row are done. “Each slice fully audited” is not “the target fully audited”, and presenting the second as the first is itself a [Y]. The ledger lives in the registry rather than beside it, so the append-only rule protects its rows like any other entry.

2. An open hypothesis carries a review_by date, and once that date passes it must record a decision: extend it with a reason and a new date, park it (status: dormant), or close it. This is the rule against the quietest way a registry rots — nobody writes a false [K], the entries simply stop moving, and a year later a hypothesis that was preregistered and never tested is indistinguishable from one that was only ever sketched. The registry still looks full. Enforced from schema 1.6.

3. A preregistration says what it locked. An entry carrying the [önkayıt] tag states three things: its data status (does the data exist yet, and have you seen it — the last two cannot be preregistered, because a hypothesis written against data you have already seen is postdiction, and saying so is the honest move), its stopping rule (when collection ends) and its exclusion rule (which observations get dropped and by what rule — “none” is a valid answer, blank is not). Schema 1.7.

4. The domain probe is recorded (probes.domain): the situations that actually occur in the field, who supplied them, and whether they were written before the work. A tier-[K] coverage claim requires them — at least one scenario, none left unchecked, and supplied_by outside {auditor, none}. That gate is R8’s arbiter logic one level up: a scenario list the auditor invented is self-report, exactly like a threshold judged by its own author.

5. The conjunction pass is recorded (probes.conjunction): feature × the existing guarantee it can reach, with the safe order when the pair is asymmetric. Atomizing destroys any defect that lives only in a pair, so a pass that leaves no row cannot be told apart from one that never ran. A green test suite is not counter-evidence: tests are written per feature.

6. Every escape names a class (probes.escaped): class_ref, the check that should have fired, or class_new, the check that now exists because of it. The scorecard already counted escapes and called that count the only measure that cannot be gamed from inside the audit. A count is not a loop.

7. A cost claim states its arm. cost_actual names the instrument that produced the numbers (a script, an invoice, a dashboard — “the model estimated it” is not one), the measurement window, the attribution (how spend was mapped to this entry, and by whom — a human judgement by construction), and baseline.kind: none · internal-phase · parallel-arm · historical. none is legal, honest, and caps the entry below [K]. What the work cost is a measurement; that the tool caused the difference is a claim, and with no comparison arm nothing separates it from the boring explanations that produce the same number — a codebase the team now knows, a smaller backlog, a second attempt at a problem already solved. R2, pointed at the claim teams most often make without a control. Schema 1.9.

Rules 23–25 came from one audit that missed four defects a user found by hand the same day — every one of them lying between slices, and none reachable by the rules above. Each closes a place the method trusted without checking.

1. MERGE reconciles the pairs nobody looked at. A MERGE row marked done carries cross_slice: the slice boundaries actually examined, each with an outcome. Empty, or every pair unchecked, fails. Slices are cut by container (module, path, surface); a defect living in a seam appears in neither slice’s findings, so reconciling the findings can never find it.

2. The auditor’s own instruments get calibrated too. A [K] entry whose metric.instrument_built_by is auditor carries instrument_validated: the record of running that instrument against a known positive and a known negative. A scanner written during an audit is not a lesser instrument; it is an uncalibrated one.

3. A runtime verdict names the artifact it ran against. A result with threshold_met: yes on a runtime arbiter carries artifact_freshness: how it is known the arbiter ran the code under audit. A stale build reports green, red and a full passing suite with equal confidence — not a weaker measurement, a false assurance. Schema 1.10.

4. Not exploited is not a pass (Mode 7). An entry resting on security_evidence: attack_failed | scanner_silence may not sit at [K]: a failed attack is one adversary’s attempt, and a quiet scanner is a statement about its own rule set. Only blocked_by promotes — a named control (control_ref) on every path to the asset, judged by someone other than the person who ran the attempt.

5. A security coverage claim maps its trust boundaries. A coverage block claiming [K] with security_scope: true needs probes.adversary: each boundary with who / controls / near_side_assumes, scenarios written as actions, none unchecked, written before the work, and supplied by someone other than the auditor. R19 with the supplier changed: nobody can name an assumption they never knew they made. Schema 1.11; procedure in references/security-probe.md.

6. The audit lists what the repository produces. A coverage block with finished phases carries coverage.produced_artifacts — each generated file and whether it was inspected (no / partial need a why_not) — or waives it in one sentence. Slices are cut from the source tree, which holds what a human typed; a generated config sits in no slice at all. Running the producer and reading its output is what catches it. Schema 1.12.

Warnings (W1–W8) advise, they do not block. An entry with no two_sided statement; an entry with no result yet and no threshold or refutation; a registry where every tiered entry sits at [K]; a coverage ledger with no MERGE row; a coverage block with no domain or conjunction probe; every phase done while the domain probe was never answered (W6 — waiving is a decision, silence is not); an adversary map with no supply-chain or CI boundary (W7); and a met runtime threshold with no failure_is_loud line saying how that arbiter fails (W8 — a hung run returns no verdict, and no verdict reads like patience). Each has a legitimate exception — a draft entry, a one-entry registry — which is why none is a rule. --strict promotes them to failures; CI runs strict on the files whose job is to model the discipline (schema, templates, examples) and advisory everywhere else, so a warning nobody has acted on yet does not block an unrelated change.

4. Common mistakes

• Turning the schema into a form: not every chat idea needs preregistration; exploratory conversation is free. Preregistration kicks in the moment you decide a claim goes to a resource-consuming test.
• Threshold shopping: “essentially met” is forbidden; a near-miss is a near-miss, with a single justified rerun allowed.
• Counting comment-existence as behavior-evidence (in code audit): each hop is verified separately — this was the system’s first live finding (discovery.py:105, a “2 levels” comment vs. an unbounded rglob).
• Treating audit output as final: an audit is also a retrospective document; read its coverage declaration, note the inaccessible sources.
• Bloating the registry into one file: one registry per project; at 100+ entries, split by area (registry-bugs.yaml, registry-feat.yaml).

5. First real task (preregistered)

Migrating your SpectralLM experiment registry (experiment_registry_and_metrics.md) to the mizan-registry.yaml schema — with the threshold locked last session: the schema carries ≥20 real entries without requiring a structural rewrite AND is read and updated at least once by a tool/model other than you → then the standardization discussion opens. Otherwise the ORP idea is [R] and the schema stays a personal tool — which is also a valid outcome.

6. File inventory

File	Contents
mizan.skill	Skill package v3.0 (7 modes + embedded schema)
mizan-registry.yaml	Schema template (standalone copy)
docs/en/project-instructions.md	Project instructions block + diff analysis
skill/mizan/SKILL.md + references/	Mode 1–5 English reference
docs/tr/	Full Turkish documentation
This guide	Install, workflows, rules