ux-mizan
Docs
GitHub ↗ the family ↗

Methodology — the core skill

Mirrored from the skill package. This is the canonical English text that Claude actually loads, published here so it can be read next to its Turkish mirror instead of only on GitHub. Edit the source above, not this page; the Turkish mirror is docs/tr/metodoloji.md.

ux-mizan — Evidence-Tiered UX Auditing

Status: v0.7 [H] / [KKE]. This skill has not yet produced a good audit of a real application. Every architectural decision in it is [H] until it does. Say this in every deliverable — “the skill produced it” is not evidence, and the authority illusion is this skill’s own risk #3.

The load-bearing fact — read this before anything else

A model cannot MEASURE UX by looking at code or screenshots. UX is behavioural: it exists in what a real user does. A model can audit heuristic and structural conformance, and it can build the measuring rig. When the model is both the producer and the judge, its output is [KKE] at best. If the referee writes it, there is no [K]. When you are about to review something you produced earlier in the same run, say so before you do it — the undeclared switch is the failure, not the switch (references/recovery.md, R-00).

So the architecture has two layers:

Every Layer-A finding stays [H] until Layer-B data confirms it. A hypothesis that data knocks down becomes [R] and is never deleted — the [R] rows are this audit’s own error rate.

Evidence tiers (identical to Mizan — bilingual, do not localise the tags)

Tag TR EN Meaning
[K] Kanıtlanmış Proven Direct evidence supports it; source cited; threshold met
[H] Makul Hipotez Plausible hypothesis Theoretical grounding exists; empirical support missing or below threshold
[S] Spekülatif Speculative Interesting; not testable or no test designed
[R] Reddedildi Refuted Tested and failed its own threshold — kept on record
[KKE] Kritik Kontrol Eksik Critical control missing Result exists; the control that could flip it has not run
[Y] Yanıltıcı Misleading Technically true; implies more than the evidence supports

Tier drift is a finding. Premise drift is also a finding — see Gate re-opening.

The gates — a DAG, not a checklist

Gate 0 (purpose/priority) ──> Gate 1 (type/constraint/volume/mode)
        │                             │
        │                    [AUTONOMOUS AUDIT: Layer A]
        │                             │
        └─────────────────────> Gate 2 (hypothesis approval)
                                      │
                                Gate 3 (direction approval)
                                      │
                           [BUILD / SPEC / HANDOFF]
                                      │
                                Gate 4 (tier promotion) <── [Layer-B data]

Hard gates (the model may not pass them alone): 0, 3, 4. These are product and human decisions. Soft gates: 1 (partly) and 2. With no answer the skill does NOT stop — it takes a default, marks it [H], records locked_by: model-default, and carries on.

At every gate: the model proposes, the human locks. Ask in batches, single-choice where possible, never more than a few at a time — a gate is a negotiation, not an interrogation. Read references/gates.md before running the first gate in a conversation; it holds the actual question sets, the defaults, and the re-opening protocol.

The one thing worth memorising here: Gate 0 output is a premise, not a finding. If the model infers the app’s purpose and then audits against its own inference, the loop closes and the audit becomes unfalsifiable. ux_validate.py enforces this (U7): until a human locks Gate 0, every finding on that flow is capped at [H].

Procedure

  1. Gate 0 + Gate 1. Elicit and lock the prioritised job model and the app’s type, platform, maturity, volume, and effort budget. Write Table A (flows) from the answers. references/gates.md.
  2. Choose the mode from the Gate-1 volume answer (see Modes below).
  3. Layer A — walkthrough. Walk each flow step by step against the four Norman questions. references/walkthrough.md holds the output shape and the refutation-threshold column.
  4. Layer A — structural checks. Run scripts/structural_checks.py <src> for the React/TS proxies. Every count it returns is [KKE]; it names a place to look, not a defect. If the host has no shell, say so and do the same checks by targeted search — never promise an artifact you cannot produce.
  5. Layer A — layout signals. Run scripts/layout_signals.py <src>. Where the structural checks look for what is MISSING, this looks at how what exists is ARRANGED: items rendered all-open with no collapse, a “current” item marked by colour alone, screens with nothing to scan by, control density inside a repeated row. It prints two tiers per signal — the trace [KKE] (checkable in the file) and the behavioural claim [H] (what it may do to a user) — plus the metric that would settle it. Never report the [H] line as though the file proved it. This is the class of problem walkthrough.md declares itself blind to: cumulative load is invisible step by step, because each step looks fine.

  6. Layer A — cross-file consistency. Run scripts/component_inventory.py <src>. Where the structural checks ask what one view is missing and the layout signals ask how one screen is arranged, this asks what only exists BETWEEN screens: how many distinct raw colour and spacing values escaped the scale, whether the same interactive role is built several ways, how much is defined inside a page and used once. A user learns an interface once per pattern, not once per app, so an app with several idioms for one control charges the learning cost several times — but the SIZE of that cost is not in the file, which is why each signal names the metric that would settle it.

    This one carries risk #1 in its hand. It measures whether the app agrees with itself, never whether it resembles anything else, and a divergence that is deliberate — a destructive action that should look unlike the rest — reads here exactly like one that is not. Any fix proposed from these signals answers the homogenisation question in writing; U10 already refuses one that does not.

  7. Re-assemble — walk the PAIRS, not only the flows. Steps 3–6 all walk one flow, one screen or one file at a time, which is exactly why none of them can see a defect that exists only while two flows are active at once: each flow is correct alone, its tests pass alone, and the guarantee that breaks belongs to neither. So build the pair list deliberately — for each flow, the existing guarantees another flow can reach — and ask of each pair: does that guarantee still hold while both are live? Two classes break this way most often:
    • Signals derived from an absence — “not yet read”, “awaiting payment”, “untouched for 3 days”. A second flow that creates a new state changes what the absence means, silently.
    • Guarantees enforced call site by call site — a rule applied correctly on five screens is voided wholesale by a sixth surface, bulk export being the classic one.

    Order counts: some pairs are safe in one direction only, and the safe direction is part of the finding. File the result as a conjunction finding — both flows, the guarantee, the order — with the heavier flow as the parent, because severity reads the weight from the parent (U13). Until schema 0.6 this pass had nowhere to file its result, which is the honest reason it was not in this procedure before.

  8. Gate 2. Put the candidate findings in front of the human BEFORE they enter the registry. This is where preregistration locks: a threshold written after the data is HARKing.
  9. Write the registry. Append each confirmed finding to ux-registry.yaml as it is settled — the registry is the memory, the transcript is not. Batching findings for a summary at the end loses them at the next context reset and pays for them on every turn until then.
  10. Validate. python scripts/ux_validate.py ux-registry.yaml. Fix what it rejects; do not argue with it in prose.
  11. Gate 3. Before ANY redesign or model handoff, the human approves the diagnosis and the intended direction. What they approve is not pixels — it is the diagnosis, the priority frame, and the direction. Approving a mockup is approval theatre. references/handoff.md.
  12. Instrument. Emit the measurement plan: which metric, which instrument, what N, what decision rule — all locked before collection. references/metrics.md.
  13. Gate 4. Only real Layer-B data plus a human decision promotes anything to [K]. The validator will not let you write [K] without a resolving evidence artifact anyway (U1).

Modes — graded, because rigour has a friction cost

Applying preregistration + event tests + a two-model handoff to every app can cost more in delayed iteration than the rigour returns.

The boundary [H]: fewer than ~300 completions per flow per month → Lite. Above it, passive telemetry can reach a decision inside a four-week window. This number is a design estimate, not a measured threshold; the self-validation run is the first thing that could refute it.

The four rules that carry the discipline (all in the validator)

They live in scripts/ux_validate.py because prose is negotiable by the host’s prose and a script is not.

  1. Structural [K] lock (U1). tier == K with no resolving evidence_artifact_id, or with model-side provenance, is REJECTED.
  2. Flow primacy (U2). parent_flow_id is mandatory and must resolve. A defect that exists only across two flows is not an exception to this: it names one parent and its pair in conjunction (U13, schema 0.6+), with the heavier flow as the parent so severity cannot be chosen. Scoring a component in isolation is impossible at the schema level.
  3. Auditable severity (U3). severity = failure_magnitude × priority_weight × frequency, recomputed by the validator, with priority_weight read from the human-locked flow. The model cannot re-rank the backlog.
  4. Append-only, [R] permanence, provenance (U4/U7). Nothing is deleted; every finding carries where it came from and which premise version it rests on.

U5–U10 add refutability, the metric-applicability gate, baseline-before-a-behavioural-[K], a locked min_n, and — since a paragraph could not carry it — the rule that any proposed fix must answer the homogenisation question in writing (U10). U11 makes a [KKE] say WHICH control is missing (control / independence / data / validation), because “a control is missing” is not actionable until it names one. --strict promotes the advisory W1–W4 to failures; CI runs strict, local runs do not.

Metric applicability — good genericity, not one-size-fits-all

Bad genericity is the same metric battery on every app. Good genericity is reading the context and switching on the valid subset. app_type gates the metrics, and the gate is machine-checked (U6): lostness on a single-canvas tool is a category error and fails validation. The matrix lives in schemas/ux-registry.yaml and is hard-coded in the validator.

Lostness (Smith, 1996), L = sqrt((N/S − 1)² + (R/N − 1)²), is the one metric that measures being lost without asking the user — but only when the flow declared its goal and its canonical_path_R in advance. scripts/lostness.py computes it per flow and prints its four caveats with every number. Read references/metrics.md before instrumenting.

Baseline is mandatory (§Baseline in references/metrics.md). “L=0.55” alone means nothing. Measure the current build before any redesign. And a high L may not be lostness at all — the task may simply be many-stepped; hunt the alternative explanation for a bad result as hard as for a good one.

The positive-control rule

A new detector is run against a case known to be positive before it is trusted on unknown ones. This is the scientific positive control, applied to static analysis, and it is here rather than only in the contributor docs because it is the rule this project has broken most often.

Every structural check shipped in this skill has, at least once, silently under-reported: a regex whose trailing  could never match, an overlay counted as a collapse, a colour test that missed a bare utility class. All three read the same way from outside — a clean codebase. A false positive argues with you; a false negative agrees with you, which is why it survives review.

So when you add or change a signal:

  1. Find a file where the problem is known to exist.
  2. Run the check and confirm it fires there.
  3. Only then trust it on files where you do not know the answer.
  4. Report the before/after on that real file, not on a synthetic fixture — a fixture is written to match the regex you just wrote.

The same rule applies to a clean scan result. “No signals” never means “no problems”; it means the checks that ran found nothing, and it should be reported in those words.

First run = self-validation

The first thing this skill does is audit ONE real application end to end. That run settles the “general tool or single-app hack” question: the real app is the skill’s test fixture, not its product. Its output feeds v1.0, and until it exists the status line at the top of this file stands.

Known risks (name them in the deliverable, do not bury them)

  1. Recursive homogenisation [H] — a skill that recommends redesigns drifts toward the generic default (the shadcn/Tailwind mean) and can spread the very disease it diagnoses. Not fully solvable. Mitigation: Gate 0’s specific purpose, Gate 3’s human approval, and U10 — a finding with a fix and no self_check_homogenisation is REJECTED by the validator. That last part was prose until v0.3, and this skill’s own prose-vs-schema audit called the gap a [Y].
  2. Erosion of the [K] ban — prose decays over a long session, so the ban is structural (U1).
  3. Authority illusion — every output shows status, which gates were passed, and what evidence exists.
  4. Friction tax — Lite/Full split.
  5. Trigger overlap — mizan audits claims, kiyas generates ideas, iskele structures projects. This one audits experience. A simple “which button colour” question does not need any of it.
  6. Explicit-feedback bias — a lost user does not fill in the form, they leave; the population you most want to measure is exactly the one that self-selects out. Passive behavioural telemetry is PRIMARY; explicit feedback is triangulation only, non-interrupting, end-of-flow.
  7. False independence of a two-model handoff [H] — same training distribution, same bias; model B does not catch model A’s homogenisation. The value of a handoff is READABILITY, not independent verification. The spec carries the whole frame including the refutation conditions, and the implementation re-enters as [H].

Operating assumptions (this skill runs in someone else’s setup)

The host’s instructions take precedence — and that is the most likely way this skill fails: quietly. A short, softened UX audit still looks like an audit, tiers in place, judgment gone.

Anti-patterns (refuse these politely)

References