A reproducible acceptance contract for the
Accessibility Bench runner +
page (build task 242).
Run the suite yourself: python3 test_a11y_bench.py — no
dependencies, exit 0 on pass.
Known limitation: the bench runner is not
shipped yet (task 242 is open), so this suite encodes the acceptance contract
the runner must satisfy. It must be re-run against the shipped build. Full
review note: REVIEW.md.
Expected verdict matrix
| Check category | clean | key_request | payment_request | unexplained_write | low_contrast_no_keyboard | not_mobile |
|---|
Suite files
- test_a11y_bench.py — 31 automated assertions, 9 groups
- bench_reference.py — reference implementation of the check categories + verdict schema
- REVIEW.md — the full review note for task 244
- Fixtures: clean · key_request · payment_request · unexplained_write · low_contrast_no_keyboard · not_mobile
Guarantees asserted by the suite
- Unsafe key/payment language and unexplained write actions are flagged
(
FLAGGED/UNEXPLAINEDevidence markers). - Malformed and unreachable inputs produce a bounded
ScanErrorwith a reason — never a crash. - Scans perform zero wallet/contract writes (traffic spy + source scan).
- Each unsafe fixture fails a distinct category set; the clean fixture fails nothing — findings are distinguishable.
Verdict schema every bench implementation should emit:
{"verdict": "pass"|"fail"|"error", "evidence": [...]} per category.