Metrics
A coverage claim needs more than wall time.
Coverage
- First-party chunks read divided by first-party chunks.
- First-party tokens read divided by first-party tokens.
- Unexplained unread chunks.
- Skipped chunks by reason.
- Chunks with findings divided by chunks read.
Freshness
- Index lag after a file change.
- Stale findings after a change.
- Audit drift: changed, added, and removed files.
Retrieval
- Correct partition for a fixed question.
- Correct chunk for a fixed question.
- Citation support: a cited path was content-read.
- Query latency and result count.
Answer quality
- Correctness on a fixed question set.
- Citation coverage.
- Unsupported claims.
- Meander: repeated reads, no-progress turns, and segments without a finding.
Efficiency
- Time to full coverage.
- Tokens per covered chunk.
- Segments and compactions.
- Model calls per segment.
- Cache read and fresh input.
Current status
The quality run scores an eight-question set about Auwai's own coverage path.
The 2026-11-09 run scored 30 of 32 points with Auwai and 28 of 32 with the
vanilla control arm. A separate full-window validation sent the largest 64K
segment to the same local model and resolved all three citations. The question
set and both results are in benchmarks/quality/.