Results
All results on this page were produced locally from an explicit
AUWAI_BENCH_URL. The release commands are in
local replication. Historical hosted-proxy runs are
kept for context only.
Release 1.0 evidence
Measured on 2026-11-08 with the local v6 regression and workflow probes.
| Gate | Result |
|---|---|
| v6 matrix cells | 96 / 96 |
| Parseable cells | 96 / 96 |
| Auwai + knowledge cells at 64/64 | 48 / 48 |
| Knowledge coverage | 1.00 |
| Workflow findings through one SQLite service | 12,800 |
| Workflow claims through one SQLite service | 12,800 |
| Stable facts | 796 |
| 64-worktree p99 | 998 ms |
The v6 run used one local endpoint and all three protocol shapes, supplied
through AUWAI_BENCH_URL. No public endpoint is part of the release.
Stage 9 validation
Measured on 2026-11-09 against one local model endpoint. The same eight-question rubric and the same model were used for both arms.
| Run | Score | Schema | Notes |
|---|---|---|---|
| Auwai arm | 30 / 32 (93.8%) | valid | Skill and index present |
| Vanilla control | 28 / 32 (87.5%) | valid | Copied without .auwai |
| Full-window segment | 64K segment | valid | 3 of 3 citations resolved |
The full-window run read the largest 64K segment (read-3, 10 chunks,
35,344 tokens) in 160.5 seconds. The structured answer contained three findings
and resolved every citation.
Evidence:
The control arm is not a no-tools arm. It can read source; it lacks the Auwai index and skill. That is the comparison the stage gate asks for.
Auwai on itself
Measured on 2026-11-03:
| Measure | Result |
|---|---|
| Files scanned | 253 |
| Chunks | 281 |
| Included tokens | 282,315 |
| Index time | about 2.2 s |
| Repeated ledger sync | about 0.035 s |
| Read segments at 64K | 9 |
| Read segments at 128K | 4 |
| Read segments at 256K | 2 |
The segment counts include one final synthesis segment.
Answer quality
The local model answered a fixed eight-question rubric about Auwai's own
coverage path. See the rubric
and the recorded run under benchmarks/quality/results/.
| Measure | Result |
|---|---|
| Answers | 8 of 8 |
| Rubric score | 30 / 32 (93.8%) |
| API calls | 4 |
| Input tokens | 155,348 |
| Fresh input tokens | 63,907 |
| Output tokens | 3,186 |
| Wall time | 3m56s |
The two lost points were content criteria: question 6 did not name the interpreter escape that limits the guard, and question 8 did not name the included-scope filter. Every citation criterion passed.
The run used the installed skill copy, so the answers cite paths relative to the skill root. The rubric accepts both that form and the checkout form. The score is answer correctness and citation support, not speed.
Linux kernel
The kernel was indexed on 2026-11-03:
| Measure | Result |
|---|---|
| Chunks | 177,872 |
| Included chunks | 177,864 |
| Included tokens | 364,909,391 |
| First ledger sync | 154.8 s |
| Unchanged resync | 14.8 s |
| Changed-path ledger sync | 1.1 to 2.0 s |
| Watcher end to end | 16.9 s |
| Ledger size | about 492 MB |
The kernel is public. It is used for scale, not for answer quality.
Limits
- The quality run is one rubric on one repository and has no control arm.
- The kernel ledger was measured for sync cost and coverage, not answer quality.
- A full-read segment has not been measured end to end on a model.
- The v6 release run is a knowledge-service regression, not a human review of every answer.
- Workflow probes measure concurrency and storage behavior, not model quality.