nicweyand/argand-site-registry
1
0
Fork 0
Code Issues Pull requests Projects Releases 7 Packages Wiki Activity Actions

release: implement site registry v0.4 trust pipeline

Browse source
This commit is contained in:
Nic Weyand 2026-09-13 12:22:05 -04:00
commit e26efc19fa
Signed by: nicweyand
SSH key fingerprint: SHA256:2te+ycJIQON/Wo/dH6+ZkFSQ4HnHWpetV2azx9E65dQ
67 changed files with 10698 additions and 640 deletions
Download patch file Download diff file Expand all files Collapse all files

40
docs/EVALUATION.md
View file

@ -1,23 +1,20 @@
# Resolver evaluation
`evaluate` replays authored judgments through the same native resolver used by
consumers. It opens one externally pinned immutable generation, streams up to the
configured case limit and reports correctness plus native p50/p95 query latency.
Pass an explicit `--at` time when the report must be replayable across approval
expiry boundaries; the chosen clock is included in the report.
The input is bounded to 16 MiB, each line to 64 KiB and case IDs must be unique.
Each nonempty JSONL line has this form:
`evaluate` streams authored JSONL judgments through the same pinned native reader
used by consumers. It reports correctness and p50/p95 latency. Pass `--at` for a
reproducible policy clock; approval expiry and future votes otherwise depend on
current time. Inputs are bounded to 16 MiB, lines to 64 KiB, and case IDs must be
unique.
```json
{"id":"facebook-primary","query":"facebook","locale":null,"country":null,"expected_status":"resolved","expected_entity_id":"argand:entity:SOURCE:ID","expected_url":"https://www.facebook.com/"}
```
`locale`, `country`, `expected_entity_id` and `expected_url` are optional. Status
is required and is one of `resolved`, `no_name_match`, `ambiguous_identity`,
`safety_limit_exceeded`, `no_eligible_destination`, `no_active_review`,
`region_mismatch` or `ambiguous_destination`. Expected URLs must be copied from a
pinned registry, including normalization such as a trailing slash.
Optional fields are `locale`, `country`, `expected_entity_id`, and `expected_url`.
Current statuses are `resolved`, `no_name_match`, `ambiguous_identity`,
`no_active_name_review`, `safety_limit_exceeded`, `no_eligible_destination`,
`no_active_review`, `region_mismatch`, and `ambiguous_destination`. Copy expected
URLs from a pinned registry, including normalization such as trailing slash.
```bash
argand-site-registry evaluate --generation /data/registry/reviewed \
@ -27,9 +24,14 @@ argand-site-registry evaluate --generation /data/registry/reviewed \
jq -e '.failed == 0 and .passed == .total' /data/evaluation/report.json
```
Keep the corpus version and digest with the report. Include canonical names,
aliases, Unicode normalization, multiple scripts, every served country/locale,
unknown names, deceptive lookalikes, ambiguous entities, expired/revoked reviews
and tied destinations. Use synthetic or authorized query material; do not commit
private user logs. Latency values are process and hardware measurements. Compare
them only with an equivalent environment and sufficient sample size.
Keep corpus version and digest, registry pin, policy digest, reviewer-trust digest,
process build and hardware with the report. Cover canonical names, aliases,
Unicode normalization, scripts, regions, unknown names, lookalikes, ambiguity,
missing name votes, missing edge votes, correlated reviewers, stale policy,
stale observations, expiry, revocation, sticky supersession and tied routes.
Segment failures by safe abstention and wrong resolved route; any wrong resolved
route is a release blocker.
Use synthetic or authorized queries and do not commit private user logs. Latency is
process and hardware evidence. Compare only equivalent native configurations with
sufficient samples.