argand-site-registry/docs/EVALUATION.md

Resolver evaluation

evaluate streams authored JSONL judgments through the same pinned native reader used by consumers. Its v2 report includes every active source snapshot and declared lineage (explicit null for legacy unknown lineage), plus correctness and p50/p95 latency. Pass --at for a reproducible policy clock; approval expiry and future votes otherwise depend on current time. Inputs are bounded to 16 MiB, lines to 64 KiB, and case IDs must be unique.

{"id":"facebook-primary","query":"facebook","locale":null,"country":null,"expected_status":"resolved","expected_entity_id":"argand:entity:SOURCE:ID","expected_url":"https://www.facebook.com/"}

Optional fields are locale, country, expected_entity_id, and expected_url. Current statuses are resolved, no_name_match, ambiguous_identity, no_active_name_review, safety_limit_exceeded, no_eligible_destination, no_active_review, region_mismatch, and ambiguous_destination. Copy expected URLs from a pinned registry, including normalization such as trailing slash.

argand-site-registry evaluate --generation /data/registry/reviewed \
  --pin "$REVIEWED_PIN" --cases /data/evaluation/navigation.jsonl \
  --maximum-cases 10000 --at 2026-09-13T00:00:00Z \
  > /data/evaluation/report.json
jq -e '.failed == 0 and .passed == .total' /data/evaluation/report.json

Keep corpus version and digest, registry pin, policy digest, reviewer-trust digest, process build and hardware with the report. Cover canonical names, aliases, Unicode normalization, scripts, regions, unknown names, lookalikes, ambiguity, missing name votes, missing edge votes, correlated reviewers, stale policy, stale observations, expiry, revocation, sticky supersession and tied routes. Segment failures by safe abstention and wrong resolved route; any wrong resolved route is a release blocker.

Use synthetic or authorized queries and do not commit private user logs. Latency is process and hardware evidence. Compare only equivalent native configurations with sufficient samples.