Resolver evaluation

evaluate replays authored judgments through the same native resolver used by consumers. It opens one externally pinned immutable generation, streams up to the configured case limit and reports correctness plus native p50/p95 query latency. Pass an explicit --at time when the report must be replayable across approval expiry boundaries; the chosen clock is included in the report. The input is bounded to 16 MiB, each line to 64 KiB and case IDs must be unique.

Each nonempty JSONL line has this form:

{"id":"facebook-primary","query":"facebook","locale":null,"country":null,"expected_status":"resolved","expected_entity_id":"argand:entity:SOURCE:ID","expected_url":"https://www.facebook.com/"}

locale, country, expected_entity_id and expected_url are optional. Status is required and is one of resolved, no_name_match, ambiguous_identity, safety_limit_exceeded, no_eligible_destination, no_active_review, region_mismatch or ambiguous_destination. Expected URLs must be copied from a pinned registry, including normalization such as a trailing slash.

argand-site-registry evaluate --generation /data/registry/reviewed \
  --pin "$REVIEWED_PIN" --cases /data/evaluation/navigation.jsonl \
  --maximum-cases 10000 --at 2026-09-13T00:00:00Z \
  > /data/evaluation/report.json
jq -e '.failed == 0 and .passed == .total' /data/evaluation/report.json

Keep the corpus version and digest with the report. Include canonical names, aliases, Unicode normalization, multiple scripts, every served country/locale, unknown names, deceptive lookalikes, ambiguous entities, expired/revoked reviews and tied destinations. Use synthetic or authorized query material; do not commit private user logs. Latency values are process and hardware measurements. Compare them only with an equivalent environment and sufficient sample size.

1.8 KiB Raw Blame History

Resolver evaluation

1.8 KiB

Raw Blame History