feat: harden reviewed registry releases
All checks were successful
Standalone registry checks / check (push) Successful in 3m43s

This commit is contained in:
Nic Weyand 2026-09-13 01:19:27 -04:00
commit e83f43d00f
Signed by: nicweyand
SSH key fingerprint: SHA256:2te+ycJIQON/Wo/dH6+ZkFSQ4HnHWpetV2azx9E65dQ
37 changed files with 2856 additions and 286 deletions

View file

@ -4,10 +4,11 @@
Use `argand_site_registry::query::Registry::open(generation, trusted_pin)` once per
immutable generation and reuse the reader. `lookup(query, limit)` returns evidence
and complete ambiguity counts; `resolve(query, locale, country, now)` returns an
optional reviewed candidate. Check the compiled example and API docs for exact
types. `selection_context` binds the full alternative set for downstream query
review. Preserve the returned provenance, scopes, counts and attribution.
and complete ambiguity counts; `resolve_explained` returns a reviewed candidate or
a typed abstention reason with counts. Exact reverse views cover entity IDs,
URLs/domains, popularity and Curlie categories. Check the compiled example and API
docs for exact types. `selection_context` binds the full alternative set for
downstream query review. Preserve returned provenance, scopes, counts and attribution.
For local integration, point a Cargo dependency at
`crates/argand-site-registry` inside an extracted standalone source tree. Once an
@ -17,7 +18,8 @@ Both crates remain in this workspace; the atomic helper is a relative dependency
## CLI and other languages
`lookup`, `resolve`, `verify` and the other commands emit JSON. The Python example
`lookup`, `resolve`, `entity`, `lookup-web`, `popularity`, `category`, `stats`,
`evaluate`, `verify` and the other commands emit JSON. The Python example
passes arguments directly to the native executable, preserving query text and the
entire response. A nonzero exit is an error. `destination: null` is a successful
abstention, not a request to pick the first lookup candidate. Render source names
@ -37,8 +39,10 @@ omits descriptions and includes fact provenance plus an attribution envelope.
It is an assertion export, not a self-contained signed list of admitted routes.
Raw SQL inspection is useful for audit; it does not implement resolution policy.
Code version 0.1.0 is an initial interface. Schema/rule contracts are versioned
independently in receipts. Unsupported contracts fail closed. Pin source releases,
Code version 0.2.0 uses schema version 2 and `argand.site-rules/v2`.
It adds authenticated reviewer proofs, exact immutable SQLite opening, typed diffs
and audit/evaluation APIs. Schema/rule contracts remain versioned independently in
receipts. Unsupported contracts fail closed. Pin source releases,
compile consumers and replay fixed fixtures before upgrades. Preserve import and
review history; never mutate complete generations to migrate them.

35
docs/EVALUATION.md Normal file
View file

@ -0,0 +1,35 @@
# Resolver evaluation
`evaluate` replays authored judgments through the same native resolver used by
consumers. It opens one externally pinned immutable generation, streams up to the
configured case limit and reports correctness plus native p50/p95 query latency.
Pass an explicit `--at` time when the report must be replayable across approval
expiry boundaries; the chosen clock is included in the report.
The input is bounded to 16 MiB, each line to 64 KiB and case IDs must be unique.
Each nonempty JSONL line has this form:
```json
{"id":"facebook-primary","query":"facebook","locale":null,"country":null,"expected_status":"resolved","expected_entity_id":"argand:entity:SOURCE:ID","expected_url":"https://www.facebook.com/"}
```
`locale`, `country`, `expected_entity_id` and `expected_url` are optional. Status
is required and is one of `resolved`, `no_name_match`, `ambiguous_identity`,
`safety_limit_exceeded`, `no_eligible_destination`, `no_active_review`,
`region_mismatch` or `ambiguous_destination`. Expected URLs must be copied from a
pinned registry, including normalization such as a trailing slash.
```bash
argand-site-registry evaluate --generation /data/registry/reviewed \
--pin "$REVIEWED_PIN" --cases /data/evaluation/navigation.jsonl \
--maximum-cases 10000 --at 2026-09-13T00:00:00Z \
> /data/evaluation/report.json
jq -e '.failed == 0 and .passed == .total' /data/evaluation/report.json
```
Keep the corpus version and digest with the report. Include canonical names,
aliases, Unicode normalization, multiple scripts, every served country/locale,
unknown names, deceptive lookalikes, ambiguous entities, expired/revoked reviews
and tied destinations. Use synthetic or authorized query material; do not commit
private user logs. Latency values are process and hardware measurements. Compare
them only with an equivalent environment and sufficient sample size.

View file

@ -6,6 +6,8 @@
- [Source licenses](../crates/argand-site-registry/LICENSE_SOURCES.md): exact terms and attribution.
- [Consumers](CONSUMERS.md): Rust, Python/CLI, data distribution and Argand transition.
- [Trust](TRUST.md): enforced checks and publisher/consumer responsibilities.
- [Publishing](PUBLISHING.md): reviewer keys, candidate acceptance and activation.
- [Evaluation](EVALUATION.md): bounded JSONL judgments and result interpretation.
- [Releasing](RELEASING.md): CI, source signing and archive verification.
- [Validation](VALIDATION.md): independent builds and native acceptance evidence.
- [Contributing](../CONTRIBUTING.md), [governance](../GOVERNANCE.md),

75
docs/PUBLISHING.md Normal file
View file

@ -0,0 +1,75 @@
# Dataset publisher runbook
This runbook creates a reviewable candidate and a signed activation. Source update,
review and release keys are separate authorities. Scheduled jobs may acquire,
import and build candidates; they do not approve, sign or activate destinations.
## Trust roots and local state
Keep cache, mutable store, immutable generations, reviewer keys, release keys and
consumer trust files outside the checkout. An OpenSSH reviewer trust file contains
one accepted principal and public key per line:
```text
operator@example.org ssh-ed25519 REVIEWER_PUBLIC_KEY
```
Distribute the release publisher public key to consumers through an independent
authenticated channel. Do not put private keys, production trust files or source
datasets in Git or CI. The isolated source CI runner has none of these files.
## Candidate acceptance
Run `update` or the explicit download/import/build commands from the operator guide.
For every candidate generation:
1. Verify its externally recorded receipt pin with `verify`.
2. Run `stats` and compare source snapshots, selected sources, facts, rejections,
reviews, database bytes and upcoming expiry with the prior accepted generation.
3. Run `diff` against the prior pin. Investigate every source-selection, identity,
name, property, edge, popularity, review and equivalence change.
4. Use `lookup`, `entity`, `lookup-web`, `popularity` and `category` to inspect exact
source evidence and conflicts. Popularity never proves ownership.
5. Replay the maintained evaluation corpus with `evaluate`; require zero judgment
mismatches and compare latency with a documented hardware/process baseline.
## Authenticated decisions
Create a bounded review JSON from the exact candidate fingerprint and evidence.
Sign its exact bytes and append it through the CLI:
```bash
ssh-keygen -Y sign -n argand-site-registry-review \
-f /secure/reviewer-key review.json
argand-site-registry review --database /data/registry/import.sqlite \
--generation /data/registry/candidate --pin "$CANDIDATE_PIN" \
--decision review.json --signature review.json.sig \
--allowed-reviewers /secure/reviewer-allowed-signers \
--identity operator@example.org
```
Identity equivalence decisions use the same signed JSON and reviewer namespace.
Rebuild after appending decisions, then repeat the complete diff and evaluation.
The release command re-verifies every retained reviewer signature against the
current reviewer trust file. Missing, altered or no-longer-trusted proofs stop it.
## Sign and activate
```bash
argand-site-registry sign --generation /data/registry/reviewed \
--pin "$REVIEWED_PIN" --key /secure/release-key \
--allowed-reviewers /secure/reviewer-allowed-signers
argand-site-registry activate --generation /data/registry/reviewed \
--current /data/registry/current.json \
--allowed-signers /secure/release-allowed-signers \
--identity registry-publisher
```
Record the source commit, candidate and accepted receipt pins, typed diff, evaluation
report, reviewer trust-file digest, release signer identity and activation receipt
in an immutable operator log. Activation refuses a rollback that drops a distributed
revocation. Deliver a new current pointer/pin to every consumer and bound their caches.
For an incident, append a signed revocation, rebuild with full history, inspect,
evaluate, sign and activate. Preserve the suspect source bytes, generation and proofs.
Follow [SECURITY.md](../SECURITY.md) for private reporting and key compromise.

View file

@ -14,7 +14,8 @@ define the build behavior. A lockfile pins dependencies, not the host compiler.
The Forgejo workflow uses the documented [workflow and context syntax](https://forgejo.org/docs/latest/user/actions/reference/).
Register `site-registry-isolated` only on a disposable, repository-scoped runner
with the above tools, two build jobs and at least 4 GiB memory. Use a pinned,
reviewed runner image. Do not mount production directories, share signing keys,
reviewed runner image; the exact local build is in [ci/Dockerfile](../ci/Dockerfile).
Do not mount production directories, share signing keys,
or use Argand's host runners. Follow Forgejo's [runner security guidance](https://forgejo.org/docs/latest/user/actions/security/).
The label is a deployment requirement, not a provisioned runner supplied by this
repository. The workflow runs on trusted main pushes or manual dispatch, fetches
@ -85,6 +86,8 @@ receipt; the initial release tool packages source only.
Source releases contain no provider datasets or real approvals. Dataset publishers
follow the operator guide: import, inspect, review, build, diff, sign and activate.
Follow the [publisher runbook](PUBLISHING.md); release signing requires the external
reviewer trust file and re-verifies every stored decision signature.
The dataset namespace `argand-site-registry` is distinct from the source namespace
above. Keep `LICENSE_SOURCES.md` and `ATTRIBUTION.json` with the database and receipt.
The weekly update example creates candidates. It never approves, renews, signs or

View file

@ -12,18 +12,22 @@ Destination and identity decisions bind exact evidence fingerprints, including
names, assertions and normalization context. Reviews expire within 90 days.
Changed evidence invalidates earlier approvals. Resolution abstains on ambiguity,
ties, missing approval or ineligible claims; regional scopes must explicitly
match. The local writer owns the append-only review log.
match. Reviewer decisions are signed under a dedicated SSH namespace and the local
writer retains their exact decision/signature bytes in the append-only review log.
Generations bind the database, license document and attribution to a completion
receipt. Consumers provide a trusted hash or verify an external publisher key.
Activation checks signatures and refuses rollback that loses distributed
revocations. Updates build candidates and cannot approve, sign or activate them.
Release signing re-verifies every stored decision against an external reviewer
trust file. Activation checks publisher signatures, rejects structurally incomplete
review proofs and refuses rollback that loses distributed revocations. Updates
build candidates and cannot approve, sign or activate them.
## What a publisher must establish
The reviewer name and evidence locator in a decision are operator assertions.
The CLI validates their structure and evidence binding; it does not independently
authenticate the reviewer, retrieve their evidence or prove website ownership.
The CLI authenticates exact decision bytes to an allowed SSH signer and validates
their structure and evidence binding; it does not retrieve the cited evidence or
prove website ownership.
Protect the writer database and signing key with separate operating permissions.
Restrict who can author decisions and require human review before release signing.
@ -37,8 +41,9 @@ to review. Confidence values are assertion scores, not calibrated probabilities.
Choose expiry based on volatility, within the enforced maximum. Do not renew
blindly on a timer. Expired approval should lead to abstention until evidence is
reviewed. Disclose editorial conflicts and use an independent reviewer for a
disputed claim when possible. The initial implementation is a local single-writer
tool; it does not provide authenticated reviewer accounts or an enforced quorum.
disputed claim when possible. The implementation is a local single-writer tool
with externally authenticated reviewer keys; it does not provide accounts or an
enforced quorum.
## What consumers must preserve