feat: harden reviewed registry releases
All checks were successful
Standalone registry checks / check (push) Successful in 3m43s
All checks were successful
Standalone registry checks / check (push) Successful in 3m43s
This commit is contained in:
parent
0bcd4a2fc9
commit
e83f43d00f
37 changed files with 2856 additions and 286 deletions
|
|
@ -4,10 +4,11 @@
|
|||
|
||||
Use `argand_site_registry::query::Registry::open(generation, trusted_pin)` once per
|
||||
immutable generation and reuse the reader. `lookup(query, limit)` returns evidence
|
||||
and complete ambiguity counts; `resolve(query, locale, country, now)` returns an
|
||||
optional reviewed candidate. Check the compiled example and API docs for exact
|
||||
types. `selection_context` binds the full alternative set for downstream query
|
||||
review. Preserve the returned provenance, scopes, counts and attribution.
|
||||
and complete ambiguity counts; `resolve_explained` returns a reviewed candidate or
|
||||
a typed abstention reason with counts. Exact reverse views cover entity IDs,
|
||||
URLs/domains, popularity and Curlie categories. Check the compiled example and API
|
||||
docs for exact types. `selection_context` binds the full alternative set for
|
||||
downstream query review. Preserve returned provenance, scopes, counts and attribution.
|
||||
|
||||
For local integration, point a Cargo dependency at
|
||||
`crates/argand-site-registry` inside an extracted standalone source tree. Once an
|
||||
|
|
@ -17,7 +18,8 @@ Both crates remain in this workspace; the atomic helper is a relative dependency
|
|||
|
||||
## CLI and other languages
|
||||
|
||||
`lookup`, `resolve`, `verify` and the other commands emit JSON. The Python example
|
||||
`lookup`, `resolve`, `entity`, `lookup-web`, `popularity`, `category`, `stats`,
|
||||
`evaluate`, `verify` and the other commands emit JSON. The Python example
|
||||
passes arguments directly to the native executable, preserving query text and the
|
||||
entire response. A nonzero exit is an error. `destination: null` is a successful
|
||||
abstention, not a request to pick the first lookup candidate. Render source names
|
||||
|
|
@ -37,8 +39,10 @@ omits descriptions and includes fact provenance plus an attribution envelope.
|
|||
It is an assertion export, not a self-contained signed list of admitted routes.
|
||||
Raw SQL inspection is useful for audit; it does not implement resolution policy.
|
||||
|
||||
Code version 0.1.0 is an initial interface. Schema/rule contracts are versioned
|
||||
independently in receipts. Unsupported contracts fail closed. Pin source releases,
|
||||
Code version 0.2.0 uses schema version 2 and `argand.site-rules/v2`.
|
||||
It adds authenticated reviewer proofs, exact immutable SQLite opening, typed diffs
|
||||
and audit/evaluation APIs. Schema/rule contracts remain versioned independently in
|
||||
receipts. Unsupported contracts fail closed. Pin source releases,
|
||||
compile consumers and replay fixed fixtures before upgrades. Preserve import and
|
||||
review history; never mutate complete generations to migrate them.
|
||||
|
||||
|
|
|
|||
35
docs/EVALUATION.md
Normal file
35
docs/EVALUATION.md
Normal file
|
|
@ -0,0 +1,35 @@
|
|||
# Resolver evaluation
|
||||
|
||||
`evaluate` replays authored judgments through the same native resolver used by
|
||||
consumers. It opens one externally pinned immutable generation, streams up to the
|
||||
configured case limit and reports correctness plus native p50/p95 query latency.
|
||||
Pass an explicit `--at` time when the report must be replayable across approval
|
||||
expiry boundaries; the chosen clock is included in the report.
|
||||
The input is bounded to 16 MiB, each line to 64 KiB and case IDs must be unique.
|
||||
|
||||
Each nonempty JSONL line has this form:
|
||||
|
||||
```json
|
||||
{"id":"facebook-primary","query":"facebook","locale":null,"country":null,"expected_status":"resolved","expected_entity_id":"argand:entity:SOURCE:ID","expected_url":"https://www.facebook.com/"}
|
||||
```
|
||||
|
||||
`locale`, `country`, `expected_entity_id` and `expected_url` are optional. Status
|
||||
is required and is one of `resolved`, `no_name_match`, `ambiguous_identity`,
|
||||
`safety_limit_exceeded`, `no_eligible_destination`, `no_active_review`,
|
||||
`region_mismatch` or `ambiguous_destination`. Expected URLs must be copied from a
|
||||
pinned registry, including normalization such as a trailing slash.
|
||||
|
||||
```bash
|
||||
argand-site-registry evaluate --generation /data/registry/reviewed \
|
||||
--pin "$REVIEWED_PIN" --cases /data/evaluation/navigation.jsonl \
|
||||
--maximum-cases 10000 --at 2026-09-13T00:00:00Z \
|
||||
> /data/evaluation/report.json
|
||||
jq -e '.failed == 0 and .passed == .total' /data/evaluation/report.json
|
||||
```
|
||||
|
||||
Keep the corpus version and digest with the report. Include canonical names,
|
||||
aliases, Unicode normalization, multiple scripts, every served country/locale,
|
||||
unknown names, deceptive lookalikes, ambiguous entities, expired/revoked reviews
|
||||
and tied destinations. Use synthetic or authorized query material; do not commit
|
||||
private user logs. Latency values are process and hardware measurements. Compare
|
||||
them only with an equivalent environment and sufficient sample size.
|
||||
|
|
@ -6,6 +6,8 @@
|
|||
- [Source licenses](../crates/argand-site-registry/LICENSE_SOURCES.md): exact terms and attribution.
|
||||
- [Consumers](CONSUMERS.md): Rust, Python/CLI, data distribution and Argand transition.
|
||||
- [Trust](TRUST.md): enforced checks and publisher/consumer responsibilities.
|
||||
- [Publishing](PUBLISHING.md): reviewer keys, candidate acceptance and activation.
|
||||
- [Evaluation](EVALUATION.md): bounded JSONL judgments and result interpretation.
|
||||
- [Releasing](RELEASING.md): CI, source signing and archive verification.
|
||||
- [Validation](VALIDATION.md): independent builds and native acceptance evidence.
|
||||
- [Contributing](../CONTRIBUTING.md), [governance](../GOVERNANCE.md),
|
||||
|
|
|
|||
75
docs/PUBLISHING.md
Normal file
75
docs/PUBLISHING.md
Normal file
|
|
@ -0,0 +1,75 @@
|
|||
# Dataset publisher runbook
|
||||
|
||||
This runbook creates a reviewable candidate and a signed activation. Source update,
|
||||
review and release keys are separate authorities. Scheduled jobs may acquire,
|
||||
import and build candidates; they do not approve, sign or activate destinations.
|
||||
|
||||
## Trust roots and local state
|
||||
|
||||
Keep cache, mutable store, immutable generations, reviewer keys, release keys and
|
||||
consumer trust files outside the checkout. An OpenSSH reviewer trust file contains
|
||||
one accepted principal and public key per line:
|
||||
|
||||
```text
|
||||
operator@example.org ssh-ed25519 REVIEWER_PUBLIC_KEY
|
||||
```
|
||||
|
||||
Distribute the release publisher public key to consumers through an independent
|
||||
authenticated channel. Do not put private keys, production trust files or source
|
||||
datasets in Git or CI. The isolated source CI runner has none of these files.
|
||||
|
||||
## Candidate acceptance
|
||||
|
||||
Run `update` or the explicit download/import/build commands from the operator guide.
|
||||
For every candidate generation:
|
||||
|
||||
1. Verify its externally recorded receipt pin with `verify`.
|
||||
2. Run `stats` and compare source snapshots, selected sources, facts, rejections,
|
||||
reviews, database bytes and upcoming expiry with the prior accepted generation.
|
||||
3. Run `diff` against the prior pin. Investigate every source-selection, identity,
|
||||
name, property, edge, popularity, review and equivalence change.
|
||||
4. Use `lookup`, `entity`, `lookup-web`, `popularity` and `category` to inspect exact
|
||||
source evidence and conflicts. Popularity never proves ownership.
|
||||
5. Replay the maintained evaluation corpus with `evaluate`; require zero judgment
|
||||
mismatches and compare latency with a documented hardware/process baseline.
|
||||
|
||||
## Authenticated decisions
|
||||
|
||||
Create a bounded review JSON from the exact candidate fingerprint and evidence.
|
||||
Sign its exact bytes and append it through the CLI:
|
||||
|
||||
```bash
|
||||
ssh-keygen -Y sign -n argand-site-registry-review \
|
||||
-f /secure/reviewer-key review.json
|
||||
argand-site-registry review --database /data/registry/import.sqlite \
|
||||
--generation /data/registry/candidate --pin "$CANDIDATE_PIN" \
|
||||
--decision review.json --signature review.json.sig \
|
||||
--allowed-reviewers /secure/reviewer-allowed-signers \
|
||||
--identity operator@example.org
|
||||
```
|
||||
|
||||
Identity equivalence decisions use the same signed JSON and reviewer namespace.
|
||||
Rebuild after appending decisions, then repeat the complete diff and evaluation.
|
||||
The release command re-verifies every retained reviewer signature against the
|
||||
current reviewer trust file. Missing, altered or no-longer-trusted proofs stop it.
|
||||
|
||||
## Sign and activate
|
||||
|
||||
```bash
|
||||
argand-site-registry sign --generation /data/registry/reviewed \
|
||||
--pin "$REVIEWED_PIN" --key /secure/release-key \
|
||||
--allowed-reviewers /secure/reviewer-allowed-signers
|
||||
argand-site-registry activate --generation /data/registry/reviewed \
|
||||
--current /data/registry/current.json \
|
||||
--allowed-signers /secure/release-allowed-signers \
|
||||
--identity registry-publisher
|
||||
```
|
||||
|
||||
Record the source commit, candidate and accepted receipt pins, typed diff, evaluation
|
||||
report, reviewer trust-file digest, release signer identity and activation receipt
|
||||
in an immutable operator log. Activation refuses a rollback that drops a distributed
|
||||
revocation. Deliver a new current pointer/pin to every consumer and bound their caches.
|
||||
|
||||
For an incident, append a signed revocation, rebuild with full history, inspect,
|
||||
evaluate, sign and activate. Preserve the suspect source bytes, generation and proofs.
|
||||
Follow [SECURITY.md](../SECURITY.md) for private reporting and key compromise.
|
||||
|
|
@ -14,7 +14,8 @@ define the build behavior. A lockfile pins dependencies, not the host compiler.
|
|||
The Forgejo workflow uses the documented [workflow and context syntax](https://forgejo.org/docs/latest/user/actions/reference/).
|
||||
Register `site-registry-isolated` only on a disposable, repository-scoped runner
|
||||
with the above tools, two build jobs and at least 4 GiB memory. Use a pinned,
|
||||
reviewed runner image. Do not mount production directories, share signing keys,
|
||||
reviewed runner image; the exact local build is in [ci/Dockerfile](../ci/Dockerfile).
|
||||
Do not mount production directories, share signing keys,
|
||||
or use Argand's host runners. Follow Forgejo's [runner security guidance](https://forgejo.org/docs/latest/user/actions/security/).
|
||||
The label is a deployment requirement, not a provisioned runner supplied by this
|
||||
repository. The workflow runs on trusted main pushes or manual dispatch, fetches
|
||||
|
|
@ -85,6 +86,8 @@ receipt; the initial release tool packages source only.
|
|||
|
||||
Source releases contain no provider datasets or real approvals. Dataset publishers
|
||||
follow the operator guide: import, inspect, review, build, diff, sign and activate.
|
||||
Follow the [publisher runbook](PUBLISHING.md); release signing requires the external
|
||||
reviewer trust file and re-verifies every stored decision signature.
|
||||
The dataset namespace `argand-site-registry` is distinct from the source namespace
|
||||
above. Keep `LICENSE_SOURCES.md` and `ATTRIBUTION.json` with the database and receipt.
|
||||
The weekly update example creates candidates. It never approves, renews, signs or
|
||||
|
|
|
|||
|
|
@ -12,18 +12,22 @@ Destination and identity decisions bind exact evidence fingerprints, including
|
|||
names, assertions and normalization context. Reviews expire within 90 days.
|
||||
Changed evidence invalidates earlier approvals. Resolution abstains on ambiguity,
|
||||
ties, missing approval or ineligible claims; regional scopes must explicitly
|
||||
match. The local writer owns the append-only review log.
|
||||
match. Reviewer decisions are signed under a dedicated SSH namespace and the local
|
||||
writer retains their exact decision/signature bytes in the append-only review log.
|
||||
|
||||
Generations bind the database, license document and attribution to a completion
|
||||
receipt. Consumers provide a trusted hash or verify an external publisher key.
|
||||
Activation checks signatures and refuses rollback that loses distributed
|
||||
revocations. Updates build candidates and cannot approve, sign or activate them.
|
||||
Release signing re-verifies every stored decision against an external reviewer
|
||||
trust file. Activation checks publisher signatures, rejects structurally incomplete
|
||||
review proofs and refuses rollback that loses distributed revocations. Updates
|
||||
build candidates and cannot approve, sign or activate them.
|
||||
|
||||
## What a publisher must establish
|
||||
|
||||
The reviewer name and evidence locator in a decision are operator assertions.
|
||||
The CLI validates their structure and evidence binding; it does not independently
|
||||
authenticate the reviewer, retrieve their evidence or prove website ownership.
|
||||
The CLI authenticates exact decision bytes to an allowed SSH signer and validates
|
||||
their structure and evidence binding; it does not retrieve the cited evidence or
|
||||
prove website ownership.
|
||||
Protect the writer database and signing key with separate operating permissions.
|
||||
Restrict who can author decisions and require human review before release signing.
|
||||
|
||||
|
|
@ -37,8 +41,9 @@ to review. Confidence values are assertion scores, not calibrated probabilities.
|
|||
Choose expiry based on volatility, within the enforced maximum. Do not renew
|
||||
blindly on a timer. Expired approval should lead to abstention until evidence is
|
||||
reviewed. Disclose editorial conflicts and use an independent reviewer for a
|
||||
disputed claim when possible. The initial implementation is a local single-writer
|
||||
tool; it does not provide authenticated reviewer accounts or an enforced quorum.
|
||||
disputed claim when possible. The implementation is a local single-writer tool
|
||||
with externally authenticated reviewer keys; it does not provide accounts or an
|
||||
enforced quorum.
|
||||
|
||||
## What consumers must preserve
|
||||
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue