release: implement site registry v0.4 trust pipeline

docs/superpowers/plans/2026-09-13-v0.4-and-beyond.md

@@ -0,0 +1,880 @@
+# Argand Site Registry v0.4 and Beyond Plan
+
+> **Status:** Version 0.4 phases 0 through 2 implemented and security-reviewed.
+> Phases 3 and later remain the sequenced roadmap.
+>
+> **Baseline:** Clean `main` at `2861337`; runtime behavior is the tagged
+> `v0.3.0` release at `ac82820`. The complete v0.3 acceptance suite passes.
+>
+> **Execution constraint:** Work in this standalone repository only. Do not use
+> subagents, edit Argand's main checkout, share its build cache, acquire paid data,
+> publish a dataset, sign with production keys, or deploy without the authority
+> already established for that specific action.
+
+## Goal
+
+Turn the v0.3 evidence and release substrate into a production-grade, reusable
+entity-to-website authority system that can publish a useful signed reference
+registry while preserving conflict, provenance, commercial-reuse terms, human
+review, abstention, revocation, and deterministic regional resolution.
+
+The implementation must remain useful in two modes:
+
+1. A local publisher builds and reviews its own registry from allowed sources.
+2. A consumer verifies and queries a separately distributed signed reference
+   generation without trusting an unauthenticated download location.
+
+The work is divided into independently releasable stages. Correct active-state
+semantics and review controls come before broader ingestion. Provider-scale proof
+comes before a public reference dataset. A hosted service remains optional.
+
+## Product thesis
+
+Popularity lists answer which domains receive attention. They do not reliably
+answer which entity controls a domain, whether a destination is current, or which
+regional property is appropriate. Argand Site Registry should compile independent
+source assertions, observed site relationships, and authenticated reviewer votes
+into a signed generation that resolves only when policy is satisfied.
+
+The principal product outcome is a signed, explainable answer:
+
+```text
+query + locale/country
+  -> reviewed name-to-entity binding
+  -> reviewed entity-to-property edge
+  -> active regional-selection policy
+  -> URL or typed abstention
+```
+
+A larger assertion database is not the goal by itself. Release quality is measured
+by resolved-route correctness, useful coverage, freshness, and revocation speed.
+
+## Non-negotiable invariants
+
+- Keep raw inputs, normalized facts, crawler observations, reviewer decisions,
+  policy decisions, popularity, and release authority logically separate.
+- Never infer entity equivalence or website ownership from similar names, domains,
+  redirects, TLS, DNS, `sameAs`, popularity, or shared upstream data alone.
+- Preserve conflicting and rejected evidence with its source-native identity.
+- Every imported or derived fact retains source, source identifier, license,
+  license evidence URL, retrieval timestamp, confidence, and derivation version.
+- Every source adapter must be rights-reviewed, format-pinned, streaming, bounded,
+  reproducible, resumable where feasible, and idempotent.
+- Automated updates may download, import, evaluate, and build candidates. They may
+  not approve, renew, sign, activate, or publish them.
+- Resolution remains fail closed. Ambiguity, missing policy, expired evidence,
+  missing votes, unsupported locale semantics, or conflicting equal candidates
+  produces a typed abstention.
+- Popularity remains source-separated evidence and never becomes an ownership vote.
+- Existing v0.3 data remains auditable. Migrations must not reinterpret an old
+  approval, source scope, or timestamp as if it had been created under a new rule.
+- Curlie descriptions remain redacted unless the exact distribution surface meets
+  its attribution obligations.
+- Public submissions are untrusted proposals. They never mutate an active release.
+- Cloudflare Radar, default Tranco, Cisco Umbrella, and other unverified sources
+  remain excluded.
+
+## Target architecture
+
+```mermaid
+flowchart LR
+    A[Rights-reviewed source adapters] --> B[Immutable source manifests and raw cache]
+    B --> C[Writer store: records, facts, conflicts, tombstones]
+    C --> D[Normalized assertion graph]
+    E[Bounded candidate-site observer] --> F[Immutable observation bundles]
+    F --> D
+    D --> G[Review queue and evidence diff]
+    G --> H[Signed reviewer votes]
+    H --> I[Versioned publisher policy compiler]
+    I --> J[Immutable active projection]
+    J --> K[Signed full generation]
+    J --> L[Signed delta and revocation feed]
+    K --> M[Verified local library and CLI]
+    L --> M
+    M --> N[Exact lookup and regional resolve]
+```
+
+The writer store remains append-oriented and audit-capable. Runtime generations
+contain the selected facts, active decisions, required proof material, and signed
+references to cold audit bundles. They do not duplicate every historical raw row.
+
+## Release sequence
+
+| Release | Purpose | Exit condition |
+| --- | --- | --- |
+| v0.3.x | Correct active-state and review-time semantics | Historical export is explicit, scope overlap fails closed, review acceptance time is trusted, and legacy behavior is regression-tested. |
+| v0.4 | Scalable trust and observation pipeline | Quorum policy, granular name/edge decisions, sticky revocations, structured evidence bundles, and review queues work end to end. |
+| v0.5 | Provider-scale and source expansion | Real-format scale canaries pass; incremental Wikidata and the first new rights-approved adapters are reproducible and bounded. |
+| v0.6 | Signed public reference registry | A reviewed dataset, full/delta releases, revocation delivery, coverage report, and consumer verification are published independently of source releases. |
+| Later | Wider resolver/product surface | Locale fallback, typed destination roles, candidate discovery, platform portability, and an optional read-only service are justified by real use. |
+
+## Phase 0: Freeze contracts and record design decisions
+
+### Task 0.1: Capture the v0.3 compatibility baseline
+
+- [x] Record the exact v0.3 CLI output schemas, generation schema, source-manifest
+      schema, rules version, review JSON, release receipts, and Python/Rust examples.
+- [x] Add golden fixtures for a complete v0.3 generation and a mutable schema-v3
+      writer database without committing provider data or private keys.
+- [x] Verify that current `lookup`, `resolve`, `diff`, `export`, `stats`, signature
+      verification, rollback refusal, and migrations behave exactly as documented.
+- [x] Preserve the current five-source all-synthetic fixture as a compatibility gate.
+
+Likely files:
+
+- `crates/argand-site-registry/tests/common/`
+- `crates/argand-site-registry/tests/registry.rs`
+- `crates/argand-site-registry/tests/failures.rs`
+- `crates/argand-site-registry/tests/cli.rs`
+- `docs/VALIDATION.md`
+
+Acceptance:
+
+- A v0.3 reader fixture remains readable or fails with a precise documented version
+  error after each later schema change.
+- No later test can silently regenerate the baseline fixture from new behavior.
+
+### Task 0.2: Write architecture decisions before migrations
+
+- [x] Add an ADR for active source coverage and supersession.
+- [x] Add an ADR for separating name bindings, website edges, and observations.
+- [x] Add an ADR for reviewer votes, revocation precedence, and publisher policy.
+- [x] Add an ADR for runtime projection versus cold audit retention.
+- [x] Add an ADR for full and delta dataset release identities.
+- [x] Add an ADR for source lineage so copied upstream evidence does not count twice.
+- [x] Document whether broad proprietary embedding is a goal. If so, evaluate a
+      small permissively licensed format/verifier crate without changing the AGPL
+      publisher engine or source-driven dataset licenses automatically.
+
+Acceptance:
+
+- Every later migration and public contract links to an approved decision.
+- The ADRs explain rejected alternatives and compatibility consequences.
+
+## Phase 1: v0.3.x active-state and trust hardening
+
+### Task 1.1: Replace free-form snapshot replacement semantics with typed coverage
+
+Problem: v0.3 selects the latest completed snapshot per exact `(source, scope)`
+string. Different strings are treated as additive even when one is a full snapshot
+or overlaps another partition.
+
+- [x] Introduce a source-manifest version with explicit coverage semantics:
+      collection identity, coverage kind, stable partition identity, base snapshot
+      where applicable, and explicit supersession references.
+- [x] Preserve v1 manifest parsing. Map every legacy `(source, scope)` to an isolated
+      legacy partition so migration does not silently change its active facts.
+- [x] Require operators to provide an explicit mapping before a legacy partial
+      snapshot can join or be replaced by a typed full collection.
+- [x] Select one coherent coverage set per source collection:
+  - a complete full snapshot supersedes older declared partitions;
+  - disjoint partitions compose only when their identities are explicit;
+  - deltas require an authenticated base and ordered continuity;
+  - overlapping or missing coverage relationships abort the build;
+  - deletions use explicit tombstones and remain auditable.
+- [x] Include the complete selected-coverage graph in the build receipt.
+- [x] Make `stats` and `diff` report selected, superseded, incomplete, and conflicting
+      source snapshots separately.
+
+Likely files:
+
+- `crates/argand-site-registry/src/model.rs`
+- `crates/argand-site-registry/src/store.rs`
+- `crates/argand-site-registry/src/build.rs`
+- `crates/argand-site-registry/src/diff.rs`
+- `crates/argand-site-registry/src/query.rs`
+- `crates/argand-site-registry/src/cli.rs`
+- `crates/argand-site-registry/migrations/004.sql`
+
+Tests:
+
+- Latest snapshot replaces an older snapshot in the same partition.
+- A declared full snapshot supersedes earlier partitions.
+- Two explicit disjoint partitions compose.
+- An undeclared full-plus-partial combination fails.
+- Overlapping partitions fail.
+- Missing delta bases, skipped deltas, and forked delta histories fail.
+- Tombstoned facts disappear from the active projection but remain in audit history.
+- Failed or partial imports never replace selected complete coverage.
+- Reimporting identical manifests and deltas is byte-for-byte idempotent.
+
+### Task 1.2: Split active export from audit-history export
+
+Problem: v0.3 JSONL export emits facts from every complete retained source snapshot,
+including superseded snapshots, without a per-row active-state marker.
+
+- [x] Change the normal export contract to emit selected active facts only.
+- [x] Version the export envelope and include generation identity, selected source
+      IDs, coverage-policy version, derivation version, and attribution identity.
+- [x] Add explicit selection state to every assertion.
+- [x] Add a separate audit export that includes active, superseded, rejected, and
+      tombstoned facts with replacement links.
+- [x] Keep Curlie descriptions redacted by default in both modes.
+- [x] Make consumers reject unknown export schema versions.
+- [x] Document that neither export bypasses `resolve` admission policy.
+
+Likely files:
+
+- `crates/argand-site-registry/src/release.rs`
+- `crates/argand-site-registry/src/cli.rs`
+- `crates/argand-site-registry/src/diff.rs`
+- `README.md`
+- `docs/CONSUMERS.md`
+- `docs/TRUST.md`
+
+Tests:
+
+- Superseded facts are absent from active export.
+- Audit export retains and labels the same facts.
+- Active export and native generation agree exactly on selected source IDs.
+- Description redaction and attribution survive both modes.
+- Python and Rust consumers refuse unknown versions and preserve null abstention.
+
+### Task 1.3: Bind decisions to trusted acceptance time
+
+Problem: the reviewer signs `reviewed_at`, but the writer does not persist a trusted
+append time. That self-declared time influences approval validity and reviewer-key
+validity-epoch checks.
+
+- [x] Record `accepted_at` from the writer when exact signature verification and
+      candidate validation succeed.
+- [x] Make approval validity begin no earlier than trusted acceptance.
+- [x] Bound effective expiry by both the signed review duration and the enforced
+      maximum measured from acceptance, preventing backdating or future dating from
+      extending authority.
+- [x] Evaluate reviewer eligibility at acceptance for new decisions. Preserve the
+      signed claimed decision time for audit, without treating it as trusted time.
+- [x] Include acceptance time and the time-policy version in generation receipts.
+- [x] Retain legacy approvals for audit but require an explicit migration policy or
+      fresh decision before they are active under the new rules. Never synthesize a
+      historical acceptance time. Preserve legacy revocations regardless.
+- [x] Test clock skew, future dates, backdates, expired keys, removed keys, offline
+      signing followed by later acceptance, and reproducible builds at fixed clocks.
+
+Likely files:
+
+- `crates/argand-site-registry/src/review.rs`
+- `crates/argand-site-registry/src/query.rs`
+- `crates/argand-site-registry/src/resolution.rs`
+- `crates/argand-site-registry/src/release.rs`
+- `crates/argand-site-registry/migrations/004.sql`
+- `docs/TRUST.md`
+- `docs/PUBLISHING.md`
+
+### Task 1.4: Make revocation precedence explicit before quorum work
+
+- [x] Treat a valid revocation as sticky for its exact subject.
+- [x] Require an explicit signed supersession that references the revocation before
+      the same destination can become active again.
+- [x] Prevent a later ordinary approval from overriding a revocation by sequence
+      order alone.
+- [x] Preserve existing activation rollback checks and strengthen them to compare
+      revocation identities and supersession relationships.
+- [x] Add a machine-readable emergency revocation export suitable for cached
+      consumers.
+
+Acceptance:
+
+- Reordering, forking, or appending an ordinary approval cannot erase a revocation.
+- A deliberately superseded revocation remains visible in lookup, diff, and audit.
+- An offline consumer can apply the revocation export before receiving a full build.
+
+### Phase 1 release gate
+
+- [x] Run `cargo fetch --locked` once, then the complete offline `scripts/check.sh`.
+- [x] Run migrations from empty, schema v1, v2, and v3 databases.
+- [x] Build the same generation twice and compare every output byte.
+- [x] Verify source release determinism from a clean signed commit.
+- [x] Update changelog, trust docs, format contracts, and migration guidance.
+- [x] Run `cargo audit --deny warnings` and record it separately from the offline gate.
+- [x] Confirm no provider data, keys, approval logs, or private paths entered Git.
+
+## Phase 2: v0.4 scalable review and structured evidence
+
+### Task 2.1: Separate name-to-entity trust from entity-to-property trust
+
+Problem: v0.3 binds each website approval to the entity's full name set. This stops
+a newly injected alias from inheriting a route, but benign alias or label changes
+invalidate every website edge. A global Wikidata revision also changes edge identity
+when the P856 statement itself did not materially change.
+
+- [x] Create independent material fingerprints for:
+  - source name or alias assertion to stable entity;
+  - explicit entity equivalence;
+  - entity-to-normalized-web-property assertion;
+  - normalized regional role and scope;
+  - observation bundle;
+  - reviewer vote and policy result.
+- [x] Bind website-edge identity only to material website evidence: exact source
+      statement identity/value/rank/relevant qualifiers, normalized URL result, and
+      derivation rule version.
+- [x] Retain source record revision and unrelated entity metadata in provenance and
+      diffs without making them part of website-edge identity.
+- [x] Require `resolve` to satisfy both an admitted name binding and an admitted
+      website edge. `lookup` continues to expose unreviewed names and edges.
+- [x] Ensure a newly imported alias cannot inherit an existing approved route.
+- [x] Ensure a benign accepted alias update does not invalidate an unchanged route.
+- [x] Give canonical labels, aliases, and source-specific names distinct evidence
+      identities and policy treatment.
+
+Tests:
+
+- Adding an unreviewed alias never creates a resolvable query.
+- Removing or changing an approved alias affects only that binding.
+- An unrelated Wikidata `lastrevid` change preserves the route fingerprint.
+- A P856 URL, rank, end qualifier, or relevant regional qualifier change invalidates
+  the route approval.
+- Confusable and bidi-control names remain rejected.
+- Explicit entity equivalence never carries unstated name or route authority.
+
+### Task 2.2: Replace latest-decision-wins with signed votes and policy compilation
+
+- [x] Represent decisions as immutable signed votes by authenticated reviewer.
+- [x] Permit one current vote per signer, subject, decision type, and policy epoch;
+      preserve superseded votes in the append-only log.
+- [x] Add a versioned publisher policy that defines:
+  - required approval threshold by decision type and risk class;
+  - revocation threshold and sticky-revocation behavior;
+  - publisher/reviewer separation;
+  - reviewer groups or independence constraints where configured;
+  - maximum approval age and evidence-freshness requirements;
+  - treatment of source conflicts and unresolved observations.
+- [x] Hash the exact policy and selected reviewer trust roots into the release receipt.
+- [x] Compile votes deterministically into approved, revoked, expired, disputed,
+      probationary, or insufficient-review state.
+- [x] Keep policy configurable for independent publishers while shipping a strict,
+      documented reference policy for Argand's own releases.
+- [x] Preserve the ability to operate locally without accounts or a network service.
+
+Tests:
+
+- One signer cannot satisfy a two-independent-reviewer policy.
+- Duplicate keys or aliases for the same reviewer do not create extra votes.
+- Publisher/reviewer separation is enforced when enabled.
+- One authorized revocation blocks resolution under the reference policy.
+- Conflicting votes produce a disputed state and abstention.
+- Changing policy invalidates the old compiled result without changing source facts.
+- Release verification fails on policy, signer, vote, or receipt tampering.
+
+### Task 2.3: Turn observation types into a stored evidence pipeline
+
+- [x] Extend the writer schema with immutable observation batches and observations.
+- [x] Preserve the existing distinction between observations and ownership claims.
+- [x] Add an adapter contract for externally captured observations, including source,
+      rights declaration, capture ID, retrieval time, content hash, exact selector,
+      from/to URLs, relation, and confidence.
+- [x] Project redirect, canonical, hreflang, JSON-LD `sameAs`, sitemap, and country
+      selector evidence without automatically creating an entity edge.
+- [x] Add observation lookup, reverse lookup, generation diff, and evidence-bundle
+      output for review.
+- [x] Record negative or failed observations with bounded error classes so absence is
+      not confused with a fetch that never succeeded.
+- [x] Define retention that stores hashes and necessary bounded extracts rather than
+      copied page bodies unless rights and need are established.
+
+Likely files:
+
+- `crates/argand-site-registry/src/observation.rs`
+- `crates/argand-site-registry/src/store.rs`
+- `crates/argand-site-registry/src/build.rs`
+- `crates/argand-site-registry/src/query.rs`
+- `crates/argand-site-registry/src/diff.rs`
+- `crates/argand-site-registry/migrations/005.sql`
+
+### Task 2.4: Add a bounded candidate-site observer
+
+- [x] Observe only imported/reviewed candidate URLs. Do not start an open-web crawler.
+- [x] Enforce scheme, redirect-count, response-size, header-size, decompression,
+      timeout, DNS-result, address-range, and per-host request bounds.
+- [x] Block credentials, local/private/link-local targets, unsafe redirect transitions,
+      non-HTTP protocols, and host confusion.
+- [x] Capture redirect chains, final URL, status, canonical, hreflang, sameAs, sitemap
+      references, country selectors, DNS answers, and TLS certificate fingerprints as
+      separate evidence classes.
+- [x] Do not treat TLS, DNS, redirects, or site self-assertions as ownership proof.
+- [x] Emit immutable observation manifests compatible with Task 2.3.
+- [x] Support cache-only replay so parser and policy tests never require the network.
+- [x] Keep acquisition cadence, concurrency, bandwidth, and data path configurable.
+
+Security tests:
+
+- SSRF attempts, redirect loops, compression bombs, oversized markup, malformed HTML,
+  DNS rebinding, mixed encodings, invalid certificates, and cross-scheme redirects.
+- Parser fuzz/property tests for headers, hreflang, canonical, JSON-LD, and sitemaps.
+- Deterministic replay from captured fixtures with no network access.
+
+### Task 2.5: Build review queues and evidence bundles before a graphical UI
+
+- [x] Add deterministic queue output ordered by risk and material change, not source
+      popularity alone.
+- [x] Queue new routes, new aliases, source conflicts, changed website statements,
+      redirects across registrable domains, observation drift, expiring approvals,
+      unresolved regional scopes, and revoked destinations proposed for reinstatement.
+- [x] Produce a bounded review bundle containing exact claims, conflicts, observation
+      diffs, source licenses, capture hashes, requested role, and candidate fingerprint.
+- [x] Add commands to prepare a vote, verify exact vote bytes, append it, and show the
+      compiled policy result.
+- [x] Keep the queue read-only and deterministic. Never let viewing evidence mutate
+      approval state.
+- [x] Defer a browser workbench until CLI bundles have proven the workflow.
+
+### Task 2.6: Monitor approved routes and classify drift
+
+- [x] Schedule observation refresh independently from source import cadence.
+- [x] Classify material changes: unreachable, cross-domain redirect, DNS/TLS change,
+      content/canonical shift, domain expiry indicators, malware-policy result, or no
+      material change.
+- [x] Use risk policy to shorten review intervals. Never auto-extend approvals.
+- [x] Put materially changed routes into probation or revoke them according to signed
+      publisher policy; default to abstention where evidence is insufficient.
+- [x] Record every transition and make revocation candidates immediately exportable.
+- [x] Design malware-feed adapters only after exact commercial-reuse and redistribution
+      terms are verified. Do not hard-code an unreviewed vendor.
+
+### Phase 2 release gate
+
+- [x] Complete a signed two-reviewer fixture from source assertion through name vote,
+      edge vote, observation bundle, policy compilation, release, resolution, drift,
+      revocation, signed delta, and consumer application.
+- [x] Prove a malicious contributor cannot submit directly into active state.
+- [x] Prove one compromised reviewer cannot approve under the reference policy.
+- [x] Prove an emergency revocation reaches a pinned offline consumer without a full
+      source reimport.
+- [x] Preserve all v0.3 conflict, normalization, regional, and rollback tests.
+
+Completion note: the version 0.4 implementation satisfies the Phase 0 through 2
+acceptance boundary, including the complete two-reviewer fixture and security
+review. The immutable v0.3 contract is frozen by exact signed commit and golden
+schema/field coordinates rather than a committed SQLite generation, because this
+repository does not admit generated datasets or approval logs. General registry
+deltas remain explicitly deferred by ADR 0005; version 0.4 supplies typed source
+deltas and a separately signed, cumulative emergency block feed.
+
+## Phase 3: v0.5 storage, scale, and current-source improvements
+
+### Task 3.1: Separate runtime projections from cold audit history
+
+- [ ] Keep raw cache objects immutable and content-addressed outside Git.
+- [ ] Package completed source imports into content-addressed audit bundles with
+      manifest, record/fact indices, hashes, format version, and attribution.
+- [ ] Make a runtime generation contain selected facts, normalized projections,
+      active policy results, required votes/revocations, and signed bundle references.
+- [ ] Do not copy all historical records and facts into every runtime generation.
+- [ ] Add audit verification that streams referenced bundles and detects absence,
+      truncation, substitution, or mismatched attribution.
+- [ ] Add retention/checkpoint tooling that never deletes the only authenticated copy
+      of evidence and produces a signed deletion/retention report.
+- [ ] Measure query latency and generation size before and after the split.
+
+### Task 3.2: Add provider-scale benchmark and recovery tooling
+
+- [ ] Define repeatable small, medium, and provider-representative import profiles.
+- [ ] Record wall time, CPU time, peak RSS, compressed and expanded bytes, database
+      growth, facts/second, checkpoint frequency, restart time, build size, and query
+      latency.
+- [ ] Interrupt imports at multiple checkpoints and prove idempotent resumption.
+- [ ] Exercise disk-full, truncated input, cache corruption, duplicate records, and
+      interrupted generation publication.
+- [ ] Make benchmark reports name exact source snapshot hashes and hardware without
+      committing source data.
+- [ ] Treat synthetic performance as development evidence, not provider capacity.
+
+### Task 3.3: Harden full Wikidata ingestion and add incremental refresh
+
+- [ ] Confirm current official full and incremental formats from Wikidata documentation
+      and inspected fixtures before changing the adapter.
+- [ ] Replace the assumption that every useful entity fits in one in-memory 16 MiB
+      line with bounded disk-spooling or an explicitly receipted oversized-record path.
+- [ ] Never silently skip an oversized entity that may contain a relevant fact.
+- [ ] Add incremental add/change ingestion with authenticated base snapshot identity,
+      ordered application, checkpoints, and reconciliation against later full dumps.
+- [ ] Define how deletions and removed P856 statements become tombstones.
+- [ ] Keep full raw assertion/qualifier/reference provenance for consumed fields.
+- [ ] Make unrelated `lastrevid` changes visible in audit diffs without invalidating
+      unchanged material edge fingerprints.
+- [ ] Test real-format pathological entities and multistream compression boundaries.
+
+Authoritative format reference:
+
+- <https://www.wikidata.org/wiki/Wikidata:Database_download>
+
+### Task 3.4: Strengthen current-source acquisition verification
+
+- [ ] Prefer provider-published checksums or signatures when officially available and
+      bind verification method into the source manifest.
+- [ ] Keep HTTPS allowlists, manual redirect validation, byte bounds, strong-validator
+      resume rules, and immutable local cache behavior.
+- [ ] Detect and report source format drift before partial import can replace a source.
+- [ ] Add format-version canaries for Majestic, CrUX, Curlie, PSL, and Wikidata.
+- [ ] Preserve CrUX billing as explicit opt-in configuration and record job identity,
+      query, result period, and actual cost outside public fixtures.
+- [ ] Continue frequent PSL refresh and include exact PSL hash in normalization proofs.
+- [ ] Preserve Curlie attribution and description-redaction tests on every export path.
+
+### Task 3.5: Add source lineage and independence metadata
+
+- [ ] Record direct provider, upstream/origin dataset, transformation, snapshot, and
+      known dependency relationships for each fact source.
+- [ ] Prevent policy from counting two assertions as independent corroboration when
+      one republishes the other.
+- [ ] Expose lineage in lookup, review bundles, export, diff, and evaluation.
+- [ ] Keep unknown lineage explicit rather than assuming independence.
+
+## Phase 4: Rights-gated additional source adapters
+
+Every source follows the same gate:
+
+1. Verify authoritative download, schema, update cadence, license, attribution,
+   redistribution, database-right, and commercial-use documentation.
+2. Record the exact decision and URLs in `LICENSE_SOURCES.md`.
+3. Inspect current official fixtures. Do not infer fields from third-party examples.
+4. Add a source enum/format only with a streaming adapter and bounded failure tests.
+5. Preserve native IDs, raw relevant records, selectors, lineage, and confidence.
+6. Import into a separate logical source layer.
+7. Demonstrate that the source cannot auto-create an approved route.
+8. Run deterministic, idempotent, interrupted, malformed, and conflict fixtures.
+
+### Task 4.1: Add ROR first
+
+Rationale: ROR is CC0 and directly supplies stable organization IDs, names, aliases,
+status, locations, links, and domains. It is compact and well aligned with the model.
+
+- [ ] Verify the current ROR schema version and official release asset from the ROR
+      data-dump documentation at implementation time.
+- [ ] Consume only fields confirmed in that inspected schema.
+- [ ] Map names and aliases without merging ROR entities into Wikidata entities unless
+      an exact external identifier or reviewed equivalence supports the join.
+- [ ] Preserve links and domains as ROR assertions, not approvals.
+- [ ] Preserve status, type, country/location, external IDs, and upstream lineage.
+- [ ] Test domain conflicts, former/inactive organizations, aliases, multiple links,
+      missing fields, duplicate input, schema drift, and exact-ID equivalence.
+
+References:
+
+- <https://ror.readme.io/docs/data-dump>
+- <https://ror.readme.io/docs/ror-data-structure>
+- <https://ror.readme.io/docs/fields>
+
+### Task 4.2: Add MusicBrainz core snapshots second
+
+- [ ] Use only the CC0 core database snapshot and its verified checksums/signatures.
+- [ ] Do not use the CC BY-NC-SA live replication feed in the commercial-safe default
+      pipeline.
+- [ ] Consume exact URL entities and documented URL relationship types, including
+      official-homepage and ended-state metadata.
+- [ ] Keep artists, labels, places, and events separate by stable MusicBrainz ID.
+- [ ] Preserve relationship begin/end dates and link types as material evidence.
+- [ ] Treat community-curated URLs as assertions requiring normal Argand review.
+- [ ] Test removed URLs, ended relationships, entity redirects/merges, duplicate URLs,
+      malicious/taken-over sites, and snapshot replacement.
+
+References:
+
+- <https://musicbrainz.org/doc/MusicBrainz_Database/Download>
+- <https://musicbrainz.org/doc/Style/Relationships/URLs>
+- <https://musicbrainz.org/doc/Live_Data_Feed>
+
+### Task 4.3: Validate GND as the third adapter
+
+- [ ] Verify the exact current CC0 declaration for the selected GND files.
+- [ ] Inspect current JSON-LD or RDF schema and confirm homepage predicates before
+      implementing an adapter.
+- [ ] Preserve authority IDs, preferred/variant names, types, countries, external IDs,
+      and exact homepage assertions where present.
+- [ ] Keep its regional and language focus visible in provenance and evaluation.
+- [ ] Stop after the rights/schema spike if homepage coverage does not justify the
+      adapter cost.
+
+Reference:
+
+- <https://data.dnb.de/opendata/>
+
+### Task 4.4: Keep lower-priority candidates behind explicit holds
+
+- [ ] ORCID: research a low-confidence individuals-only adapter. Its public file is
+      CC0, but links are self-declared and require privacy, impersonation, and
+      volatility policy. Never auto-approve. Reference:
+      <https://info.orcid.org/public-data-file-use-policy/>.
+- [ ] OpenAlex: use only for research-activity/popularity metadata if useful. Record
+      ROR as upstream lineage and never count its institution website as independent
+      corroboration. Reference: <https://help.openalex.org/data/institutions/>.
+- [ ] OpenStreetMap: do not ingest until an ODbL-compatible distribution and
+      attribution architecture is approved. Reference:
+      <https://osmfoundation.org/wiki/Licence_and_Legal_FAQ>.
+- [ ] Government/corporate registries: assess jurisdiction by jurisdiction. Prefer
+      stable identity crosswalks; do not infer a website where no authoritative field
+      exists.
+- [ ] DNS, RDAP, certificate transparency, package registries, and web crawl data:
+      evaluate as observation sources only after exact terms are verified.
+- [ ] Open Library and other sources with unresolved underlying rights remain excluded.
+
+### Phase 4 acceptance
+
+- Each admitted source has authoritative license evidence, current fixture evidence,
+  exact fields consumed, attribution behavior, source lineage, and format-drift tests.
+- Full source-specific provenance appears in lookup, review bundles, export, and diff.
+- Removing any new adapter leaves existing source identities and release verification
+  deterministic.
+- No new source changes an existing edge's approval merely by corroborating it.
+
+## Phase 5: Resolver, evaluation, and consumer improvements
+
+### Task 5.1: Canonicalize locale and country semantics
+
+- [ ] Add standards-based BCP 47 parsing/canonicalization after reviewing the chosen
+      library and current specification behavior.
+- [ ] Define deterministic precedence for exact locale/country, country-only,
+      language-parent, and global-primary candidates.
+- [ ] Abstain on equal candidates at the same specificity.
+- [ ] Preserve the requested and normalized locale in the explanation envelope.
+- [ ] Add tests for `en-GB`, `en`, script subtags, case, deprecated aliases, malformed
+      tags, country-only properties, multi-country sites, and conflicting scopes.
+
+### Task 5.2: Extend property roles without weakening default navigation
+
+- [ ] Define a versioned role vocabulary covering at least global primary, regional
+      primary, product, support, developer, careers, login, status, and other reviewed
+      roles justified by real cases.
+- [ ] Keep default `resolve` restricted to the requested navigation role.
+- [ ] Require role-specific evidence and votes; a support site cannot become primary
+      because it shares a domain.
+- [ ] Preserve unknown source roles as evidence without admitting them.
+
+### Task 5.3: Add candidate discovery separately from resolution
+
+- [ ] Add prefix/fuzzy discovery only as an audit/candidate operation.
+- [ ] Preserve exact normalized matching for final admission.
+- [ ] Return candidate score components and ambiguity rather than hiding a rewrite.
+- [ ] Never let similarity bypass reviewed name-to-entity bindings.
+- [ ] Test homographs, typosquatting, short names, multilingual aliases, and popular
+      entities with colliding names.
+
+### Task 5.4: Expand evaluation into a release gate
+
+- [ ] Extend judgments to cover expected abstention reasons, selected name binding,
+      route edge, regional precedence, and policy state.
+- [ ] Report resolved-route errors separately from safe abstentions.
+- [ ] Measure coverage, abstention taxonomy, active-evidence age, expiring approvals,
+      conflict rate, reviewer agreement, review turnaround, observation drift, and
+      revocation propagation.
+- [ ] Add adversarial suites for source poisoning, alias injection, correlated sources,
+      malicious redirects, domain takeover, compromised reviewer, compromised
+      publisher, stale cache, rollback, and policy downgrade.
+- [ ] Segment evaluation by source, entity type, language, country, popularity band,
+      destination role, and evidence age without collapsing source signals.
+- [ ] Establish release thresholds only after an audited baseline exists. Any known
+      wrong resolved destination is a release blocker for the reference dataset.
+
+## Phase 6: v0.6 signed public reference dataset
+
+### Task 6.1: Define reference publisher governance
+
+- [ ] Publish reviewer eligibility, independence, conflict-of-interest, evidence,
+      expiry, appeals, correction, key rotation, incident, and emergency-revocation
+      policies.
+- [ ] Separate source maintainers, reviewers, and release publishers where practical.
+- [ ] Publish the exact policy hash and reviewer trust roots with each release.
+- [ ] Define a transparent proposal process in which contributors submit signed
+      evidence bundles rather than direct active-dataset edits.
+- [ ] Record disputed claims and abstain until policy is met.
+- [ ] Publish change logs and correction history without exposing sensitive reviewer
+      material unnecessarily.
+
+### Task 6.2: Build a deliberately bounded starter registry
+
+- [ ] Select a high-value initial coverage set using source-separated popularity and
+      declared entity classes only for prioritization.
+- [ ] Publish the selection methodology and its biases.
+- [ ] Review name bindings, entity equivalences, website edges, regional roles, and
+      current observations under the reference policy.
+- [ ] Do not claim comprehensive web, country, language, or entity-type coverage.
+- [ ] Require every resolvable destination to have current votes, unexpired evidence,
+      complete provenance, and an active monitoring schedule.
+- [ ] Run the full evaluation, diff, license, source-lineage, and release gates.
+
+### Task 6.3: Publish full releases, deltas, and revocations
+
+- [ ] Keep dataset releases separate from source-code releases and raw provider cache.
+- [ ] Produce a deterministic full generation, receipt, attribution bundle, policy,
+      reviewer trust roots, evaluation report, coverage report, and detached publisher
+      signature.
+- [ ] Produce ordered, signed deltas bound to exact base and target generation IDs.
+- [ ] Publish a small signed revocation feed with monotonic continuity and rollback
+      protection.
+- [ ] Make full and delta application atomic and recoverable after interruption.
+- [ ] Provide a documented mirror-independent verification procedure.
+- [ ] Never make an unauthenticated `latest` URL the trust root. A convenience current
+      pointer must itself be signed and rollback protected.
+
+### Task 6.4: Make immediate use simple
+
+- [ ] Add a beginner workflow that downloads a release, verifies its publisher and
+      policy, pins it, and resolves `facebook` locally.
+- [ ] Show an entity with multiple reviewed regional properties and an abstention.
+- [ ] Provide identical native CLI, Rust library, and Python subprocess examples.
+- [ ] Add a machine-readable capability/version command.
+- [ ] Explain code license separately from each dataset source license and attribution.
+- [ ] Provide update and emergency-revocation examples for systemd and cron.
+
+### Reference release acceptance
+
+- Every resolved route is reproducible from selected source facts, name/edge votes,
+  policy, and observations contained in or authenticated by the release.
+- Every conflict, rejection, abstention, and supersession remains inspectable.
+- An untrusted mirror cannot substitute a generation, policy, reviewer set, delta, or
+  revocation feed without verification failure.
+- A clean consumer machine can verify and query the release without provider
+  credentials, reviewer keys, publisher private keys, or the mutable writer database.
+- The coverage and evaluation reports distinguish implemented capability from actual
+  reviewed data coverage.
+
+## Phase 7: Optional service and portability
+
+### Task 7.1: Cross-platform immutable generation reads
+
+- [ ] Implement equivalent no-follow, regular-file, bounded-copy, hash-before-open,
+      and private-temporary-file behavior for macOS and Windows.
+- [ ] Add platform CI only on isolated runners with no release authority.
+- [ ] Preserve Linux behavior and reject platforms without a safe implementation.
+
+### Task 7.2: Read-only service only after the dataset proves useful
+
+- [ ] Expose the existing verified query envelope through a minimal read-only local
+      HTTP or Unix-socket service if consumer demand justifies it.
+- [ ] Pin one generation per process/request context and swap only after full signature,
+      policy, reviewer, and rollback verification.
+- [ ] Bound query size, output size, concurrency, and time.
+- [ ] Preserve typed abstention and explanation; do not add silent query rewriting.
+- [ ] Keep publication, review, acquisition, and private keys out of the serving process.
+
+## Operational security and supply-chain work
+
+- [ ] Add property tests or fuzz targets for URL/domain normalization, PSL behavior,
+      JSON duplicate keys, each parser, coverage selection, observation parsing,
+      signed-delta application, and locale handling.
+- [ ] Add dependency vulnerability and license-policy checks to a network-enabled,
+      isolated scheduled workflow. Preserve the offline acceptance gate separately.
+- [ ] Pin and document CI images and external actions by immutable identity.
+- [ ] Verify official provider checksums/signatures where available.
+- [ ] Add threat-model cases for reviewer-key theft, publisher-key theft, source
+      compromise, malicious proposal bundles, cache substitution, stale mirrors,
+      rollback, denial of service, and domain takeover.
+- [ ] Add key rotation and emergency response drills using disposable test keys.
+- [ ] Keep signed receipts and validation artifacts outside the source tree unless they
+      are deliberately public, non-sensitive release evidence.
+
+## Documentation deliverables
+
+- [ ] Keep the README focused on the first successful verified lookup.
+- [ ] Add an architecture document explaining assertion, observation, decision, policy,
+      generation, delta, and consumer boundaries.
+- [ ] Expand `LICENSE_SOURCES.md` for every admitted source with exact fields consumed,
+      source URLs, licenses, attribution, redistribution, update cadence, and lineage.
+- [ ] Document active versus audit export semantics.
+- [ ] Document migrations and legacy decision handling.
+- [ ] Publish the reference review and incident policies.
+- [ ] Publish provider-scale measurements without implying serving or corpus coverage.
+- [ ] Maintain a source-candidate table showing approved, research, held, and rejected
+      sources with the reason for each state.
+
+## SWOT-driven checks
+
+### Preserve strengths
+
+- Deterministic, source-separated facts and normalization.
+- Fail-closed resolution and explicit abstention.
+- Signed reviews/releases and rollback-safe revocations.
+- Complete licenses, attribution, provenance, and conflict evidence.
+- Streaming, bounded, resumable, idempotent imports.
+- Meaningful adversarial and cross-consumer tests.
+
+### Correct weaknesses
+
+- Publish an immediately usable dataset after trust and scale gates pass.
+- Replace one-latest-decision semantics with policy-compiled votes.
+- Reduce benign review churn without allowing alias inheritance.
+- Separate active runtime state from unbounded audit history.
+- Add provider-scale proof, monitoring, queues, and richer metrics.
+- Improve regional semantics and platform support after the core release.
+
+### Capture opportunities
+
+- Establish an open evidence-bundle and signed-generation interchange format.
+- Let independent publishers share facts while choosing different trust policies.
+- Supply explainable navigation authority to search engines, assistants, browsers,
+  bookmarks, enterprise catalogs, and safety products.
+- Publish an evaluation benchmark for entity-to-site resolution and abstention.
+- Use ROR, MusicBrainz, and GND to expand high-quality vertical coverage.
+
+### Mitigate threats
+
+- Compromised reviewers: quorum, independence, expiry, and sticky revocation.
+- Compromised publishers: independent reviewer roots and consumer policy verification.
+- Domain takeover: active observation, probation, and rapid revocation delivery.
+- Source poisoning: no auto-approval, source lineage, evidence diffs, and review queues.
+- Correlated evidence: upstream lineage and independence-aware policy.
+- License drift: exact rights gates and fail-closed format/license changes.
+- Cache/mirror staleness: signed continuity, expiry, and rollback protection.
+- Governance capture: public policies, disputes, appeals, and transparent changes.
+
+## Explicit deferrals
+
+Do not prioritize these before a reviewed reference dataset exists:
+
+- A general hosted API or multi-tenant account system.
+- ML or LLM approval of ownership or regional roles.
+- Open-web crawling.
+- Fuzzy matching in the final resolver.
+- Additional popularity feeds merely to enlarge source count.
+- Sources with unclear commercial reuse, database rights, or redistribution terms.
+- ODbL data without an approved packaging architecture.
+- A graphical review UI before deterministic CLI bundles and queues are proven.
+- Automatic approval renewal based on unchanged popularity, TLS, DNS, or redirects.
+
+## Definition of complete
+
+The roadmap is complete when:
+
+1. Source coverage and exports cannot confuse historical facts with active facts.
+2. Review authority is bound to trusted acceptance time and a signed policy.
+3. Name bindings and website edges change independently without creating inherited
+   authority.
+4. Multiple authenticated reviewers and sticky revocations protect the reference
+   policy from one compromised contributor or reviewer.
+5. Candidate-site observations are reproducible, bounded, rights-declared evidence
+   and material drift reaches reviewers promptly.
+6. Runtime generations remain compact while complete audit evidence stays verifiable.
+7. Current real source formats import and resume within documented modest-hardware
+   bounds, with no silent oversized-record loss.
+8. Every new source has verified commercial-reuse rights, exact consumed fields,
+   lineage, attribution, and adversarial tests.
+9. Regional resolution uses deterministic locale/country precedence and abstains on
+   unresolved ambiguity.
+10. A separately distributed signed reference dataset produces useful reviewed
+    results, including `facebook -> Facebook -> facebook.com` and multiple regional
+    properties, from a clean consumer machine.
+11. Full releases, deltas, and revocations authenticate independently of mirrors and
+    refuse rollback.
+12. Published evaluation and coverage reports state what is actually reviewed,
+    current, relevant, and resolvable without conflating implementation with data.
+13. Formatter, compiler checks, strict lints, documentation, unit/integration tests,
+    adversarial tests, source-release determinism, provider-scale canaries, and a
+    clean final diff all pass for every release boundary.
+
+## First implementation slice after approval
+
+Begin only with Phase 0 and Phase 1. Do not combine the initial migration with the
+crawler, new sources, public data acquisition, or dataset publication.
+
+The first concrete change set should contain:
+
+1. Frozen v0.3 compatibility fixtures.
+2. Approved active-coverage and export ADRs.
+3. Typed source coverage with conservative legacy behavior.
+4. Active-only export plus explicit audit export.
+5. Trusted review acceptance time and sticky revocation semantics.
+6. Schema/rules migration, documentation, and complete offline acceptance evidence.
+
+Review that diff and release it before beginning granular name/edge votes or crawler
+observations. This keeps the highest-risk semantic corrections small enough to audit
+and makes every later phase build on an unambiguous active registry.