argand-site-registry/docs/superpowers/plans/2026-09-13-v0.4-and-beyond.md

# Argand Site Registry v0.4 and Beyond Plan

> **Status:** Version 0.5 phases 0 through 3 and the ROR adapter in task 4.1 are
> implemented, provider-scale tested, and security-reviewed. MusicBrainz and GND
> remain rights/schema-gated candidates. Phases 5 through 7 describe later releases,
> including the human-reviewed public dataset; they are not version 0.5 exit work.
>
> **Current implementation:** signed commit `557ba7c` (version 0.5.0). The plan's
> original audited baseline was clean `main` at `2861337` with tagged `v0.3.0`
> runtime behavior at `ac82820`.
>
> **Execution constraint:** Work in this standalone repository only. Do not use
> subagents, edit Argand's main checkout, share its build cache, acquire paid data,
> publish a dataset, sign with production keys, or deploy without the authority
> already established for that specific action.

## Goal

Turn the v0.3 evidence and release substrate into a production-grade, reusable
entity-to-website authority system that can publish a useful signed reference
registry while preserving conflict, provenance, commercial-reuse terms, human
review, abstention, revocation, and deterministic regional resolution.

The implementation must remain useful in two modes:

1. A local publisher builds and reviews its own registry from allowed sources.
2. A consumer verifies and queries a separately distributed signed reference
   generation without trusting an unauthenticated download location.

The work is divided into independently releasable stages. Correct active-state
semantics and review controls come before broader ingestion. Provider-scale proof
comes before a public reference dataset. A hosted service remains optional.

## Product thesis

Popularity lists answer which domains receive attention. They do not reliably
answer which entity controls a domain, whether a destination is current, or which
regional property is appropriate. Argand Site Registry should compile independent
source assertions, observed site relationships, and authenticated reviewer votes
into a signed generation that resolves only when policy is satisfied.

The principal product outcome is a signed, explainable answer:

```text
query + locale/country
  -> reviewed name-to-entity binding
  -> reviewed entity-to-property edge
  -> active regional-selection policy
  -> URL or typed abstention
```

A larger assertion database is not the goal by itself. Release quality is measured
by resolved-route correctness, useful coverage, freshness, and revocation speed.

## Non-negotiable invariants

- Keep raw inputs, normalized facts, crawler observations, reviewer decisions,
  policy decisions, popularity, and release authority logically separate.
- Never infer entity equivalence or website ownership from similar names, domains,
  redirects, TLS, DNS, `sameAs`, popularity, or shared upstream data alone.
- Preserve conflicting and rejected evidence with its source-native identity.
- Every imported or derived fact retains source, source identifier, license,
  license evidence URL, retrieval timestamp, confidence, and derivation version.
- Every source adapter must be rights-reviewed, format-pinned, streaming, bounded,
  reproducible, resumable where feasible, and idempotent.
- Automated updates may download, import, evaluate, and build candidates. They may
  not approve, renew, sign, activate, or publish them.
- Resolution remains fail closed. Ambiguity, missing policy, expired evidence,
  missing votes, unsupported locale semantics, or conflicting equal candidates
  produces a typed abstention.
- Popularity remains source-separated evidence and never becomes an ownership vote.
- Existing v0.3 data remains auditable. Migrations must not reinterpret an old
  approval, source scope, or timestamp as if it had been created under a new rule.
- Curlie descriptions remain redacted unless the exact distribution surface meets
  its attribution obligations.
- Public submissions are untrusted proposals. They never mutate an active release.
- Cloudflare Radar, default Tranco, Cisco Umbrella, and other unverified sources
  remain excluded.

## Target architecture

```mermaid
flowchart LR
    A[Rights-reviewed source adapters] --> B[Immutable source manifests and raw cache]
    B --> C[Writer store: records, facts, conflicts, tombstones]
    C --> D[Normalized assertion graph]
    E[Bounded candidate-site observer] --> F[Immutable observation bundles]
    F --> D
    D --> G[Review queue and evidence diff]
    G --> H[Signed reviewer votes]
    H --> I[Versioned publisher policy compiler]
    I --> J[Immutable active projection]
    J --> K[Signed full generation]
    J --> L[Signed delta and revocation feed]
    K --> M[Verified local library and CLI]
    L --> M
    M --> N[Exact lookup and regional resolve]
```

The writer store remains append-oriented and audit-capable. Runtime generations
contain the selected facts, active decisions, required proof material, and signed
references to cold audit bundles. They do not duplicate every historical raw row.

## Release sequence

| Release | Purpose | Exit condition |
| --- | --- | --- |
| v0.3.x | Correct active-state and review-time semantics | Historical export is explicit, scope overlap fails closed, review acceptance time is trusted, and legacy behavior is regression-tested. |
| v0.4 | Scalable trust and observation pipeline | Quorum policy, granular name/edge decisions, sticky revocations, structured evidence bundles, and review queues work end to end. |
| v0.5 | Provider-scale and source expansion | Real-format scale canaries pass; incremental Wikidata and the first new rights-approved adapters are reproducible and bounded. |
| v0.6 | Signed public reference registry | A reviewed dataset, full/delta releases, revocation delivery, coverage report, and consumer verification are published independently of source releases. |
| Later | Wider resolver/product surface | Locale fallback, typed destination roles, candidate discovery, platform portability, and an optional read-only service are justified by real use. |

## Phase 0: Freeze contracts and record design decisions

### Task 0.1: Capture the v0.3 compatibility baseline

- [x] Record the exact v0.3 CLI output schemas, generation schema, source-manifest
      schema, rules version, review JSON, release receipts, and Python/Rust examples.
- [x] Add golden fixtures for a complete v0.3 generation and a mutable schema-v3
      writer database without committing provider data or private keys.
- [x] Verify that current `lookup`, `resolve`, `diff`, `export`, `stats`, signature
      verification, rollback refusal, and migrations behave exactly as documented.
- [x] Preserve the current five-source all-synthetic fixture as a compatibility gate.

Likely files:

- `crates/argand-site-registry/tests/common/`
- `crates/argand-site-registry/tests/registry.rs`
- `crates/argand-site-registry/tests/failures.rs`
- `crates/argand-site-registry/tests/cli.rs`
- `docs/VALIDATION.md`

Acceptance:

- A v0.3 reader fixture remains readable or fails with a precise documented version
  error after each later schema change.
- No later test can silently regenerate the baseline fixture from new behavior.

### Task 0.2: Write architecture decisions before migrations

- [x] Add an ADR for active source coverage and supersession.
- [x] Add an ADR for separating name bindings, website edges, and observations.
- [x] Add an ADR for reviewer votes, revocation precedence, and publisher policy.
- [x] Add an ADR for runtime projection versus cold audit retention.
- [x] Add an ADR for full and delta dataset release identities.
- [x] Add an ADR for source lineage so copied upstream evidence does not count twice.
- [x] Document whether broad proprietary embedding is a goal. If so, evaluate a
      small permissively licensed format/verifier crate without changing the AGPL
      publisher engine or source-driven dataset licenses automatically.

Acceptance:

- Every later migration and public contract links to an approved decision.
- The ADRs explain rejected alternatives and compatibility consequences.

## Phase 1: v0.3.x active-state and trust hardening

### Task 1.1: Replace free-form snapshot replacement semantics with typed coverage

Problem: v0.3 selects the latest completed snapshot per exact `(source, scope)`
string. Different strings are treated as additive even when one is a full snapshot
or overlaps another partition.

- [x] Introduce a source-manifest version with explicit coverage semantics:
      collection identity, coverage kind, stable partition identity, base snapshot
      where applicable, and explicit supersession references.
- [x] Preserve v1 manifest parsing. Map every legacy `(source, scope)` to an isolated
      legacy partition so migration does not silently change its active facts.
- [x] Require operators to provide an explicit mapping before a legacy partial
      snapshot can join or be replaced by a typed full collection.
- [x] Select one coherent coverage set per source collection:
  - a complete full snapshot supersedes older declared partitions;
  - disjoint partitions compose only when their identities are explicit;
  - deltas require an authenticated base and ordered continuity;
  - overlapping or missing coverage relationships abort the build;
  - deletions use explicit tombstones and remain auditable.
- [x] Include the complete selected-coverage graph in the build receipt.
- [x] Make `stats` and `diff` report selected, superseded, incomplete, and conflicting
      source snapshots separately.

Likely files:

- `crates/argand-site-registry/src/model.rs`
- `crates/argand-site-registry/src/store.rs`
- `crates/argand-site-registry/src/build.rs`
- `crates/argand-site-registry/src/diff.rs`
- `crates/argand-site-registry/src/query.rs`
- `crates/argand-site-registry/src/cli.rs`
- `crates/argand-site-registry/migrations/004.sql`

Tests:

- Latest snapshot replaces an older snapshot in the same partition.
- A declared full snapshot supersedes earlier partitions.
- Two explicit disjoint partitions compose.
- An undeclared full-plus-partial combination fails.
- Overlapping partitions fail.
- Missing delta bases, skipped deltas, and forked delta histories fail.
- Tombstoned facts disappear from the active projection but remain in audit history.
- Failed or partial imports never replace selected complete coverage.
- Reimporting identical manifests and deltas is byte-for-byte idempotent.

### Task 1.2: Split active export from audit-history export

Problem: v0.3 JSONL export emits facts from every complete retained source snapshot,
including superseded snapshots, without a per-row active-state marker.

- [x] Change the normal export contract to emit selected active facts only.
- [x] Version the export envelope and include generation identity, selected source
      IDs, coverage-policy version, derivation version, and attribution identity.
- [x] Add explicit selection state to every assertion.
- [x] Add a separate audit export that includes active, superseded, rejected, and
      tombstoned facts with replacement links.
- [x] Keep Curlie descriptions redacted by default in both modes.
- [x] Make consumers reject unknown export schema versions.
- [x] Document that neither export bypasses `resolve` admission policy.

Likely files:

- `crates/argand-site-registry/src/release.rs`
- `crates/argand-site-registry/src/cli.rs`
- `crates/argand-site-registry/src/diff.rs`
- `README.md`
- `docs/CONSUMERS.md`
- `docs/TRUST.md`

Tests:

- Superseded facts are absent from active export.
- Audit export retains and labels the same facts.
- Active export and native generation agree exactly on selected source IDs.
- Description redaction and attribution survive both modes.
- Python and Rust consumers refuse unknown versions and preserve null abstention.

### Task 1.3: Bind decisions to trusted acceptance time

Problem: the reviewer signs `reviewed_at`, but the writer does not persist a trusted
append time. That self-declared time influences approval validity and reviewer-key
validity-epoch checks.

- [x] Record `accepted_at` from the writer when exact signature verification and
      candidate validation succeed.
- [x] Make approval validity begin no earlier than trusted acceptance.
- [x] Bound effective expiry by both the signed review duration and the enforced
      maximum measured from acceptance, preventing backdating or future dating from
      extending authority.
- [x] Evaluate reviewer eligibility at acceptance for new decisions. Preserve the
      signed claimed decision time for audit, without treating it as trusted time.
- [x] Include acceptance time and the time-policy version in generation receipts.
- [x] Retain legacy approvals for audit but require an explicit migration policy or
      fresh decision before they are active under the new rules. Never synthesize a
      historical acceptance time. Preserve legacy revocations regardless.
- [x] Test clock skew, future dates, backdates, expired keys, removed keys, offline
      signing followed by later acceptance, and reproducible builds at fixed clocks.

Likely files:

- `crates/argand-site-registry/src/review.rs`
- `crates/argand-site-registry/src/query.rs`
- `crates/argand-site-registry/src/resolution.rs`
- `crates/argand-site-registry/src/release.rs`
- `crates/argand-site-registry/migrations/004.sql`
- `docs/TRUST.md`
- `docs/PUBLISHING.md`

### Task 1.4: Make revocation precedence explicit before quorum work

- [x] Treat a valid revocation as sticky for its exact subject.
- [x] Require an explicit signed supersession that references the revocation before
      the same destination can become active again.
- [x] Prevent a later ordinary approval from overriding a revocation by sequence
      order alone.
- [x] Preserve existing activation rollback checks and strengthen them to compare
      revocation identities and supersession relationships.
- [x] Add a machine-readable emergency revocation export suitable for cached
      consumers.

Acceptance:

- Reordering, forking, or appending an ordinary approval cannot erase a revocation.
- A deliberately superseded revocation remains visible in lookup, diff, and audit.
- An offline consumer can apply the revocation export before receiving a full build.

### Phase 1 release gate

- [x] Run `cargo fetch --locked` once, then the complete offline `scripts/check.sh`.
- [x] Run migrations from empty, schema v1, v2, and v3 databases.
- [x] Build the same generation twice and compare every output byte.
- [x] Verify source release determinism from a clean signed commit.
- [x] Update changelog, trust docs, format contracts, and migration guidance.
- [x] Run `cargo audit --deny warnings` and record it separately from the offline gate.
- [x] Confirm no provider data, keys, approval logs, or private paths entered Git.

## Phase 2: v0.4 scalable review and structured evidence

### Task 2.1: Separate name-to-entity trust from entity-to-property trust

Problem: v0.3 binds each website approval to the entity's full name set. This stops
a newly injected alias from inheriting a route, but benign alias or label changes
invalidate every website edge. A global Wikidata revision also changes edge identity
when the P856 statement itself did not materially change.

- [x] Create independent material fingerprints for:
  - source name or alias assertion to stable entity;
  - explicit entity equivalence;
  - entity-to-normalized-web-property assertion;
  - normalized regional role and scope;
  - observation bundle;
  - reviewer vote and policy result.
- [x] Bind website-edge identity only to material website evidence: exact source
      statement identity/value/rank/relevant qualifiers, normalized URL result, and
      derivation rule version.
- [x] Retain source record revision and unrelated entity metadata in provenance and
      diffs without making them part of website-edge identity.
- [x] Require `resolve` to satisfy both an admitted name binding and an admitted
      website edge. `lookup` continues to expose unreviewed names and edges.
- [x] Ensure a newly imported alias cannot inherit an existing approved route.
- [x] Ensure a benign accepted alias update does not invalidate an unchanged route.
- [x] Give canonical labels, aliases, and source-specific names distinct evidence
      identities and policy treatment.

Tests:

- Adding an unreviewed alias never creates a resolvable query.
- Removing or changing an approved alias affects only that binding.
- An unrelated Wikidata `lastrevid` change preserves the route fingerprint.
- A P856 URL, rank, end qualifier, or relevant regional qualifier change invalidates
  the route approval.
- Confusable and bidi-control names remain rejected.
- Explicit entity equivalence never carries unstated name or route authority.

### Task 2.2: Replace latest-decision-wins with signed votes and policy compilation

- [x] Represent decisions as immutable signed votes by authenticated reviewer.
- [x] Permit one current vote per signer, subject, decision type, and policy epoch;
      preserve superseded votes in the append-only log.
- [x] Add a versioned publisher policy that defines:
  - required approval threshold by decision type and risk class;
  - revocation threshold and sticky-revocation behavior;
  - publisher/reviewer separation;
  - reviewer groups or independence constraints where configured;
  - maximum approval age and evidence-freshness requirements;
  - treatment of source conflicts and unresolved observations.
- [x] Hash the exact policy and selected reviewer trust roots into the release receipt.
- [x] Compile votes deterministically into approved, revoked, expired, disputed,
      probationary, or insufficient-review state.
- [x] Keep policy configurable for independent publishers while shipping a strict,
      documented reference policy for Argand's own releases.
- [x] Preserve the ability to operate locally without accounts or a network service.

Tests:

- One signer cannot satisfy a two-independent-reviewer policy.
- Duplicate keys or aliases for the same reviewer do not create extra votes.
- Publisher/reviewer separation is enforced when enabled.
- One authorized revocation blocks resolution under the reference policy.
- Conflicting votes produce a disputed state and abstention.
- Changing policy invalidates the old compiled result without changing source facts.
- Release verification fails on policy, signer, vote, or receipt tampering.

### Task 2.3: Turn observation types into a stored evidence pipeline

- [x] Extend the writer schema with immutable observation batches and observations.
- [x] Preserve the existing distinction between observations and ownership claims.
- [x] Add an adapter contract for externally captured observations, including source,
      rights declaration, capture ID, retrieval time, content hash, exact selector,
      from/to URLs, relation, and confidence.
- [x] Project redirect, canonical, hreflang, JSON-LD `sameAs`, sitemap, and country
      selector evidence without automatically creating an entity edge.
- [x] Add observation lookup, reverse lookup, generation diff, and evidence-bundle
      output for review.
- [x] Record negative or failed observations with bounded error classes so absence is
      not confused with a fetch that never succeeded.
- [x] Define retention that stores hashes and necessary bounded extracts rather than
      copied page bodies unless rights and need are established.

Likely files:

- `crates/argand-site-registry/src/observation.rs`
- `crates/argand-site-registry/src/store.rs`
- `crates/argand-site-registry/src/build.rs`
- `crates/argand-site-registry/src/query.rs`
- `crates/argand-site-registry/src/diff.rs`
- `crates/argand-site-registry/migrations/005.sql`

### Task 2.4: Add a bounded candidate-site observer

- [x] Observe only imported/reviewed candidate URLs. Do not start an open-web crawler.
- [x] Enforce scheme, redirect-count, response-size, header-size, decompression,
      timeout, DNS-result, address-range, and per-host request bounds.
- [x] Block credentials, local/private/link-local targets, unsafe redirect transitions,
      non-HTTP protocols, and host confusion.
- [x] Capture redirect chains, final URL, status, canonical, hreflang, sameAs, sitemap
      references, country selectors, DNS answers, and TLS certificate fingerprints as
      separate evidence classes.
- [x] Do not treat TLS, DNS, redirects, or site self-assertions as ownership proof.
- [x] Emit immutable observation manifests compatible with Task 2.3.
- [x] Support cache-only replay so parser and policy tests never require the network.
- [x] Keep acquisition cadence, concurrency, bandwidth, and data path configurable.

Security tests:

- SSRF attempts, redirect loops, compression bombs, oversized markup, malformed HTML,
  DNS rebinding, mixed encodings, invalid certificates, and cross-scheme redirects.
- Parser fuzz/property tests for headers, hreflang, canonical, JSON-LD, and sitemaps.
- Deterministic replay from captured fixtures with no network access.

### Task 2.5: Build review queues and evidence bundles before a graphical UI

- [x] Add deterministic queue output ordered by risk and material change, not source
      popularity alone.
- [x] Queue new routes, new aliases, source conflicts, changed website statements,
      redirects across registrable domains, observation drift, expiring approvals,
      unresolved regional scopes, and revoked destinations proposed for reinstatement.
- [x] Produce a bounded review bundle containing exact claims, conflicts, observation
      diffs, source licenses, capture hashes, requested role, and candidate fingerprint.
- [x] Add commands to prepare a vote, verify exact vote bytes, append it, and show the
      compiled policy result.
- [x] Keep the queue read-only and deterministic. Never let viewing evidence mutate
      approval state.
- [x] Defer a browser workbench until CLI bundles have proven the workflow.

### Task 2.6: Monitor approved routes and classify drift

- [x] Schedule observation refresh independently from source import cadence.
- [x] Classify material changes: unreachable, cross-domain redirect, DNS/TLS change,
      content/canonical shift, domain expiry indicators, malware-policy result, or no
      material change.
- [x] Use risk policy to shorten review intervals. Never auto-extend approvals.
- [x] Put materially changed routes into probation or revoke them according to signed
      publisher policy; default to abstention where evidence is insufficient.
- [x] Record every transition and make revocation candidates immediately exportable.
- [x] Design malware-feed adapters only after exact commercial-reuse and redistribution
      terms are verified. Do not hard-code an unreviewed vendor.

### Phase 2 release gate

- [x] Complete a signed two-reviewer fixture from source assertion through name vote,
      edge vote, observation bundle, policy compilation, release, resolution, drift,
      revocation, signed delta, and consumer application.
- [x] Prove a malicious contributor cannot submit directly into active state.
- [x] Prove one compromised reviewer cannot approve under the reference policy.
- [x] Prove an emergency revocation reaches a pinned offline consumer without a full
      source reimport.
- [x] Preserve all v0.3 conflict, normalization, regional, and rollback tests.

Completion note: the version 0.4 implementation satisfies the Phase 0 through 2
acceptance boundary, including the complete two-reviewer fixture and security
review. The immutable v0.3 contract is frozen by exact signed commit and golden
schema/field coordinates rather than a committed SQLite generation, because this
repository does not admit generated datasets or approval logs. General registry
deltas remain explicitly deferred by ADR 0005; version 0.4 supplies typed source
deltas and a separately signed, cumulative emergency block feed.

## Phase 3: v0.5 storage, scale, and current-source improvements

### Task 3.1: Separate runtime projections from cold audit history

- [x] Keep raw cache objects immutable and content-addressed outside Git.
- [x] Package completed source imports into content-addressed audit bundles with
      manifest, record/fact indices, hashes, format version, and attribution.
- [x] Make a runtime generation contain selected facts, normalized projections,
      active policy results, required votes/revocations, and signed bundle references.
- [x] Do not copy all historical records and facts into every runtime generation.
- [x] Add audit verification that streams referenced bundles and detects absence,
      truncation, substitution, or mismatched attribution.
- [x] Add retention/checkpoint tooling that never deletes the only authenticated copy
      of evidence and produces a signed deletion/retention report.
- [x] Measure query latency and generation size before and after the split.

### Task 3.2: Add provider-scale benchmark and recovery tooling

- [x] Define repeatable small, medium, and provider-representative import profiles.
- [x] Record wall time, CPU time, peak RSS, compressed and expanded bytes, database
      growth, facts/second, checkpoint frequency, restart time, build size, and query
      latency.
- [x] Interrupt imports at multiple checkpoints and prove idempotent resumption.
- [x] Exercise disk-full, truncated input, cache corruption, duplicate records, and
      interrupted generation publication.
- [x] Make benchmark reports name exact source snapshot hashes and hardware without
      committing source data.
- [x] Treat synthetic performance as development evidence, not provider capacity.

### Task 3.3: Harden full Wikidata ingestion and add incremental refresh

- [x] Confirm current official full and incremental formats from Wikidata documentation
      and inspected fixtures before changing the adapter.
- [x] Replace the assumption that every useful entity fits in one in-memory 16 MiB
      line with bounded disk-spooling or an explicitly receipted oversized-record path.
- [x] Never silently skip an oversized entity that may contain a relevant fact.
- [x] Add incremental add/change ingestion with authenticated base snapshot identity,
      ordered application, checkpoints, and reconciliation against later full dumps.
- [x] Define how deletions and removed P856 statements become tombstones.
- [x] Keep full raw assertion/qualifier/reference provenance for consumed fields.
- [x] Make unrelated `lastrevid` changes visible in audit diffs without invalidating
      unchanged material edge fingerprints.
- [x] Test real-format pathological entities and multistream compression boundaries.

Authoritative format reference:

- <https://www.wikidata.org/wiki/Wikidata:Database_download>

Implementation note: the inspected Add/Change distribution is XML and the provider warns that embedded JSON in XML is unstable. Version 0.5 therefore implements ordered typed deltas from official Wikibase entity JSON and deliberately rejects the XML incremental artifact.

### Task 3.4: Strengthen current-source acquisition verification

- [x] Prefer provider-published checksums or signatures when officially available and
      bind verification method into the source manifest.
- [x] Keep HTTPS allowlists, manual redirect validation, byte bounds, strong-validator
      resume rules, and immutable local cache behavior.
- [x] Detect and report source format drift before partial import can replace a source.
- [x] Add format-version canaries for Majestic, CrUX, Curlie, PSL, and Wikidata.
- [x] Preserve CrUX billing as explicit opt-in configuration and record job identity,
      query, result period, and actual cost outside public fixtures.
- [x] Continue frequent PSL refresh and include exact PSL hash in normalization proofs.
- [x] Preserve Curlie attribution and description-redaction tests on every export path.

### Task 3.5: Add source lineage and independence metadata

- [x] Record direct provider, upstream/origin dataset, transformation, snapshot, and
      known dependency relationships for each fact source.
- [x] Prevent policy from counting two assertions as independent corroboration when
      one republishes the other.
- [x] Expose lineage in lookup, review bundles, export, diff, and evaluation.
- [x] Keep unknown lineage explicit rather than assuming independence.

Phase 3 completion note: compact v3 generations, audit bundle verification and no-delete retention, the source-pinned benchmark runner, two-checkpoint process-kill recovery, provider-scale ROR/PSL measurements, streamed Wikidata JSON arrays, authenticated record ceilings, typed JSON tombstones, source-bound checksums, current format canaries and explicit source lineage are implemented and tested in version 0.5.

## Phase 4: Rights-gated additional source adapters

Every source follows the same gate:

1. Verify authoritative download, schema, update cadence, license, attribution,
   redistribution, database-right, and commercial-use documentation.
2. Record the exact decision and URLs in `LICENSE_SOURCES.md`.
3. Inspect current official fixtures. Do not infer fields from third-party examples.
4. Add a source enum/format only with a streaming adapter and bounded failure tests.
5. Preserve native IDs, raw relevant records, selectors, lineage, and confidence.
6. Import into a separate logical source layer.
7. Demonstrate that the source cannot auto-create an approved route.
8. Run deterministic, idempotent, interrupted, malformed, and conflict fixtures.

### Task 4.1: Add ROR first

Rationale: ROR is CC0 and directly supplies stable organization IDs, names, aliases,
status, locations, links, and domains. It is compact and well aligned with the model.

- [x] Verify the current ROR schema version and official release asset from the ROR
      data-dump documentation at implementation time.
- [x] Consume only fields confirmed in that inspected schema.
- [x] Map names and aliases without merging ROR entities into Wikidata entities unless
      an exact external identifier or reviewed equivalence supports the join.
- [x] Preserve links and domains as ROR assertions, not approvals.
- [x] Preserve status, type, country/location, external IDs, and upstream lineage.
- [x] Test domain conflicts, former/inactive organizations, aliases, multiple links,
      missing fields, duplicate input, schema drift, and exact-ID equivalence.

References:

- <https://ror.readme.io/docs/data-dump>
- <https://ror.readme.io/docs/ror-data-structure>
- <https://ror.readme.io/docs/fields>

### Task 4.2: Add MusicBrainz core snapshots second

- [ ] Use only the CC0 core database snapshot and its verified checksums/signatures.
- [ ] Do not use the CC BY-NC-SA live replication feed in the commercial-safe default
      pipeline.
- [ ] Consume exact URL entities and documented URL relationship types, including
      official-homepage and ended-state metadata.
- [ ] Keep artists, labels, places, and events separate by stable MusicBrainz ID.
- [ ] Preserve relationship begin/end dates and link types as material evidence.
- [ ] Treat community-curated URLs as assertions requiring normal Argand review.
- [ ] Test removed URLs, ended relationships, entity redirects/merges, duplicate URLs,
      malicious/taken-over sites, and snapshot replacement.

References:

- <https://musicbrainz.org/doc/MusicBrainz_Database/Download>
- <https://musicbrainz.org/doc/Style/Relationships/URLs>
- <https://musicbrainz.org/doc/Live_Data_Feed>

### Task 4.3: Validate GND as the third adapter

- [ ] Verify the exact current CC0 declaration for the selected GND files.
- [ ] Inspect current JSON-LD or RDF schema and confirm homepage predicates before
      implementing an adapter.
- [ ] Preserve authority IDs, preferred/variant names, types, countries, external IDs,
      and exact homepage assertions where present.
- [ ] Keep its regional and language focus visible in provenance and evaluation.
- [ ] Stop after the rights/schema spike if homepage coverage does not justify the
      adapter cost.

Reference:

- <https://data.dnb.de/opendata/>

### Task 4.4: Keep lower-priority candidates behind explicit holds

- [x] ORCID: research a low-confidence individuals-only adapter. Its public file is
      CC0, but links are self-declared and require privacy, impersonation, and
      volatility policy. Never auto-approve. Reference:
      <https://info.orcid.org/public-data-file-use-policy/>.
- [x] OpenAlex: use only for research-activity/popularity metadata if useful. Record
      ROR as upstream lineage and never count its institution website as independent
      corroboration. Reference: <https://help.openalex.org/data/institutions/>.
- [x] OpenStreetMap: do not ingest until an ODbL-compatible distribution and
      attribution architecture is approved. Reference:
      <https://osmfoundation.org/wiki/Licence_and_Legal_FAQ>.
- [x] Government/corporate registries: assess jurisdiction by jurisdiction. Prefer
      stable identity crosswalks; do not infer a website where no authoritative field
      exists.
- [x] DNS, RDAP, certificate transparency, package registries, and web crawl data:
      evaluate as observation sources only after exact terms are verified.
- [x] Open Library and other sources with unresolved underlying rights remain excluded.

Task 4.4 completion note: the reviewed hold states and their next admission gates
are maintained in `docs/SOURCE-CANDIDATES.md`; none of these sources is ingestible.

### Phase 4 acceptance

- Each admitted source has authoritative license evidence, current fixture evidence,
  exact fields consumed, attribution behavior, source lineage, and format-drift tests.
- Full source-specific provenance appears in lookup, review bundles, export, and diff.
- Removing any new adapter leaves existing source identities and release verification
  deterministic.
- No new source changes an existing edge's approval merely by corroborating it.

## Phase 5: Resolver, evaluation, and consumer improvements

### Task 5.1: Canonicalize locale and country semantics

- [ ] Add standards-based BCP 47 parsing/canonicalization after reviewing the chosen
      library and current specification behavior.
- [ ] Define deterministic precedence for exact locale/country, country-only,
      language-parent, and global-primary candidates.
- [ ] Abstain on equal candidates at the same specificity.
- [ ] Preserve the requested and normalized locale in the explanation envelope.
- [ ] Add tests for `en-GB`, `en`, script subtags, case, deprecated aliases, malformed
      tags, country-only properties, multi-country sites, and conflicting scopes.

### Task 5.2: Extend property roles without weakening default navigation

- [ ] Define a versioned role vocabulary covering at least global primary, regional
      primary, product, support, developer, careers, login, status, and other reviewed
      roles justified by real cases.
- [ ] Keep default `resolve` restricted to the requested navigation role.
- [ ] Require role-specific evidence and votes; a support site cannot become primary
      because it shares a domain.
- [ ] Preserve unknown source roles as evidence without admitting them.

### Task 5.3: Add candidate discovery separately from resolution

- [ ] Add prefix/fuzzy discovery only as an audit/candidate operation.
- [ ] Preserve exact normalized matching for final admission.
- [ ] Return candidate score components and ambiguity rather than hiding a rewrite.
- [ ] Never let similarity bypass reviewed name-to-entity bindings.
- [ ] Test homographs, typosquatting, short names, multilingual aliases, and popular
      entities with colliding names.

### Task 5.4: Expand evaluation into a release gate

- [ ] Extend judgments to cover expected abstention reasons, selected name binding,
      route edge, regional precedence, and policy state.
- [ ] Report resolved-route errors separately from safe abstentions.
- [ ] Measure coverage, abstention taxonomy, active-evidence age, expiring approvals,
      conflict rate, reviewer agreement, review turnaround, observation drift, and
      revocation propagation.
- [ ] Add adversarial suites for source poisoning, alias injection, correlated sources,
      malicious redirects, domain takeover, compromised reviewer, compromised
      publisher, stale cache, rollback, and policy downgrade.
- [ ] Segment evaluation by source, entity type, language, country, popularity band,
      destination role, and evidence age without collapsing source signals.
- [ ] Establish release thresholds only after an audited baseline exists. Any known
      wrong resolved destination is a release blocker for the reference dataset.

## Phase 6: v0.6 signed public reference dataset

### Task 6.1: Define reference publisher governance

- [ ] Publish reviewer eligibility, independence, conflict-of-interest, evidence,
      expiry, appeals, correction, key rotation, incident, and emergency-revocation
      policies.
- [ ] Separate source maintainers, reviewers, and release publishers where practical.
- [ ] Publish the exact policy hash and reviewer trust roots with each release.
- [ ] Define a transparent proposal process in which contributors submit signed
      evidence bundles rather than direct active-dataset edits.
- [ ] Record disputed claims and abstain until policy is met.
- [ ] Publish change logs and correction history without exposing sensitive reviewer
      material unnecessarily.

### Task 6.2: Build a deliberately bounded starter registry

- [ ] Select a high-value initial coverage set using source-separated popularity and
      declared entity classes only for prioritization.
- [ ] Publish the selection methodology and its biases.
- [ ] Review name bindings, entity equivalences, website edges, regional roles, and
      current observations under the reference policy.
- [ ] Do not claim comprehensive web, country, language, or entity-type coverage.
- [ ] Require every resolvable destination to have current votes, unexpired evidence,
      complete provenance, and an active monitoring schedule.
- [ ] Run the full evaluation, diff, license, source-lineage, and release gates.

### Task 6.3: Publish full releases, deltas, and revocations

- [ ] Keep dataset releases separate from source-code releases and raw provider cache.
- [ ] Produce a deterministic full generation, receipt, attribution bundle, policy,
      reviewer trust roots, evaluation report, coverage report, and detached publisher
      signature.
- [ ] Produce ordered, signed deltas bound to exact base and target generation IDs.
- [ ] Publish a small signed revocation feed with monotonic continuity and rollback
      protection.
- [ ] Make full and delta application atomic and recoverable after interruption.
- [ ] Provide a documented mirror-independent verification procedure.
- [ ] Never make an unauthenticated `latest` URL the trust root. A convenience current
      pointer must itself be signed and rollback protected.

### Task 6.4: Make immediate use simple

- [ ] Add a beginner workflow that downloads a release, verifies its publisher and
      policy, pins it, and resolves `facebook` locally.
- [ ] Show an entity with multiple reviewed regional properties and an abstention.
- [ ] Provide identical native CLI, Rust library, and Python subprocess examples.
- [ ] Add a machine-readable capability/version command.
- [ ] Explain code license separately from each dataset source license and attribution.
- [ ] Provide update and emergency-revocation examples for systemd and cron.

### Reference release acceptance

- Every resolved route is reproducible from selected source facts, name/edge votes,
  policy, and observations contained in or authenticated by the release.
- Every conflict, rejection, abstention, and supersession remains inspectable.
- An untrusted mirror cannot substitute a generation, policy, reviewer set, delta, or
  revocation feed without verification failure.
- A clean consumer machine can verify and query the release without provider
  credentials, reviewer keys, publisher private keys, or the mutable writer database.
- The coverage and evaluation reports distinguish implemented capability from actual
  reviewed data coverage.

## Phase 7: Optional service and portability

### Task 7.1: Cross-platform immutable generation reads

- [ ] Implement equivalent no-follow, regular-file, bounded-copy, hash-before-open,
      and private-temporary-file behavior for macOS and Windows.
- [ ] Add platform CI only on isolated runners with no release authority.
- [ ] Preserve Linux behavior and reject platforms without a safe implementation.

### Task 7.2: Read-only service only after the dataset proves useful

- [ ] Expose the existing verified query envelope through a minimal read-only local
      HTTP or Unix-socket service if consumer demand justifies it.
- [ ] Pin one generation per process/request context and swap only after full signature,
      policy, reviewer, and rollback verification.
- [ ] Bound query size, output size, concurrency, and time.
- [ ] Preserve typed abstention and explanation; do not add silent query rewriting.
- [ ] Keep publication, review, acquisition, and private keys out of the serving process.

## Operational security and supply-chain work

- [ ] Add property tests or fuzz targets for URL/domain normalization, PSL behavior,
      JSON duplicate keys, each parser, coverage selection, observation parsing,
      signed-delta application, and locale handling.
- [ ] Add dependency vulnerability and license-policy checks to a network-enabled,
      isolated scheduled workflow. Preserve the offline acceptance gate separately.
- [ ] Pin and document CI images and external actions by immutable identity.
- [ ] Verify official provider checksums/signatures where available.
- [ ] Add threat-model cases for reviewer-key theft, publisher-key theft, source
      compromise, malicious proposal bundles, cache substitution, stale mirrors,
      rollback, denial of service, and domain takeover.
- [ ] Add key rotation and emergency response drills using disposable test keys.
- [ ] Keep signed receipts and validation artifacts outside the source tree unless they
      are deliberately public, non-sensitive release evidence.

## Documentation deliverables

- [x] Keep the README focused on the first successful verified lookup.
- [x] Add an architecture document explaining assertion, observation, decision, policy,
      generation, delta, and consumer boundaries.
- [x] Expand `LICENSE_SOURCES.md` for every admitted source with exact fields consumed,
      source URLs, licenses, attribution, redistribution, update cadence, and lineage.
- [x] Document active versus audit export semantics.
- [x] Document migrations and legacy decision handling.
- [ ] Publish the reference review and incident policies.
- [x] Publish provider-scale measurements without implying serving or corpus coverage.
- [x] Maintain a source-candidate table showing approved, research, held, and rejected
      sources with the reason for each state.

## SWOT-driven checks

### Preserve strengths

- Deterministic, source-separated facts and normalization.
- Fail-closed resolution and explicit abstention.
- Signed reviews/releases and rollback-safe revocations.
- Complete licenses, attribution, provenance, and conflict evidence.
- Streaming, bounded, resumable, idempotent imports.
- Meaningful adversarial and cross-consumer tests.

### Correct weaknesses

- Publish an immediately usable dataset after trust and scale gates pass.
- Replace one-latest-decision semantics with policy-compiled votes.
- Reduce benign review churn without allowing alias inheritance.
- Separate active runtime state from unbounded audit history.
- Add provider-scale proof, monitoring, queues, and richer metrics.
- Improve regional semantics and platform support after the core release.

### Capture opportunities

- Establish an open evidence-bundle and signed-generation interchange format.
- Let independent publishers share facts while choosing different trust policies.
- Supply explainable navigation authority to search engines, assistants, browsers,
  bookmarks, enterprise catalogs, and safety products.
- Publish an evaluation benchmark for entity-to-site resolution and abstention.
- Use ROR, MusicBrainz, and GND to expand high-quality vertical coverage.

### Mitigate threats

- Compromised reviewers: quorum, independence, expiry, and sticky revocation.
- Compromised publishers: independent reviewer roots and consumer policy verification.
- Domain takeover: active observation, probation, and rapid revocation delivery.
- Source poisoning: no auto-approval, source lineage, evidence diffs, and review queues.
- Correlated evidence: upstream lineage and independence-aware policy.
- License drift: exact rights gates and fail-closed format/license changes.
- Cache/mirror staleness: signed continuity, expiry, and rollback protection.
- Governance capture: public policies, disputes, appeals, and transparent changes.

## Explicit deferrals

Do not prioritize these before a reviewed reference dataset exists:

- A general hosted API or multi-tenant account system.
- ML or LLM approval of ownership or regional roles.
- Open-web crawling.
- Fuzzy matching in the final resolver.
- Additional popularity feeds merely to enlarge source count.
- Sources with unclear commercial reuse, database rights, or redistribution terms.
- ODbL data without an approved packaging architecture.
- A graphical review UI before deterministic CLI bundles and queues are proven.
- Automatic approval renewal based on unchanged popularity, TLS, DNS, or redirects.

## Definition of complete

The roadmap is complete when:

1. Source coverage and exports cannot confuse historical facts with active facts.
2. Review authority is bound to trusted acceptance time and a signed policy.
3. Name bindings and website edges change independently without creating inherited
   authority.
4. Multiple authenticated reviewers and sticky revocations protect the reference
   policy from one compromised contributor or reviewer.
5. Candidate-site observations are reproducible, bounded, rights-declared evidence
   and material drift reaches reviewers promptly.
6. Runtime generations remain compact while complete audit evidence stays verifiable.
7. Current real source formats import and resume within documented modest-hardware
   bounds, with no silent oversized-record loss.
8. Every new source has verified commercial-reuse rights, exact consumed fields,
   lineage, attribution, and adversarial tests.
9. Regional resolution uses deterministic locale/country precedence and abstains on
   unresolved ambiguity.
10. A separately distributed signed reference dataset produces useful reviewed
    results, including `facebook -> Facebook -> facebook.com` and multiple regional
    properties, from a clean consumer machine.
11. Full releases, deltas, and revocations authenticate independently of mirrors and
    refuse rollback.
12. Published evaluation and coverage reports state what is actually reviewed,
    current, relevant, and resolvable without conflating implementation with data.
13. Formatter, compiler checks, strict lints, documentation, unit/integration tests,
    adversarial tests, source-release determinism, provider-scale canaries, and a
    clean final diff all pass for every release boundary.

## First implementation slice after approval

Begin only with Phase 0 and Phase 1. Do not combine the initial migration with the
crawler, new sources, public data acquisition, or dataset publication.

The first concrete change set should contain:

1. Frozen v0.3 compatibility fixtures.
2. Approved active-coverage and export ADRs.
3. Typed source coverage with conservative legacy behavior.
4. Active-only export plus explicit audit export.
5. Trusted review acceptance time and sticky revocation semantics.
6. Schema/rules migration, documentation, and complete offline acceptance evidence.

Review that diff and release it before beginning granular name/edge votes or crawler
observations. This keeps the highest-risk semantic corrections small enough to audit
and makes every later phase build on an unambiguous active registry.