argand-site-registry/docs/superpowers/plans/2026-09-13-v0.4-and-beyond.md

Argand Site Registry v0.4 and Beyond Plan

Status: Version 0.4 phases 0 through 2 implemented and security-reviewed. Phases 3 and later remain the sequenced roadmap.

Baseline: Clean main at 2861337; runtime behavior is the tagged v0.3.0 release at ac82820. The complete v0.3 acceptance suite passes.

Execution constraint: Work in this standalone repository only. Do not use subagents, edit Argand's main checkout, share its build cache, acquire paid data, publish a dataset, sign with production keys, or deploy without the authority already established for that specific action.

Goal

Turn the v0.3 evidence and release substrate into a production-grade, reusable entity-to-website authority system that can publish a useful signed reference registry while preserving conflict, provenance, commercial-reuse terms, human review, abstention, revocation, and deterministic regional resolution.

The implementation must remain useful in two modes:

1. A local publisher builds and reviews its own registry from allowed sources.
2. A consumer verifies and queries a separately distributed signed reference generation without trusting an unauthenticated download location.

The work is divided into independently releasable stages. Correct active-state semantics and review controls come before broader ingestion. Provider-scale proof comes before a public reference dataset. A hosted service remains optional.

Product thesis

Popularity lists answer which domains receive attention. They do not reliably answer which entity controls a domain, whether a destination is current, or which regional property is appropriate. Argand Site Registry should compile independent source assertions, observed site relationships, and authenticated reviewer votes into a signed generation that resolves only when policy is satisfied.

The principal product outcome is a signed, explainable answer:

query + locale/country
  -> reviewed name-to-entity binding
  -> reviewed entity-to-property edge
  -> active regional-selection policy
  -> URL or typed abstention

A larger assertion database is not the goal by itself. Release quality is measured by resolved-route correctness, useful coverage, freshness, and revocation speed.

Non-negotiable invariants

• Keep raw inputs, normalized facts, crawler observations, reviewer decisions, policy decisions, popularity, and release authority logically separate.
• Never infer entity equivalence or website ownership from similar names, domains, redirects, TLS, DNS, sameAs, popularity, or shared upstream data alone.
• Preserve conflicting and rejected evidence with its source-native identity.
• Every imported or derived fact retains source, source identifier, license, license evidence URL, retrieval timestamp, confidence, and derivation version.
• Every source adapter must be rights-reviewed, format-pinned, streaming, bounded, reproducible, resumable where feasible, and idempotent.
• Automated updates may download, import, evaluate, and build candidates. They may not approve, renew, sign, activate, or publish them.
• Resolution remains fail closed. Ambiguity, missing policy, expired evidence, missing votes, unsupported locale semantics, or conflicting equal candidates produces a typed abstention.
• Popularity remains source-separated evidence and never becomes an ownership vote.
• Existing v0.3 data remains auditable. Migrations must not reinterpret an old approval, source scope, or timestamp as if it had been created under a new rule.
• Curlie descriptions remain redacted unless the exact distribution surface meets its attribution obligations.
• Public submissions are untrusted proposals. They never mutate an active release.
• Cloudflare Radar, default Tranco, Cisco Umbrella, and other unverified sources remain excluded.

Target architecture

flowchart LR
    A[Rights-reviewed source adapters] --> B[Immutable source manifests and raw cache]
    B --> C[Writer store: records, facts, conflicts, tombstones]
    C --> D[Normalized assertion graph]
    E[Bounded candidate-site observer] --> F[Immutable observation bundles]
    F --> D
    D --> G[Review queue and evidence diff]
    G --> H[Signed reviewer votes]
    H --> I[Versioned publisher policy compiler]
    I --> J[Immutable active projection]
    J --> K[Signed full generation]
    J --> L[Signed delta and revocation feed]
    K --> M[Verified local library and CLI]
    L --> M
    M --> N[Exact lookup and regional resolve]

The writer store remains append-oriented and audit-capable. Runtime generations contain the selected facts, active decisions, required proof material, and signed references to cold audit bundles. They do not duplicate every historical raw row.

Release sequence

Release	Purpose	Exit condition
v0.3.x	Correct active-state and review-time semantics	Historical export is explicit, scope overlap fails closed, review acceptance time is trusted, and legacy behavior is regression-tested.
v0.4	Scalable trust and observation pipeline	Quorum policy, granular name/edge decisions, sticky revocations, structured evidence bundles, and review queues work end to end.
v0.5	Provider-scale and source expansion	Real-format scale canaries pass; incremental Wikidata and the first new rights-approved adapters are reproducible and bounded.
v0.6	Signed public reference registry	A reviewed dataset, full/delta releases, revocation delivery, coverage report, and consumer verification are published independently of source releases.
Later	Wider resolver/product surface	Locale fallback, typed destination roles, candidate discovery, platform portability, and an optional read-only service are justified by real use.

Phase 0: Freeze contracts and record design decisions

Task 0.1: Capture the v0.3 compatibility baseline

• Record the exact v0.3 CLI output schemas, generation schema, source-manifest schema, rules version, review JSON, release receipts, and Python/Rust examples.
• Add golden fixtures for a complete v0.3 generation and a mutable schema-v3 writer database without committing provider data or private keys.
• Verify that current lookup, resolve, diff, export, stats, signature verification, rollback refusal, and migrations behave exactly as documented.
• Preserve the current five-source all-synthetic fixture as a compatibility gate.

Likely files:

• crates/argand-site-registry/tests/common/
• crates/argand-site-registry/tests/registry.rs
• crates/argand-site-registry/tests/failures.rs
• crates/argand-site-registry/tests/cli.rs
• docs/VALIDATION.md

Acceptance:

• A v0.3 reader fixture remains readable or fails with a precise documented version error after each later schema change.
• No later test can silently regenerate the baseline fixture from new behavior.

Task 0.2: Write architecture decisions before migrations

• Add an ADR for active source coverage and supersession.
• Add an ADR for separating name bindings, website edges, and observations.
• Add an ADR for reviewer votes, revocation precedence, and publisher policy.
• Add an ADR for runtime projection versus cold audit retention.
• Add an ADR for full and delta dataset release identities.
• Add an ADR for source lineage so copied upstream evidence does not count twice.
• Document whether broad proprietary embedding is a goal. If so, evaluate a small permissively licensed format/verifier crate without changing the AGPL publisher engine or source-driven dataset licenses automatically.

Acceptance:

• Every later migration and public contract links to an approved decision.
• The ADRs explain rejected alternatives and compatibility consequences.

Phase 1: v0.3.x active-state and trust hardening

Task 1.1: Replace free-form snapshot replacement semantics with typed coverage

Problem: v0.3 selects the latest completed snapshot per exact (source, scope) string. Different strings are treated as additive even when one is a full snapshot or overlaps another partition.

• Introduce a source-manifest version with explicit coverage semantics: collection identity, coverage kind, stable partition identity, base snapshot where applicable, and explicit supersession references.
• Preserve v1 manifest parsing. Map every legacy (source, scope) to an isolated legacy partition so migration does not silently change its active facts.
• Require operators to provide an explicit mapping before a legacy partial snapshot can join or be replaced by a typed full collection.
• Select one coherent coverage set per source collection:
  • a complete full snapshot supersedes older declared partitions;
  • disjoint partitions compose only when their identities are explicit;
  • deltas require an authenticated base and ordered continuity;
  • overlapping or missing coverage relationships abort the build;
  • deletions use explicit tombstones and remain auditable.
• Include the complete selected-coverage graph in the build receipt.
• Make stats and diff report selected, superseded, incomplete, and conflicting source snapshots separately.

Likely files:

• crates/argand-site-registry/src/model.rs
• crates/argand-site-registry/src/store.rs
• crates/argand-site-registry/src/build.rs
• crates/argand-site-registry/src/diff.rs
• crates/argand-site-registry/src/query.rs
• crates/argand-site-registry/src/cli.rs
• crates/argand-site-registry/migrations/004.sql

Tests:

• Latest snapshot replaces an older snapshot in the same partition.
• A declared full snapshot supersedes earlier partitions.
• Two explicit disjoint partitions compose.
• An undeclared full-plus-partial combination fails.
• Overlapping partitions fail.
• Missing delta bases, skipped deltas, and forked delta histories fail.
• Tombstoned facts disappear from the active projection but remain in audit history.
• Failed or partial imports never replace selected complete coverage.
• Reimporting identical manifests and deltas is byte-for-byte idempotent.

Task 1.2: Split active export from audit-history export

Problem: v0.3 JSONL export emits facts from every complete retained source snapshot, including superseded snapshots, without a per-row active-state marker.

• Change the normal export contract to emit selected active facts only.
• Version the export envelope and include generation identity, selected source IDs, coverage-policy version, derivation version, and attribution identity.
• Add explicit selection state to every assertion.
• Add a separate audit export that includes active, superseded, rejected, and tombstoned facts with replacement links.
• Keep Curlie descriptions redacted by default in both modes.
• Make consumers reject unknown export schema versions.
• Document that neither export bypasses resolve admission policy.

Likely files:

• crates/argand-site-registry/src/release.rs
• crates/argand-site-registry/src/cli.rs
• crates/argand-site-registry/src/diff.rs
• README.md
• docs/CONSUMERS.md
• docs/TRUST.md

Tests:

• Superseded facts are absent from active export.
• Audit export retains and labels the same facts.
• Active export and native generation agree exactly on selected source IDs.
• Description redaction and attribution survive both modes.
• Python and Rust consumers refuse unknown versions and preserve null abstention.

Task 1.3: Bind decisions to trusted acceptance time

Problem: the reviewer signs reviewed_at, but the writer does not persist a trusted append time. That self-declared time influences approval validity and reviewer-key validity-epoch checks.

• Record accepted_at from the writer when exact signature verification and candidate validation succeed.
• Make approval validity begin no earlier than trusted acceptance.
• Bound effective expiry by both the signed review duration and the enforced maximum measured from acceptance, preventing backdating or future dating from extending authority.
• Evaluate reviewer eligibility at acceptance for new decisions. Preserve the signed claimed decision time for audit, without treating it as trusted time.
• Include acceptance time and the time-policy version in generation receipts.
• Retain legacy approvals for audit but require an explicit migration policy or fresh decision before they are active under the new rules. Never synthesize a historical acceptance time. Preserve legacy revocations regardless.
• Test clock skew, future dates, backdates, expired keys, removed keys, offline signing followed by later acceptance, and reproducible builds at fixed clocks.

Likely files:

• crates/argand-site-registry/src/review.rs
• crates/argand-site-registry/src/query.rs
• crates/argand-site-registry/src/resolution.rs
• crates/argand-site-registry/src/release.rs
• crates/argand-site-registry/migrations/004.sql
• docs/TRUST.md
• docs/PUBLISHING.md

Task 1.4: Make revocation precedence explicit before quorum work

• Treat a valid revocation as sticky for its exact subject.
• Require an explicit signed supersession that references the revocation before the same destination can become active again.
• Prevent a later ordinary approval from overriding a revocation by sequence order alone.
• Preserve existing activation rollback checks and strengthen them to compare revocation identities and supersession relationships.
• Add a machine-readable emergency revocation export suitable for cached consumers.

Acceptance:

• Reordering, forking, or appending an ordinary approval cannot erase a revocation.
• A deliberately superseded revocation remains visible in lookup, diff, and audit.
• An offline consumer can apply the revocation export before receiving a full build.

Phase 1 release gate

• Run cargo fetch --locked once, then the complete offline scripts/check.sh.
• Run migrations from empty, schema v1, v2, and v3 databases.
• Build the same generation twice and compare every output byte.
• Verify source release determinism from a clean signed commit.
• Update changelog, trust docs, format contracts, and migration guidance.
• Run cargo audit --deny warnings and record it separately from the offline gate.
• Confirm no provider data, keys, approval logs, or private paths entered Git.

Phase 2: v0.4 scalable review and structured evidence

Task 2.1: Separate name-to-entity trust from entity-to-property trust

Problem: v0.3 binds each website approval to the entity's full name set. This stops a newly injected alias from inheriting a route, but benign alias or label changes invalidate every website edge. A global Wikidata revision also changes edge identity when the P856 statement itself did not materially change.

• Create independent material fingerprints for:
  • source name or alias assertion to stable entity;
  • explicit entity equivalence;
  • entity-to-normalized-web-property assertion;
  • normalized regional role and scope;
  • observation bundle;
  • reviewer vote and policy result.
• Bind website-edge identity only to material website evidence: exact source statement identity/value/rank/relevant qualifiers, normalized URL result, and derivation rule version.
• Retain source record revision and unrelated entity metadata in provenance and diffs without making them part of website-edge identity.
• Require resolve to satisfy both an admitted name binding and an admitted website edge. lookup continues to expose unreviewed names and edges.
• Ensure a newly imported alias cannot inherit an existing approved route.
• Ensure a benign accepted alias update does not invalidate an unchanged route.
• Give canonical labels, aliases, and source-specific names distinct evidence identities and policy treatment.

Tests:

• Adding an unreviewed alias never creates a resolvable query.
• Removing or changing an approved alias affects only that binding.
• An unrelated Wikidata lastrevid change preserves the route fingerprint.
• A P856 URL, rank, end qualifier, or relevant regional qualifier change invalidates the route approval.
• Confusable and bidi-control names remain rejected.
• Explicit entity equivalence never carries unstated name or route authority.

Task 2.2: Replace latest-decision-wins with signed votes and policy compilation

• Represent decisions as immutable signed votes by authenticated reviewer.
• Permit one current vote per signer, subject, decision type, and policy epoch; preserve superseded votes in the append-only log.
• Add a versioned publisher policy that defines:
  • required approval threshold by decision type and risk class;
  • revocation threshold and sticky-revocation behavior;
  • publisher/reviewer separation;
  • reviewer groups or independence constraints where configured;
  • maximum approval age and evidence-freshness requirements;
  • treatment of source conflicts and unresolved observations.
• Hash the exact policy and selected reviewer trust roots into the release receipt.
• Compile votes deterministically into approved, revoked, expired, disputed, probationary, or insufficient-review state.
• Keep policy configurable for independent publishers while shipping a strict, documented reference policy for Argand's own releases.
• Preserve the ability to operate locally without accounts or a network service.

Tests:

• One signer cannot satisfy a two-independent-reviewer policy.
• Duplicate keys or aliases for the same reviewer do not create extra votes.
• Publisher/reviewer separation is enforced when enabled.
• One authorized revocation blocks resolution under the reference policy.
• Conflicting votes produce a disputed state and abstention.
• Changing policy invalidates the old compiled result without changing source facts.
• Release verification fails on policy, signer, vote, or receipt tampering.

Task 2.3: Turn observation types into a stored evidence pipeline

• Extend the writer schema with immutable observation batches and observations.
• Preserve the existing distinction between observations and ownership claims.
• Add an adapter contract for externally captured observations, including source, rights declaration, capture ID, retrieval time, content hash, exact selector, from/to URLs, relation, and confidence.
• Project redirect, canonical, hreflang, JSON-LD sameAs, sitemap, and country selector evidence without automatically creating an entity edge.
• Add observation lookup, reverse lookup, generation diff, and evidence-bundle output for review.
• Record negative or failed observations with bounded error classes so absence is not confused with a fetch that never succeeded.
• Define retention that stores hashes and necessary bounded extracts rather than copied page bodies unless rights and need are established.

Likely files:

• crates/argand-site-registry/src/observation.rs
• crates/argand-site-registry/src/store.rs
• crates/argand-site-registry/src/build.rs
• crates/argand-site-registry/src/query.rs
• crates/argand-site-registry/src/diff.rs
• crates/argand-site-registry/migrations/005.sql

Task 2.4: Add a bounded candidate-site observer

• Observe only imported/reviewed candidate URLs. Do not start an open-web crawler.
• Enforce scheme, redirect-count, response-size, header-size, decompression, timeout, DNS-result, address-range, and per-host request bounds.
• Block credentials, local/private/link-local targets, unsafe redirect transitions, non-HTTP protocols, and host confusion.
• Capture redirect chains, final URL, status, canonical, hreflang, sameAs, sitemap references, country selectors, DNS answers, and TLS certificate fingerprints as separate evidence classes.
• Do not treat TLS, DNS, redirects, or site self-assertions as ownership proof.
• Emit immutable observation manifests compatible with Task 2.3.
• Support cache-only replay so parser and policy tests never require the network.
• Keep acquisition cadence, concurrency, bandwidth, and data path configurable.

Security tests:

• SSRF attempts, redirect loops, compression bombs, oversized markup, malformed HTML, DNS rebinding, mixed encodings, invalid certificates, and cross-scheme redirects.
• Parser fuzz/property tests for headers, hreflang, canonical, JSON-LD, and sitemaps.
• Deterministic replay from captured fixtures with no network access.

Task 2.5: Build review queues and evidence bundles before a graphical UI

• Add deterministic queue output ordered by risk and material change, not source popularity alone.
• Queue new routes, new aliases, source conflicts, changed website statements, redirects across registrable domains, observation drift, expiring approvals, unresolved regional scopes, and revoked destinations proposed for reinstatement.
• Produce a bounded review bundle containing exact claims, conflicts, observation diffs, source licenses, capture hashes, requested role, and candidate fingerprint.
• Add commands to prepare a vote, verify exact vote bytes, append it, and show the compiled policy result.
• Keep the queue read-only and deterministic. Never let viewing evidence mutate approval state.
• Defer a browser workbench until CLI bundles have proven the workflow.

Task 2.6: Monitor approved routes and classify drift

• Schedule observation refresh independently from source import cadence.
• Classify material changes: unreachable, cross-domain redirect, DNS/TLS change, content/canonical shift, domain expiry indicators, malware-policy result, or no material change.
• Use risk policy to shorten review intervals. Never auto-extend approvals.
• Put materially changed routes into probation or revoke them according to signed publisher policy; default to abstention where evidence is insufficient.
• Record every transition and make revocation candidates immediately exportable.
• Design malware-feed adapters only after exact commercial-reuse and redistribution terms are verified. Do not hard-code an unreviewed vendor.

Phase 2 release gate

• Complete a signed two-reviewer fixture from source assertion through name vote, edge vote, observation bundle, policy compilation, release, resolution, drift, revocation, signed delta, and consumer application.
• Prove a malicious contributor cannot submit directly into active state.
• Prove one compromised reviewer cannot approve under the reference policy.
• Prove an emergency revocation reaches a pinned offline consumer without a full source reimport.
• Preserve all v0.3 conflict, normalization, regional, and rollback tests.

Completion note: the version 0.4 implementation satisfies the Phase 0 through 2 acceptance boundary, including the complete two-reviewer fixture and security review. The immutable v0.3 contract is frozen by exact signed commit and golden schema/field coordinates rather than a committed SQLite generation, because this repository does not admit generated datasets or approval logs. General registry deltas remain explicitly deferred by ADR 0005; version 0.4 supplies typed source deltas and a separately signed, cumulative emergency block feed.

Phase 3: v0.5 storage, scale, and current-source improvements

Task 3.1: Separate runtime projections from cold audit history

• Keep raw cache objects immutable and content-addressed outside Git.
• Package completed source imports into content-addressed audit bundles with manifest, record/fact indices, hashes, format version, and attribution.
• Make a runtime generation contain selected facts, normalized projections, active policy results, required votes/revocations, and signed bundle references.
• Do not copy all historical records and facts into every runtime generation.
• Add audit verification that streams referenced bundles and detects absence, truncation, substitution, or mismatched attribution.
• Add retention/checkpoint tooling that never deletes the only authenticated copy of evidence and produces a signed deletion/retention report.
• Measure query latency and generation size before and after the split.

Task 3.2: Add provider-scale benchmark and recovery tooling

• Define repeatable small, medium, and provider-representative import profiles.
• Record wall time, CPU time, peak RSS, compressed and expanded bytes, database growth, facts/second, checkpoint frequency, restart time, build size, and query latency.
• Interrupt imports at multiple checkpoints and prove idempotent resumption.
• Exercise disk-full, truncated input, cache corruption, duplicate records, and interrupted generation publication.
• Make benchmark reports name exact source snapshot hashes and hardware without committing source data.
• Treat synthetic performance as development evidence, not provider capacity.

Task 3.3: Harden full Wikidata ingestion and add incremental refresh

• Confirm current official full and incremental formats from Wikidata documentation and inspected fixtures before changing the adapter.
• Replace the assumption that every useful entity fits in one in-memory 16 MiB line with bounded disk-spooling or an explicitly receipted oversized-record path.
• Never silently skip an oversized entity that may contain a relevant fact.
• Add incremental add/change ingestion with authenticated base snapshot identity, ordered application, checkpoints, and reconciliation against later full dumps.
• Define how deletions and removed P856 statements become tombstones.
• Keep full raw assertion/qualifier/reference provenance for consumed fields.
• Make unrelated lastrevid changes visible in audit diffs without invalidating unchanged material edge fingerprints.
• Test real-format pathological entities and multistream compression boundaries.

Authoritative format reference:

• https://www.wikidata.org/wiki/Wikidata:Database_download

Task 3.4: Strengthen current-source acquisition verification

• Prefer provider-published checksums or signatures when officially available and bind verification method into the source manifest.
• Keep HTTPS allowlists, manual redirect validation, byte bounds, strong-validator resume rules, and immutable local cache behavior.
• Detect and report source format drift before partial import can replace a source.
• Add format-version canaries for Majestic, CrUX, Curlie, PSL, and Wikidata.
• Preserve CrUX billing as explicit opt-in configuration and record job identity, query, result period, and actual cost outside public fixtures.
• Continue frequent PSL refresh and include exact PSL hash in normalization proofs.
• Preserve Curlie attribution and description-redaction tests on every export path.

Task 3.5: Add source lineage and independence metadata

• Record direct provider, upstream/origin dataset, transformation, snapshot, and known dependency relationships for each fact source.
• Prevent policy from counting two assertions as independent corroboration when one republishes the other.
• Expose lineage in lookup, review bundles, export, diff, and evaluation.
• Keep unknown lineage explicit rather than assuming independence.

Phase 4: Rights-gated additional source adapters

Every source follows the same gate:

1. Verify authoritative download, schema, update cadence, license, attribution, redistribution, database-right, and commercial-use documentation.
2. Record the exact decision and URLs in LICENSE_SOURCES.md.
3. Inspect current official fixtures. Do not infer fields from third-party examples.
4. Add a source enum/format only with a streaming adapter and bounded failure tests.
5. Preserve native IDs, raw relevant records, selectors, lineage, and confidence.
6. Import into a separate logical source layer.
7. Demonstrate that the source cannot auto-create an approved route.
8. Run deterministic, idempotent, interrupted, malformed, and conflict fixtures.

Task 4.1: Add ROR first

Rationale: ROR is CC0 and directly supplies stable organization IDs, names, aliases, status, locations, links, and domains. It is compact and well aligned with the model.

• Verify the current ROR schema version and official release asset from the ROR data-dump documentation at implementation time.
• Consume only fields confirmed in that inspected schema.
• Map names and aliases without merging ROR entities into Wikidata entities unless an exact external identifier or reviewed equivalence supports the join.
• Preserve links and domains as ROR assertions, not approvals.
• Preserve status, type, country/location, external IDs, and upstream lineage.
• Test domain conflicts, former/inactive organizations, aliases, multiple links, missing fields, duplicate input, schema drift, and exact-ID equivalence.

References:

• https://ror.readme.io/docs/data-dump
• https://ror.readme.io/docs/ror-data-structure
• https://ror.readme.io/docs/fields

Task 4.2: Add MusicBrainz core snapshots second

• Use only the CC0 core database snapshot and its verified checksums/signatures.
• Do not use the CC BY-NC-SA live replication feed in the commercial-safe default pipeline.
• Consume exact URL entities and documented URL relationship types, including official-homepage and ended-state metadata.
• Keep artists, labels, places, and events separate by stable MusicBrainz ID.
• Preserve relationship begin/end dates and link types as material evidence.
• Treat community-curated URLs as assertions requiring normal Argand review.
• Test removed URLs, ended relationships, entity redirects/merges, duplicate URLs, malicious/taken-over sites, and snapshot replacement.

References:

• https://musicbrainz.org/doc/MusicBrainz_Database/Download
• https://musicbrainz.org/doc/Style/Relationships/URLs
• https://musicbrainz.org/doc/Live_Data_Feed

Task 4.3: Validate GND as the third adapter

• Verify the exact current CC0 declaration for the selected GND files.
• Inspect current JSON-LD or RDF schema and confirm homepage predicates before implementing an adapter.
• Preserve authority IDs, preferred/variant names, types, countries, external IDs, and exact homepage assertions where present.
• Keep its regional and language focus visible in provenance and evaluation.
• Stop after the rights/schema spike if homepage coverage does not justify the adapter cost.

Reference:

• https://data.dnb.de/opendata/

Task 4.4: Keep lower-priority candidates behind explicit holds

• ORCID: research a low-confidence individuals-only adapter. Its public file is CC0, but links are self-declared and require privacy, impersonation, and volatility policy. Never auto-approve. Reference: https://info.orcid.org/public-data-file-use-policy/.
• OpenAlex: use only for research-activity/popularity metadata if useful. Record ROR as upstream lineage and never count its institution website as independent corroboration. Reference: https://help.openalex.org/data/institutions/.
• OpenStreetMap: do not ingest until an ODbL-compatible distribution and attribution architecture is approved. Reference: https://osmfoundation.org/wiki/Licence_and_Legal_FAQ.
• Government/corporate registries: assess jurisdiction by jurisdiction. Prefer stable identity crosswalks; do not infer a website where no authoritative field exists.
• DNS, RDAP, certificate transparency, package registries, and web crawl data: evaluate as observation sources only after exact terms are verified.
• Open Library and other sources with unresolved underlying rights remain excluded.

Phase 4 acceptance

• Each admitted source has authoritative license evidence, current fixture evidence, exact fields consumed, attribution behavior, source lineage, and format-drift tests.
• Full source-specific provenance appears in lookup, review bundles, export, and diff.
• Removing any new adapter leaves existing source identities and release verification deterministic.
• No new source changes an existing edge's approval merely by corroborating it.

Phase 5: Resolver, evaluation, and consumer improvements

Task 5.1: Canonicalize locale and country semantics

• Add standards-based BCP 47 parsing/canonicalization after reviewing the chosen library and current specification behavior.
• Define deterministic precedence for exact locale/country, country-only, language-parent, and global-primary candidates.
• Abstain on equal candidates at the same specificity.
• Preserve the requested and normalized locale in the explanation envelope.
• Add tests for en-GB, en, script subtags, case, deprecated aliases, malformed tags, country-only properties, multi-country sites, and conflicting scopes.

Task 5.2: Extend property roles without weakening default navigation

• Define a versioned role vocabulary covering at least global primary, regional primary, product, support, developer, careers, login, status, and other reviewed roles justified by real cases.
• Keep default resolve restricted to the requested navigation role.
• Require role-specific evidence and votes; a support site cannot become primary because it shares a domain.
• Preserve unknown source roles as evidence without admitting them.

Task 5.3: Add candidate discovery separately from resolution

• Add prefix/fuzzy discovery only as an audit/candidate operation.
• Preserve exact normalized matching for final admission.
• Return candidate score components and ambiguity rather than hiding a rewrite.
• Never let similarity bypass reviewed name-to-entity bindings.
• Test homographs, typosquatting, short names, multilingual aliases, and popular entities with colliding names.

Task 5.4: Expand evaluation into a release gate

• Extend judgments to cover expected abstention reasons, selected name binding, route edge, regional precedence, and policy state.
• Report resolved-route errors separately from safe abstentions.
• Measure coverage, abstention taxonomy, active-evidence age, expiring approvals, conflict rate, reviewer agreement, review turnaround, observation drift, and revocation propagation.
• Add adversarial suites for source poisoning, alias injection, correlated sources, malicious redirects, domain takeover, compromised reviewer, compromised publisher, stale cache, rollback, and policy downgrade.
• Segment evaluation by source, entity type, language, country, popularity band, destination role, and evidence age without collapsing source signals.
• Establish release thresholds only after an audited baseline exists. Any known wrong resolved destination is a release blocker for the reference dataset.

Phase 6: v0.6 signed public reference dataset

Task 6.1: Define reference publisher governance

• Publish reviewer eligibility, independence, conflict-of-interest, evidence, expiry, appeals, correction, key rotation, incident, and emergency-revocation policies.
• Separate source maintainers, reviewers, and release publishers where practical.
• Publish the exact policy hash and reviewer trust roots with each release.
• Define a transparent proposal process in which contributors submit signed evidence bundles rather than direct active-dataset edits.
• Record disputed claims and abstain until policy is met.
• Publish change logs and correction history without exposing sensitive reviewer material unnecessarily.

Task 6.2: Build a deliberately bounded starter registry

• Select a high-value initial coverage set using source-separated popularity and declared entity classes only for prioritization.
• Publish the selection methodology and its biases.
• Review name bindings, entity equivalences, website edges, regional roles, and current observations under the reference policy.
• Do not claim comprehensive web, country, language, or entity-type coverage.
• Require every resolvable destination to have current votes, unexpired evidence, complete provenance, and an active monitoring schedule.
• Run the full evaluation, diff, license, source-lineage, and release gates.

Task 6.3: Publish full releases, deltas, and revocations

• Keep dataset releases separate from source-code releases and raw provider cache.
• Produce a deterministic full generation, receipt, attribution bundle, policy, reviewer trust roots, evaluation report, coverage report, and detached publisher signature.
• Produce ordered, signed deltas bound to exact base and target generation IDs.
• Publish a small signed revocation feed with monotonic continuity and rollback protection.
• Make full and delta application atomic and recoverable after interruption.
• Provide a documented mirror-independent verification procedure.
• Never make an unauthenticated latest URL the trust root. A convenience current pointer must itself be signed and rollback protected.

Task 6.4: Make immediate use simple

• Add a beginner workflow that downloads a release, verifies its publisher and policy, pins it, and resolves facebook locally.
• Show an entity with multiple reviewed regional properties and an abstention.
• Provide identical native CLI, Rust library, and Python subprocess examples.
• Add a machine-readable capability/version command.
• Explain code license separately from each dataset source license and attribution.
• Provide update and emergency-revocation examples for systemd and cron.

Reference release acceptance

• Every resolved route is reproducible from selected source facts, name/edge votes, policy, and observations contained in or authenticated by the release.
• Every conflict, rejection, abstention, and supersession remains inspectable.
• An untrusted mirror cannot substitute a generation, policy, reviewer set, delta, or revocation feed without verification failure.
• A clean consumer machine can verify and query the release without provider credentials, reviewer keys, publisher private keys, or the mutable writer database.
• The coverage and evaluation reports distinguish implemented capability from actual reviewed data coverage.

Phase 7: Optional service and portability

Task 7.1: Cross-platform immutable generation reads

• Implement equivalent no-follow, regular-file, bounded-copy, hash-before-open, and private-temporary-file behavior for macOS and Windows.
• Add platform CI only on isolated runners with no release authority.
• Preserve Linux behavior and reject platforms without a safe implementation.

Task 7.2: Read-only service only after the dataset proves useful

• Expose the existing verified query envelope through a minimal read-only local HTTP or Unix-socket service if consumer demand justifies it.
• Pin one generation per process/request context and swap only after full signature, policy, reviewer, and rollback verification.
• Bound query size, output size, concurrency, and time.
• Preserve typed abstention and explanation; do not add silent query rewriting.
• Keep publication, review, acquisition, and private keys out of the serving process.

Operational security and supply-chain work

• Add property tests or fuzz targets for URL/domain normalization, PSL behavior, JSON duplicate keys, each parser, coverage selection, observation parsing, signed-delta application, and locale handling.
• Add dependency vulnerability and license-policy checks to a network-enabled, isolated scheduled workflow. Preserve the offline acceptance gate separately.
• Pin and document CI images and external actions by immutable identity.
• Verify official provider checksums/signatures where available.
• Add threat-model cases for reviewer-key theft, publisher-key theft, source compromise, malicious proposal bundles, cache substitution, stale mirrors, rollback, denial of service, and domain takeover.
• Add key rotation and emergency response drills using disposable test keys.
• Keep signed receipts and validation artifacts outside the source tree unless they are deliberately public, non-sensitive release evidence.

Documentation deliverables

• Keep the README focused on the first successful verified lookup.
• Add an architecture document explaining assertion, observation, decision, policy, generation, delta, and consumer boundaries.
• Expand LICENSE_SOURCES.md for every admitted source with exact fields consumed, source URLs, licenses, attribution, redistribution, update cadence, and lineage.
• Document active versus audit export semantics.
• Document migrations and legacy decision handling.
• Publish the reference review and incident policies.
• Publish provider-scale measurements without implying serving or corpus coverage.
• Maintain a source-candidate table showing approved, research, held, and rejected sources with the reason for each state.

SWOT-driven checks

Preserve strengths

• Deterministic, source-separated facts and normalization.
• Fail-closed resolution and explicit abstention.
• Signed reviews/releases and rollback-safe revocations.
• Complete licenses, attribution, provenance, and conflict evidence.
• Streaming, bounded, resumable, idempotent imports.
• Meaningful adversarial and cross-consumer tests.

Correct weaknesses

• Publish an immediately usable dataset after trust and scale gates pass.
• Replace one-latest-decision semantics with policy-compiled votes.
• Reduce benign review churn without allowing alias inheritance.
• Separate active runtime state from unbounded audit history.
• Add provider-scale proof, monitoring, queues, and richer metrics.
• Improve regional semantics and platform support after the core release.

Capture opportunities

• Establish an open evidence-bundle and signed-generation interchange format.
• Let independent publishers share facts while choosing different trust policies.
• Supply explainable navigation authority to search engines, assistants, browsers, bookmarks, enterprise catalogs, and safety products.
• Publish an evaluation benchmark for entity-to-site resolution and abstention.
• Use ROR, MusicBrainz, and GND to expand high-quality vertical coverage.

Mitigate threats

• Compromised reviewers: quorum, independence, expiry, and sticky revocation.
• Compromised publishers: independent reviewer roots and consumer policy verification.
• Domain takeover: active observation, probation, and rapid revocation delivery.
• Source poisoning: no auto-approval, source lineage, evidence diffs, and review queues.
• Correlated evidence: upstream lineage and independence-aware policy.
• License drift: exact rights gates and fail-closed format/license changes.
• Cache/mirror staleness: signed continuity, expiry, and rollback protection.
• Governance capture: public policies, disputes, appeals, and transparent changes.

Explicit deferrals

Do not prioritize these before a reviewed reference dataset exists:

• A general hosted API or multi-tenant account system.
• ML or LLM approval of ownership or regional roles.
• Open-web crawling.
• Fuzzy matching in the final resolver.
• Additional popularity feeds merely to enlarge source count.
• Sources with unclear commercial reuse, database rights, or redistribution terms.
• ODbL data without an approved packaging architecture.
• A graphical review UI before deterministic CLI bundles and queues are proven.
• Automatic approval renewal based on unchanged popularity, TLS, DNS, or redirects.

Definition of complete

The roadmap is complete when:

1. Source coverage and exports cannot confuse historical facts with active facts.
2. Review authority is bound to trusted acceptance time and a signed policy.
3. Name bindings and website edges change independently without creating inherited authority.
4. Multiple authenticated reviewers and sticky revocations protect the reference policy from one compromised contributor or reviewer.
5. Candidate-site observations are reproducible, bounded, rights-declared evidence and material drift reaches reviewers promptly.
6. Runtime generations remain compact while complete audit evidence stays verifiable.
7. Current real source formats import and resume within documented modest-hardware bounds, with no silent oversized-record loss.
8. Every new source has verified commercial-reuse rights, exact consumed fields, lineage, attribution, and adversarial tests.
9. Regional resolution uses deterministic locale/country precedence and abstains on unresolved ambiguity.
10. A separately distributed signed reference dataset produces useful reviewed results, including facebook -> Facebook -> facebook.com and multiple regional properties, from a clean consumer machine.
11. Full releases, deltas, and revocations authenticate independently of mirrors and refuse rollback.
12. Published evaluation and coverage reports state what is actually reviewed, current, relevant, and resolvable without conflating implementation with data.
13. Formatter, compiler checks, strict lints, documentation, unit/integration tests, adversarial tests, source-release determinism, provider-scale canaries, and a clean final diff all pass for every release boundary.

First implementation slice after approval

Begin only with Phase 0 and Phase 1. Do not combine the initial migration with the crawler, new sources, public data acquisition, or dataset publication.

The first concrete change set should contain:

1. Frozen v0.3 compatibility fixtures.
2. Approved active-coverage and export ADRs.
3. Typed source coverage with conservative legacy behavior.
4. Active-only export plus explicit audit export.
5. Trusted review acceptance time and sticky revocation semantics.
6. Schema/rules migration, documentation, and complete offline acceptance evidence.

Review that diff and release it before beginning granular name/edge votes or crawler observations. This keeps the highest-risk semantic corrections small enough to audit and makes every later phase build on an unambiguous active registry.