argand-site-registry/README.md

# Argand Site Registry

Argand Site Registry is a Rust toolkit for building a trustworthy map from names
to entities to official websites. It is useful for navigational search, regional
site selection, link directories, and any application that needs to explain why
it trusts a destination.

```text
facebook -> Facebook (Wikidata Q355) -> https://www.facebook.com/
                                          hostname: www.facebook.com
                                          registrable domain: facebook.com
                                          public suffix: com
```

The repository contains the library, CLI, schemas, migrations, synthetic
fixtures, and independently authenticated public trust roots. It does not place a
mutable production database in Git. Publishers import source evidence, collect
signed reviews, and distribute immutable signed registry generations. The current
public catalog generation is available as a release asset; see
[Public catalog](docs/PUBLIC_CATALOG.md).

## The basic idea

The registry keeps four things separate:

1. **Assertions:** Wikidata, Curlie, and other reviewed sources say that a name or
   website is associated with an entity.
2. **Observations:** a bounded observer records redirects, canonical links,
   hreflang, JSON-LD `sameAs`, sitemaps, DNS, TLS, and failures. These are evidence,
   not proof of ownership.
3. **Votes:** authenticated reviewers approve or revoke one exact name, website
   edge, or entity equivalence under a versioned policy.
4. **Releases:** a separate publisher signs an immutable generation. Consumers
   verify it before resolving names.

`lookup` shows source evidence and conflicts. `resolve` returns a destination only
when both the name binding and website edge satisfy the configured review policy.
Otherwise it returns a typed abstention such as `no_active_name_review`,
`no_active_review`, or `ambiguous_destination`.

One entity can have several independently reviewed regional properties:

```text
Example Store
  primary             -> https://example.com/
  country=GB          -> https://example.co.uk/
  country=DE          -> https://example.de/
```

The relationship must come from evidence and review. Similar names, ccTLDs, and
hostnames never merge entities or create regional relationships by themselves.

## Install and try it

Linux is the validated platform. You need Rust 1.97 or newer, a C toolchain,
CMake, Perl, Python 3.11 or newer, and OpenSSH `ssh-keygen`.

```bash
git clone https://git.argand.org/nicweyand/argand-site-registry.git
cd argand-site-registry
cargo fetch --locked
cargo build --release --locked --offline -p argand-site-registry
./target/release/argand-site-registry --help
```

Run the complete source-shaped fixture without downloading provider data:

```bash
ARGAND_REGISTRY_E2E_OUTPUT=/tmp/site-registry-example \
  cargo test -p argand-site-registry --test cli --locked --offline -- --nocapture
```

Run the strict 0.4 trust and observer acceptance flow:

```bash
cargo test -p argand-site-registry --test v04 --locked --offline -- --nocapture
```

The first fixture imports synthetic Wikidata, ROR, Majestic Million, CrUX, Curlie,
and Public Suffix List inputs. The second proves two-reviewer name and edge votes,
policy epochs, stale evidence, observation replay, drift, sticky revocation,
publisher separation, signed activation, and resistance to signature tampering.
All keys and approvals created by tests are disposable.

## Build and use a registry

Production data and keys belong outside the checkout. A typical lifecycle is:

```text
download/manifest -> import -> build candidate -> inspect/observe
 -> prepare and sign votes -> rebuild -> evaluate -> sign -> activate
```

The strict default policy requires two independent reviewer identities and groups
for names, website edges, and entity equivalences. A build binds both the policy
and exact reviewer trust file into its receipt.

```bash
argand-site-registry build \
  --database /data/site-registry/import.sqlite \
  --output /data/site-registry/candidate \
  --reviewer-trust /secure/site-registry/reviewer-allowed-signers

PIN=$(sha256sum /data/site-registry/candidate/COMPLETE.json | cut -d' ' -f1)

argand-site-registry lookup \
  --generation /data/site-registry/candidate --pin "$PIN" --query facebook

argand-site-registry review-queue \
  --generation /data/site-registry/candidate --pin "$PIN" \
  --at 2026-09-13T00:00:00Z
```

For a queued name or edge, prepare canonical vote JSON and sign its exact bytes:

```bash
argand-site-registry prepare-vote \
  --generation /data/site-registry/candidate --pin "$PIN" \
  --subject-kind edge --fingerprint "$EDGE_FINGERPRINT" \
  --decision approve --reviewer reviewer-one \
  --reason "Verified exact entity, URL, and global role" \
  --reviewed-at 2026-09-13T00:00:00Z \
  --expires-at 2026-10-13T00:00:00Z --role primary \
  --output /secure/site-registry/edge-vote.json

ssh-keygen -Y sign -n argand-site-registry-vote \
  -f /secure/site-registry/reviewer-one /secure/site-registry/edge-vote.json

argand-site-registry vote \
  --database /data/site-registry/import.sqlite \
  --generation /data/site-registry/candidate --pin "$PIN" \
  --decision /secure/site-registry/edge-vote.json \
  --signature /secure/site-registry/edge-vote.json.sig \
  --allowed-reviewers /secure/site-registry/reviewer-allowed-signers \
  --identity reviewer-one
```

Repeat independently for the second reviewer and for the selected name binding,
then rebuild. `prepare-equivalence-vote`, `verify-equivalence-vote`, and
`equivalence-vote` provide the same flow for explicit cross-source entity links.
The [operator guide](crates/argand-site-registry/README.md) documents source
downloads, typed full/partition/delta coverage, observation commands, regional
roles, release signing, activation, and recovery.

Consumers should use `resolve`, not the first result from `lookup`:

```bash
argand-site-registry resolve \
  --generation /data/site-registry/reviewed --pin "$REVIEWED_PIN" \
  --query facebook
```

The [Rust example](crates/argand-site-registry/examples/lookup.rs) reuses a
verified reader. The [Python example](examples/lookup.py) invokes the same native
CLI and preserves its JSON contract.

## Sources and licenses

| Source | Consumed evidence | Data license |
| --- | --- | --- |
| Wikidata | IDs, labels, aliases, P856 statements, selected locale/country metadata | CC0 1.0 Universal |
| Research Organization Registry (ROR) | organization IDs, names, aliases, websites, domains, status, type, locations and external IDs | CC0 1.0 Universal; GeoNames location lineage is attributed separately |
| Majestic Million | source-specific domain rank and supplied metrics | CC BY 3.0 Unported |
| Chrome UX Report | origin popularity bucket, month, optional audience country | CC BY 4.0 International |
| Curlie | site titles, categories, descriptions retained for audit | CC BY 3.0 Unported |
| Public Suffix List | ICANN and PRIVATE suffix rules | MPL 2.0 |
| Common Crawl Web Graph | domain harmonic-centrality/PageRank and member-host count | Common Crawl Terms of Use (`LicenseRef-Common-Crawl-Terms-of-Use`) |

Popularity never proves identity or ownership. Curlie attribution applies to
names and categories as well as descriptions; copied descriptions are redacted
from compact display surfaces unless the caller explicitly exports them and
satisfies the display obligations. The scheduled updater can bind the Web Graph
to the exact public-identity domain frontier; it never approves or activates routes.
Read [LICENSE_SOURCES.md](LICENSE_SOURCES.md) before distributing provider data.

Cloudflare Radar, default Tranco, Cisco Umbrella, arbitrary mirrors, and sources
without verified commercial reuse rights are unsupported. Adding a provider
requires an explicit adapter, current format inspection, provenance, rights
review, attribution rules, and tests.

## Safety and reproducibility

- Raw sources remain logically separate and content-addressed outside Git.
- Every imported or derived fact keeps source, source identifier, license,
  retrieval time, selector, and confidence.
- Typed coverage declares full snapshots, partitions, deltas, bases, and exact
  supersession; conflicts fail closed.
- Active export contains selected, nonrejected facts. Audit export also retains
  superseded and rejected evidence.
- Compact generations keep query projections local and bind complete historical
  records/facts through verified, content-addressed external audit bundles.
- Votes bind assertion, evidence-bundle, policy, reviewer, scope, and expiry.
- Revocations remain sticky until every member of a fresh quorum explicitly
  supersedes them.
- Small publisher-signed cumulative revocation feeds can block a compatible
  pinned offline registry before its replacement generation arrives. They expire
  after seven days and cannot restore a route across generations.
- The observer follows at most five redirects, uses public DNS pinning, blocks
  private and link-local networks, rejects HTTPS downgrade and nondefault ports,
  and caps headers, bodies, time, and extracted links.
- Automated updates may download, import, observe, and build candidates. They do
  not approve, renew, sign, activate, or silently choose a destination.

The code is **AGPL-3.0-or-later**. Provider data retains its own license. See the
[documentation index](docs/INDEX.md), [architecture](docs/ARCHITECTURE.md),
[trust model](docs/TRUST.md), [publishing runbook](docs/PUBLISHING.md), and
[consumer contract](docs/CONSUMERS.md).

## Development

```bash
cargo fetch --locked
bash scripts/check.sh
```

The gate covers formatting, all-target compilation, strict Clippy, Rust tests,
API documentation, Python release checks, deterministic packaging, and native
Rust/Python consumer parity. Security validation is recorded separately in
[docs/VALIDATION.md](docs/VALIDATION.md).