argand-site-registry/README.md

# Argand Site Registry

Argand Site Registry connects an organization, product or other named entity to
its official websites. It is built for navigational search and for choosing the
right regional site without hiding where each claim came from.

Given `facebook`, a registry can return:

```text
facebook  ->  Facebook (Wikidata Q355)  ->  https://www.facebook.com/
                                                registrable domain: facebook.com
```

An entity may have several legitimate properties. For example, one Amazon entity
can carry separately evidenced properties for `amazon.com`, `amazon.co.uk` and
`amazon.de`. The registry never joins entities merely because their names or
hostnames look alike.

This repository provides the Rust library, command-line tools and synthetic test
fixtures. It does **not** publish a ready-to-use approved-link dataset. A registry
publisher imports source data, reviews destinations and distributes a signed
generation; a consumer verifies that generation before using it.

## What it answers

The two main query operations have different purposes:

| Operation | Answer | Trust behavior |
| --- | --- | --- |
| `lookup` | What entities and websites do the sources associate with this name? | Returns evidence for inspection, including conflicts and unreviewed claims. |
| `resolve` | Which destination is approved for this name, locale and country? | Returns only an unambiguous, signed-review-backed, unexpired destination; otherwise it abstains. |

The CLI can also inspect an entity by stable ID, reverse-lookup a URL or domain,
show source-separated popularity signals, inspect redacted Curlie categories,
compare generations and evaluate a pinned generation against JSONL judgments.

Source confidence is evidence metadata, not a malware-safety score. An imported
official-site claim cannot become a `resolve` destination until an authorized
reviewer approves that exact claim. Applications should still apply their own
security and content policy.

## How a registry is built

```text
source downloads -> provenance-preserving imports -> immutable candidate
       -> signed human reviews -> rebuilt generation -> signed activation
       -> lookup / regional resolve
```

| Stage | Main commands | What changes |
| --- | --- | --- |
| Acquire | `download`, `crux-download`, `manifest` | Stores source bytes and a receipt describing their origin. |
| Import | `import` | Streams normalized facts into the local SQLite writer database. |
| Build | `build` | Produces a new immutable, content-pinned generation. |
| Review | `review`, `equivalence` | Records signed destination or entity-equivalence decisions against exact evidence fingerprints. |
| Publish | `sign`, `activate` | Verifies reviewer trust, signs a generation and atomically selects it. |
| Consume | `lookup`, `resolve`, `lookup-web`, `entity` | Reads a verified generation without changing it. |

Automated updates stop after building a candidate. They cannot approve a link,
sign a release or activate it.

## Try it locally

Linux is the currently validated platform. You need Rust 1.97 or newer, a C
compiler, CMake, Perl and OpenSSH (`ssh-keygen`). Python 3.11 or newer is needed
for release checks and the Python example. SQLite is compiled into the binary.

```bash
git clone https://git.argand.org/nicweyand/argand-site-registry.git
cd argand-site-registry
cargo fetch --locked
cargo build --release --locked --offline -p argand-site-registry
./target/release/argand-site-registry --help
```

To see the complete lifecycle without downloading provider data, run the native
fixture in a new directory outside the checkout:

```bash
ARGAND_REGISTRY_E2E_OUTPUT=/tmp/site-registry-example \
  cargo test -p argand-site-registry --test cli --locked --offline -- --nocapture
```

The fixture imports source-shaped Wikidata, Majestic Million, CrUX, Curlie and
Public Suffix List records. It proves repeatable imports, alias lookup, explicit
entity equivalence, destination review, signing, activation, revocation and
rollback protection. Its keys and approvals are disposable test material and
must never be used for a real registry.

Production inputs belong in a configurable data/cache directory outside the
source checkout. Installation and tests do not download datasets or start
scheduled jobs.

## Use a generation from Rust or Python

A consumer needs the generation directory and a trusted pin for its
`COMPLETE.json` receipt. Obtain that pin through a trusted publisher channel, or
verify the publisher signature with an independently configured key. A hash found
beside an untrusted download does not authenticate the download.

The [Rust example](crates/argand-site-registry/examples/lookup.rs) reuses the
native reader. The [Python example](examples/lookup.py) calls the native CLI so it
keeps the same validation and response contract:

```bash
cargo run --locked --offline -p argand-site-registry --example lookup -- \
  --generation /data/registry/generation \
  --pin "$REGISTRY_TRUSTED_PIN" \
  --query facebook

python3 examples/lookup.py \
  --binary ./target/release/argand-site-registry \
  --generation /data/registry/generation \
  --pin "$REGISTRY_TRUSTED_PIN" \
  --query facebook
```

Do not turn `lookup.candidates[0]` into an automatic redirect. Use `resolve` and
preserve `destination: null` as a deliberate abstention. The
[consumer contract](docs/CONSUMERS.md) documents the Rust API, CLI JSON, signed
generation files, compatibility rules and Argand's pinned integration.

## Operate or contribute

The [operator guide](crates/argand-site-registry/README.md) contains the complete
commands for acquiring each source, importing large files, reviewing regional
properties, signing releases, scheduling candidate updates and recovering state.
CrUX acquisition always requires explicit credentials and a billing cap.

The registry currently supports these source adapters:

| Source | Purpose | Data license |
| --- | --- | --- |
| Wikidata | Entity names, aliases, official websites and locale/country evidence | CC0 1.0 Universal |
| Majestic Million | Domain popularity rank | CC BY 3.0 Unported |
| Chrome UX Report (CrUX) | Popular-origin rank bucket | CC BY 4.0 International |
| Curlie | Human-curated names and categories | CC BY 3.0 Unported |
| Public Suffix List | Public-suffix and registrable-domain parsing | Mozilla Public License 2.0 |

Read [LICENSE_SOURCES.md](LICENSE_SOURCES.md) before redistributing data. Curlie
attribution applies to names and categories as well as descriptions. The registry
keeps each source logically separate and retains provenance, licenses, retrieval
timestamps, source identifiers and conflicting evidence.

For the security model and release process, see:

- [Trust and evidence policy](docs/TRUST.md)
- [Publishing runbook](docs/PUBLISHING.md)
- [Security policy](SECURITY.md)
- [Governance](GOVERNANCE.md)
- [Full documentation index](docs/INDEX.md)

Pull requests may add evidence or code, but cannot directly approve a production
destination.

## Development

```bash
cargo fetch --locked
bash scripts/check.sh
```

The check covers formatting, all-target compilation, strict Clippy, Rust tests,
API documentation, Python release tests and native Rust/Python consumer parity.
[RELEASING.md](docs/RELEASING.md) explains deterministic source archives and
independent signature verification; [VALIDATION.md](docs/VALIDATION.md) records
the current independent build and acceptance evidence.

The code is licensed under **AGPL-3.0-or-later**; see [LICENSE](LICENSE). Provider
data keeps its own license. [UPSTREAM.json](UPSTREAM.json) records the signed
Argand extraction revision and original file hashes. Argand pins the signed
`v0.3.0` release by full Git revision and no longer carries an embedded source
copy.