nicweyand/argand-site-registry
1
0
Fork 0
Code Issues Pull requests Projects Releases 7 Packages Wiki Activity Actions

docs: make registry README easier to follow
All checks were successful
Standalone registry checks / check (push) Successful in 4m24s
Details

Browse source
This commit is contained in:
Nic Weyand 2026-09-13 09:14:10 -04:00
commit 2861337a45
Signed by: nicweyand
SSH key fingerprint: SHA256:2te+ycJIQON/Wo/dH6+ZkFSQ4HnHWpetV2azx9E65dQ
1 changed files with 132 additions and 87 deletions
Download patch file Download diff file Expand all files Collapse all files

217
README.md
View file

@ -1,128 +1,173 @@
# Argand Site Registry # Argand Site Registry
Evidence-backed website identity and regional resolution, as a Rust library and CLI. Argand Site Registry connects an organization, product or other named entity to
its official websites. It is built for navigational search and for choosing the
right regional site without hiding where each claim came from.
Look up an entity's names, aliases and websites; inspect each source assertion; Given `facebook`, a registry can return:
resolve an explicitly reviewed primary or regional destination. Sources, conflicting
claims, licenses and review history remain available for audit. Everything runs
locally with SQLite. No search engine, hosted account or GPU is required.
`facebook → Facebook → facebook.com` is a name-to-entity-to-registrable-domain ```text
lookup. The retained real Wikidata Facebook record has `www.facebook.com` and facebook -> Facebook (Wikidata Q355) -> https://www.facebook.com/
`m.facebook.com` as distinct properties. Its Amazon record includes `amazon.com`, registrable domain: facebook.com
`amazon.co.uk` and `amazon.de` on the same entity. Similar hostname spelling never ```
establishes common ownership. Imported links await review before `resolve` can
return a destination; source confidence is not a malware-safety guarantee.
## Build and try it An entity may have several legitimate properties. For example, one Amazon entity
can carry separately evidenced properties for `amazon.com`, `amazon.co.uk` and
`amazon.de`. The registry never joins entities merely because their names or
hostnames look alike.
Linux is the currently validated platform. Install Rust (tested with 1.98.1; the This repository provides the Rust library, command-line tools and synthetic test
inherited minimum is 1.97), a C compiler, CMake, Perl and OpenSSH (`ssh-keygen`). fixtures. It does **not** publish a ready-to-use approved-link dataset. A registry
Python 3.11+ is needed for release tooling and the Python example. SQLite is built publisher imports source data, reviews destinations and distributes a signed
with the binary. Get the public source, or use a verified source release: generation; a consumer verifies that generation before using it.
## What it answers
The two main query operations have different purposes:
| Operation | Answer | Trust behavior |
| --- | --- | --- |
| `lookup` | What entities and websites do the sources associate with this name? | Returns evidence for inspection, including conflicts and unreviewed claims. |
| `resolve` | Which destination is approved for this name, locale and country? | Returns only an unambiguous, signed-review-backed, unexpired destination; otherwise it abstains. |
The CLI can also inspect an entity by stable ID, reverse-lookup a URL or domain,
show source-separated popularity signals, inspect redacted Curlie categories,
compare generations and evaluate a pinned generation against JSONL judgments.
Source confidence is evidence metadata, not a malware-safety score. An imported
official-site claim cannot become a `resolve` destination until an authorized
reviewer approves that exact claim. Applications should still apply their own
security and content policy.
## How a registry is built
```text
source downloads -> provenance-preserving imports -> immutable candidate
-> signed human reviews -> rebuilt generation -> signed activation
-> lookup / regional resolve
```
| Stage | Main commands | What changes |
| --- | --- | --- |
| Acquire | `download`, `crux-download`, `manifest` | Stores source bytes and a receipt describing their origin. |
| Import | `import` | Streams normalized facts into the local SQLite writer database. |
| Build | `build` | Produces a new immutable, content-pinned generation. |
| Review | `review`, `equivalence` | Records signed destination or entity-equivalence decisions against exact evidence fingerprints. |
| Publish | `sign`, `activate` | Verifies reviewer trust, signs a generation and atomically selects it. |
| Consume | `lookup`, `resolve`, `lookup-web`, `entity` | Reads a verified generation without changing it. |
Automated updates stop after building a candidate. They cannot approve a link,
sign a release or activate it.
## Try it locally
Linux is the currently validated platform. You need Rust 1.97 or newer, a C
compiler, CMake, Perl and OpenSSH (`ssh-keygen`). Python 3.11 or newer is needed
for release checks and the Python example. SQLite is compiled into the binary.
```bash ```bash
git clone https://git.argand.org/nicweyand/argand-site-registry.git git clone https://git.argand.org/nicweyand/argand-site-registry.git
cd argand-site-registry cd argand-site-registry
```
From the root of the checkout or extracted release:
```bash
cargo fetch --locked cargo fetch --locked
cargo build --release --locked --offline -p argand-site-registry cargo build --release --locked --offline -p argand-site-registry
./target/release/argand-site-registry --help ./target/release/argand-site-registry --help
``` ```
If your Cargo configuration sets a different target directory, set To see the complete lifecycle without downloading provider data, run the native
`CARGO_TARGET_DIR="$PWD/target"` before these commands. To install the CLI: fixture in a new directory outside the checkout:
```bash
cargo install --path crates/argand-site-registry --locked --offline
argand-site-registry --help
```
Run the native all-source example in a **new directory outside the checkout**:
```bash ```bash
ARGAND_REGISTRY_E2E_OUTPUT=/tmp/site-registry-example \ ARGAND_REGISTRY_E2E_OUTPUT=/tmp/site-registry-example \
cargo test -p argand-site-registry --test cli --locked --offline -- --nocapture cargo test -p argand-site-registry --test cli --locked --offline -- --nocapture
``` ```
It imports synthetic Wikidata, Majestic, CrUX, Curlie and PSL fixtures, checks The fixture imports source-shaped Wikidata, Majestic Million, CrUX, Curlie and
idempotency, joins explicitly reviewed identities, approves a destination, signs Public Suffix List records. It proves repeatable imports, alias lookup, explicit
and activates it, revokes it, and rejects rollback past that revocation. Fixture entity equivalence, destination review, signing, activation, revocation and
signing keys and approvals are disposable test material. All production inputs rollback protection. Its keys and approvals are disposable test material and
belong in a configurable data/cache directory outside the source checkout. must never be used for a real registry.
Follow the [operator guide](crates/argand-site-registry/README.md) for actual Production inputs belong in a configurable data/cache directory outside the
downloads, imports, regional reviews, signed datasets and weekly candidate updates. source checkout. Installation and tests do not download datasets or start
CrUX acquisition requires explicit credentials and a billing cap. No source scheduled jobs.
downloads or scheduled jobs run during installation or tests.
## Use it in another project ## Use a generation from Rust or Python
The [Rust example](crates/argand-site-registry/examples/lookup.rs) opens a generation A consumer needs the generation directory and a trusted pin for its
once with an externally trusted receipt hash and returns the full lookup envelope. `COMPLETE.json` receipt. Obtain that pin through a trusted publisher channel, or
The [Python example](examples/lookup.py) calls the same native CLI; it does not verify the publisher signature with an independently configured key. A hash found
implement a second resolver. Both preserve attribution and fact provenance. beside an untrusted download does not authenticate the download.
The [Rust example](crates/argand-site-registry/examples/lookup.rs) reuses the
native reader. The [Python example](examples/lookup.py) calls the native CLI so it
keeps the same validation and response contract:
```bash ```bash
cargo run --locked --offline -p argand-site-registry --example lookup -- \ cargo run --locked --offline -p argand-site-registry --example lookup -- \
--generation /data/registry/generation --pin "$REGISTRY_TRUSTED_PIN" --query facebook --generation /data/registry/generation \
--pin "$REGISTRY_TRUSTED_PIN" \
--query facebook
python3 examples/lookup.py --binary ./target/release/argand-site-registry \ python3 examples/lookup.py \
--generation /data/registry/generation --pin "$REGISTRY_TRUSTED_PIN" --query facebook --binary ./target/release/argand-site-registry \
--generation /data/registry/generation \
--pin "$REGISTRY_TRUSTED_PIN" \
--query facebook
``` ```
Obtain the pin from a trusted publisher channel or verify the release signature Do not turn `lookup.candidates[0]` into an automatic redirect. Use `resolve` and
against an independently configured key. Reading a hash from the same untrusted preserve `destination: null` as a deliberate abstention. The
download does not authenticate it. Never turn `lookup.candidates[0]` into an [consumer contract](docs/CONSUMERS.md) documents the Rust API, CLI JSON, signed
automatic destination: use the native `resolve` result and your application's generation files, compatibility rules and Argand's pinned integration.
own admission policy. A null destination is a meaningful abstention.
The [consumer contract](docs/CONSUMERS.md) covers Rust dependencies, CLI JSON, ## Operate or contribute
SQLite/JSONL distribution, compatibility and Argand's pinned upstream integration.
Exact entity, URL/domain, source-separated popularity and redacted Curlie category
queries support audit tools. `resolve` also reports a stable abstention reason and
decision counts. The bounded `evaluate` command replays JSONL judgments and reports
accuracy plus native p50/p95 latency for one pinned generation.
## Sources and trust The [operator guide](crates/argand-site-registry/README.md) contains the complete
commands for acquiring each source, importing large files, reviewing regional
properties, signing releases, scheduling candidate updates and recovering state.
CrUX acquisition always requires explicit credentials and a billing cap.
Wikidata (CC0), Majestic Million (CC BY 3.0), CrUX (CC BY 4.0), Curlie (CC BY 3.0) The registry currently supports these source adapters:
and the Public Suffix List (MPL 2.0) remain logically separate. Read the exact
[source licenses and attribution rules](LICENSE_SOURCES.md) before redistribution.
Curlie attribution applies to names and categories as well as descriptions.
The [trust policy](docs/TRUST.md) explains enforced checks, publisher responsibilities, | Source | Purpose | Data license |
evidence standards, expiry and revocation. The [publisher runbook](docs/PUBLISHING.md) | --- | --- | --- |
covers authenticated reviewer decisions, candidate inspection and activation. | Wikidata | Entity names, aliases, official websites and locale/country evidence | CC0 1.0 Universal |
[CONTRIBUTING.md](CONTRIBUTING.md), | Majestic Million | Domain popularity rank | CC BY 3.0 Unported |
[GOVERNANCE.md](GOVERNANCE.md) and [SECURITY.md](SECURITY.md) cover contributions, | Chrome UX Report (CrUX) | Popular-origin rank bucket | CC BY 4.0 International |
decisions, disputes and incidents. Pull requests cannot directly approve destinations. | Curlie | Human-curated names and categories | CC BY 3.0 Unported |
| Public Suffix List | Public-suffix and registrable-domain parsing | Mozilla Public License 2.0 |
## Development and releases Read [LICENSE_SOURCES.md](LICENSE_SOURCES.md) before redistributing data. Curlie
attribution applies to names and categories as well as descriptions. The registry
keeps each source logically separate and retains provenance, licenses, retrieval
timestamps, source identifiers and conflicting evidence.
For the security model and release process, see:
- [Trust and evidence policy](docs/TRUST.md)
- [Publishing runbook](docs/PUBLISHING.md)
- [Security policy](SECURITY.md)
- [Governance](GOVERNANCE.md)
- [Full documentation index](docs/INDEX.md)
Pull requests may add evidence or code, but cannot directly approve a production
destination.
## Development
```bash ```bash
cargo fetch --locked cargo fetch --locked
bash scripts/check.sh bash scripts/check.sh
``` ```
The check runs formatting, all-target compilation, strict Clippy, Rust tests and The check covers formatting, all-target compilation, strict Clippy, Rust tests,
documentation, Python release tests, and native Rust/Python consumer parity. API documentation, Python release tests and native Rust/Python consumer parity.
[RELEASING.md](docs/RELEASING.md) covers deterministic source archives, signature [RELEASING.md](docs/RELEASING.md) explains deterministic source archives and
verification and rebuilding outside the checkout. The Forgejo workflow requires independent signature verification; [VALIDATION.md](docs/VALIDATION.md) records
a dedicated isolated runner; it has no signing or dataset-promotion authority. the current independent build and acceptance evidence.
The [validation record](docs/VALIDATION.md) reports independent builds and native The code is licensed under **AGPL-3.0-or-later**; see [LICENSE](LICENSE). Provider
acceptance. Hosted checks use the repository's isolated Forgejo runner label and data keeps its own license. [UPSTREAM.json](UPSTREAM.json) records the signed
have no dataset, review, signing or activation authority. Argand extraction revision and original file hashes. Argand pins the signed
`v0.3.0` release by full Git revision and no longer carries an embedded source
The code remains **AGPL-3.0-or-later**; the complete license is in [LICENSE](LICENSE). copy.
Original attribution is retained. [UPSTREAM.json](UPSTREAM.json) records the signed
Argand extraction revision and original file hashes. Version 0.3 advances the
standalone database to schema version 3; the migration and consumer rules are
documented in [CONSUMERS.md](docs/CONSUMERS.md). Argand pins this signed `v0.3.0`
release by full Git revision and no longer carries an embedded source copy. This
repository does not operate a public approved-link dataset.