nicweyand/argand-site-registry
1
0
Fork 0
Code Issues Pull requests Projects Releases 7 Packages Wiki Activity Actions

docs: make registry README easier to follow
All checks were successful
Standalone registry checks / check (push) Successful in 4m24s
Details

Browse source
This commit is contained in:
Nic Weyand 2026-09-13 09:14:10 -04:00
commit 2861337a45
Signed by: nicweyand
SSH key fingerprint: SHA256:2te+ycJIQON/Wo/dH6+ZkFSQ4HnHWpetV2azx9E65dQ
1 changed files with 132 additions and 87 deletions
Download patch file Download diff file Expand all files Collapse all files

217
README.md
View file

@ -1,128 +1,173 @@
# Argand Site Registry
Evidence-backed website identity and regional resolution, as a Rust library and CLI.
Argand Site Registry connects an organization, product or other named entity to
its official websites. It is built for navigational search and for choosing the
right regional site without hiding where each claim came from.
Look up an entity's names, aliases and websites; inspect each source assertion;
resolve an explicitly reviewed primary or regional destination. Sources, conflicting
claims, licenses and review history remain available for audit. Everything runs
locally with SQLite. No search engine, hosted account or GPU is required.
Given `facebook`, a registry can return:
`facebook → Facebook → facebook.com` is a name-to-entity-to-registrable-domain
lookup. The retained real Wikidata Facebook record has `www.facebook.com` and
`m.facebook.com` as distinct properties. Its Amazon record includes `amazon.com`,
`amazon.co.uk` and `amazon.de` on the same entity. Similar hostname spelling never
establishes common ownership. Imported links await review before `resolve` can
return a destination; source confidence is not a malware-safety guarantee.
```text
facebook -> Facebook (Wikidata Q355) -> https://www.facebook.com/
registrable domain: facebook.com
```
## Build and try it
An entity may have several legitimate properties. For example, one Amazon entity
can carry separately evidenced properties for `amazon.com`, `amazon.co.uk` and
`amazon.de`. The registry never joins entities merely because their names or
hostnames look alike.
Linux is the currently validated platform. Install Rust (tested with 1.98.1; the
inherited minimum is 1.97), a C compiler, CMake, Perl and OpenSSH (`ssh-keygen`).
Python 3.11+ is needed for release tooling and the Python example. SQLite is built
with the binary. Get the public source, or use a verified source release:
This repository provides the Rust library, command-line tools and synthetic test
fixtures. It does **not** publish a ready-to-use approved-link dataset. A registry
publisher imports source data, reviews destinations and distributes a signed
generation; a consumer verifies that generation before using it.
## What it answers
The two main query operations have different purposes:
| Operation | Answer | Trust behavior |
| --- | --- | --- |
| `lookup` | What entities and websites do the sources associate with this name? | Returns evidence for inspection, including conflicts and unreviewed claims. |
| `resolve` | Which destination is approved for this name, locale and country? | Returns only an unambiguous, signed-review-backed, unexpired destination; otherwise it abstains. |
The CLI can also inspect an entity by stable ID, reverse-lookup a URL or domain,
show source-separated popularity signals, inspect redacted Curlie categories,
compare generations and evaluate a pinned generation against JSONL judgments.
Source confidence is evidence metadata, not a malware-safety score. An imported
official-site claim cannot become a `resolve` destination until an authorized
reviewer approves that exact claim. Applications should still apply their own
security and content policy.
## How a registry is built
```text
source downloads -> provenance-preserving imports -> immutable candidate
-> signed human reviews -> rebuilt generation -> signed activation
-> lookup / regional resolve
```
| Stage | Main commands | What changes |
| --- | --- | --- |
| Acquire | `download`, `crux-download`, `manifest` | Stores source bytes and a receipt describing their origin. |
| Import | `import` | Streams normalized facts into the local SQLite writer database. |
| Build | `build` | Produces a new immutable, content-pinned generation. |
| Review | `review`, `equivalence` | Records signed destination or entity-equivalence decisions against exact evidence fingerprints. |
| Publish | `sign`, `activate` | Verifies reviewer trust, signs a generation and atomically selects it. |
| Consume | `lookup`, `resolve`, `lookup-web`, `entity` | Reads a verified generation without changing it. |
Automated updates stop after building a candidate. They cannot approve a link,
sign a release or activate it.
## Try it locally
Linux is the currently validated platform. You need Rust 1.97 or newer, a C
compiler, CMake, Perl and OpenSSH (`ssh-keygen`). Python 3.11 or newer is needed
for release checks and the Python example. SQLite is compiled into the binary.
```bash
git clone https://git.argand.org/nicweyand/argand-site-registry.git
cd argand-site-registry
```
From the root of the checkout or extracted release:
```bash
cargo fetch --locked
cargo build --release --locked --offline -p argand-site-registry
./target/release/argand-site-registry --help
```
If your Cargo configuration sets a different target directory, set
`CARGO_TARGET_DIR="$PWD/target"` before these commands. To install the CLI:
```bash
cargo install --path crates/argand-site-registry --locked --offline
argand-site-registry --help
```
Run the native all-source example in a **new directory outside the checkout**:
To see the complete lifecycle without downloading provider data, run the native
fixture in a new directory outside the checkout:
```bash
ARGAND_REGISTRY_E2E_OUTPUT=/tmp/site-registry-example \
cargo test -p argand-site-registry --test cli --locked --offline -- --nocapture
```
It imports synthetic Wikidata, Majestic, CrUX, Curlie and PSL fixtures, checks
idempotency, joins explicitly reviewed identities, approves a destination, signs
and activates it, revokes it, and rejects rollback past that revocation. Fixture
signing keys and approvals are disposable test material. All production inputs
belong in a configurable data/cache directory outside the source checkout.
The fixture imports source-shaped Wikidata, Majestic Million, CrUX, Curlie and
Public Suffix List records. It proves repeatable imports, alias lookup, explicit
entity equivalence, destination review, signing, activation, revocation and
rollback protection. Its keys and approvals are disposable test material and
must never be used for a real registry.
Follow the [operator guide](crates/argand-site-registry/README.md) for actual
downloads, imports, regional reviews, signed datasets and weekly candidate updates.
CrUX acquisition requires explicit credentials and a billing cap. No source
downloads or scheduled jobs run during installation or tests.
Production inputs belong in a configurable data/cache directory outside the
source checkout. Installation and tests do not download datasets or start
scheduled jobs.
## Use it in another project
## Use a generation from Rust or Python
The [Rust example](crates/argand-site-registry/examples/lookup.rs) opens a generation
once with an externally trusted receipt hash and returns the full lookup envelope.
The [Python example](examples/lookup.py) calls the same native CLI; it does not
implement a second resolver. Both preserve attribution and fact provenance.
A consumer needs the generation directory and a trusted pin for its
`COMPLETE.json` receipt. Obtain that pin through a trusted publisher channel, or
verify the publisher signature with an independently configured key. A hash found
beside an untrusted download does not authenticate the download.
The [Rust example](crates/argand-site-registry/examples/lookup.rs) reuses the
native reader. The [Python example](examples/lookup.py) calls the native CLI so it
keeps the same validation and response contract:
```bash
cargo run --locked --offline -p argand-site-registry --example lookup -- \
--generation /data/registry/generation --pin "$REGISTRY_TRUSTED_PIN" --query facebook
--generation /data/registry/generation \
--pin "$REGISTRY_TRUSTED_PIN" \
--query facebook
python3 examples/lookup.py --binary ./target/release/argand-site-registry \
--generation /data/registry/generation --pin "$REGISTRY_TRUSTED_PIN" --query facebook
python3 examples/lookup.py \
--binary ./target/release/argand-site-registry \
--generation /data/registry/generation \
--pin "$REGISTRY_TRUSTED_PIN" \
--query facebook
```
Obtain the pin from a trusted publisher channel or verify the release signature
against an independently configured key. Reading a hash from the same untrusted
download does not authenticate it. Never turn `lookup.candidates[0]` into an
automatic destination: use the native `resolve` result and your application's
own admission policy. A null destination is a meaningful abstention.
Do not turn `lookup.candidates[0]` into an automatic redirect. Use `resolve` and
preserve `destination: null` as a deliberate abstention. The
[consumer contract](docs/CONSUMERS.md) documents the Rust API, CLI JSON, signed
generation files, compatibility rules and Argand's pinned integration.
The [consumer contract](docs/CONSUMERS.md) covers Rust dependencies, CLI JSON,
SQLite/JSONL distribution, compatibility and Argand's pinned upstream integration.
Exact entity, URL/domain, source-separated popularity and redacted Curlie category
queries support audit tools. `resolve` also reports a stable abstention reason and
decision counts. The bounded `evaluate` command replays JSONL judgments and reports
accuracy plus native p50/p95 latency for one pinned generation.
## Operate or contribute
## Sources and trust
The [operator guide](crates/argand-site-registry/README.md) contains the complete
commands for acquiring each source, importing large files, reviewing regional
properties, signing releases, scheduling candidate updates and recovering state.
CrUX acquisition always requires explicit credentials and a billing cap.
Wikidata (CC0), Majestic Million (CC BY 3.0), CrUX (CC BY 4.0), Curlie (CC BY 3.0)
and the Public Suffix List (MPL 2.0) remain logically separate. Read the exact
[source licenses and attribution rules](LICENSE_SOURCES.md) before redistribution.
Curlie attribution applies to names and categories as well as descriptions.
The registry currently supports these source adapters:
The [trust policy](docs/TRUST.md) explains enforced checks, publisher responsibilities,
evidence standards, expiry and revocation. The [publisher runbook](docs/PUBLISHING.md)
covers authenticated reviewer decisions, candidate inspection and activation.
[CONTRIBUTING.md](CONTRIBUTING.md),
[GOVERNANCE.md](GOVERNANCE.md) and [SECURITY.md](SECURITY.md) cover contributions,
decisions, disputes and incidents. Pull requests cannot directly approve destinations.
| Source | Purpose | Data license |
| --- | --- | --- |
| Wikidata | Entity names, aliases, official websites and locale/country evidence | CC0 1.0 Universal |
| Majestic Million | Domain popularity rank | CC BY 3.0 Unported |
| Chrome UX Report (CrUX) | Popular-origin rank bucket | CC BY 4.0 International |
| Curlie | Human-curated names and categories | CC BY 3.0 Unported |
| Public Suffix List | Public-suffix and registrable-domain parsing | Mozilla Public License 2.0 |
## Development and releases
Read [LICENSE_SOURCES.md](LICENSE_SOURCES.md) before redistributing data. Curlie
attribution applies to names and categories as well as descriptions. The registry
keeps each source logically separate and retains provenance, licenses, retrieval
timestamps, source identifiers and conflicting evidence.
For the security model and release process, see:
- [Trust and evidence policy](docs/TRUST.md)
- [Publishing runbook](docs/PUBLISHING.md)
- [Security policy](SECURITY.md)
- [Governance](GOVERNANCE.md)
- [Full documentation index](docs/INDEX.md)
Pull requests may add evidence or code, but cannot directly approve a production
destination.
## Development
```bash
cargo fetch --locked
bash scripts/check.sh
```
The check runs formatting, all-target compilation, strict Clippy, Rust tests and
documentation, Python release tests, and native Rust/Python consumer parity.
[RELEASING.md](docs/RELEASING.md) covers deterministic source archives, signature
verification and rebuilding outside the checkout. The Forgejo workflow requires
a dedicated isolated runner; it has no signing or dataset-promotion authority.
The check covers formatting, all-target compilation, strict Clippy, Rust tests,
API documentation, Python release tests and native Rust/Python consumer parity.
[RELEASING.md](docs/RELEASING.md) explains deterministic source archives and
independent signature verification; [VALIDATION.md](docs/VALIDATION.md) records
the current independent build and acceptance evidence.
The [validation record](docs/VALIDATION.md) reports independent builds and native
acceptance. Hosted checks use the repository's isolated Forgejo runner label and
have no dataset, review, signing or activation authority.
The code remains **AGPL-3.0-or-later**; the complete license is in [LICENSE](LICENSE).
Original attribution is retained. [UPSTREAM.json](UPSTREAM.json) records the signed
Argand extraction revision and original file hashes. Version 0.3 advances the
standalone database to schema version 3; the migration and consumer rules are
documented in [CONSUMERS.md](docs/CONSUMERS.md). Argand pins this signed `v0.3.0`
release by full Git revision and no longer carries an embedded source copy. This
repository does not operate a public approved-link dataset.
The code is licensed under **AGPL-3.0-or-later**; see [LICENSE](LICENSE). Provider
data keeps its own license. [UPSTREAM.json](UPSTREAM.json) records the signed
Argand extraction revision and original file hashes. Argand pins the signed
`v0.3.0` release by full Git revision and no longer carries an embedded source
copy.