release: add Web Graph authority evidence for v0.6
Some checks failed
Standalone registry checks / check (push) Has been cancelled

This commit is contained in:
Nic Weyand 2026-09-22 08:25:24 -04:00
commit 3e0cc1bc50
Signed by: nicweyand
SSH key fingerprint: SHA256:2te+ycJIQON/Wo/dH6+ZkFSQ4HnHWpetV2azx9E65dQ
21 changed files with 698 additions and 15 deletions

View file

@ -34,7 +34,7 @@ source attribution and application-specific malware/content policy.
## Current contracts
Code version 0.5.0 uses writer schema 5 and `argand.site-rules/v4`.
Code version 0.6.0 uses writer schema 5 and `argand.site-rules/v4`.
New compact `COMPLETE.json` files use `argand.site-registry/v3` and bind:
- authenticated `registry.sqlite` bytes;
@ -101,8 +101,9 @@ revocation history. Never mutate a complete generation to migrate it. See
## Argand integration
`UPSTREAM.json` records the original Argand extraction baseline and file hashes.
Argand pins signed v0.5.0 revision
`3d3e08cdfd303df9fbd347a9bab2ba52ad575759`. The public beta uses Site Registry as
Argand's prior integration pinned signed v0.5.0 revision
`3d3e08cdfd303df9fbd347a9bab2ba52ad575759`; the v0.6 downstream pin is recorded
by the Argand integration commit after this source release. The public beta uses Site Registry as
Navigate's authoritative auto-route catalog. Its native `navigation-catalog/v2`
file is only a collection- and content-policy-bound serving projection compiled
from one exact registry generation; it is not a second independently curated

View file

@ -1,8 +1,18 @@
# Versioned formats
Version 0.5 uses writer schema 5 and `argand.site-rules/v4`. Schema identifiers
Version 0.6 uses writer schema 5 and `argand.site-rules/v4`. Schema identifiers
are independent from the crate version. Unknown schemas and rules fail closed.
The Common Crawl domain-rank adapter is new in 0.6, so no older v4 store can
contain one of its source manifests. Its replacement scope is
`candidate-domains:<sha256>`, where the digest covers the canonical JSON encoding
of the sorted set of registrable domains derived from retained complete website
assertions. Retained superseded evidence may enlarge this conservative set but
cannot create a route or make a retired assertion active.
The importer authenticates and validates every graph row but persists only the
selected domains. A changed identity frontier therefore produces a new immutable
source identity instead of silently reusing a stale projection.
| Artifact | Current schema | Purpose |
| --- | --- | --- |
| Source manifest | `argand.site-source/v3` | Exact source object, integrity proof, lineage, parser bound and typed coverage |

View file

@ -7,13 +7,14 @@ all been reviewed. A research entry below is not permission to ingest it.
| Source | Status | Decision and next gate |
| --- | --- | --- |
| ROR | Admitted in 0.5 | The official CC0 schema 2.1 ZIP is streamed with exact Zenodo checksum evidence. Organization websites remain assertions; inactive/withdrawn edges are ineligible. GeoNames location lineage is explicit. |
| Common Crawl domain Web Graph ranks | Admitted after 0.5 as authority evidence | The official six-column domain-rank object is streamed from an exact allowlisted release URL under Common Crawl's Terms of Use. Harmonic centrality, PageRank and member-host count remain source separated. The source cannot create identities, official-site edges, reviews, or routes. Full-graph acquisition and production-catalog selection remain separate operational gates. |
| MusicBrainz | Next adapter; held | The [official download documentation](https://musicbrainz.org/doc/MusicBrainz_Database/Download) identifies the core `mbdump.tar.bz2` snapshot as CC0. The live replication/edit/statistics material with noncommercial terms is excluded. Admission still needs a current core snapshot/checksum canary and a bounded relational-table adapter for documented [URL relationships](https://musicbrainz.org/doc/Style/Relationships/URLs). |
| GND | Research hold | The [DNB open-data distribution](https://data.dnb.de/opendata/) must be checked at implementation time for the exact file license, current JSON-LD/RDF predicates, checksum and useful homepage coverage. Stop the adapter if explicit homepage coverage does not justify it. |
| ORCID public data | Research hold | Its self-declared links need an individuals-only privacy, impersonation and volatility policy in addition to the [public-file terms](https://info.orcid.org/public-data-file-use-policy/). It could never auto-approve a route. |
| OpenAlex institutions | Correlated-source hold | Institution metadata can inherit ROR. Any future use must declare ROR upstream and cannot count as independent website corroboration. See the [institution source documentation](https://help.openalex.org/data/institutions/). |
| OpenStreetMap | License-architecture hold | No ingestion until an ODbL-compatible attribution, database-right and redistribution design is accepted. See the [OSMF license FAQ](https://osmfoundation.org/wiki/Licence_and_Legal_FAQ). |
| Government/corporate registries | Jurisdiction hold | Review one jurisdiction and exact field at a time. Stable identifiers may support crosswalks; the registry cannot infer a website absent an authoritative field. |
| DNS, RDAP, certificate transparency, package registries, web crawl data | Observation-only research | Exact commercial reuse terms and retention rules must be approved first. These sources describe current infrastructure and cannot establish entity ownership alone. |
| DNS, RDAP, certificate transparency, package registries, other web crawl data | Observation-only research | Exact commercial reuse terms and retention rules must be approved first. These sources describe current infrastructure and cannot establish entity ownership alone. |
| Open Library and unresolved-rights sources | Excluded | Keep excluded until the underlying data rights and redistribution obligations are clear enough for commercial reuse. |
Cloudflare Radar, default Tranco, Cisco Umbrella, arbitrary mirrors, and any

View file

@ -1,5 +1,27 @@
# Validation
# Version 0.6.0 Web Graph and updater validation, 2026-09-22
The evaluation contract was written before implementation. The new Common Crawl
domain-rank integration passed five default integration tests; its sixth test is
an explicit resource benchmark. The benchmark parsed 100,000 valid provider-shaped
rows, retained exactly one identity-matched domain, and completed in 0.52 seconds
of test time. The enclosing warm Cargo process used 79,944 KiB peak RSS, wrote
9,136 filesystem blocks, and used no swap on the development machine. These are
engineering bounds, not a full 2 GiB provider-object throughput claim.
`scripts/check.sh` passed after the v0.6 version and Rustls lockfile updates. It
covered formatting, offline all-target checks, strict Clippy, all Rust and CLI
tests, documentation, Python release-package tests, and native/Python consumer
parity. The focused graph suite proves full-stream schema/value validation,
source-line provenance, gzip authentication, exact URL allowlisting, deterministic
builds, compact candidate-domain selection, and zero route authorization from
rank evidence.
The networked `cargo audit --deny warnings` gate initially detected
RUSTSEC-2026-0285 in Rustls 0.23.43. The lockfile was updated to Rustls 0.23.45;
the repeated audit passed with no findings.
## Version 0.5.0 release and security validation, 2026-09-13
Implementation commit: `557ba7cd6982b02754d34fb99cba5a116f78f153`, signed by