release: add Web Graph authority evidence for v0.6
Some checks failed
Standalone registry checks / check (push) Has been cancelled
Some checks failed
Standalone registry checks / check (push) Has been cancelled
This commit is contained in:
parent
16c740a98f
commit
3e0cc1bc50
21 changed files with 698 additions and 15 deletions
|
|
@ -34,7 +34,7 @@ source attribution and application-specific malware/content policy.
|
|||
|
||||
## Current contracts
|
||||
|
||||
Code version 0.5.0 uses writer schema 5 and `argand.site-rules/v4`.
|
||||
Code version 0.6.0 uses writer schema 5 and `argand.site-rules/v4`.
|
||||
New compact `COMPLETE.json` files use `argand.site-registry/v3` and bind:
|
||||
|
||||
- authenticated `registry.sqlite` bytes;
|
||||
|
|
@ -101,8 +101,9 @@ revocation history. Never mutate a complete generation to migrate it. See
|
|||
## Argand integration
|
||||
|
||||
`UPSTREAM.json` records the original Argand extraction baseline and file hashes.
|
||||
Argand pins signed v0.5.0 revision
|
||||
`3d3e08cdfd303df9fbd347a9bab2ba52ad575759`. The public beta uses Site Registry as
|
||||
Argand's prior integration pinned signed v0.5.0 revision
|
||||
`3d3e08cdfd303df9fbd347a9bab2ba52ad575759`; the v0.6 downstream pin is recorded
|
||||
by the Argand integration commit after this source release. The public beta uses Site Registry as
|
||||
Navigate's authoritative auto-route catalog. Its native `navigation-catalog/v2`
|
||||
file is only a collection- and content-policy-bound serving projection compiled
|
||||
from one exact registry generation; it is not a second independently curated
|
||||
|
|
|
|||
|
|
@ -1,8 +1,18 @@
|
|||
# Versioned formats
|
||||
|
||||
Version 0.5 uses writer schema 5 and `argand.site-rules/v4`. Schema identifiers
|
||||
Version 0.6 uses writer schema 5 and `argand.site-rules/v4`. Schema identifiers
|
||||
are independent from the crate version. Unknown schemas and rules fail closed.
|
||||
|
||||
The Common Crawl domain-rank adapter is new in 0.6, so no older v4 store can
|
||||
contain one of its source manifests. Its replacement scope is
|
||||
`candidate-domains:<sha256>`, where the digest covers the canonical JSON encoding
|
||||
of the sorted set of registrable domains derived from retained complete website
|
||||
assertions. Retained superseded evidence may enlarge this conservative set but
|
||||
cannot create a route or make a retired assertion active.
|
||||
The importer authenticates and validates every graph row but persists only the
|
||||
selected domains. A changed identity frontier therefore produces a new immutable
|
||||
source identity instead of silently reusing a stale projection.
|
||||
|
||||
| Artifact | Current schema | Purpose |
|
||||
| --- | --- | --- |
|
||||
| Source manifest | `argand.site-source/v3` | Exact source object, integrity proof, lineage, parser bound and typed coverage |
|
||||
|
|
|
|||
|
|
@ -7,13 +7,14 @@ all been reviewed. A research entry below is not permission to ingest it.
|
|||
| Source | Status | Decision and next gate |
|
||||
| --- | --- | --- |
|
||||
| ROR | Admitted in 0.5 | The official CC0 schema 2.1 ZIP is streamed with exact Zenodo checksum evidence. Organization websites remain assertions; inactive/withdrawn edges are ineligible. GeoNames location lineage is explicit. |
|
||||
| Common Crawl domain Web Graph ranks | Admitted after 0.5 as authority evidence | The official six-column domain-rank object is streamed from an exact allowlisted release URL under Common Crawl's Terms of Use. Harmonic centrality, PageRank and member-host count remain source separated. The source cannot create identities, official-site edges, reviews, or routes. Full-graph acquisition and production-catalog selection remain separate operational gates. |
|
||||
| MusicBrainz | Next adapter; held | The [official download documentation](https://musicbrainz.org/doc/MusicBrainz_Database/Download) identifies the core `mbdump.tar.bz2` snapshot as CC0. The live replication/edit/statistics material with noncommercial terms is excluded. Admission still needs a current core snapshot/checksum canary and a bounded relational-table adapter for documented [URL relationships](https://musicbrainz.org/doc/Style/Relationships/URLs). |
|
||||
| GND | Research hold | The [DNB open-data distribution](https://data.dnb.de/opendata/) must be checked at implementation time for the exact file license, current JSON-LD/RDF predicates, checksum and useful homepage coverage. Stop the adapter if explicit homepage coverage does not justify it. |
|
||||
| ORCID public data | Research hold | Its self-declared links need an individuals-only privacy, impersonation and volatility policy in addition to the [public-file terms](https://info.orcid.org/public-data-file-use-policy/). It could never auto-approve a route. |
|
||||
| OpenAlex institutions | Correlated-source hold | Institution metadata can inherit ROR. Any future use must declare ROR upstream and cannot count as independent website corroboration. See the [institution source documentation](https://help.openalex.org/data/institutions/). |
|
||||
| OpenStreetMap | License-architecture hold | No ingestion until an ODbL-compatible attribution, database-right and redistribution design is accepted. See the [OSMF license FAQ](https://osmfoundation.org/wiki/Licence_and_Legal_FAQ). |
|
||||
| Government/corporate registries | Jurisdiction hold | Review one jurisdiction and exact field at a time. Stable identifiers may support crosswalks; the registry cannot infer a website absent an authoritative field. |
|
||||
| DNS, RDAP, certificate transparency, package registries, web crawl data | Observation-only research | Exact commercial reuse terms and retention rules must be approved first. These sources describe current infrastructure and cannot establish entity ownership alone. |
|
||||
| DNS, RDAP, certificate transparency, package registries, other web crawl data | Observation-only research | Exact commercial reuse terms and retention rules must be approved first. These sources describe current infrastructure and cannot establish entity ownership alone. |
|
||||
| Open Library and unresolved-rights sources | Excluded | Keep excluded until the underlying data rights and redistribution obligations are clear enough for commercial reuse. |
|
||||
|
||||
Cloudflare Radar, default Tranco, Cisco Umbrella, arbitrary mirrors, and any
|
||||
|
|
|
|||
|
|
@ -1,5 +1,27 @@
|
|||
# Validation
|
||||
|
||||
# Version 0.6.0 Web Graph and updater validation, 2026-09-22
|
||||
|
||||
The evaluation contract was written before implementation. The new Common Crawl
|
||||
domain-rank integration passed five default integration tests; its sixth test is
|
||||
an explicit resource benchmark. The benchmark parsed 100,000 valid provider-shaped
|
||||
rows, retained exactly one identity-matched domain, and completed in 0.52 seconds
|
||||
of test time. The enclosing warm Cargo process used 79,944 KiB peak RSS, wrote
|
||||
9,136 filesystem blocks, and used no swap on the development machine. These are
|
||||
engineering bounds, not a full 2 GiB provider-object throughput claim.
|
||||
|
||||
`scripts/check.sh` passed after the v0.6 version and Rustls lockfile updates. It
|
||||
covered formatting, offline all-target checks, strict Clippy, all Rust and CLI
|
||||
tests, documentation, Python release-package tests, and native/Python consumer
|
||||
parity. The focused graph suite proves full-stream schema/value validation,
|
||||
source-line provenance, gzip authentication, exact URL allowlisting, deterministic
|
||||
builds, compact candidate-domain selection, and zero route authorization from
|
||||
rank evidence.
|
||||
|
||||
The networked `cargo audit --deny warnings` gate initially detected
|
||||
RUSTSEC-2026-0285 in Rustls 0.23.43. The lockfile was updated to Rustls 0.23.45;
|
||||
the repeated audit passed with no findings.
|
||||
|
||||
## Version 0.5.0 release and security validation, 2026-09-13
|
||||
|
||||
Implementation commit: `557ba7cd6982b02754d34fb99cba5a116f78f153`, signed by
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue