diff --git a/CHANGELOG.md b/CHANGELOG.md index c89661b..a8314c9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,23 @@ # Changelog +## 0.5.0 - 2026-09-13 + +- Add a streaming ROR 2.1 ZIP adapter with exact schema checks, declared domains, + website links, names, aliases, external identifiers, status, type, country and + administrative metadata while preserving ROR identities independently. +- Advance source manifests to v3 with source-bound content/provider checksums, + explicit direct and upstream lineage, authenticated per-record limits and exact + provenance for every imported or derived fact. +- Stream multiline Wikidata JSON arrays and typed JSON deltas, including auditable + tombstones when official-website statements disappear. +- Add compact runtime generations backed by content-addressed cold audit bundles, + complete verification and export, signed no-deletion retention checkpoints and + exact active-source binding in generation receipts. +- Add lineage-aware evaluation reports and a reproducible, resource-bounded + benchmark harness with exact input, output, process and hardware evidence. +- Harden ZIP expansion limits, audit reads, foreign-key restoration, disk-full + import recovery and output-exhausted benchmark process cleanup. + ## 0.4.0 - 2026-09-13 - Add typed full/partition/delta source coverage, active-record masking, a diff --git a/docs/VALIDATION.md b/docs/VALIDATION.md index 6bb3593..8f6afb3 100644 --- a/docs/VALIDATION.md +++ b/docs/VALIDATION.md @@ -1,5 +1,72 @@ # Validation +## Version 0.5.0 release and security validation, 2026-09-13 + +Implementation commit: `557ba7cd6982b02754d34fb99cba5a116f78f153`, signed by +Nic Weyand. Version 0.5 adds the ROR 2.1 adapter, source-bound provider checksums, +explicit source lineage, multiline Wikidata dump and JSON-delta replay, compact +runtime generations with authenticated external audit bundles, signed retention +checkpoints, lineage-aware evaluation reports and a bounded benchmark harness. +Writer stores retain schema version 5; source manifests advance to v3 and compact +generation receipts to v3. + +Linux x86_64 with Rust/Cargo 1.98.1 and Python 3.14.7 passed: + +- Formatting, locked all-target compilation, Clippy with warnings denied, strict + API documentation, all 70 Rust tests and all 10 Python tests. +- Native CLI, reusable Rust and Python consumer parity across the synthetic + fixture, including trust failures. +- Adversarial checks for ZIP member and expanded-size drift, multiline and large + Wikidata records, P856 tombstones, killed and disk-full imports, audit omission + and substitution, compact/full equivalence, receipt selection mismatches, + retention signatures, benchmark output exhaustion and no-clobber publication. +- `cargo audit --deny warnings` scanned 1,243 RustSec advisories across 277 locked + dependency nodes without a finding. The final security review found no unresolved + critical, high or medium issue in its reviewed scope; see + [SECURITY-REVIEW-0.5.md](SECURITY-REVIEW-0.5.md). + +Two source packages from the clean implementation commit were byte identical. +Receipt pin: +`51dff46d846bbfe794fb33d507a694ac15fbc899591e5515d33d114db3bee553`. +Archive SHA-256: +`63a0571061385a36197beabadfd4c29e3d15bb5948570bea74b04cfb3f784abc`. +The receipt verifier accepted both copies. The archive was extracted outside the +repository without Git metadata and passed the complete acceptance gate again. + +[Hosted Forgejo Actions run 9](https://git.argand.org/nicweyand/argand-site-registry/actions/runs/9) +passed the exact implementation commit on the isolated registry runner. It fetched +the public revision without credentials, ran the offline acceptance gate, compared +and verified two deterministic source packages, extracted one without Git metadata, +and passed the complete gate again from that source tree. The runner has no dataset, +reviewer, signing or activation authority. + +A separate real-provider canary used ROR release `v2.12-2026-08-25` from Zenodo +record `22099990` and the same-day Public Suffix List. The 36,246,232-byte ROR ZIP +had SHA-256 +`5779c7baf71771fd8ea829201e7bd4343a3c68ff36c595f480b3a00292f78931` +and provider-bound MD5 `ce8807691455d4ada3216c31408e9e1a`. It expanded to +362,619,018 bytes and imported 137,398 records with 914,439 facts in 62.30 seconds, +using 25,032 KiB peak RSS. Exact replay produced zero changes. A deliberately +killed import resumed from its last committed checkpoint and converged on the same +record and fact totals. + +The resulting compact registry contained 137,398 entities, 131,591 properties, +133,398 edges and 169 rejected facts. Its generation pin was +`b4f639e3d4f759833914a34aded2389442eeca142952d78fb7f66343f6f4224d`. +The 824,705,024-byte compact database was 38.69% smaller than the full audit-bearing +database. Two compact builds were byte identical. Full verification covered +686,468,048 audit bytes, 137,399 source records and 914,440 facts. A 1,000-case +evaluation produced 1,000 expected safe abstentions with 72 microsecond median and +96 microsecond p95 lookup latency. These single-machine import and warm-read +measurements characterize this release canary, not production serving capacity; +see [BENCHMARKING.md](BENCHMARKING.md). + +The final inventory was reviewed for credentials, private paths, datasets, +generated artifacts, unsafe Rust and unrelated changes. Provider data, generated +registries and audit bundles remain outside Git. No production review or signing +key was used, and no Argand source tree, build cache, service or public route was +changed during implementation or validation. + ## Version 0.4.0 release and security validation, 2026-09-13 Implementation commit: `e26efc19fa7f73e63cd98cb32b446d1fe10eed40`, signed by diff --git a/docs/superpowers/plans/2026-09-13-v0.4-and-beyond.md b/docs/superpowers/plans/2026-09-13-v0.4-and-beyond.md index 7dbac92..503068c 100644 --- a/docs/superpowers/plans/2026-09-13-v0.4-and-beyond.md +++ b/docs/superpowers/plans/2026-09-13-v0.4-and-beyond.md @@ -1,10 +1,13 @@ # Argand Site Registry v0.4 and Beyond Plan -> **Status:** Version 0.4 phases 0 through 2 implemented and security-reviewed. -> Phases 3 and later remain the sequenced roadmap. +> **Status:** Version 0.5 phases 0 through 3 and the ROR adapter in task 4.1 are +> implemented, provider-scale tested, and security-reviewed. MusicBrainz and GND +> remain rights/schema-gated candidates. Phases 5 through 7 describe later releases, +> including the human-reviewed public dataset; they are not version 0.5 exit work. > -> **Baseline:** Clean `main` at `2861337`; runtime behavior is the tagged -> `v0.3.0` release at `ac82820`. The complete v0.3 acceptance suite passes. +> **Current implementation:** signed commit `557ba7c` (version 0.5.0). The plan's +> original audited baseline was clean `main` at `2861337` with tagged `v0.3.0` +> runtime behavior at `ac82820`. > > **Execution constraint:** Work in this standalone repository only. Do not use > subagents, edit Argand's main checkout, share its build cache, acquire paid data, @@ -595,22 +598,25 @@ Reference: ### Task 4.4: Keep lower-priority candidates behind explicit holds -- [ ] ORCID: research a low-confidence individuals-only adapter. Its public file is +- [x] ORCID: research a low-confidence individuals-only adapter. Its public file is CC0, but links are self-declared and require privacy, impersonation, and volatility policy. Never auto-approve. Reference: . -- [ ] OpenAlex: use only for research-activity/popularity metadata if useful. Record +- [x] OpenAlex: use only for research-activity/popularity metadata if useful. Record ROR as upstream lineage and never count its institution website as independent corroboration. Reference: . -- [ ] OpenStreetMap: do not ingest until an ODbL-compatible distribution and +- [x] OpenStreetMap: do not ingest until an ODbL-compatible distribution and attribution architecture is approved. Reference: . -- [ ] Government/corporate registries: assess jurisdiction by jurisdiction. Prefer +- [x] Government/corporate registries: assess jurisdiction by jurisdiction. Prefer stable identity crosswalks; do not infer a website where no authoritative field exists. -- [ ] DNS, RDAP, certificate transparency, package registries, and web crawl data: +- [x] DNS, RDAP, certificate transparency, package registries, and web crawl data: evaluate as observation sources only after exact terms are verified. -- [ ] Open Library and other sources with unresolved underlying rights remain excluded. +- [x] Open Library and other sources with unresolved underlying rights remain excluded. + +Task 4.4 completion note: the reviewed hold states and their next admission gates +are maintained in `docs/SOURCE-CANDIDATES.md`; none of these sources is ingestible. ### Phase 4 acceptance @@ -769,16 +775,16 @@ Reference: ## Documentation deliverables -- [ ] Keep the README focused on the first successful verified lookup. -- [ ] Add an architecture document explaining assertion, observation, decision, policy, +- [x] Keep the README focused on the first successful verified lookup. +- [x] Add an architecture document explaining assertion, observation, decision, policy, generation, delta, and consumer boundaries. -- [ ] Expand `LICENSE_SOURCES.md` for every admitted source with exact fields consumed, +- [x] Expand `LICENSE_SOURCES.md` for every admitted source with exact fields consumed, source URLs, licenses, attribution, redistribution, update cadence, and lineage. -- [ ] Document active versus audit export semantics. -- [ ] Document migrations and legacy decision handling. +- [x] Document active versus audit export semantics. +- [x] Document migrations and legacy decision handling. - [ ] Publish the reference review and incident policies. -- [ ] Publish provider-scale measurements without implying serving or corpus coverage. -- [ ] Maintain a source-candidate table showing approved, research, held, and rejected +- [x] Publish provider-scale measurements without implying serving or corpus coverage. +- [x] Maintain a source-candidate table showing approved, research, held, and rejected sources with the reason for each state. ## SWOT-driven checks