release: implement site registry v0.5

docs/SECURITY-REVIEW-0.5.md

@@ -0,0 +1,105 @@
+# Version 0.5 security review
+
+Date: 2026-09-13. Scope: all changes from signed version 0.4.0 through the 0.5
+implementation, including source manifest v3, ROR ZIP acquisition/import, streamed
+Wikidata JSON, compact generations, external audit bundles, retention signatures,
+evaluation lineage, benchmark tooling, dependencies and operator documentation.
+
+No unresolved critical, high, or medium-severity finding remained at release
+validation. This is a source review and adversarial test result, not a claim that
+an imported URL is safe or that a future public registry has been reviewed.
+
+## Boundaries reviewed
+
+- Provider responses, manifests, JSON/CSV/XML/ZIP/archive members, source records,
+  names, URLs, lineages and audit-store entries are untrusted input.
+- A source assertion cannot authorize a route. Name, edge and equivalence authority
+  still comes from policy-qualified reviewer signatures; release authority remains
+  a separate publisher signature and activation step.
+- The cache, writer, audit store, reviewer trust file and keys are operator-owned
+  local resources. Consumer trust starts with an independently obtained receipt
+  pin or publisher trust root.
+- The project ships no hosted API, browser rendering surface, credential store,
+  production key, preapproved dataset, automatic signer, or automatic activation.
+
+## Findings fixed
+
+| ID | Severity before fix | Resolution |
+| --- | --- | --- |
+| SR-05-01 | Medium | Audit-store discovery previously used the 512 MiB fact-line ceiling for an untrusted unmatched header. Header reads now stop at 1 MiB before JSON materialization; facts remain bounded and content-authenticated. |
+| SR-05-02 | Medium | SQLite can auto-rollback a transaction on `SQLITE_FULL`; an unconditional second rollback could return before cleanup and leave a committed incomplete checkpoint. Recovery now detects autocommit, always attempts source discard and growth-ceiling restoration, and has a real growth-cap regression. |
+| SR-05-03 | Medium | A syntactically valid compact receipt could omit a selected audit reference while still naming other bundles. Opening now validates every source declaration, exact PSL identity, receipt-to-database selected-source equality, and exact equality between selected bundle references and active receipt sources. |
+| SR-05-04 | Medium | Initial ROR ZIP limits trusted member size metadata. JSON and CSV decompression now have independent actual-byte ceilings in addition to record, count, database-growth, member-count, compression-method, encryption and name checks. |
+| SR-05-05 | Medium | Foreign keys were disabled during an indexed compaction transaction and the first error on rollback could bypass restoration. Every exit now attempts rollback as needed and restores foreign-key enforcement before returning. Publication still requires `integrity_check` and a zero-row `foreign_key_check`. |
+| SR-05-06 | Low | The first benchmark stdout cap applied `RLIMIT_FSIZE` to the whole child and could kill SQLite writes. The final runner uses bounded stdout/stderr pipes, kills the isolated process group only on output overflow, records the condition, and never limits dataset/database files. The induced hot journal was recovered and the import resumed to the clean-import totals. |
+
+The review also capped authenticated custom records at 256 MiB so generated cold
+bundle lines remain within their verifier ceiling. Removed Wikidata statements
+carry their exact superseded source IDs in tombstone exports. ROR inactive and
+withdrawn website assertions remain auditable but ineligible.
+
+## Acquisition and archive checks
+
+Source URLs use an HTTPS provider allowlist, reject credentials/fragments/custom
+ports and revalidate every redirect. Resumption requires a strong unchanged ETag,
+exact final URL, matching total and Content-Range. Downloads have caller-supplied
+byte ceilings and immutable completion receipts. Provider checksum evidence is
+accepted only for reviewed source-bound locations: the matching Zenodo record for
+ROR or the same Wikimedia dump directory for Wikidata. ROR's MD5 is used only to
+match Zenodo's published field; the local content identity and all downstream
+references use SHA-256.
+
+The ROR reader accepts exactly one version-matched JSON and CSV member, Stored or
+Deflated, with no path components, encryption or extra members. It streams schema
+2.1 JSON with duplicate-key rejection and fails on unknown top-level fields. Every
+source import re-hashes the exact no-follow input descriptor after parsing and
+publishes completion only after all records commit.
+
+## Compact history and signing checks
+
+Cold bundles use digest-derived names and are created without replacement. A
+reference binds exact bytes, source declaration, selected state, record/fact
+counts, coverage and attribution. Verification opens with `O_NOFOLLOW`, checks
+length and SHA-256 on the same descriptor, then parses bounded JSONL in required
+record-before-fact order. Audit export and retention checkpoint first verify every
+object. Retention signatures use the distinct
+`argand-site-registry-audit-retention` namespace, and their authenticated document
+always requires `deletion_authorized: false`; the software provides no deletion
+command.
+
+Runtime compaction occurs only after all bundles are durable and verified. It
+uses an indexed exact fact keep-set, commits a deterministic runtime copy, restores
+foreign keys, then runs integrity and foreign-key checks before `COMPLETE.json` is
+created. Readers authenticate and copy SQLite into a private unlinked file before
+opening immutable mode, preserving the existing sidecar and post-open mutation
+defenses.
+
+## Automated and manual evidence
+
+- Workspace formatting, locked all-target checks, Clippy with warnings denied,
+  strict rustdoc, Rust/Python tests, consumer parity, reproducible source packaging
+  and extracted-source acceptance are release gates.
+- `cargo audit --deny warnings` loaded 1,243 RustSec advisories and reported no
+  finding across 277 locked dependency nodes.
+- The repository and diff contain no private key, token, password, dataset,
+  production review, or generated registry. SQL values remain parameterized;
+  dynamic SQL fragments are fixed internal table/predicate vocabularies.
+- Adversarial tests cover duplicate JSON keys/records, schema/member drift,
+  symlinks, truncation/substitution, oversized records, malformed coverage,
+  process termination at multiple checkpoints, SQLite growth exhaustion,
+  incomplete update/publication, receipt bundle omission, signature tampering,
+  reviewer/publisher separation, revocation continuity and rollback.
+
+## Residual operator responsibilities
+
+Protect cache, writer and audit directories from untrusted local writers and
+replicate cold objects by exact digest. A deliberately configured 256 MiB record
+can require comparable memory; choose the smallest inspected limit and enforce
+process/storage quotas around provider jobs. Measure cold authenticated startup on
+deployment storage, because every registry open copies the full runtime database.
+
+Obtain provider checksums and publisher/reviewer trust roots independently. A valid
+hash or signature authenticates bytes and a decision, not ownership, current site
+safety, commercial fitness, or reviewer competence. Continue malware/content,
+drift, expiry, revocation, backup/restore and incident controls before any public
+dataset or Argand route is activated.