argand-site-registry/docs/SECURITY-REVIEW-0.5.md
nicweyand 557ba7cd69
All checks were successful
Standalone registry checks / check (push) Successful in 5m58s
release: implement site registry v0.5
2026-09-13 14:42:39 -04:00

7.3 KiB

Version 0.5 security review

Date: 2026-09-13. Scope: all changes from signed version 0.4.0 through the 0.5 implementation, including source manifest v3, ROR ZIP acquisition/import, streamed Wikidata JSON, compact generations, external audit bundles, retention signatures, evaluation lineage, benchmark tooling, dependencies and operator documentation.

No unresolved critical, high, or medium-severity finding remained at release validation. This is a source review and adversarial test result, not a claim that an imported URL is safe or that a future public registry has been reviewed.

Boundaries reviewed

  • Provider responses, manifests, JSON/CSV/XML/ZIP/archive members, source records, names, URLs, lineages and audit-store entries are untrusted input.
  • A source assertion cannot authorize a route. Name, edge and equivalence authority still comes from policy-qualified reviewer signatures; release authority remains a separate publisher signature and activation step.
  • The cache, writer, audit store, reviewer trust file and keys are operator-owned local resources. Consumer trust starts with an independently obtained receipt pin or publisher trust root.
  • The project ships no hosted API, browser rendering surface, credential store, production key, preapproved dataset, automatic signer, or automatic activation.

Findings fixed

ID Severity before fix Resolution
SR-05-01 Medium Audit-store discovery previously used the 512 MiB fact-line ceiling for an untrusted unmatched header. Header reads now stop at 1 MiB before JSON materialization; facts remain bounded and content-authenticated.
SR-05-02 Medium SQLite can auto-rollback a transaction on SQLITE_FULL; an unconditional second rollback could return before cleanup and leave a committed incomplete checkpoint. Recovery now detects autocommit, always attempts source discard and growth-ceiling restoration, and has a real growth-cap regression.
SR-05-03 Medium A syntactically valid compact receipt could omit a selected audit reference while still naming other bundles. Opening now validates every source declaration, exact PSL identity, receipt-to-database selected-source equality, and exact equality between selected bundle references and active receipt sources.
SR-05-04 Medium Initial ROR ZIP limits trusted member size metadata. JSON and CSV decompression now have independent actual-byte ceilings in addition to record, count, database-growth, member-count, compression-method, encryption and name checks.
SR-05-05 Medium Foreign keys were disabled during an indexed compaction transaction and the first error on rollback could bypass restoration. Every exit now attempts rollback as needed and restores foreign-key enforcement before returning. Publication still requires integrity_check and a zero-row foreign_key_check.
SR-05-06 Low The first benchmark stdout cap applied RLIMIT_FSIZE to the whole child and could kill SQLite writes. The final runner uses bounded stdout/stderr pipes, kills the isolated process group only on output overflow, records the condition, and never limits dataset/database files. The induced hot journal was recovered and the import resumed to the clean-import totals.

The review also capped authenticated custom records at 256 MiB so generated cold bundle lines remain within their verifier ceiling. Removed Wikidata statements carry their exact superseded source IDs in tombstone exports. ROR inactive and withdrawn website assertions remain auditable but ineligible.

Acquisition and archive checks

Source URLs use an HTTPS provider allowlist, reject credentials/fragments/custom ports and revalidate every redirect. Resumption requires a strong unchanged ETag, exact final URL, matching total and Content-Range. Downloads have caller-supplied byte ceilings and immutable completion receipts. Provider checksum evidence is accepted only for reviewed source-bound locations: the matching Zenodo record for ROR or the same Wikimedia dump directory for Wikidata. ROR's MD5 is used only to match Zenodo's published field; the local content identity and all downstream references use SHA-256.

The ROR reader accepts exactly one version-matched JSON and CSV member, Stored or Deflated, with no path components, encryption or extra members. It streams schema 2.1 JSON with duplicate-key rejection and fails on unknown top-level fields. Every source import re-hashes the exact no-follow input descriptor after parsing and publishes completion only after all records commit.

Compact history and signing checks

Cold bundles use digest-derived names and are created without replacement. A reference binds exact bytes, source declaration, selected state, record/fact counts, coverage and attribution. Verification opens with O_NOFOLLOW, checks length and SHA-256 on the same descriptor, then parses bounded JSONL in required record-before-fact order. Audit export and retention checkpoint first verify every object. Retention signatures use the distinct argand-site-registry-audit-retention namespace, and their authenticated document always requires deletion_authorized: false; the software provides no deletion command.

Runtime compaction occurs only after all bundles are durable and verified. It uses an indexed exact fact keep-set, commits a deterministic runtime copy, restores foreign keys, then runs integrity and foreign-key checks before COMPLETE.json is created. Readers authenticate and copy SQLite into a private unlinked file before opening immutable mode, preserving the existing sidecar and post-open mutation defenses.

Automated and manual evidence

  • Workspace formatting, locked all-target checks, Clippy with warnings denied, strict rustdoc, Rust/Python tests, consumer parity, reproducible source packaging and extracted-source acceptance are release gates.
  • cargo audit --deny warnings loaded 1,243 RustSec advisories and reported no finding across 277 locked dependency nodes.
  • The repository and diff contain no private key, token, password, dataset, production review, or generated registry. SQL values remain parameterized; dynamic SQL fragments are fixed internal table/predicate vocabularies.
  • Adversarial tests cover duplicate JSON keys/records, schema/member drift, symlinks, truncation/substitution, oversized records, malformed coverage, process termination at multiple checkpoints, SQLite growth exhaustion, incomplete update/publication, receipt bundle omission, signature tampering, reviewer/publisher separation, revocation continuity and rollback.

Residual operator responsibilities

Protect cache, writer and audit directories from untrusted local writers and replicate cold objects by exact digest. A deliberately configured 256 MiB record can require comparable memory; choose the smallest inspected limit and enforce process/storage quotas around provider jobs. Measure cold authenticated startup on deployment storage, because every registry open copies the full runtime database.

Obtain provider checksums and publisher/reviewer trust roots independently. A valid hash or signature authenticates bytes and a decision, not ownership, current site safety, commercial fitness, or reviewer competence. Continue malware/content, drift, expiry, revocation, backup/restore and incident controls before any public dataset or Argand route is activated.