feat: establish standalone Argand Site Registry
This commit is contained in:
commit
2a0fe1714b
60 changed files with 10494 additions and 0 deletions
57
docs/CONSUMERS.md
Normal file
57
docs/CONSUMERS.md
Normal file
|
|
@ -0,0 +1,57 @@
|
|||
# Consumer and compatibility contract
|
||||
|
||||
## Rust library
|
||||
|
||||
Use `argand_site_registry::query::Registry::open(generation, trusted_pin)` once per
|
||||
immutable generation and reuse the reader. `lookup(query, limit)` returns evidence
|
||||
and complete ambiguity counts; `resolve(query, locale, country, now)` returns an
|
||||
optional reviewed candidate. Check the compiled example and API docs for exact
|
||||
types. `selection_context` binds the full alternative set for downstream query
|
||||
review. Preserve the returned provenance, scopes, counts and attribution.
|
||||
|
||||
For local integration, point a Cargo dependency at
|
||||
`crates/argand-site-registry` inside an extracted standalone source tree. Once an
|
||||
upstream repository is published, use its actual Git URL and a full reviewed `rev`
|
||||
pin. Do not invent a crates.io version or track a mutable branch in production.
|
||||
Both crates remain in this workspace; the atomic helper is a relative dependency.
|
||||
|
||||
## CLI and other languages
|
||||
|
||||
`lookup`, `resolve`, `verify` and the other commands emit JSON. The Python example
|
||||
passes arguments directly to the native executable, preserving query text and the
|
||||
entire response. A nonzero exit is an error. `destination: null` is a successful
|
||||
abstention, not a request to pick the first lookup candidate. Render source names
|
||||
as untrusted text and satisfy their source-specific attribution requirements.
|
||||
|
||||
Use a bounded process or service wrapper appropriate to your workload. The Python
|
||||
example has a 60-second timeout and invokes a fresh reader per request; for repeated
|
||||
low-latency queries, use the reusable Rust reader. No hosted API or Python package
|
||||
registry publication is claimed by this repository.
|
||||
|
||||
## SQLite, JSONL and license scope
|
||||
|
||||
Distribute `registry.sqlite`, `COMPLETE.json`, `LICENSE_SOURCES.md` and
|
||||
`ATTRIBUTION.json` together, plus the publisher signature when applicable. The
|
||||
database contains audit records and source descriptions. Default JSONL export
|
||||
omits descriptions and includes fact provenance plus an attribution envelope.
|
||||
It is an assertion export, not a self-contained signed list of admitted routes.
|
||||
Raw SQL inspection is useful for audit; it does not implement resolution policy.
|
||||
|
||||
Code version 0.1.0 is an initial interface. Schema/rule contracts are versioned
|
||||
independently in receipts. Unsupported contracts fail closed. Pin source releases,
|
||||
compile consumers and replay fixed fixtures before upgrades. Preserve import and
|
||||
review history; never mutate complete generations to migrate them.
|
||||
|
||||
## Argand transition
|
||||
|
||||
UPSTREAM.json pins the exact Argand source baseline, including the beta agent's
|
||||
selection-context API. The first standalone extraction preserves runtime Rust and
|
||||
migration bytes. Argand currently consumes its embedded workspace crate; this
|
||||
package does not silently redirect that dependency.
|
||||
|
||||
At cutover, coordinate with the Argand source/build owner, compare both trees with
|
||||
the recorded baseline, carry any subsequent fixes forward, replace the embedded
|
||||
dependency with a reviewed standalone revision, and run Argand's complete engine
|
||||
and navigation-compiler gates. Preserve existing registry receipts and public
|
||||
navigation admission. After cutover, develop the library upstream and update
|
||||
Argand through explicit pinned dependency changes, avoiding permanent dual copies.
|
||||
13
docs/INDEX.md
Normal file
13
docs/INDEX.md
Normal file
|
|
@ -0,0 +1,13 @@
|
|||
# Documentation
|
||||
|
||||
- [README](../README.md): build, examples and scope.
|
||||
- [Operator guide](../crates/argand-site-registry/README.md): all source commands,
|
||||
schema, reviews, regional resolution, releases and update configuration.
|
||||
- [Source licenses](../crates/argand-site-registry/LICENSE_SOURCES.md): exact terms and attribution.
|
||||
- [Consumers](CONSUMERS.md): Rust, Python/CLI, data distribution and Argand transition.
|
||||
- [Trust](TRUST.md): enforced checks and publisher/consumer responsibilities.
|
||||
- [Releasing](RELEASING.md): CI, source signing and archive verification.
|
||||
- [Contributing](../CONTRIBUTING.md), [governance](../GOVERNANCE.md),
|
||||
[security](../SECURITY.md): proposals, decisions and incidents.
|
||||
- [Extraction design](superpowers/specs/2026-09-12-standalone-design.md) and
|
||||
[implementation plan](superpowers/plans/2026-09-12-standalone.md).
|
||||
92
docs/RELEASING.md
Normal file
92
docs/RELEASING.md
Normal file
|
|
@ -0,0 +1,92 @@
|
|||
# Source and dataset releases
|
||||
|
||||
## Build environment and CI
|
||||
|
||||
The two-crate workspace requires the normal Rust/C build toolchain, OpenSSH and
|
||||
Python 3.11+ for checks and packaging. The initial extraction declares Rust 1.97
|
||||
and was tested on Linux with Rust 1.98.1. Install rustfmt and Clippy alongside the
|
||||
compiler. TLS dependencies may require CMake and Perl. Keep Cargo.lock tracked.
|
||||
Run `cargo fetch --locked` once, then `bash scripts/check.sh` offline. Cargo's
|
||||
[workspace inheritance](https://doc.rust-lang.org/cargo/reference/workspaces.html)
|
||||
and [locked/offline options](https://doc.rust-lang.org/cargo/commands/cargo-test.html)
|
||||
define the build behavior. A lockfile pins dependencies, not the host compiler.
|
||||
|
||||
The Forgejo workflow uses the documented [workflow and context syntax](https://forgejo.org/docs/latest/user/actions/reference/).
|
||||
Register `site-registry-isolated` only on a disposable, repository-scoped runner
|
||||
with the above tools, two build jobs and at least 4 GiB memory. Use a pinned,
|
||||
reviewed runner image. Do not mount production directories, share signing keys,
|
||||
or use Argand's host runners. Follow Forgejo's [runner security guidance](https://forgejo.org/docs/latest/user/actions/security/).
|
||||
The label is a deployment requirement, not a provisioned runner supplied by this
|
||||
repository. The workflow runs on trusted main pushes or manual dispatch, fetches
|
||||
the exact public commit without credentials, runs offline gates, compares two
|
||||
source archives and reruns acceptance on extracted source. It neither publishes
|
||||
artifacts nor signs them. Review outside contributions before allowing them to
|
||||
execute on infrastructure. No hosted CI pass is claimed by a local test run.
|
||||
|
||||
## Deterministic source archive
|
||||
|
||||
From a reviewed, clean committed tree, choose a new output directory outside the
|
||||
checkout (its parent must exist):
|
||||
|
||||
```bash
|
||||
python3 scripts/source_release.py create --output /data/releases/site-registry-source
|
||||
```
|
||||
|
||||
The command emits the full receipt `pin` and creates `source.tar.gz` plus
|
||||
`RELEASE.json`. The archive contains only allowed tracked source files from the
|
||||
exact commit, including both crates, lockfile and license texts. Tar metadata and
|
||||
gzip timestamps are fixed; identical source commits and Python/zlib packaging
|
||||
versions produce identical bytes. The receipt records those packaging versions.
|
||||
Links, submodules, datasets, private-key extensions and unsafe paths are refused.
|
||||
The complete source tree is limited to 2,000 files and 32 MiB; review these limits
|
||||
before expanding them. The allowlist is not a secret scanner: review source contents
|
||||
for embedded credentials and unrelated material before signing or publication.
|
||||
|
||||
The receipt records the Git commit/tree and each path's hash, size and mode.
|
||||
It is written last and fsynced with its directory. Existing output is never
|
||||
overwritten. An interrupted directory without a valid receipt is incomplete;
|
||||
inspect it and choose a fresh output path. The verifier bounds decompression,
|
||||
rejects unexpected/missing/duplicate files, and checks every member before any
|
||||
extraction. It accepts an external receipt pin; a hash found in the same download
|
||||
is insufficient authentication.
|
||||
|
||||
## Signing and consuming source
|
||||
|
||||
After all checks and source review, sign with an operator-controlled SSH key:
|
||||
|
||||
```bash
|
||||
ssh-keygen -Y sign -n argand-site-registry-source -f /secure/source-signing-key \
|
||||
/data/releases/site-registry-source/RELEASE.json
|
||||
```
|
||||
|
||||
Distribute the archive, receipt and `RELEASE.json.sig` together. Consumers obtain
|
||||
the accepted identity and allowed-signers file through an independent trusted
|
||||
channel, verify the signature, then compute the now-authenticated receipt pin:
|
||||
|
||||
```bash
|
||||
ssh-keygen -Y verify -n argand-site-registry-source \
|
||||
-f /secure/source-allowed-signers -I registry-source-publisher \
|
||||
-s /data/releases/site-registry-source/RELEASE.json.sig \
|
||||
< /data/releases/site-registry-source/RELEASE.json
|
||||
sha256sum /data/releases/site-registry-source/RELEASE.json
|
||||
python3 scripts/source_release.py verify \
|
||||
--release /data/releases/site-registry-source --pin "$REGISTRY_TRUSTED_PIN"
|
||||
```
|
||||
|
||||
Set `REGISTRY_TRUSTED_PIN` to that authenticated receipt hash. Use a verifier you
|
||||
already trust, not one extracted from an unchecked archive. After verification,
|
||||
extract into an empty directory and run `cargo fetch --locked` and
|
||||
`bash scripts/check.sh`. Preserve the receipt with the installation. A source
|
||||
signature authenticates reviewed source bytes, not an arbitrary executable built
|
||||
elsewhere. Binary distribution needs its own artifact hash and build-environment
|
||||
receipt; the initial release tool packages source only.
|
||||
|
||||
## Dataset publication
|
||||
|
||||
Source releases contain no provider datasets or real approvals. Dataset publishers
|
||||
follow the operator guide: import, inspect, review, build, diff, sign and activate.
|
||||
The dataset namespace `argand-site-registry` is distinct from the source namespace
|
||||
above. Keep `LICENSE_SOURCES.md` and `ATTRIBUTION.json` with the database and receipt.
|
||||
The weekly update example creates candidates. It never approves, renews, signs or
|
||||
activates them. Record actual acquisition cost and dataset size separately from
|
||||
small-fixture test results. Follow SECURITY.md for incidents and revocation delivery.
|
||||
63
docs/TRUST.md
Normal file
63
docs/TRUST.md
Normal file
|
|
@ -0,0 +1,63 @@
|
|||
# Trust and evidence policy
|
||||
|
||||
## What the implementation enforces
|
||||
|
||||
Inputs use reviewed source adapters and explicit manifests. URL normalization,
|
||||
PSL parsing and stable identities are deterministic. Fact provenance, conflicting
|
||||
evidence and source-specific popularity remain separate. Names and similar
|
||||
hostnames never silently join entities. Invalid URLs and malformed imports fail
|
||||
validation; a partial import does not replace a complete source selection.
|
||||
|
||||
Destination and identity decisions bind exact evidence fingerprints, including
|
||||
names, assertions and normalization context. Reviews expire within 90 days.
|
||||
Changed evidence invalidates earlier approvals. Resolution abstains on ambiguity,
|
||||
ties, missing approval or ineligible claims; regional scopes must explicitly
|
||||
match. The local writer owns the append-only review log.
|
||||
|
||||
Generations bind the database, license document and attribution to a completion
|
||||
receipt. Consumers provide a trusted hash or verify an external publisher key.
|
||||
Activation checks signatures and refuses rollback that loses distributed
|
||||
revocations. Updates build candidates and cannot approve, sign or activate them.
|
||||
|
||||
## What a publisher must establish
|
||||
|
||||
The reviewer name and evidence locator in a decision are operator assertions.
|
||||
The CLI validates their structure and evidence binding; it does not independently
|
||||
authenticate the reviewer, retrieve their evidence or prove website ownership.
|
||||
Protect the writer database and signing key with separate operating permissions.
|
||||
Restrict who can author decisions and require human review before release signing.
|
||||
|
||||
Publish dated evidence supporting the exact entity, URL, relationship, role and
|
||||
country/locale. Prefer independently corroborated primary evidence with immutable
|
||||
capture identifiers. Record contrary evidence and uncertainty. TLS, DNS control,
|
||||
registrable-domain spelling, redirects, `sameAs` or popularity alone cannot
|
||||
establish every identity or role claim. Future crawler observations remain inputs
|
||||
to review. Confidence values are assertion scores, not calibrated probabilities.
|
||||
|
||||
Choose expiry based on volatility, within the enforced maximum. Do not renew
|
||||
blindly on a timer. Expired approval should lead to abstention until evidence is
|
||||
reviewed. Disclose editorial conflicts and use an independent reviewer for a
|
||||
disputed claim when possible. The initial implementation is a local single-writer
|
||||
tool; it does not provide authenticated reviewer accounts or an enforced quorum.
|
||||
|
||||
## What consumers must preserve
|
||||
|
||||
Authenticate a release before opening it. Keep the full receipt pin and required
|
||||
attribution with caches and exports. Use `resolve` for reviewed destinations, keep
|
||||
null as abstention, and enforce application-specific malware/content/navigation
|
||||
policy separately. A verified signature authenticates the publisher, not the truth
|
||||
of every assertion. An official website may later be compromised.
|
||||
|
||||
Deliver revocations to every active consumer and derived catalogue, and bound cache
|
||||
lifetimes. The resolver checks review expiry at query time; a detached cached URL
|
||||
does not recheck itself. Preserve the current review history during rollback.
|
||||
Copying SQLite rows or the JSONL export into a second resolver can bypass these
|
||||
checks; use the native API or CLI for admission decisions.
|
||||
|
||||
## Community changes
|
||||
|
||||
Treat public submissions as untrusted evidence. Do not execute submitted content
|
||||
or copy review decisions into production automatically. Source allowlist and
|
||||
attribution changes need both implementation tests and documented rights review.
|
||||
Reject unknown sources until those checks are complete. Every publisher may apply
|
||||
stricter admission rules, and must state its actual review and incident procedures.
|
||||
37
docs/superpowers/plans/2026-09-12-standalone.md
Normal file
37
docs/superpowers/plans/2026-09-12-standalone.md
Normal file
|
|
@ -0,0 +1,37 @@
|
|||
# Standalone Site Registry Implementation Plan
|
||||
|
||||
> **For agentic workers:** Use superpowers:executing-plans inline. No subagents or Argand worktrees.
|
||||
|
||||
**Goal:** Produce a reusable standalone repository and verifiable source release.
|
||||
|
||||
**Architecture:** Retain the registry and atomic helper as two workspace crates.
|
||||
Preserve runtime bytes and wire packaging, consumer examples and release gates
|
||||
around their existing APIs. Use an isolated build directory.
|
||||
|
||||
**Tech Stack:** Rust, SQLite, Python 3.11 standard library tooling, OpenSSH, Forgejo Actions.
|
||||
|
||||
### Task 1: Extract the verified baseline
|
||||
- [x] Verify signed Argand commit 47911062b00d87f215ba61c41965faf8a7f4b7f7.
|
||||
- [x] Export only `engine/crates/argand-site-registry` and `engine/crates/argand-atomic`.
|
||||
- [x] Write `UPSTREAM.json` with original path/blob/hash evidence and a two-crate `Cargo.toml`.
|
||||
- [ ] Prune the inherited lock with `cargo metadata --offline --format-version 1`; retain exact dependency versions.
|
||||
|
||||
### Task 2: Make standalone use and trust policy concrete
|
||||
- [ ] Add root README, LICENSE, source-license entrypoint, CONTRIBUTING, SECURITY and governance docs.
|
||||
- [ ] Add Rust `examples/lookup.rs` and Python `examples/lookup.py` consumers of existing query contracts.
|
||||
- [ ] Run both consumers on the native fixture and compare full lookup outputs, including attribution.
|
||||
|
||||
### Task 3: Package and verify source releases
|
||||
- [ ] Add deterministic source archive and verification commands in `scripts/source_release.py`.
|
||||
- [ ] Test determinism, dirty-tree refusal, no-clobber, tampering and unsafe archive members in `tests/test_source_release.py`.
|
||||
- [ ] Add `scripts/check.sh` and `.forgejo/workflows/ci.yml` using an isolated runner without release secrets.
|
||||
- [ ] Document explicit release signing, downstream pins and incident response in `docs/RELEASING.md` and `docs/TRUST.md`.
|
||||
|
||||
### Task 4: Validate and land the bounded task
|
||||
- [ ] Run `cargo fmt --all -- --check`, `cargo check --workspace --all-targets --locked --offline`,
|
||||
`cargo clippy --workspace --all-targets --locked --offline -- -D warnings`,
|
||||
`cargo test --workspace --locked --offline` and strict `cargo doc`.
|
||||
- [ ] Build the native executable and retain the all-five-source CLI fixture outside the repo.
|
||||
- [ ] Rebuild/run tests from the release archive outside the original workspace.
|
||||
- [ ] Review every initial tracked file and dependency change, sign the local commit and release receipt.
|
||||
- [ ] Record exact validation, publication and Argand cutover state; release any coordination window.
|
||||
37
docs/superpowers/specs/2026-09-12-standalone-design.md
Normal file
37
docs/superpowers/specs/2026-09-12-standalone-design.md
Normal file
|
|
@ -0,0 +1,37 @@
|
|||
# Standalone Site Registry design
|
||||
|
||||
The user approved continuing standalone packaging, release CI and public governance
|
||||
on 2026-09-12. Preserve the existing AGPL-3.0-or-later license and all data terms.
|
||||
|
||||
Create a separate two-crate workspace containing the current registry and atomic
|
||||
file helper. Preserve Rust module boundaries and the existing source adapters,
|
||||
SQLite schema, receipt contracts, explicit reviews and regional resolution.
|
||||
Record the exact signed Argand baseline and original file hashes in UPSTREAM.json.
|
||||
Do not copy Argand history, deployment configuration, datasets or signing keys.
|
||||
The existing Argand checkout and shared build caches remain owned by the beta agent.
|
||||
|
||||
A separate package is preferable to a permanent second implementation or a new
|
||||
hosted service: it reuses the existing contracts and runs locally. Extracting only
|
||||
the CLI would lose the reusable Rust API. Preserve both CLI and library, and prove
|
||||
non-Rust consumption through the JSON CLI without creating a second resolver.
|
||||
|
||||
Ship a focused Cargo.lock, complete code license, beginner quickstart, native Rust
|
||||
and Python examples, contribution requirements, trust policy and incident/revocation
|
||||
procedure. Source releases come from an exact committed tree, with deterministic
|
||||
archives and hash receipts; signing is an explicit local release action. CI checks
|
||||
formatting, all targets, strict lints, tests, documentation and a native all-source
|
||||
fixture, and exercises the archive outside its Git checkout. No CI signing key,
|
||||
provider credentials, production runner or automatic dataset promotion is included.
|
||||
|
||||
Keep Argand's current consumer operational. Until an independently hosted release
|
||||
is accepted into Argand, its embedded copy remains the active consumer; document
|
||||
the cutover and verify baseline parity. The initial standalone package has no
|
||||
runtime behavior changes. Each dataset publisher chooses its own external trust
|
||||
roots; review policy is public and cannot be replaced by popularity or hostname
|
||||
similarity. An authenticated release is not proof of source accuracy or malware safety.
|
||||
|
||||
Acceptance: preserve upstream Rust/schema bytes; build with only this workspace;
|
||||
run the complete two-crate offline gates plus fresh CLI and Rust/Python examples;
|
||||
produce the same source archive twice; reject modified releases, unsafe paths and
|
||||
release overwrites; inspect the initial Git inventory for private/unrelated content.
|
||||
Remote publication and runner activation are separate from local package readiness.
|
||||
Loading…
Add table
Add a link
Reference in a new issue