nicweyand
e83f43d00f
All checks were successful
Standalone registry checks / check (push) Successful in 3m43s
95 lines
5.3 KiB
Markdown
95 lines
5.3 KiB
Markdown
# Source and dataset releases
|
|
|
|
## Build environment and CI
|
|
|
|
The two-crate workspace requires the normal Rust/C build toolchain, OpenSSH and
|
|
Python 3.11+ for checks and packaging. The initial extraction declares Rust 1.97
|
|
and was tested on Linux with Rust 1.98.1. Install rustfmt and Clippy alongside the
|
|
compiler. TLS dependencies may require CMake and Perl. Keep Cargo.lock tracked.
|
|
Run `cargo fetch --locked` once, then `bash scripts/check.sh` offline. Cargo's
|
|
[workspace inheritance](https://doc.rust-lang.org/cargo/reference/workspaces.html)
|
|
and [locked/offline options](https://doc.rust-lang.org/cargo/commands/cargo-test.html)
|
|
define the build behavior. A lockfile pins dependencies, not the host compiler.
|
|
|
|
The Forgejo workflow uses the documented [workflow and context syntax](https://forgejo.org/docs/latest/user/actions/reference/).
|
|
Register `site-registry-isolated` only on a disposable, repository-scoped runner
|
|
with the above tools, two build jobs and at least 4 GiB memory. Use a pinned,
|
|
reviewed runner image; the exact local build is in [ci/Dockerfile](../ci/Dockerfile).
|
|
Do not mount production directories, share signing keys,
|
|
or use Argand's host runners. Follow Forgejo's [runner security guidance](https://forgejo.org/docs/latest/user/actions/security/).
|
|
The label is a deployment requirement, not a provisioned runner supplied by this
|
|
repository. The workflow runs on trusted main pushes or manual dispatch, fetches
|
|
the exact public commit without credentials, runs offline gates, compares two
|
|
source archives and reruns acceptance on extracted source. It neither publishes
|
|
artifacts nor signs them. Review outside contributions before allowing them to
|
|
execute on infrastructure. No hosted CI pass is claimed by a local test run.
|
|
|
|
## Deterministic source archive
|
|
|
|
From a reviewed, clean committed tree, choose a new output directory outside the
|
|
checkout (its parent must exist):
|
|
|
|
```bash
|
|
python3 scripts/source_release.py create --output /data/releases/site-registry-source
|
|
```
|
|
|
|
The command emits the full receipt `pin` and creates `source.tar.gz` plus
|
|
`RELEASE.json`. The archive contains only allowed tracked source files from the
|
|
exact commit, including both crates, lockfile and license texts. Tar metadata and
|
|
gzip timestamps are fixed; identical source commits and Python/zlib packaging
|
|
versions produce identical bytes. The receipt records those packaging versions.
|
|
Links, submodules, datasets, private-key extensions and unsafe paths are refused.
|
|
The complete source tree is limited to 2,000 files and 32 MiB; review these limits
|
|
before expanding them. The allowlist is not a secret scanner: review source contents
|
|
for embedded credentials and unrelated material before signing or publication.
|
|
|
|
The receipt records the Git commit/tree and each path's hash, size and mode.
|
|
It is written last and fsynced with its directory. Existing output is never
|
|
overwritten. An interrupted directory without a valid receipt is incomplete;
|
|
inspect it and choose a fresh output path. The verifier bounds decompression,
|
|
rejects unexpected/missing/duplicate files, and checks every member before any
|
|
extraction. It accepts an external receipt pin; a hash found in the same download
|
|
is insufficient authentication.
|
|
|
|
## Signing and consuming source
|
|
|
|
After all checks and source review, sign with an operator-controlled SSH key:
|
|
|
|
```bash
|
|
ssh-keygen -Y sign -n argand-site-registry-source -f /secure/source-signing-key \
|
|
/data/releases/site-registry-source/RELEASE.json
|
|
```
|
|
|
|
Distribute the archive, receipt and `RELEASE.json.sig` together. Consumers obtain
|
|
the accepted identity and allowed-signers file through an independent trusted
|
|
channel, verify the signature, then compute the now-authenticated receipt pin:
|
|
|
|
```bash
|
|
ssh-keygen -Y verify -n argand-site-registry-source \
|
|
-f /secure/source-allowed-signers -I registry-source-publisher \
|
|
-s /data/releases/site-registry-source/RELEASE.json.sig \
|
|
< /data/releases/site-registry-source/RELEASE.json
|
|
sha256sum /data/releases/site-registry-source/RELEASE.json
|
|
python3 scripts/source_release.py verify \
|
|
--release /data/releases/site-registry-source --pin "$REGISTRY_TRUSTED_PIN"
|
|
```
|
|
|
|
Set `REGISTRY_TRUSTED_PIN` to that authenticated receipt hash. Use a verifier you
|
|
already trust, not one extracted from an unchecked archive. After verification,
|
|
extract into an empty directory and run `cargo fetch --locked` and
|
|
`bash scripts/check.sh`. Preserve the receipt with the installation. A source
|
|
signature authenticates reviewed source bytes, not an arbitrary executable built
|
|
elsewhere. Binary distribution needs its own artifact hash and build-environment
|
|
receipt; the initial release tool packages source only.
|
|
|
|
## Dataset publication
|
|
|
|
Source releases contain no provider datasets or real approvals. Dataset publishers
|
|
follow the operator guide: import, inspect, review, build, diff, sign and activate.
|
|
Follow the [publisher runbook](PUBLISHING.md); release signing requires the external
|
|
reviewer trust file and re-verifies every stored decision signature.
|
|
The dataset namespace `argand-site-registry` is distinct from the source namespace
|
|
above. Keep `LICENSE_SOURCES.md` and `ATTRIBUTION.json` with the database and receipt.
|
|
The weekly update example creates candidates. It never approves, renews, signs or
|
|
activates them. Record actual acquisition cost and dataset size separately from
|
|
small-fixture test results. Follow SECURITY.md for incidents and revocation delivery.
|