feat: establish standalone Argand Site Registry
This commit is contained in:
commit
2a0fe1714b
60 changed files with 10494 additions and 0 deletions
60
crates/argand-site-registry/LICENSE_SOURCES.md
Normal file
60
crates/argand-site-registry/LICENSE_SOURCES.md
Normal file
|
|
@ -0,0 +1,60 @@
|
|||
# Site Registry source licenses
|
||||
|
||||
Reviewed against the primary distribution and licensing pages on 2026-09-12.
|
||||
The Rust code uses the workspace's **AGPL-3.0-or-later** license. Imported data keeps
|
||||
its own licenses; neither the code license nor a merged export relicenses it.
|
||||
Commercial reuse is supported subject to the following obligations. A provider's
|
||||
listing is evidence of an assertion, not a guarantee of ownership or safety.
|
||||
|
||||
| Source | Exact data license and evidence | Distribution and consumed fields |
|
||||
| --- | --- | --- |
|
||||
| Wikidata | [CC0 1.0 Universal](https://creativecommons.org/publicdomain/zero/1.0/), [`CC0-1.0`](https://www.wikidata.org/wiki/Wikidata:Licensing) | [JSON dumps](https://www.wikidata.org/wiki/Wikidata:Database_download), [entity JSON](https://www.wikidata.org/wiki/Special:EntityData/Q355.json). Entity ID, revision, all labels/aliases, full P856 statements including ranks, qualifiers and references; P17, P159, P407, P1001 and country/language code mappings P297/P218/P219/P220. Relevant raw entity records remain available for audit. |
|
||||
| Majestic Million | [CC BY 3.0 Unported](https://creativecommons.org/licenses/by/3.0/), [`CC-BY-3.0`](https://majestic.com/reports/majestic-million) | [Official CSV](https://downloads.majestic.com/majestic_million.csv). Domain/IDN/TLD, global/TLD ranks, referring subnet/IP counts and their previous values. Ranks remain source-specific signals; no entity ownership is inferred. |
|
||||
| Chrome UX Report (CrUX), Google | [CC BY 4.0 International](https://creativecommons.org/licenses/by/4.0/), [`CC-BY-4.0`](https://developer.chrome.com/docs/crux/methodology) | [Monthly BigQuery dataset](https://developer.chrome.com/docs/crux/bigquery/): `origin`, `experimental.popularity.rank`, observation month, optional audience-country dataset code. The adapter produces `origin,rank,yyyymm,country_code` CSV. Rank is a coarse bucket, not a precise visit count. Audience country is not website jurisdiction. No API key or OAuth token is retained. |
|
||||
| Curlie | [CC BY 3.0 Unported](https://creativecommons.org/licenses/by/3.0/), [`CC-BY-3.0`](https://curlie.org/docs/en/license.html), including the attribution placement prescribed on that page | [Format documentation](https://curlie.org/docs/en/rdf.html), [official download redirect](https://curlie.org/directory-dl), currently [Passau-hosted archive](https://share.innkube.fim.uni-passau.de/curlie-rdf/curlie-rdf-all.tar.gz). Despite its RDF name, the current archive contains **literal TSV**. Content: URL, title, description, category ID. Structure: category ID, full category path, entry count, description, latitude, longitude. Archive notices are retained. |
|
||||
| Public Suffix List contributors | [Mozilla Public License 2.0](https://mozilla.org/MPL/2.0/), [`MPL-2.0`](https://publicsuffix.org/list/public_suffix_list.dat) | [Official list](https://publicsuffix.org/list/public_suffix_list.dat). All ICANN and PRIVATE rules, wildcard/exception rules, version/commit comments and notices. Used for hostname, registrable-domain and public-suffix derivations. Download at most once per day. |
|
||||
|
||||
## Attribution and distribution
|
||||
|
||||
* **Wikidata:** CC0 imposes no attribution condition. Keep Wikidata IDs, source
|
||||
links and revision evidence for traceability. CC0 structured data does not
|
||||
extend to unrelated Wikipedia prose, images or linked websites.
|
||||
* **Majestic:** credit “Majestic Million, Majestic”, link the source and CC BY 3.0,
|
||||
retain supplied notices, and identify Argand's changes. The current distribution
|
||||
page governs this import; older blog posts describe different historic terms.
|
||||
* **CrUX:** credit “Chrome UX Report, Google”, link the source and CC BY 4.0,
|
||||
retain notices and indicate the projection/normalization. BigQuery access and
|
||||
billing are separate from the data license. This adapter requires an explicit
|
||||
project and positive maximum-bytes-billed limit.
|
||||
* **Curlie:** the requirement applies to **names, categories and descriptions**.
|
||||
Every public page using Curlie content must include its prescribed HTML credit:
|
||||
|
||||
```html
|
||||
<div title="Curlie Directory Attribution">
|
||||
With content from <a style="text-decoration:none; color:#cd4932;" href="https://curlie.org/" title="Curlie - the largest human-edited directory of the web">Curlie.org</a> - the largest human-edited directory of the web. Contribute by submitting a website or becoming an editor.
|
||||
</div>
|
||||
```
|
||||
|
||||
Also retain the source and license links and indicate modifications. Imports
|
||||
retain descriptions as audit data. Lookup/resolve never expose descriptions;
|
||||
JSONL export omits them by default. `--include-descriptions` is an explicit
|
||||
distribution choice: receiving applications must satisfy these obligations
|
||||
before displaying the text. Treat directory descriptions as untrusted text,
|
||||
never as executable HTML. A JSON attribution object alone does not satisfy
|
||||
Curlie's public-page placement requirement.
|
||||
* **PSL:** retain the list's notices, MPL license and access to its source form
|
||||
when redistributing it. Generations retain the unmodified list in `records`
|
||||
and `facts`; exports include the PSL fact and original download locator.
|
||||
Changes to covered PSL source files must remain available under MPL 2.0.
|
||||
The Rust `publicsuffix` parser is MIT/Apache-2.0; that is separate from the list.
|
||||
|
||||
Every generation contains this document and `ATTRIBUTION.json`, both hash-bound
|
||||
by its signed completion receipt. Exports carry source manifests, licenses,
|
||||
retrieval times, assertion selectors and confidence, plus a required attribution
|
||||
envelope. Consumers must preserve these when extracting subsets or redistributing
|
||||
derived facts. Source credits do not imply endorsement by any provider.
|
||||
|
||||
Cloudflare Radar, default Tranco, Cisco Umbrella, arbitrary mirrors and any other
|
||||
unreviewed source are not supported. Adding a source requires verified commercial
|
||||
reuse rights, its own adapter, provenance and attribution policy. A source format
|
||||
or distribution-host change fails validation until the adapter is reviewed.
|
||||
Loading…
Add table
Add a link
Reference in a new issue