Rosa Starter Kit
================

This bundle lets your own AI assistant (ChatGPT, Claude, or any MCP client) run
your data through Rosa and validate the results itself. You have a Rosa account
and an API key; your portal's API / MCP page has the endpoint.

Start here
----------
1. Give "Rosa-BYO-LLM-Guide.txt" to your AI assistant as context. It explains
   how to connect to Rosa, describe a dataset, read the results, and validate
   them. Everything else in this kit is referenced from that guide.
2. "Rosa-Glossary.pdf" defines every term Rosa emits, for a human to read.

What is in here
---------------
  Rosa-BYO-LLM-Guide.txt    the guide (machine-readable) - start here
  Rosa-Glossary.pdf         definitions of every term Rosa emits (for a human)
  VERSION.txt               this kit's version + release date
  MANIFEST.sha256           SHA-256 of every file in the kit
  Test 1/                   the Test 1 worked example
  COMPAS/                   the COMPAS worked example (ProPublica two-year cohort)
  Synthetic/                build + score your own known-answer dataset
  EVIDENCE/                 the versioned benchmark evidence (replay any published band)

Each subfolder holds its datasets, Rosa configs, and its reproduction script
together, so a script runs in place from inside its own folder (its default file
paths point at its siblings). All scripts are standard Python + scikit-learn and
none import Rosa - you can read exactly what they do.

- Test 1/       test1-population.csv, test1.csv, test1-inference.csv, the two
                test1-config-*.json, and reproduce-test1-downstream.py.
- COMPAS/       compas-two-year-train.csv, compas-two-year-test.csv, the two
                compas-two-year-config-*.json, and reproduce-compas-downstream.py.
                This is ProPublica's two-year recidivism cohort (group-aware
                50/50 split by defendant, seed 42, ~3,075 rows each side) - the
                honest, weak-signal hard case: Rosa removes most of the racial
                disparity, and because the protected attribute carried much of
                the model's predictive power on this data, it does so at a
                visible accuracy cost, which the reproduction script reports.
- Synthetic/    synthetic_test_generator.py (build a dataset with a KNOWN fair
                answer) and reproduce-synthetic-downstream.py (score Models
                A/B/C against it).
- EVIDENCE/     benchmark-evidence.json (the exact engine version, container
                digest, every job ID and per-run value - 10 for Test 1,
                82 for COMPAS - and the training input + config hashes
                behind each published figure), the
                reproduce environment lock, the COMPAS source-to-split provenance
                chain (including the runnable build_compas_benchmark.py that
                rebuilds the splits from the raw ProPublica source), the Test 1
                dataset provenance, and data-checks/ (Rosa's own
                Data Checker findings for each shipped dataset plus a declared
                datasheet). This is what lets a third party replay every published
                band - and see the data anomalies Rosa flagged - from this kit alone.
                EVIDENCE/interpreting-results.md also carries the RESIDUAL-RISK
                STATEMENT: what a "no recoverable bias" result does and does not
                establish. Read it before you rely on one.
                EVIDENCE/validate-your-own-run.md is the companion for a job you
                ran on YOUR data: six checks covering manifest hashes, row
                identity, pass-through integrity, lineage, downstream utility and
                an external recovery probe. Run its first three before you trust a
                Rosa output in a pipeline.

Not in this kit
---------------
The Rosa Security and Compliance Guide - the control-by-control statement of what
is evidenced today, what is on the roadmap, and how customer data is handled - is
NOT bundled here. It is supplied on request; ask whoever gave you your trial. This
kit is the technical evidence pack; that document is the security and compliance
one, and it is a deliberate split rather than an omission.

Verify this download
--------------------
Every file in the kit is listed with its SHA-256 in MANIFEST.sha256 (standard
sha256sum format). To confirm nothing was altered in transit, from inside the
extracted folder run:

    cd Rosa-Starter-Kit
    sha256sum -c MANIFEST.sha256

Every line should report "OK". (The hashes recorded inside
EVIDENCE/benchmark-evidence.json carry a "sha256:" prefix instead - those are
Rosa's Run-Manifest hashes, written to match what your own Rosa run reports.)

A note on reproducibility
-------------------------
Rosa's engine is stochastic and exposes no training seed on purpose, so a single
run varies; the published figures are N-run means with a range, stamped to a
Rosa version. Run it several times and take the mean. Section 5 of the guide, the
script headers, and EVIDENCE/benchmark-evidence.json explain this.

A suggested plan for the trial
------------------------------
The trial gives you 20 jobs a month, renewing each month, so the question is not
how to ration one budget but what to learn in what order.

Month one, 15 jobs: learn the instrument on known answers. Test 1, COMPAS and
Binary synthetic, five jobs each: one diagnose, then two training-and-inference
pairs. A training job runs the detection test first, so a second diagnose on a
file you have already diagnosed tells you nothing new. The five spare absorb a
retry or a second look at whatever surprised you. COMPAS is the honest hard case:
weak-signal real data where the fairness gaps shrink sharply and the model's
ranking power falls towards the majority baseline, at about a 14-point accuracy
cost. Most vendors do not show you this dataset.

Month two: find out whether Rosa fits your data, before you send any. Read your
own geometry off section 3 of the guide; generate synthetic data shaped like
yours with the bundled generator; run a multi-class characteristic above and
below the line; then diagnose your real data.

Month three: scope the proof of concept. Your own data, the characteristics you
have to evidence, and the edge cases you already know are coming.

Two pairs is a demonstration, not a measurement. Rosa is stochastic and exposes
no seed, so our published figures are multi-run means with a range.

What this plan cannot tell you, stated plainly. A few pairs is a targeted
evidence budget, not a certification campaign. Our own testing needed twenty
pairs per arm to detect a large difference between two configurations and could
not distinguish twenty passes in twenty from seventeen at that size. Read three
consistent pairs on your own data as encouraging; read one multi-class pair as a
smoke test; and do not read two failures as a verdict on the product - on some
datasets the downstream ordering fails repeatedly with nothing wrong, because the
utility comparison is closest exactly where the raw data carried the least bias
to begin with. Before you spend a job on your own file, run the Data Checker and
note the row count of your smallest protected group: if it is very small, expect
a decline and read section 4 of the guide before concluding anything from it.
