Test the ranking on your own terms.

Run it yourself

Four quarters, scored by the published evaluator.

Out-of-sample rankings from the SEC-only model for 2025-06-30 to 2026-03-31: 28,092 candidate rows across 55 managers, with the realised outcome and a peer-count baseline. Identifiers are one-way hashes, so the sample measures the method without publishing holdings.

curl -O https://priormoves.com/data/sec-eval/predictions.csv
curl -O https://priormoves.com/data/sec-eval/outcomes.csv
curl -O https://priormoves.com/data/sec-eval/pm_evaluate.py
python3 pm_evaluate.py predictions.csv outcomes.csv \
  --sha256 a770c66a9365ca341b4e2e368111c9142933735b2cabbc4efc6aadf1370f7cd5

Timestamps in this historical sample are simulated: a prediction is dated 45 days after its quarter, when its inputs were last filed, and the outcome 45 days after the next quarter. The same model configuration was evaluated on these quarters before, so this is a reproduction, not a fresh holdout.

Controls

Shuffled labels and simple baselines.

Over 25 quarters (2020-03-31 to 2026-03-31) the SEC-only model averages 0.6635 within-manager AUC. Refitting it 5 times on labels shuffled inside each manager and quarter gives 0.5365 on average (best 0.5583); random scores give 0.5143. The real model beats every shuffle in 18 of 25 quarters. The strongest single input, how many other managers hold the issuer, scores 0.6368 alone, and the model beats the best single input in 15 of 25 quarters. Most of the ranking is peer popularity; the model adds a modest, inconsistent margin over it.

A shuffled-label control rules out skill produced by manager identity, base rates or fold structure. It cannot detect leakage in which rows exist. Source: runs/gates/q25_sec_shuffle_and_baselines.json.

Model cards

Each model, with its own target and evidence.

These measure different targets on different populations and are not interchangeable. The US figure is an upper estimate: survivors carry +0.051 AUC per investor, paired, on scored names (28 of 33), a lower bound.

How much to trust these numbers

Every figure, with the tries behind it.

The project has registered 513 hypotheses and settled 498 of them, failures included. The 15-name backtest was chosen among 33 configurations of the same backtest (names per quarter, filing lag, leak-free variants). Corrected for that many tries, its deflated Sharpe ratio is 0.339: the probability that its Sharpe ratio of 0.444 a year beats what the luckiest of 33 skill-free configurations would show. Uncorrected, the probability that it beats zero is 0.957. Telling skill from that much choice needs 22.7 years of quarters; the backtest has 8.8.

• Delistings: 0 of 525 backtest picks lacked a price over their holding quarter, so none left the average that way. The open risk sits upstream: names without a row in data/tickers/cusip_to_ticker.parquet never enter the ranking; if that map was built from today's listings, names that later delisted are missing from early quarters.
• Survivorship in the ranking: survivors score higher per manager (28 of 33), so the US figure is an upper estimate.
• Probability of backtest overfitting: not computed yet. CSCV needs a per-quarter return series for every configuration; the configuration files on disk hold aggregates only.

Method: Bailey and López de Prado (2014). Source: runs/gates/selection_adjust.json; register runs/hypotheses/INDEX.json.

Source filings

Every forecast row names the filing it rests on.

Each of the 138 manager-quarters in the sample cites the 13F its inputs came from and the 13F that graded it. 225 of 225 cited accession numbers appear in the SEC EDGAR full index. 0 manager-quarters carry a flag (no grading filing, or one filed after 45 days). The free sample stays hashed, because an accession number begins with the filer’s CIK; the licence key table carries the manager, CIK, source accession and grading accession for every row.

What a citation looks like: Berkshire Hathaway, quarter ending 2026-06-30, accession 0001193125-26-352200, filed 2026-08-14.

Source: runs/gates/filing_citations.json.

What was knowable then

A point-in-time audit of every input.

Each input classed by whether it was observable at the simulated forecast date. This is an audit, not a certificate: the served US model still has hindsight in its candidate universe and price history.

Forward commitment

A sealed cohort, graded after the November filings.

On 2026-10-09 the SEC-only model ranked 10,448 candidates for 51 managers’ September-quarter filings. The file stays private until outcomes are graded; its hash and an OpenTimestamps proof commit to it now.

SHA-256 515a9bddd92de91a8673cf69b1eee71ba3fff3b175750ad4f6095a8ab79c04c4 · proof file · status sealed predictions outcome pending. A proof is final only once its Bitcoin attestation confirms.