Changelog
Each version has a dated analysis record and a report with its gate log. The timing evidence for analysis plans varies by round; the manuscript distinguishes planned analyses from exploratory controls. Frozen rows never change; later rows carry the version in which they entered.
Major review corrections
These changes repair the released metadata, tools and presentation while preserving the historical score record. They are not a new benchmark evaluation.
- T3 exposure metadata. The released
time_travel_v1.npzfieldexposure_2023held a Boolean prior-annotation flag instead of publication counts. Applying<= 5to that field selected all 347 evaluation genes rather than the intended 137. The field now agrees with the released gene table; labels and stored scores are unchanged. v0 receives the same correction. - GO AUROC ties. The original GO scoring helper assigned ordered ranks to tied scores. The corrected helper uses average ranks, so a constant predictor scores 0.5. Historical prediction arrays are absent from this checkout: the effect on published model scores and contrasts cannot yet be quantified. Recomputed values must be released separately with the corrected metric.
- Corpus aliases. Seven reused GenePT aliases map to multiple current genes. Corpus construction now rejects ambiguous keys by default; reproducing the historical mapping requires an explicit legacy option. A future corrected-corpus experiment is needed to measure its effect.
- Uncertainty and ranks. Family-clustered intervals now appear with their own round-14 regenerated points and ranks in a separate sensitivity table, rather than over frozen v1 scores. Chart positions distinguish current sorting from historical rank intervals over the original 20 Track O rows.
- Website fidelity. Track F details resolve by track and tag; downloads record stratum, probe, task weight, score source and rank population. T2/T3 curation dependencies, summary-removal estimands, missing artifacts and proxy temporal eligibility are stated explicitly. Generated records carry source and download SHA-256 checksums.
The saved numerical records and accompanying review are in round4/review_major_2026-10-25/. This review does not publish the paper or Zenodo deposit.
26 October 2026 · website
This site: the leaderboard with task profiles, intervals, family-clustered intervals, T1-weight views and row declarations, built from the frozen record by round4/scripts/r16_site_data.py; design, submission and data pages; a submission issue form. No score changed.
25 October 2026 · metadata correction
The exposure column in genes_v0.tsv and genes_v1.tsv now holds the filtered publication count that defines the exposure bins (19,141 values changed; 8,081 had contradicted their bin). Gene order, bins, folds, labels and every stored score are unchanged. Manuscript revised to v1.1 (author-review draft).
17 October 2026 · v1.1 (round 14)
Six rows appended in response to external comments: rich summary-filtered text without UniProt function text; GB.Protein-16B concatenated with that text; ESM2-650M concatenated with rich text with and without UniProt function text; the GO annotation vector at the 2023 cutoff and its concatenation with GB.Protein-16B (T3-only references). Every embedding lost with the scratch disk was re-made and every out-of-fold array regenerated under a reproduction gate: 350 of 366 v1 cells within 0.002 AUROC; the twelve ESM2-650M cells (re-embedded under a different library version) and four MLP T2 cells are listed as exceptions and the v1 values stand. Added: family-clustered bootstrap intervals, T1-weight sensitivity, coverage by exposure bin, recall at 10 for T4, the zero-publication bin under three label sets, a term-frequency baseline.
11 October 2026 · v1 (round 11)
The GeneProv Index (mean chance-corrected skill over T1, T2 and T4; T3 beside it), seven new Track O rows (GB.DNA-300M, GB.RNA-1.6B, ESM1b, ProtST-ESM1b, ProTrek-650M, UniProt function text only, GB.Protein-16B ⊕ rich summary-filtered text), T2 and T4 for the six Track F embedders, paired bootstrap index and rank intervals for Track O (Track F index and rank intervals were not recorded), the leaderboard figure and the first static leaderboard page. Four pre-registered gates committed before computation; all passed, including that every v0 cell reproduces (218 of 218).
10 October 2026 · v0 (round 10)
The frozen first table: thirteen Track O rows and six Track F embedders on the four tasks, the universe, strata, folds and rules; four pre-registered gates, all passed. Task labels and stored scores are retained; later metadata corrections are recorded above.