Leaderboard
One score per representation: the GeneProv Index, the mean over three tasks of the chance-corrected skill 100 × (2 AUROC − 1), with its 95% paired bootstrap interval and rank interval. Frozen v1 rank intervals refer only to the original 20 Track O rows. Track F has no index or rank intervals; appended rows have gene-bootstrap index intervals but no joint rank intervals. Switch the stratum, probe, track and rows; click a bar or row for the full declaration and task profile. Frozen v1 rows never change; appended rows carry their version.
Review status. The scores below are frozen historical results. A later code review found missing tie correction in the GO AUROC helper; its numerical impact on these scores remains unquantified because the original prediction arrays are unavailable here. The revised helper is tested, but these results have not been rerun with it. The T3 exposure metadata has also been corrected. See the correction record.
How to read it
Tasks
T1, GO recovery. 4,949 GO terms (no IEA or ND evidence, propagated, 20–1,000 genes each); multi-label probe, five shared folds, per-gene AUROC of the gene's terms against the terms it lacks. Track F scores it on summary-filtered text; Track O scores each declared input as it is.
T2, non-GO labels. Eleven binary endpoints spanning assays, population variation and clinical curation: DepMap common essentiality, gnomAD constraint, ClinGen haploinsufficiency, six HPA subcellular locations, GTEx tissue specificity and HPA tissue enrichment; one probe per label with folds grouped by HGNC gene family; mean AUROC over the labels with at least five genes of each class in the stratum (ClinGen drops out of the 0–10 stratum).
T3, annotation holdout. Experimental GO annotations added after 1 January 2023 to genes that had none then (347 genes; 137 with 0–5 publications at the cutoff). Representations are admitted under a declared annotation-access rule, not a verified historical training cutoff. Sequence and expression models qualify by an input-type assumption regardless of release date; GenePT uses summary tag dates as a proxy, and scELMo uses a reported training-date assumption. Reported beside the index, not in it.
T4, co-essentiality. Each gene's ten most correlated genes by DepMap CRISPR effect, retrieved by cosine similarity of the standardised embedding; AUROC of the ten partners against all other genes; no probe.
Index, intervals, probes
Index. Mean over T1, T2 and T4 of 100 × (2 AUROC − 1): chance is 0, perfect discrimination 100, below-chance negative. The transform does not depend on which other rows exist, so a row's score never changes when rows are added. Index-4 adds T3 for eligible rows.
Intervals. Paired bootstrap over the genes of the stratum (1,000 resamples, seed 0), recomputing the three task means per resample; rank intervals come from the same resamples over the original 20 Track O v1 rows. Appended v1.1 rows have no comparable rank intervals. The separate family sensitivity table uses round-14 regenerated point estimates and HGNC-family bootstrap intervals, which are several-fold wider. Those intervals are not attached to the frozen v1 points: ESM2 was re-embedded, and its scores changed.
Probes. Linear is a 300-epoch logistic probe; MLP has one hidden layer of 512 units. The headline ranking is robust to the probe (Spearman between the two indices) and to replacing the mean skill by the mean rank (), but not to dropping T1 ( with a T2+T4-only index), under which single-cell models lead.
Coverage. The Embedded genes column counts available embeddings across the whole universe; missing genes receive the mean embedding during evaluation. The row detail lists missingness by exposure bin and task-specific evaluated counts where recorded. Universe size, embedding coverage and the number of scored genes are different denominators.