The missing question
When we introduced EnergyFingerprint in June, the obvious question was: how does it compare to established predictors? CADD, REVEL, and AlphaMissense are the industry standard — trained on hundreds of thousands of variants across tens of thousands of genes, with massive functional annotations. EnergyFingerprint is a per-gene model of ~228K parameters using only two signals: mRNA physics and protein evolution.
Now we have the quantitative answer.
Methods: a fair comparison
The benchmark uses the "Likely" ClinVar subset (Likely pathogenic vs Likely benign) — variants with consistent clinical classification. For each variant, we obtained CADD, REVEL, and AlphaMissense scores via dbNSFP (myvariant.info, GRCh38). Coverage: CADD 99.9%, REVEL 99.1%, AlphaMissense 98.6%.
All comparisons use the complete-case subset — only variants where all four predictors have a score. Each comparison is paired on the same variants. The statistical test is DeLong's for correlated ROC curves, and pooled inference uses inverse-variance meta-analysis stratified by gene — not naive score concatenation, which would produce a Simpson's paradox.
Why not concatenate scores? EnergyFingerprint trains a separate model per gene. BRCA1 scores and SCN1A scores are not on the same scale. Concatenating them and computing a global AUC artificially penalizes EF. Gene-stratified meta-analysis is the correct test: it combines evidence from each gene weighted by precision.
Results: within 2-4% AUC
| Gene | n | EF | CADD | REVEL | AlphaMissense |
|---|---|---|---|---|---|
| BRCA1 | 302 | 0.929 | 0.950 | 0.927 | 0.944 |
| TP53 | 145 | 0.886 | 0.889 | 0.921 | 0.939 |
| PTEN | 111 | 0.898 | 0.996 | 1.000 | 0.999 |
| PALB2 | 48 | 0.902 | 1.000 | 0.880 | 0.957 |
| CFTR | 103 | 0.925 | 1.000 | 0.991 | 0.966 |
| HBB | 34 | 0.586 | 0.751 | 0.777 | 0.762 |
| SCN1A | 162 | 0.913 | 0.832 | 0.957 | 0.975 |
| Weighted mean | 905 | 0.901 | 0.926 | 0.940 | 0.952 |
MECP2 is excluded from the benchmark: its 4 Likely-pathogenic variants lack REVEL and AlphaMissense coverage, leaving 0 positives in the complete-case subset.
The finding: SCN1A
The highlighted row in the table is the most important result of the benchmark. On SCN1A — a voltage-gated sodium channel with 24 transmembrane segments — EnergyFingerprint achieves 0.913 against CADD's 0.832. A +0.081 AUC difference in favor of a 228K-parameter model over a predictor that integrates dozens of genomic annotations.
On SCN1A, mRNA physics captures something that genomic annotations don't see. Ion channels have their own thermodynamic grammar: ΔG barriers that pause the ribosome for cotranslational folding coupled to the membrane insertion machinery.
This is not a statistical accident. It is consistent with the transferability gradient we described in June: transmembrane proteins have highly informative ΔG profiles, with thermodynamic constraints that protein-level predictors don't model.
Meta-analysis: the statistical verdict
A note on the AUCs: they are computed on ClinVar, where the value may reflect gene identity or variant position rather than the effect of the variant itself. This affects EF and the compared predictors alike.
| Comparison | Pooled ΔAUC | p | p (BH adj.) | Interpretation |
|---|---|---|---|---|
| EF vs CADD | -0.024 | 0.048 | 0.048 | Significantly below (p = 0.048) |
| EF vs REVEL | -0.028 | 0.022 | 0.033 | Marginally behind |
| EF vs AlphaMissense | -0.040 | 0.0003 | 0.0009 | Behind |
EnergyFingerprint falls 2.4 points behind CADD, 2.8 points behind REVEL, and 4 points behind AlphaMissense in mean AUC. With ~228K parameters per gene versus predictors trained on massive cross-gene datasets, the result demonstrates that the mRNA thermodynamic signal is competitive as an independent dimension.
Other highlights
- BRCA1: EF (0.929) matches REVEL (0.927) — DeLong p = 0.90, statistically indistinguishable.
- PALB2: EF (0.902) exceeds REVEL (0.880), though with only 2 pathogenic variants the power is limited.
- HBB: EF's weakest gene (0.586). Hemoglobin is an atypical protein — very small, no membrane regions or complex folding. Consistent with the transferability gradient: globins have weak thermodynamic pressure.
What this means for diagnostics
The benchmark leaves open, without proving, the hypothesis that EnergyFingerprint provides an orthogonal signal to existing predictors. It contains no additivity experiment: it measures each predictor separately, not EF combined with the others. If the hypothesis holds, EF is not a replacement but a complement. Thermodynamic information from the mRNA is a dimension that CADD, REVEL, and AlphaMissense do not model, and in certain contexts (membrane ion channels, proteins with cotranslational folding constraints) that dimension dominates.
Were the hypothesis confirmed, the value for a genetic diagnostics lab would lie in the ensemble: adding EF to the existing predictor panel would contribute additional signal precisely in the genes where current predictors are weakest. Testing that requires an additive-value study we have not published.
And for research, the SCN1A result opens a door: if mRNA physics better explains variants in ion channels, what other membrane genes would benefit from a thermodynamic predictor? The SCN1A↔CFTR cross-gene zero-shot is the next experiment.