Benchmark: EnergyFingerprint vs CADD, REVEL, AlphaMissense

The missing question

When we introduced EnergyFingerprint in June, the obvious question was: how does it compare to established predictors? CADD, REVEL, and AlphaMissense are the industry standard — trained on hundreds of thousands of variants across tens of thousands of genes, with massive functional annotations. EnergyFingerprint is a per-gene model of ~228K parameters using only two signals: mRNA physics and protein evolution.

Now we have the quantitative answer.

Methods: a fair comparison

The benchmark uses the "Likely" ClinVar subset (Likely pathogenic vs Likely benign) — variants with consistent clinical classification. For each variant, we obtained CADD, REVEL, and AlphaMissense scores via dbNSFP (myvariant.info, GRCh38). Coverage: CADD 99.9%, REVEL 99.1%, AlphaMissense 98.6%.

All comparisons use the complete-case subset — only variants where all four predictors have a score. Each comparison is paired on the same variants. The statistical test is DeLong's for correlated ROC curves, and pooled inference uses inverse-variance meta-analysis stratified by gene — not naive score concatenation, which would produce a Simpson's paradox.

Results: within 2-4% AUC

MECP2 is excluded from the benchmark: its 4 Likely-pathogenic variants lack REVEL and AlphaMissense coverage, leaving 0 positives in the complete-case subset.

The finding: SCN1A

The highlighted row in the table is the most important result of the benchmark. On SCN1A — a voltage-gated sodium channel with 24 transmembrane segments — EnergyFingerprint achieves 0.913 against CADD's 0.832. A +0.081 AUC difference in favor of a 228K-parameter model over a predictor that integrates dozens of genomic annotations.

On SCN1A, mRNA physics captures something that genomic annotations don't see. Ion channels have their own thermodynamic grammar: ΔG barriers that pause the ribosome for cotranslational folding coupled to the membrane insertion machinery.

This is not a statistical accident. It is consistent with the transferability gradient we described in June: transmembrane proteins have highly informative ΔG profiles, with thermodynamic constraints that protein-level predictors don't model.

Meta-analysis: the statistical verdict

EnergyFingerprint falls 2.4 points behind CADD, 2.8 points behind REVEL, and 4 points behind AlphaMissense in mean AUC. With ~228K parameters per gene versus predictors trained on massive cross-gene datasets, the result demonstrates that the mRNA thermodynamic signal is competitive as an independent dimension.

Other highlights

• BRCA1: EF (0.929) matches REVEL (0.927) — DeLong p = 0.90, statistically indistinguishable.
• PALB2: EF (0.902) exceeds REVEL (0.880), though with only 2 pathogenic variants the power is limited.
• HBB: EF's weakest gene (0.586). Hemoglobin is an atypical protein — very small, no membrane regions or complex folding. Consistent with the transferability gradient: globins have weak thermodynamic pressure.

What this means for diagnostics

The benchmark confirms that EnergyFingerprint provides an orthogonal signal to existing predictors. It is not a replacement — it is a complement. Thermodynamic information from the mRNA is a dimension that CADD, REVEL, and AlphaMissense do not model, and in certain contexts (membrane ion channels, proteins with cotranslational folding constraints) that dimension dominates.

For a genetic diagnostics lab, the value is in the ensemble: adding EF to the existing predictor panel would contribute additional signal precisely in the genes where current predictors are weakest.

And for research, the SCN1A result opens a door: if mRNA physics better explains variants in ion channels, what other membrane genes would benefit from a thermodynamic predictor? The SCN1A↔CFTR cross-gene zero-shot is the next experiment.