Model transparency
Every number on this page came out of the evaluation script.
This page exists to make the model auditable, not to sell it. The metrics are from a single evaluation on a held-out split of patients the model never saw during training or calibration.
Task
NORM vs MI-associated ECG morphology
Binary classification of retrospective 12-lead ECGs into 'no diagnostic abnormality (NORM)' versus 'carries the MI diagnostic superclass'. This is retrospective morphology classification, not prospective detection of an acute coronary occlusion.
- Positive class
- MI-associated morphology
- Negative class
- NORM
- Model version
- minutium-ecg-res1d-0.1.0
- Selected model
- res1d
Dataset
PTB-XL 1.0.3
PTB-XL official stratified 10-fold assignment (strat_fold), which is assigned at patient level: folds 1-8 train, fold 9 validation, fold 10 test
| Split | Records | Patients | NORM | MI |
|---|---|---|---|---|
| train | 11,622 | 10,396 | 7,243 | 4,379 |
| val | 1,454 | 1,338 | 914 | 540 |
| test | 1,462 | 1,297 | 912 | 550 |
Held-out test metrics
| Metric | Neural (test) | Neural (val) | Baseline (test) |
|---|---|---|---|
| ROC AUC | 0.9670 | 0.9732 | 0.9293 |
| PR AUC | 0.9575 | 0.9639 | 0.9094 |
| Sensitivity | 0.8709 | 0.9000 | 0.8945 |
| Specificity | 0.9221 | 0.9136 | 0.7829 |
| PPV | 0.8709 | 0.8602 | 0.7130 |
| NPV | 0.9221 | 0.9393 | 0.9249 |
| F1 | 0.8709 | 0.8796 | 0.7935 |
| Accuracy | 0.9029 | 0.9085 | 0.8249 |
| Brier score | 0.0654 | 0.0598 | 0.0969 |
| Expected calibration error | 0.0188 | 0.0171 | 0.0298 |
Baseline: Logistic regression on interpretable morphology features. Both models were trained on the same patient-level splits and evaluated once on the same held-out test fold.
ROC curve
Calibration
Calibrated Before scaling
Confusion matrix
True negative
841
False positive
71
False negative
71
True positive
479
912 NORM and 550 MI-associated records, 1462 total.
Calibration
Raw network outputs are not probabilities anyone should act on. A single temperature parameter was fitted on the validation split and divides the logits before the sigmoid. It cannot change the ranking of cases, so it leaves ROC AUC untouched, but it makes a stated probability mean what it says.
- Method
- temperature scaling
- Temperature
- 1.6295
- Fitted on
- validation split (PTB-XL strat_fold 9), never the test split
Architecture
- name
- ECGResNet1d
- input shape
- 12 · 1000
- stem
- Conv1d(12->32, k=15, s=2) + BN + ReLU
- residual stages
- ResidualBlock1d(k=7, s=2) -> 32 channels · ResidualBlock1d(k=7, s=2) -> 64 channels · ResidualBlock1d(k=7, s=2) -> 96 channels · ResidualBlock1d(k=7, s=2) -> 128 channels
- pooling
- concat(global average pool, global max pool)
- head
- Dropout -> Linear(256->64) -> ReLU -> Dropout -> Linear(64->1)
- output
- single logit; calibrated to a probability by temperature scaling
- parameters
- 411329
Distance from the real problem
Quantifies how far the headline retrospective metrics are from the problem M0 actually has to solve. Every number here is measured on the held-out test split or derived from it; none is an estimate of real-world performance.
Acute infarction versus healed scar
Recall on MI-positive test records, grouped by PTB-XL's recorded infarction stage. Stadium I is acute, Stadium III is old. A model detecting acute events specifically would score Stadium I far above Stadium III. Similar rates across stages indicate the model is substantially recognising established infarct morphology, including long-healed scar.
| Infarction stage | n | Detected | Mean probability |
|---|---|---|---|
| unknown | 328 | 84.5% | 0.816 |
| Stadium II-III | 103 | 85.4% | 0.790 |
| Stadium III | 98 | 95.9% | 0.919 |
| Stadium I | 14 | 92.9% | 0.949 |
| Stadium II | 7 | 100.0% | 0.995 |
Caveat: Stage is recorded for only part of the cohort and the acute-stage subgroup is small, so these are indicative rates and not precise subgroup estimates. Wide uncertainty on any group with n below roughly 30.
What happens when prevalence drops
Sensitivity and specificity are properties of the model and do not change with prevalence. Positive predictive value does. The test cohort is far richer in MI than a population of at-risk people activating a device at home, where most activations would be anxiety, reflux, or musculoskeletal pain.
| Prevalence | PPV | NPV | False alarms per true event |
|---|---|---|---|
| 2% | 18.6% | 99.7% | 4.4 |
| 5% | 37.1% | 99.3% | 1.7 |
| 10% | 55.4% | 98.5% | 0.8 |
| 20% | 73.7% | 96.6% | 0.4 |
| 38% | 87.3% | 92.1% | 0.1 |
Caveat: This is arithmetic on the held-out operating point, not a measurement in a real population. It shows how PPV would move if sensitivity and specificity held exactly, which itself is not a safe assumption under distribution shift.
Fewer leads, as a patch would have
Performance when only a subset of leads is available, produced by zeroing the remaining channels. The model was trained on all twelve, so this measures graceful degradation rather than the performance of a model trained natively on fewer leads, which would do better.
- full 12 lead0.967
- limb only0.842
- patch 6 lead0.932
- patch 3 lead0.810
- single lead II0.291
Caveat: An optimistic stand-in for a chest patch. Removing leads keeps the geometry of the remaining ones intact, whereas a real patch changes electrode positions and therefore the lead vectors themselves.
Noise a self-applied patch would introduce
Degradation under injected Gaussian noise plus 0.5 Hz baseline wander, at amplitudes stated in millivolts. Approximates the motion artifact and poor skin contact expected when a patient applies a patch themselves.
- 0.000 mV0.967
- 0.025 mV0.944
- 0.050 mV0.889
- 0.100 mV0.814
- 0.200 mV0.678
Caveat: Synthetic noise is a crude proxy for real motion artifact, which is correlated across leads and structured in time rather than Gaussian.
Real-time capability
Wall-clock latency for a single 10-second 12-lead recording on CPU, excluding preprocessing. Supports the claim that continuous operation over a sliding window is not compute-limited.
- Median
- 0.78 ms
- p95
- 0.87 ms
- Parameters
- 411,329
- Device
- CPU
Continuous operation would re-run this over a sliding window of the trailing ten seconds. The latency budget for a cardiac rescue decision is minutes, so compute is not the binding constraint. The genuine architectural gap is that this is a snapshot classifier: a production system would compare the current window against the patient’s stored baseline and their own recent windows, so it can see an evolving sequence rather than a still frame.
Limitations
- Trained and evaluated only on PTB-XL, a single retrospective dataset collected in Germany between 1989 and 1996. Performance on other populations, other recording hardware, and modern clinical practice is unmeasured.
- The task is retrospective classification of MI-associated ECG morphology, not prospective detection of an acute coronary occlusion. A record labelled MI may have been recorded long after the infarction.
- The model sees a 10-second, 100 Hz, 12-lead recording acquired with clinical electrode placement. The proposed M0 device would acquire a different signal, from different electrode positions, under worse conditions.
- No outcome data, no troponin, no angiography, and no clinical context are used, so the model cannot be compared against a clinical reference standard.
- Demographic performance has not been characterised. PTB-XL's age and sex distribution is not representative of any specific deployment population.
- The baseline-deviation module is illustrative. PTB-XL is cross-sectional, so no genuine longitudinal per-patient baseline exists in this data.
- Lead attribution is an occlusion measurement of model dependence. It is not a clinical explanation and must not be read as one.
Generated 2026-07-27 18:42:57 UTC. Full artifacts in artifacts/model/. See the demo overview and MODEL_CARD.md.