MRI harmonization is usually scored with one frozen site classifier: if its accuracy drops on harmonized scans, the scanner effect is taken as removed. This explorer shows, on the audit's own numbers, why that score cannot stand alone.
The frozen probe scores 0.95 on raw scans, so it passes the usual sanity check. Rerun its exact training recipe with other seeds and the same harmonized images get very different verdicts. Probes trained four times longer agree with each other, and on every output except NeuroCombat and HACA3 all five score above the frozen probe's 95% interval: the frozen probe under-reports the site information that is left.
Balanced accuracy of site probes trained on unharmonized images and tested on each output. Lower is usually read as "better harmonization". Grey band: accuracy of probes trained on shuffled labels (0.03–0.08); uniform guessing over 16 source sites gives 0.06. Hover any mark for its value.
A probe trained on raw scans may simply not recognise harmonized ones. Train the probe on the harmonized scans themselves and the site comes back: 0.87–0.91 on whole heads for histogram matching, CycleGAN and diffusion. It survives skull-stripping, and intensity histograms alone still give it away.
If a low frozen-probe score meant that scans had moved toward the target site, the lowest points below would sit furthest to the left. A plain Gaussian blur (σ = 2) moves intensities away from NYU, keeps PSNR at 24.7 dB, and still gets the lowest frozen-probe score of all: 0.07, chance level. The probe labels almost every blurred slice as one site, Yale, and none as NYU.
ΔWNYU: change in Wasserstein distance between each output's intensity distribution and that of 147 NYU training subjects (output minus input); negative means closer to NYU. Labelled points are discussed in the text; hover the others.
Train more than one site probe until validation accuracy is stable, and report the spread across seeds, not one draw.
Ask whether site is still decodable from harmonized scans, with head-shape, brain-only and histogram-only controls.
Measure movement toward the target site separately from change to the input. A blur changes the input without harmonizing it.
Every number on this page is read from the versioned results in the HarmonIt repository at tag isbi2027-submission and cross-checked against the paper's table when the page data is built. The protocol and each of its dated amendments were written before the corresponding outcome was seen. No MRI images are hosted here.