AlphaRAD

ECCV 2026 · Medical Vision–Language Pretraining

AlphaRAD: Grounded Zero-Shot Chest Radiology via α-Corrected Binary Cross-Entropy and Factorized Latent Supervision

Saying what it sees and pointing to where it sees it, with no added parameters.

Jianzhong You1,3,5,7, Yuan Gao1,2,3,5,7, Chris McIntosh1,2,3,4,5,6,7

1Peter Munk Cardiac Centre, University Health Network (UHN) · 2Toronto General Hospital Research Institute · 3Ted Rogers Centre for Heart Research · 4Computer Science, University of Toronto (U of T) · 5Medical Biophysics, U of T · 6Medical Imaging, U of T · 7Vector Institute

ArXiv Code Checkpoints BibTeX

Figure 3 — Similarity maps from FLaS on ChestXDet10. The marker sits at the pixel of highest attention: no box regressor, no detection head, no fine-tuning.

Abstract

Structured clinical semantics, and grounding that comes for free

Vision-Language Pretrained Models offer a scalable path to open-vocabulary chest radiology understanding, yet two aspects remain underexplored: how structured clinical semantics extracted from medical reports can reduce in-batch noise during contrastive learning, and how cross-modal fusion can be designed to produce more faithful spatial grounding without added complexity.

We introduce AlphaRAD, addressing these opportunities through two contributions. First, we construct a large-scale structured medical concept space from reports parsed by a Large Language Model, mitigating in-batch learning noise and removing heuristic pair matching, which positions AlphaRAD as a medical concept discriminator trained via α-Corrected Binary Cross-Entropy. Second, we propose FLaS (Factorized Latent Supervision), a simple cross-modal fusion module that factorizes representations into independent subspaces and applies dedicated alignment supervision to each, sharpening spatial grounding without introducing additional model parameters.

AlphaRAD shows strong zero-shot generalization across diverse chest radiology tasks. It establishes state-of-the-art average performance across 16 classification benchmarks, and achieves individual state-of-the-art results on 7 grounding and phrase-grounding datasets and 3 segmentation datasets.

Method

Two changes to a standard two-tower model

The encoders are unchanged. Everything below happens in how supervision is built and where the alignment loss is applied.

Contribution 1 · Supervision

α-Corrected Binary Cross-Entropy

Qwen3-30B-A3B parses every MIMIC-CXR report into atomic findings — a disease, device or abnormality, its polarity, and its anatomical context — yielding a pool of 215,698 unique clinical concepts. Each study keeps its confirmed findings as positives; everything else is a negative.

That reframes pretraining as concept discrimination rather than pair matching. A study showing pleural effusion is no longer pushed away from the phrase “pleural effusion” just because another image in the batch was assigned it. Since 200k negatives per step is neither computable nor gradient-balanced, negatives are sampled and the sum is rescaled.

Report → LLM parse → medical concept set → sampled α-BCE

L̂α(Zᵢˢ) = Σj∈Pᵢ ℓ⁺(zᵢʲ) + α Σj∈Sᵢ ℓ⁻(zᵢʲ)
E[L̂α(Zᵢˢ)] = Lβ(Zᵢᴺ), α = β·|Nᵢ| / m

The sampled loss is an unbiased estimator of the full negative-reweighted objective. For every reweighting β there is a matching α; in practice α = 1 is enough.

LAlphaRAD = LFLaS + Σi L̂α({z̃ᵢʲ}), z̃ᵢʲ = (1/F) Σh zᵢʲʰ

One objective: per-subspace alignment for dense tasks, averaged logits for coarse ones.

Contribution 2 · Fusion

FLaS: Factorized Latent Supervision

The output embedding is cut into F disjoint coordinate blocks of size d = D/F. Each block runs its own scaled cosine-similarity attention over image patches and receives its own contrastive loss.

  • No projection matrices, no concatenation, no feed-forward network, so the parameter count is unchanged.
  • Each softmax sees a different similarity landscape, and each subspace gets a separate gradient.
  • Cross-subspace mixing is given up deliberately: that mixing is what blurs attention maps in transformer decoders.
  • The eight heads are interchangeable for classification and genuinely different for localisation; averaging beats every one individually.

D = 768 → 8 × (d = 96) → 8 softmax maps → 8 losses → 0 new parameters

Figure 1 — Types of cross-modal fusion in medical VLPMs. FLaS adds no parameter overhead relative to SCSA or multi-head cross-attention.

Figure 2 — The AlphaRAD training framework. Q is the set of medical concepts sampled for the batch; each image carries its own positives and sampled negatives.

Key Results

State of the art on classification, grounding and segmentation

215,698

LLM-parsed clinical concepts

0

parameters added by FLaS

+2.5

AUROC, 16 classification benchmarks

+8.9

Pointing Game, 4 grounding datasets

+4.2

Pointing Game, 3 phrase-grounding datasets

+3.1

Dice, 3 segmentation datasets

Every number below is zero-shot, and each gain above is the average improvement over the strongest prior model, RadZero-518. At 224 × 224, AlphaRAD already averages higher than every baseline, including those evaluated at 448 and 518.

Table 2 — Zero-shot classification, AUROC (%). Best per column in blue.
Table 3 — Grounding and phrase grounding (Pointing Game, %), segmentation (Dice, %).

AlphaRAD takes the best average on all three task families. At 224 it beats RadZero-518 on CheXlocalize, both MS-CXR versions and PadChest-GR.

Table 4 — Per-disease Pointing Game accuracy (%) on ChestXDet10.
Table 12 — Per-disease Dice (%) on CheXlocalize.

Resolution helps most on small, focal findings: nodules go from 39.0 to 70.1 and rib fractures from 27.6 to 51.3 between 224 and 518. Disease keys: ATE atelectasis · CAL calcification · CAR cardiomegaly · CON consolidation · DEV support devices · EDE edema · EFF pleural effusion · EMP emphysema · ENC enlarged cardiomediastinum · FIB fibrosis · FRA rib fracture · LES lung lesion · MAS mass · NOD nodule · OPA lung opacity · PTX pneumothorax.

Qualitative Results

What the model actually points at

Randomly sampled, five per disease class, successes and failures side by side. All maps come from AlphaRAD-224, and nothing here was trained on a localisation label.

Ground-truth box Peak inside the box Peak outside the box

Figure 13 — ChestXDet10 grounding, success cases.

Figure 14 — ChestXDet10 grounding, failure cases.

Zero-shot segmentation

Masks come straight from the factorized attention maps. In each pair the annotation is shown first and the prediction directly below it.

Figure 15 — CheXlocalize, classes 1–5: atelectasis, cardiomegaly, consolidation, support devices, edema.

Figure 16 — CheXlocalize, classes 6–10: pleural effusion, enlarged cardiomediastinum, lung lesion, airspace opacity, pneumothorax.

Figure 6 — SIIM-ACR pneumothorax, sampled from cases with Dice > 0.

Figure 7 — QaTa-COV19 infection regions, sampled without filtering.

Inside the subspaces

Figure 5 — The eight factorized subspaces in FLaS, shown against the averaged map. Pleural effusion draws seven subspaces to one location while a single subspace finds the second box; pulmonary fibrosis has subspaces peaking at different points inside the same annotation; atelectasis shows the failure mode, where wrong subspaces pull the average off target.

Citation

BibTeX

          
@misc{you2026alpharad,
      title={AlphaRAD: Grounded Zero-Shot Classification in Chest Radiology via $\alpha$-Corrected Binary Cross Entropy and Factorized Latent Supervision}, 
      author={Jianzhong You and Yuan Gao and Chris McIntosh},
      year={2026},
      eprint={2609.01757},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2609.01757}, 
}

Acknowledgments. Jianzhong You is funded by the Ontario Graduate Scholarship-Doctoral. Yuan Gao holds a CIHR Canada Graduate Scholarship-Doctoral. Dr. Chris McIntosh holds the Chair in Medical Imaging at the Joint Department of Medical Imaging at University Health Network and the University of Toronto.