WAICA 2026Ophthalmic multimodal reasoning

OphVLM-R1Efficient Ophthalmic Reasoning via Curriculum RL

A 2B-parameter vision-language model that learns clinical reasoning in the order clinicians do: from visual perception to integrated diagnostic judgment.

Zishi QiXiaoya Hu*Huilin PanAng GaoJiaxin HouJiankun LiYongao Qian

School of Artificial Intelligence and Automation, Huazhong University of Science and Technology · * Corresponding author

Paper Code

Model · Hugging Face Model · ModelScope Dataset · Hugging Face Dataset · ModelScope

88.24%OmniMedVQA-Eye
out-of-domain accuracy

2Bparameters

15,418reasoning trajectories

+26.21points over SFT on Omni-Eye

OphVLM-R1 project overview with an ophthalmic AI assistant and the perception, interpretation, reasoning, and knowledge curriculum stages.

The core idea

Clinical competence is progressive. Training should be too.

Ophthalmic MLLMs are constrained by unstructured training data, flat training strategies, and model sizes that limit practical deployment. OphVLM-R1 closes the loop across data, model, and optimization.

The pipeline combines verified Chain-of-Thought data, LoRA-based supervised fine-tuning, and curriculum reinforcement learning with sequence-level optimization and hard-sample dynamic backtracking.

Verified data

OphReason-Vision

15,418 reasoning trajectories synthesized from 100K+ clinical cases and 30+ public datasets, with expert agreement κ = 0.82.

Stable optimization

Sequence-level GSPO

Sequence-level policy ratios reduce variance accumulation across long clinical reasoning chains.

Long-tail learning

Dynamic backtracking

Persistently failed prompts return with higher sampling probability while keeping every rollout on-policy.

Clinical-pathway curriculum

From seeing to reasoning

Four tasks increase in diagnostic complexity and follow the clinical workflow from perceptual grounding to knowledge-intensive decisions.

Perception

Lesion localization

Single-image visual grounding

2,700samples

Comparison

Multi-image selection

Cross-image diagnostic evidence

2,700samples

Synthesis

Report generation

Structured long-form reasoning

3,600samples

Knowledge

Knowledge Q&A

Visual findings + clinical knowledge

1,000samples

Two-stage OphVLM-R1 training pipeline: cold-start supervised fine-tuning followed by four-stage curriculum reinforcement learning with GSPO and hard-sample dynamic backtracking. — Stage 1 injects domain knowledge through LoRA-based SFT. Stage 2 progressively increases task difficulty and uses mixed verifiable and judge rewards.

OphReason-Vision

Reasoning data, closed-loop verified

Standardization, structured synthesis, and expert-collaborative optimization turn heterogeneous ophthalmic records into quality-controlled reasoning trajectories.

Three-stage data pipeline showing data standardization, structured clinical reasoning synthesis, and expert-collaborative optimization for OphReason-Vision. — Automated LVLM-as-a-Judge filtering uses a 0.7 threshold; three board-certified ophthalmologists review the difficult 18% of samples.

Dataset composition

13,418 training samples + 2,000 held-out cases

Subset	N	Token range	Images	Purpose
cold_start	3,418	1,609–2,113	1–2	Cold-start SFT
Lesion Localization	2,700	1,038–1,069	1	Stage 1
Multi-image Selection	2,700	1,115–2,240	1–2	Stage 2
Report Generation	3,600	1,012–5,985	1–5	Stage 3
Knowledge Q&A	1,000	1,012–5,985	1–5	Stage 4
eval_in_domain	2,000	1,060–1,138	1–2	Evaluation

Evaluation

A smaller model, a stronger reasoning path

OphVLM-R1 is evaluated on held-out in-domain cases, 31 fine-grained fundus tasks, and an out-of-domain ophthalmic VQA benchmark drawn from 11 independent sources.

In-Domain38.40%2,000 cases · 4 tasks

Fundus-MMBench42.58%620 samples · 31 tasks

OmniMedVQA-Eye88.24%10,044 samples · 11 sources

Reference avg.56.41%+4.46 over InternVL3.5-4B

Evaluation datasets

Three complementary dimensions of ophthalmic reasoning

Dataset	Samples	Tasks	Type
In-Domain	2,000	4	Multiple-choice
Fundus-MMBench	620	31	Fine-grained fundus
OmniMedVQA-Eye	10,044	11 sources	Out-of-domain VQA

Main results

Accuracy (%) · best and second-best are highlighted per benchmark

Model	In-Domain	Fundus	Omni-Eye	Avg.*
Generic MLLMs
InternVL3.5-2B	34.50	36.61	55.47	42.19
InternVL3.5-4B	36.23	42.10	77.51	51.95
Medical MLLMs (RLVR)
MedVLM-R1-2B	27.80	20.81	68.06	38.89
Large medical MLLMs (SFT, off-the-shelf)
Lingshu-7B	44.20	41.29	87.42	57.64
HuatuoGPT-Vision-7B	38.30	28.06	71.78	46.05
Ophthalmic MLLMs (SFT, off-the-shelf)
FundusExpert-8B	31.20	54.84	64.71	50.25
Ophthalmic MLLMs (RLVR)
OphthaReason-Intern-2B	31.00	35.48	79.61	48.70
OphthaReason-Qwen-3B	36.60	38.87	86.86	54.11
OphVLM-R1-2B (Ours)	38.40	42.58	88.24	56.41

* The cross-benchmark average is reference-only because task scales, formats, and random baselines differ. Within-benchmark ranking is the meaningful comparison.

What makes the difference?

Ablation study · ΔOmni is the drop from the full model

Configuration	In-Domain	Fundus	Omni-Eye	ΔOmni
OphVLM-R1 (Full)	38.40%	42.58%	88.24%	—
Training strategy
SFT Only	37.52%	34.47%	62.03%	−26.21
SFT + RL One-shot	37.96%	38.62%	78.14%	−10.10
SFT + RL Shuffled	37.73%	37.85%	76.48%	−11.76
Stage ablation
w/o Stage 1 (Lesion Loc.)	38.14%	40.43%	85.62%	−2.62
w/o Stage 2 (Multi-image)	38.02%	40.17%	85.13%	−3.11
w/o Stage 3 (Report Gen.)	37.88%	39.72%	84.38%	−3.86
w/o Stage 4 (Knowledge QA)	38.07%	40.31%	85.47%	−2.77
Component ablation
w/ GRPO (token-level)	37.84%	39.76%	84.52%	−3.72
w/o Hard Sample BT	38.11%	40.83%	86.12%	−2.12

Open resources

Use the model. Study the data. Reproduce the pipeline.

Model weights

OphVLM-R1

2B-parameter ophthalmic multimodal reasoning model.

Hugging Face ModelScope

Dataset

OphReason-Vision

Verified ophthalmic Chain-of-Thought reasoning trajectories.

Hugging Face ModelScope

System code

OphAgent

Open-source code for the associated ophthalmic agent system.

GitHub

Citation

Build on this work

If OphVLM-R1 or OphReason-Vision supports your research, please cite the WAICA 2026 paper.

@inproceedings{qi2026ophvlm,
  title     = {OphVLM-R1: Efficient Ophthalmic
               Reasoning via Curriculum Reinforcement Learning},
  author    = {Qi, Zishi and Hu, Xiaoya and Pan, Huilin
               and Gao, Ang and Hou, Jiaxin and Li, Jiankun
               and Qian, Yongao},
  booktitle = {Proceedings of the World Artificial
               Intelligence Conference Academic (WAICA)},
  year      = {2026}
}