You Point, I Learn is an ICLR 2026 medical-imaging paper about a practical failure mode: a segmentation model can look good during training, then become brittle when acquisition conditions, disease distribution, or imaging modality changes. The paper’s answer is to use interactive user feedback itself as a learning signal. When a user clicks to correct a segmentation mask, the model should not only fix the current image. It should adapt to the new distribution.
The method is called OAIMS: Online Adaptation for Interactive Medical-image Segmentation. It turns corrective clicks into two online updates, one after the user finishes an image and one during the click sequence. That makes the paper a useful companion to CARE: CARE asks medical reasoning systems to ground and verify their answers, while OAIMS asks medical segmentation systems to keep learning from user feedback after deployment. The connection is conceptual rather than methodological: OAIMS does not solve reasoning verification, and CARE does not solve segmentation adaptation.
| *Source: arXiv 2503.06717 — You Point, I Learn | PDF v4 | Code: WenTXuL/OAIMS* |
The problem
Interactive segmentation is already a human-in-the-loop workflow: the model predicts a mask, the user clicks where it is wrong, and the model refines the prediction. The key observation in this paper is that those corrections are not just UI events. They are high-value supervision from the target deployment distribution.
Medical imaging makes this especially important:
| Shift | Why it matters |
|---|---|
| Dataset or acquisition shift | Pixel statistics and artifacts change |
| New pathology | The model sees lesions or anatomy patterns it did not learn well |
| New modality | A model trained on one image type may react poorly to another |
| Human correction behavior | User clicks reveal where the model is failing in practice |
The usual deployment pattern wastes that signal. A user corrects the output, the case is finished, and the model goes into the next image unchanged. OAIMS changes that loop.
The method
new medical image
|
v
interactive segmentation model predicts mask
|
v
user corrective clicks
|
+--> Mid-Interaction adaptation during the click sequence
|
v
final user-refined mask
|
+--> Post-Interaction adaptation for future images
|
v
model adapts to the new distribution
The paper has three load-bearing ideas:
| Idea | Meaning |
|---|---|
| Post-Interaction adaptation | After the user finishes correcting an image, use the user-refined result as an adaptation signal for later images. |
| Mid-Interaction adaptation | Adapt during the interaction sequence so the model responds better within the same image. |
| Click-Centered Gaussian loss | Penalize wrong predictions more strongly near user-indicated correction points, using a Gaussian weighting around each click. |
That last piece matters. A click is not a generic label. It tells the model where the human believes the current prediction is wrong. Weighting the loss around clicks makes the model more responsive to the exact places where user feedback is informative.
Results to remember
The paper reports experiments across 5 fundus and 4 brain-MRI databases. The headline is not just that OAIMS improves segmentation under distribution shift. The more interesting lesson is that even a lean online update can be competitive because it uses fresh, task-specific feedback from the user’s corrections.
The authors report gains under:
- unseen imaging modalities
- unseen pathologies
- fundus distribution shifts
- brain-MRI distribution shifts
Read this as a research result, not a clinical deployment claim. The paper shows that online adaptation from user corrections is promising. It does not prove real-world clinical safety.
Why this pairs with CARE
The two papers make a useful conceptual comparison for medical AI:
| Pain point | Paper | Design response |
|---|---|---|
| Models face changed acquisition/data distributions | You Point, I Learn | Use user clicks as online adaptation signal |
| Black-box medical reasoning lacks accountability | CARE | Use an evidence-grounded medical reasoning frame |
Together they sketch a better medical-AI design pattern:
human feedback is not a correction after the fact
human feedback is part of the model's evidence and adaptation loop
For students, that is the core lesson. Human-in-the-loop AI should not mean “ask the human to clean up the model’s mistakes forever.” It should mean designing the system so corrections improve the next step.
Teaching use
Use this as a reading note in a medical AI, interactive ML, or human-in-the-loop learning module:
- Ask students to draw the inference loop before and after OAIMS.
- Mark where the user provides information.
- Discuss when user-refined adaptation signals are helpful and when they could amplify mistakes.
- Compare Post-Interaction and Mid-Interaction adaptation.
- Ask what governance checks would be required before clinical use.
The best discussion question is: when should a deployed medical model be allowed to update itself from user corrections? The answer is not “always.” It depends on auditability, data governance, expert verification, rollback, monitoring, and whether the correction truly represents ground truth.
Caveats
- User-refined adaptation signals can be wrong if the final correction is wrong.
- Online updates can drift if the feedback stream is biased.
- Medical deployment needs rollback, monitoring, privacy review, and clinical validation.
- Reported segmentation improvements do not equal patient-safety proof.
- The listed code repository may help reproducibility, but it does not make the method plug-and-play for clinical settings.
Best LearnAI use
Use You Point, I Learn as the concrete example of online adaptation from human feedback. Use CARE as the concrete example of evidence-grounded medical reasoning. Together they make a useful paired lesson: in high-stakes AI, the system should expose where it looked, why it answered, and how expert feedback changes future behavior.