CasePlay: Self-Play Reinforcement Learning from Case Reports for Medical Reasoning
To address reliance on expert-labeled questions or costly synthetic data in reinforcement learning for medical LLMs, CasePlay proposes document-grounded self-play directly using unlabeled clinical case reports, with a knowledge-conditioned proposer and a rubric-judge reasoner to mitigate knowledge-point fixation and difficulty-quality mismatch, improving an SFT warm-start model by 3.1 points on average across seven medical benchmarks.
01 ABSTRACT
CasePlay investigates whether unlabeled clinical case reports can be converted into reinforcement learning training signals to improve LLM clinical reasoning. The authors identify knowledge-point fixation and difficulty-quality mismatch in existing self-play methods and propose a knowledge-conditioned proposer and a rubric-judge reasoner to address them. Experiments across seven medically related benchmarks show gains over existing self-play baselines and a 3.1-point average improvement over an SFT warm-start model, including a 6.3-point gain on LiveClin.
02 KEY FINDINGS
- CasePlay performs document-grounded self-play directly on unlabeled clinical case reports, avoiding reliance on expert-labeled questions or costly synthetic data.
- A knowledge-conditioned proposer anchors question generation to case-specific knowledge points to broaden evidence coverage and mitigate knowledge-point fixation.
- A rubric-judge reasoner adds rubric-based feedback on clue integration, reasoning depth, and option quality beyond answer accuracy, guiding toward reasoning-intensive questions.
AI GENERATED SUMMARY / DISCOVERED BY ARXIV CS.CL · ARXIV CS.LG