The situation
Fixed practice tests give every candidate the same questions, so they say little about a learner's real level across medical subjects.
Computer adaptive testing for medical licensing exams
An adaptive test for NCLEX-RN, NCLEX-PN and USMLE candidates that adjusts question difficulty to each learner as they answer.
A US medical licensing exam preparation platform
01 · Context
02 · Challenge
What had to be true
Estimate each learner's ability as the test runs, pick questions they have about a 50% chance of answering, cover subjects by exam weightage, and stop once the estimate is reliable.
03 · What we built
The system
- A raw response matrix, cleaned and used to calibrate the item bank
- Item response theory with maximum likelihood estimation of ability
- Next-question selection at around 50% success probability, weighted across medical subjects
- Stopping rules once ability converges and error is minimised
- Two follow-on projects: peer performance estimation and a customised study calendar from the baseline test
04 Architecture
How it works, step by step.
Step through the system, or let it play.
- Response data cleaned
- Item bank calibrated
- Ability estimated after each answer
- Next question chosen
- Test stops when estimate converges
05 · Outcome
Shorter, sharper tests that adapt to each candidate, plus a study plan built from their results.
06 · Stack
Next case · Computer vision · ResearchCompNet: classification that holds up under occlusion