Random shuffling feels fair. Draw questions at random, the logic goes, and over enough papers everything evens out. But students do not sit enough papers for the law of large numbers to help them. Each individual paper is a small sample, and small random samples drift — toward too easy or too hard, almost never to the middle.
Why random drifts
A pool has a difficulty distribution. Pull from it blind and any single paper inherits the luck of that draw. One section ends up packed with the gentle questions; the next, unintentionally brutal. Students read the inconsistency as unfairness, and they are not wrong.
A difficulty filter fixes the sample, not the pool. You tell it the band you want — say, mostly medium with a tail of hard — and it draws to hit that band every time.
- Comparable papers across sections and across terms.
- A predictable curve, so practice scores actually forecast exam scores.
- Hard questions placed deliberately, not by accident.
Fairness is not randomness. Fairness is every student facing the same difficulty — which is exactly what random refuses to guarantee.
Hassan Tariq
Hassan helps schools read their own test data. He writes about item analysis, difficulty, and designing questions that actually discriminate between students.
Published