Question Quality and Point Biserial Correlation: What Makes a Good Test Question
Point biserial correlation is like a quality check for your test questions. It tells you whether each question is doing its job of separating students who understand the material from those who don’t.
The Connection:
High-Quality Questions have positive correlations (0.3 to 0.7):
- Students who do well overall tend to get these questions right
- Students who struggle overall tend to get these questions wrong
- The question successfully identifies who knows the material
Poor-Quality Questions have low or negative correlations:
- Getting the question right or wrong seems random
- Sometimes your best students miss it while struggling students get it right
- The question isn’t measuring what you think it’s measuring
Examples of High-Quality Questions (Good Correlation):
Math Example: “Solve: 4x – 12 = 20”
- Students who understand algebra get it right
- Students who don’t understand algebra get it wrong
- Result: Strong positive correlation = good question
Reading Example: “What was the main character’s primary motivation?”
- Students with good comprehension skills answer correctly
- Students with weaker comprehension struggle
- Result: Good correlation = quality question
Examples of Poor-Quality Questions (Bad Correlation):
Tricky Wording: “Calculate the area of a rectangle that is not 5 meters by 3 meters”
- Even strong math students might misread this confusing question
- Some weaker students might guess correctly
- Result: Low or negative correlation = poor question
Unfair Knowledge: “What year was the Battle of Hastings?” (on a math test)
- Math ability has nothing to do with getting this right
- Students succeed or fail based on random historical knowledge
- Result: Near-zero correlation = irrelevant question
What Different Correlations Tell You:
0.4 to 0.7: Excellent question – Keep it! 0.2 to 0.4: Decent question – Minor improvements might help 0.0 to 0.2: Weak question – Consider major revision Below 0.0: Problem question – Remove or completely rewrite
Quick Quality Check Questions:
- Do my strongest students tend to get this question right?
- Do my struggling students tend to miss it?
- Is the question testing what I actually taught?
- Could students get it wrong for reasons unrelated to understanding?
Bottom Line: Good test questions help you identify which students have mastered the material. If a question’s point biserial correlation is low, it’s not doing that job effectively and needs improvement.