What Does “Question Difficulty” Mean?
Think of question difficulty (Rasch) like the height of a hurdle in track and field. Each question on your test is like a hurdle – some are low (easy questions) and some are high (hard questions). Question difficulty (Rasch) tells you how much student ability is needed for them to have a 50-50 chance of getting that question right.
How We Measure Question Difficulty
The graph shows difficulty on a number scale that goes from about -4.5 to +4.0:
- Negative numbers (like -2.0 or -4.5) = Easier questions
- Zero = Medium difficulty
- Positive numbers (like +2.0 or +4.0) = Harder questions
🎯 What This Graph Shows You
The blue bars show you two important things:
1. Where Your Students Are
The height of each bar tells you how many of your students are at that ability level. The tallest bars (around 1.0-2.0) show where most of your students are performing.
2. How Well Your Test Matches Your Students
The goal: You want your test questions to be at the right difficulty level for your students.
Too hard: Lots of questions at +3.0 or +4.0 when most students are struggling at lower levels
Good match: Questions that line up with where most students are (the tall bars)
Too easy: Lots of questions in the negative numbers when most students are performing higher
Reading the Results
Right-Skewed Distribution:
The majority of the questions cluster in the positive difficulty range (0.5 to 3.0 logits), indicating most questions are above average difficulty.
Most Questions Are Challenging: This test has more difficult questions than easy ones. Most questions fall in the “moderately hard” to “hard” categories.
Two Main Groups of Questions:
- Largest group: Medium-hard questions (about 350+ questions)
- Second group: Hard questions (about 250 questions)
- Smallest group: Easy questions (very few)
What’s Missing
Not Enough Easy Questions: This test has very few questions that beginning students can answer correctly. This means:
- Students who are still learning basics might struggle with the entire test
- Lack of accurate information about what struggling students actually know
Good News: There aren’t too many extremely easy or extremely hard questions, which is actually ideal for most assessments.
Sparse Extreme Ranges:
- Minimal items in very easy (-3.0 to -2.0) or very hard (3.0+) ranges
- Good practice as extreme items provide little discrimination
Implications for Assessment Quality
Potential Challenges
The Test Might Be Too Hard: When most questions are difficult:
- Average students may feel overwhelmed or anxious
- Students might guess more often instead of showing their real knowledge
- You might not get a clear picture of what lower-performing students have learned
Limited Information About Beginning Learners: Your test works best for students who already have solid foundational skills, but gives less useful information about students who are still building those foundations.
What This Means for Your Classroom
✅ When your test is well-matched:
- Students feel appropriately challenged (not too frustrated, not too bored)
- You get better information about what each student actually knows
- You can more easily identify which students need extra help
- Your grades better reflect true differences in student understanding
⚠️ Signs you might need to adjust:
- Most students are getting everything right (questions too easy)
- Most students are struggling with everything (questions too hard)
- You can’t tell the difference between your medium and high performers
Actionable Recommendations
Review Your Current Test
- Look at your easiest questions: Do you have enough questions that measure basic, foundational skills?
- Check your hardest questions: Make sure they’re appropriately challenging and not just confusing or poorly written
- Consider your students: Does this difficulty mix match your actual classroom? If you have students at many different levels, you might need more variety in question difficulty
For Future Assessments
- Add more accessible questions: Include questions that test important basic concepts most students should be able to answer
- Balance your question types: Aim for a good mix across difficulty levels – some easy, some medium, some hard
- Think about your goals: What do you most need to know about each student’s learning?
Quick Tips for New Teachers
- Start with where your students are – use this data to see their current ability levels
- Aim for the “sweet spot” – most of your questions should target where most students are performing
- Include some variety – a few easier questions for confidence, a few harder ones for challenge
- Adjust as needed – if the graph shows a mismatch, consider revising questions for next time
Questions to Ask
- Do I have students who are still working on foundational skills?
- Am I getting useful information about all my students, or mainly just the higher-performing ones?
- Does this test help me identify what struggling students have mastered?
Remember
The “right” difficulty distribution depends on the students and teaching goals. If most of the students are advanced learners, having more challenging questions makes sense. But for a typical mixed-ability classroom, consider questions that can accurately measure learning at all levels. The best test isn’t the hardest test – it’s the one that gives you the clearest picture of what each student knows and can do!
Bottom Line: A good test should give clear, useful information about what every student knows and can do, not just the highest achievers.