The Reliability Coefficient and the Reliability of Assessments

An error occurred trying to load this video.

Try refreshing the page, or contact customer support.

Coming up next: Performance Assessments: Product vs. Process

You're on a roll. Keep up the good work!

Take Quiz Watch Next Lesson
 Replay
Your next lesson will play in 10 seconds
  • 0:05 Definition
  • 1:27 Conditions That Impact…
  • 2:05 Reliability Coefficient
  • 3:09 Types of Reliability
  • 5:40 Increasing Reliability…
  • 6:26 Lesson Summary
Add to Add to Add to

Want to watch this again later?

Log in or sign up to add this lesson to a Custom Course.

Login or Sign up

Timeline
Autoplay
Autoplay
Speed

Recommended Lessons and Courses for You

Lesson Transcript
Instructor: Melissa Hurst
How are test scores affected by day-to-day changes of a student? Do different people rate students' performances the same? These questions are addressed through the understanding of reliability. This lesson will define reliability, explain how reliability is measured, and explore methods to enhance reliability of assessments in the classroom.

Definition

Student One: I'm glad that is over. It's nerve racking to perform and be evaluated by three teachers.

Student Two: I agree. I also worry about how each individual teacher will score us. I hope they use the same criteria!

Student One: Oh, you are referring to reliability of the scores. Do you know about reliability?

Student Two: Not really. I've never used that term before.

Student One: Oh! I'll explain!

Reliability is defined as the extent to which an assessment yields consistent information about the knowledge, skills, or abilities being assessed. A reliable assessment is replicable, meaning it will produce consistent scores or observations of student performance.

For example, our singing performances should result in similar scores from the three teachers. If one teacher gives us a score of 10 out of 10, and the other gives us a score of 2 out of 10, the scores are not considered reliable.

Student Two: Oh, okay. So it seems like many factors could impact the reliability of a test or performance.

Student One: You are right.

Conditions That Impact Reliability

Student One: There are many conditions that impact reliability. They include:

  • Day-to-day changes in the student (such as energy level, motivation, emotional stress, and hunger)
  • Physical environment (which includes classroom temperature, outside noises, and distractions)
  • Administration of the assessment (which includes changes in test instructions and differences in how the teacher responds to questions about the test)
  • Test length (generally, the longer the test, the lower the reliability)
  • Subjectivity of the test scorer

Measurement of Reliability: Reliability Coefficient

Student Two: So, how is reliability measured?

Student One: Reliability is determined by comparing two sets of scores for a single assessment (such as two rater scores for the same person) or two scores from two tests that assess the same concept. These two scores can be derived in different ways depending on the type of reliability being assessed.

Once we have two sets of scores for a group of students or observers, we can determine how similar they are by computing a statistic known as the reliability coefficient.

The reliability coefficient is a numerical index of reliability, typically ranging from 0 to 1. A number closer to 1 indicates high reliability. A low reliability coefficient indicates more error in the assessment results, usually due to temporary factors that we previously discussed. Reliability is considered good or acceptable if the reliability coefficient is .80 or above.

Types of Reliability

Student One: There are multiple types of reliability.

Inter-Rater Reliability

In other words, do different people score students' performances similarly? This type of reliability is used to assess the degree to which different observers or scorers give consistent estimates or scores. For example, we performed in front of three teachers who scored us individually. High inter-rater reliability would indicate each teacher rated us similarly.

Test-Retest Reliability

It is used to assess the consistency of scores of an assessment from one time to another. The construct to be measured does not change - only the time at which the assessment is administered changes. For example, if we are given a test in science today and then given the same test next week, we could use those scores to determine test-retest reliability. Test-retest reliability is best used to assess things that are stable over time, such as intelligence. Reliability is typically higher when little time has passed between administrations of assessments.

To unlock this lesson you must be a Study.com Member.
Create your account

Register to view this lesson

Are you a student or a teacher?

Unlock Your Education

See for yourself why 30 million people use Study.com

Become a Study.com member and start learning now.
Become a Member  Back
What teachers are saying about Study.com
Try it risk-free for 30 days

Earning College Credit

Did you know… We have over 160 college courses that prepare you to earn credit by exam that is accepted by over 1,500 colleges and universities. You can test out of the first two years of college and save thousands off your degree. Anyone can earn credit-by-exam regardless of age or education level.

To learn more, visit our Earning Credit Page

Transferring credit to the school of your choice

Not sure what college you want to attend yet? Study.com has thousands of articles about every imaginable degree, area of study and career path that can help you find the school that's right for you.

Create an account to start this course today
Try it risk-free for 30 days!
Create An Account
Support