Yolanda has taught college Psychology and Ethics, and has a doctorate of philosophy in counselor education and supervision.
Introduction to Reliability
Suppose you are a child psychologist looking for a way to measure test anxiety in children. One of your colleagues suggests that you use the Children's Test Anxiety Scale (CTAS). You thank your colleague and purchase the test.
You find information that indicates that the CTAS is a valid measure. That is, that the CTAS is able to accurately measure the anxiety level of children. You begin to wonder if the results of the CTAS are consistent, or reliable. The CTAS would be considered reliable if it would produce the same results each time that we use it.
You do not find much information about the reliability, so you decide to measure the reliability on your own. There are four different types of reliability that you could examine:
- Inter-rater reliability: The degree to which raters are being consistent in their observations and scoring in instances where there is more than one person scoring the test results.
- Internal consistency: The degree to which all of the items on a test measure the same construct.
- Alternate forms reliability: In instances where there are two forms of a test that measure the same construct, the degree to which the results on the two tests are consistent.
- Test-retest reliability: The degree to which the results are consistent over time.
What Is Test-Retest Reliability?
Test-retest reliability is the most common measure of reliability. In order to measure the test-retest reliability, we have to give the same test to the same test respondents on two separate occasions. We can refer to the first time the test is given as T1 and the second time that the test is given as T2. The scores on the two occasions are then correlated. This correlation is known as the test-retest-reliability coefficient, or the coefficient of stability.
The closer each respondent's scores are on T1 and T2, the more reliable the test measure (and the higher the coefficient of stability will be). A coefficient of stability of 1 indicates that each respondent's scores are perfectly correlated. That is, each respondent score the exact same thing on T1 as they did on T2. A coefficient correlation of 0 indicates that the respondents' scores at T1 were completely unrelated to their scores at T2; therefore the test is not reliable.
So, how do we interpret the coefficients of stability that are between 1 and 0? The following guidelines can be used:
- 0.9 and greater: excellent reliability
- Between 0.9 and 0.8: good reliability
- Between 0.8 and 0.7: acceptable reliability
- Between 0.7 and 0.6: questionable reliability
- Between 0.6 and 0.5: poor reliability
- Less than 0.5: unacceptable reliability
The Importance of Time
When we use test-retest reliability, we are making the assumption that there is not a significant change in the construct that we are measuring between the first and second times that we measure it. If we give the same measure twice, the correlation between T1 and T2 is affected by the amount of time that passes between the two. The less time between T1 and T2, the higher the correlation; conversely, the larger the time gap between T1 and T2, the lower the correlation. This is due to the fact that the closer the time gap, the more similar the contributing factors to error.
Suppose that you took a group of 104 children and had them each complete the CTAS. One month later, you have them complete the CTAS again. You correlate their scores and get a coefficient of stability of 0.92. Your findings indicate that the CTAS has excellent test-retest reliability. Therefore, the CTAS results are consistent over time.
Suppose that a group of high school seniors took the Wechsler Adult Intelligence Scales (WAIS) at the end of the first semester of their senior year. Six months later, the same students took the WAIS again. The scores were correlated, yielding a coefficient of stability of 0.7. This is an acceptable reliability coefficient.
Upon further analysis, you notice that T2 scores were higher than T1 scores. Given that the students were provided with additional knowledge and gained new skills in those six months between T1 and T2, you would expect a slight increase in scores on the WAIS.
Test-retest reliability is one of four types of reliability. Test-retest reliability refers to the degree to which test results are consistent over time. In order to measure test-retest reliability, we must first give the same test to the same individuals on two occasions and correlate the scores. The resulting correlation is the coefficient of stability - the more similar the scores, the higher the correlation. The test-retest reliability is heavily dependent upon the amount of time that passes between the two occasions. So, now you know all about test-retest reliability. And remember, just because test results are reliable, it does not mean that they are valid; but a valid test is always reliable!
The information you glean from this lesson can be drawn upon when you attempt to:
- List the four types of reliability
- Interpret test-retest reliability
- Indicate the effects of time on test-retest reliability
- Cite pertinent examples
To unlock this lesson you must be a Study.com Member.
Create your account
Register to view this lesson
Unlock Your Education
See for yourself why 30 million people use Study.com
Become a Study.com member and start learning now.Become a Member
Already a member? Log InBack