How to Evaluate Assessment Instruments & Their Results

Instructor: Kerry Gray

Kerry has been a teacher and an administrator for more than twenty years. She has a Master of Education degree.

Assessments play a big part in the decision-making process of educators. In this lesson, we will look at key characteristics such as validity, reliability and variability, and discuss how they can help educators better evaluate the effectiveness of their instruments.

Understanding the Characteristics of Assessments

In order to effectively use a test to make classroom decisions, there are some questions teachers need to ask. Is it aligned to the state curriculum or does it consist of an assortment of national curriculum resources? Is it intended to measure content knowledge or other skills and abilities? How is it scored? Are results based on students' mastery of standards or are students compared to their peer group? The answers to these questions, as well as other information about the reliability, validity, variability, and biases in an assessment, indicate whether or not the test results may be accurately used for the intended purpose. Let's find out more about how to evaluate the use of a test instrument.

Validity

What is validity? Validity is determined by how well an assessment aligns to its intended purpose. Degrees of validity will likely exist and may change over time based on continued research. For example, if a few of the test items come from outside of the required curriculum on a criterion-referenced achievement test, those items will skew the degree of validity for the test, but will not necessarily invalidate the entire assessment. Frequently, it is not the test itself, but the way the test scores are used that determines whether or not the scores are valid. As test researchers continue to gather evidence about test results, data may surface that either supports or challenges the validity of a test.

Test items should match test goals. For example, a test that is intended to measure a student's achievement in TExES Physical Science must contain only those test items that match the standards. In addition, the weighting of questions should be aligned to both the complexity and importance of each standard. For example, if the standard requires students to be able to describe the elements using the periodic table, there will likely be some simpler questions, such as defining the atomic number of an element, but there will also be some more complex questions that are more heavily weighted, such as describing the atomic properties of halogens.

Validity also includes the social consequences of how the test is used. For example, are high school students who pass the End of Course Exam in Chemistry I ready for Chemistry II? Should students who have not taken Chemistry I, but can pass the test, be allowed to enroll in Chemistry II? This scenario outlines the importance of making sure the intended purpose of an assessment instrument is clearly outlined and that all social implications are considered.

Reliability

What is the difference between validity and reliability? Reliability is the degree to which test results would remain consistent if the test were repeated. A test may be considered reliable even if it is not valid. For example, students may receive high scores each time they take a test that is too easy. This test is reliable because it provides consistent information. It may not be considered valid, however, because the test items are not aligned with the complexity of the standards. If a test is not reliable though, it cannot be considered valid.

To unlock this lesson you must be a Study.com Member.
Create your account

Register to view this lesson

Are you a student or a teacher?

Unlock Your Education

See for yourself why 30 million people use Study.com

Become a Study.com member and start learning now.
Become a Member  Back
What teachers are saying about Study.com
Try it risk-free for 30 days

Earning College Credit

Did you know… We have over 200 college courses that prepare you to earn credit by exam that is accepted by over 1,500 colleges and universities. You can test out of the first two years of college and save thousands off your degree. Anyone can earn credit-by-exam regardless of age or education level.

To learn more, visit our Earning Credit Page

Transferring credit to the school of your choice

Not sure what college you want to attend yet? Study.com has thousands of articles about every imaginable degree, area of study and career path that can help you find the school that's right for you.

Create an account to start this course today
Try it risk-free for 30 days!
Create an account
Support