Copyright

How Novelty Effects, Test Sensitization & Measurement Timing Can Threaten External Validity

How Novelty Effects, Test Sensitization & Measurement Timing Can Threaten External Validity
Coming up next: Reactivity of Experimental Arrangements & Assessment: Threats to External Validity

You're on a roll. Keep up the good work!

Take Quiz Watch Next Lesson
 Replay
Your next lesson will play in 10 seconds
  • 0:07 External Validity
  • 1:09 Novelty Effects
  • 2:21 Test Sensitization
  • 3:45 Measurement Timing
  • 4:43 Lesson Summary
Save Save Save

Want to watch this again later?

Log in or sign up to add this lesson to a Custom Course.

Log in or Sign up

Timeline
Autoplay
Autoplay
Speed

Recommended Lessons and Courses for You

Lesson Transcript
Instructor: Natalie Boyd

Natalie is a teacher and holds an MA in English Education and is in progress on her PhD in psychology.

The purpose of research is to find things out about the world at large. But there are several things that can get in the way of external validity. In this lesson, we'll examine novelty effects, test sensitization, and measurement timing.

External Validity

Graham is a psychologist who studies how technology can help students learn. He wants to run an experiment to see if a new app can help students learn math better, so he gets several teachers to use the app in their classrooms. After the students have practiced on the app for two weeks, Graham gives them a math test and discovers that they're doing better at math.

Since the students did better after using the app, does that mean that every student should be using the app because it makes students better at math? Maybe, but there are a lot of other things to consider before Graham can say for sure that the app can make students better at math.

External validity is the extent to which the results of a research study can be generalized to the world at large. In Graham's case, he wants the results of his study to be true of all classrooms. But there are some things that can bring down the external validity of a study. Let's look closer at three threats to external validity: novelty effects, test sensitization and measurement timing.

Novelty Effects

So, the teachers of several classes give students the app, and after a couple of weeks, the kids do better. That seems pretty straightforward, right? The app must be the cause of the increase in math skills.

But wait! What if the app didn't actually cause the increase in math skills, but something else did? For example, what if the students were used to using old-fashioned flashcards and worksheets to do math? The app is new and different, so the students pay more attention to it. Because they are paying more attention, they learn the math better. It's not the app; it's that it's new and different.

Novelty effects occur when the results of a study are due to the novelty (or newness) of a treatment. Essentially, in the case of novelty effects, anything different makes a difference. Novelty effects pose a threat to external validity because they make it difficult to know if the results of the study are due to a treatment that works or due to the novelty of a treatment. In other words, does the app actually teach math, or do the students learn math just because the app is different? Sadly, there's no way that Graham can answer that question based on his study.

Test Sensitization

Another threat to external validity that Graham may face in his study is that of test sensitization, which occurs when taking a test improves the performance of the subjects. For example, remember that after the two weeks with the app, Graham gave the students a math test to see if they had improved their math skills. But what if the test questions actually ended up teaching the students math? What if they are learning math as they are taking the test, which improves their scores?

With test sensitization, the treatment wouldn't work without the test. This can happen at the end of a treatment, as with Graham's test, or it can happen at the beginning of a treatment. For example, let's say that Graham wanted to know how much math the students knew before they use the app. He might give them a math test at the beginning of the study to see what they know.

But what if the test Graham gives the students has a lot of questions about fractions on it? They might realize that they're supposed to be learning fractions, so when they play with the app, they pay attention to the fractions part of it. The app itself might not work as well as it does when the students are given a test.

To unlock this lesson you must be a Study.com Member.
Create your account

Register to view this lesson

Are you a student or a teacher?

Unlock Your Education

See for yourself why 30 million people use Study.com

Become a Study.com member and start learning now.
Become a Member  Back
What teachers are saying about Study.com
Try it risk-free for 30 days

Earning College Credit

Did you know… We have over 200 college courses that prepare you to earn credit by exam that is accepted by over 1,500 colleges and universities. You can test out of the first two years of college and save thousands off your degree. Anyone can earn credit-by-exam regardless of age or education level.

To learn more, visit our Earning Credit Page

Transferring credit to the school of your choice

Not sure what college you want to attend yet? Study.com has thousands of articles about every imaginable degree, area of study and career path that can help you find the school that's right for you.

Create an account to start this course today
Try it risk-free for 30 days!
Create an account
Support