Many psychologists and teachers complete research studies. How can you tell if a study was done well? This lesson will cover many criteria for a good quality study, including types of reliability and validity.
Imagine you're an elementary school teacher, and you want to study whether you can predict how well children will learn spelling by assessing their personalities. You could complete this study in several different ways. Let's say that you give students a questionnaire that is supposed to measure a variety of different personality traits. Then you give the students a spelling test. How do you know if the study was done well or if it was of good quality?
This lesson will discuss ways that you can assess the quality of any research study, including studies that you read in professional journals, studies you hear about in the newspaper or studies that you design yourself.
This lesson covers the concepts of reliability and validity. In a different lesson, you'll also learn about reliability and validity, but it will be in a different context. Psychologists discuss reliability and validity when they talk about whether assessments are of good quality, such as whether a certain intelligence test really measures intelligence. Assessment in school is also relevant to reliability and validity, but there are different types of reliability and validity for assessments and for research studies. This lesson focuses on research studies only.
The first criterion for whether a study is done well is called reliability. Reliability is a measure of the consistency of scores over time. There are different types of reliability in terms of research studies.
The first type of research reliability is whether the results are reliable across time; that is, if the same researcher would get the same results if he or she did the same study at a different time. Let's go back to the example of a teacher who tests whether personality predicts spelling ability. If that teacher does the study in one year and gets certain results, the teacher would want to do the exact same study the next year and the year after that to see if the results are stable across time. If they are, then the teacher can be much more confident in making conclusions. If the results are not reliable across time, then the teacher should be more cautious in making any conclusions.
A second type of reliability would be if a different teacher wanted to try doing the same study. Imagine the first teacher calls a friend in a different school and tells that friend about the results. Then, the second teacher does the same study on a different set of children in a different school. If both teachers get the same results, then we can say that the results are reliable across samples or across groups of people who are participating in the study. We could also say that the results are reliable across experimenters.
All of these types of reliability are important so researchers can be confident in saying that results are true across time and across different settings.
Besides reliability, validity is also important in research studies. Just like with reliability, there are different kinds of validity within a research study.
The first type of validity is called internal validity. Within an experiment, you are usually making conclusions about some kind of cause and effect relationship, such as concluding that personality affects spelling ability. So, internal validity is defined as making sure that the cause-effect relationship identified in the study is really there, and there are no other explanations for the results.
Let's go through an example. Let's say that a teacher decides that personality does predict spelling ability, but maybe the teacher didn't score the personality tests correctly, or maybe there was some other variable predicting spelling ability, such as the age of the children or how many books they read at home. If the teacher didn't set up the experiment correctly to account for these other variables, he or she might come to the wrong conclusion. Another lesson covers how to make a true experiment so that these other variables are covered. For now, just remember that internal validity means that you can be really sure that any conclusions you make from the results of the study are true and correct.
Another type of validity is called external validity. While internal validity was relevant to the inner workings of the experiment, external validity is about the bigger, outside world. The definition of external validity is that the results you get from the sample of participants in your study are true of people outside of the experiment as well. In other words, you want to know that your results are true of general people, not just the people in your study.
For example, let's say that a particular teacher finds that personality traits do affect spelling ability, but imagine that she works in a school that only has girls. She might be able to reach conclusions about how personality affects spelling for girls, but she can't really say that her conclusions are true for boys because she hasn't tested that. She also hasn't tested children of different ages or living in different states or even children who have different teachers. So, she must be very careful about how she generalizes her conclusions. If she concludes that personality predicts spelling ability in every child, that conclusion doesn't have good external validity because her conclusion is simply not valid without testing other types of children.
The final type of validity is called ecological validity. Ecological validity refers to whether the results have meaning in the real lives of everyday people. You could imagine a situation where an experiment is so strange and specific that it doesn't apply much to everyday life. For example, let's say a teacher designed a study to ask whether spelling ability increases or decreases when children have to feed elephants, jump up and down and eat pizza, all at the same time. While this study might be entertaining to watch, it really doesn't have much usefulness to the real world; therefore, it wouldn't have much ecological validity.
In summary, any time a researcher attempts a research study, he or she needs to consider several criteria for how well that study is done. In general, reliability measures consistency of scores across time or different contexts. There are several different types of reliability. In general, validity measures if the results of a given study are accurate, true for different kinds of people and relevant to the real world. You can use these criteria when you're designing your own research studies, or when you are reading about other people's research studies, to decide if they are of high quality.
After watching this lesson, you should be able to:
- Define reliability and validity as they relate to research studies
- Identify and describe the different types of reliability and validity