How to Test Fairness in Psychological Measurements

Instructor: Della McGuire

Della has been teaching secondary and adult education for over 20 years. She holds a BS in Sociology, MEd in Reading, and is ABD on the MComm in Storytelling.

In this lesson, we will look at how to check for fairness in psychological measurements and assess for bias using five relevant considerations. We will also look at the procedures used for eliminating bias and ensuring fairness in testing.

Psychological Measurements

Our society places high stakes on psychological tests (i.e. language tests, IQ tests, psychological evaluations, standardized tests, etc.) and performance on these tests often has significant implications. Occupational opportunities, educational opportunities, and Social Security determinations are based on these tests, so they may have intended positive or unintended negative consequences to individuals.

Problem of Bias

An IQ test has 4 images and the task is to identify which one one doesn't belong and why. The images are the Washington Monument, the Lincoln Memorial, the Vietnam War Memorial and the Statue of Liberty. Someone from outside the US might recognize these as major landmarks, but may not have enough geographical knowledge to realize that the Statue of Liberty is in New York, but the others are in DC, so the results would be skewed.

These tests are the gatekeepers for opportunity, and the results of psychological testing can have an impact on one's survival. In the testing process, individuals or groups should not be disadvantaged by unrelated factors or areas not measured by the test. For these reasons, issues of test fairness must be addressed to ensure no biases are present in these professional determinations.

Research on the test must demonstrate that members of various subgroups within the population can fairly and equivalently use these measurements. There are no tests available that can fairly represent all people in any cultural or language group. Those giving and scoring a test must acknowledge this problem and note any possible implications on the interpretation of scores.

Any test will usually represent the dominant culture, but cultural bias will generate inaccurate test results, representing an error in measurement. This kind of cultural test bias can happen when the instrument used as an assessment relies too heavily on culturally related values and variables such as ethnicity, race, class, gender, or level of education. As in the IQ test question mentioned before, measurements that rely too heavily on personal experience and background knowledge can be biased against those who do not have a wealth of experience to inform their performance on the test.

For example, in order to assess language acquisition through writing skills, a more accurate measurement would come from an essay where the topic is chosen freely, rather than a topic given to the language learner. An unfamiliar topic may require research or background knowledge, and, if the measurement is meant to determine language and writing, the instrument should not involve topics that require extensive research.

Relevant Considerations

In order to identify bias and determine fairness in psychological measurements, researchers consider 5 relevant issues that help identify cultural equivalence in psychological testing. Cultural equivalence means that when interpreting psychological measurements, they should be similar or equal across a variety of populations. Test administrators should look closely at the following factors in determining if a test is fair or if it exhibits cultural test bias.


Functional considerations insist that the construct being measured occurs with equal frequency across groups. Requiring some ethnic groups to take annual standardized tests, when other students only take them in certain grades, would skew the measurement.


Conceptual considerations determine whether the item information is familiar across groups and means the same thing in various cultures. For example, a water hose attached to an outside spigot is not a common occurrence outside the US, especially in places where there are water shortages. A test item referring to a water hose is biased as an entirely unknown concept in most of the world.


Scalar considerations determine whether average score differences reflect the same degree, intensity, or magnitude for different cultural groups in order to be fair and unbiased. Looking at the scale (extent or degree) to which an assessment represents bias can indicate its severity.


Linguistic considerations determine whether the language used has similar meaning across groups. Where there are language barriers, the test may not be valid. Even across different dialects of English, this can be an issue, like a child from England trying to imagine why someone would say pants (underwear) when they mean trousers (pants).


Metric considerations determine whether the scale measures the same behavioral qualities or characteristics and if the test has similarly measured properties in different cultures. This factor looks at whether the test itself can measure similar traits or properties across different populations.

To unlock this lesson you must be a Member.
Create your account

Register to view this lesson

Are you a student or a teacher?

Unlock Your Education

See for yourself why 30 million people use

Become a member and start learning now.
Become a Member  Back
What teachers are saying about
Try it risk-free for 30 days

Earning College Credit

Did you know… We have over 160 college courses that prepare you to earn credit by exam that is accepted by over 1,500 colleges and universities. You can test out of the first two years of college and save thousands off your degree. Anyone can earn credit-by-exam regardless of age or education level.

To learn more, visit our Earning Credit Page

Create an account to start this course today
Try it risk-free for 30 days!
Create An Account