Copyright

Linear Regression Model: Definition, Equation & Example

Instructor: Sharon Linde
Ever hear of the linear regression model? If you are confused about what linear regression is, come inside this less for an explanation, how it is used and calculated. Then test your new skills with a short quiz.

Defining the Linear Regression Model

What does the term linear regression mean? The Merriam-Webster Dictionary defines regression as 'a trend or shift toward a lower or less perfect state', or some condition or thing that used to be better and is now getting worse. In math, however, regression is used more in the sense of something going from the chaotic actual data of observations to the simpler estimate of a line fitted to the data. In other words, the linear regression model describes the process of taking observed data and getting a 'best fit' line to describe the relationship of two variables.

Calculating Linear Regression

Well, now that we know what linear regression is, how do we calculate it? This question has many answers depending on the type of data being discussed: a Google search will reveal perhaps a dozen different techniques for fitting this hypothetical regression line. However, by far the most common approach is to use the least squares method, which has the following steps:

1.) Assume there is a linear relationship between the two variables, so Y' = mX + b, where Y' is the predicted, or fitted value.

2.) For every observed point there will be a difference between the observed value Y and the predicted value Y'. This difference is expressed as (Y-Y').

3.) (Y-Y') is then multiplied by itself, or squared.

4.) Vary the slope and y-intercept, or m and b in the equation, to produce the lowest sum of these squares.

If this sounds complicated that's because it is quite time consuming to perform this method by hand calculations - even for just a few data points. In practice, this is all done very quickly by computer software. Through some very complicated math, which we won't get into here, we can use the least squares method to calculate the slope and y-intercept, which are shown below. Both are non-standard formulas specific to the values we're working with.

Slope
Formula for Slope2

Y-Intercept
Formula for Y-Intercept

Linear Regression in Real Life

Let's suppose you work for a toy manufacturer, and they're trying to create a cost-volume analysis for a particular product line. They give you the following data:

Volume (1000 units) Total Costs ($1000)
1 0.9
2 1.6
3 1.9
4 2.6
5 3.0

You're asked to come up with an equation that will estimate, as accurately as possible, the total costs given a certain proposed volume of production. You decide to use the least squares method to compute the slope and y--intercept of the linear regression line for this data.

To do this, you'll have to compute the following five values with the formulas provided. The formula for standard deviation includes an (n - 1) denominator, which is useful for working with limited data sets. When computing correlation, you'll also have to compute covariance. Covariance shows how a change in one variable relates to a change in another variable, or a measure of their linear relationship.

1.) Mean, or average, of X

2.) Mean, or average, of Y

3.) Standard deviation, or numerical spread, around the mean of X

4.) Standard deviation, or numerical spread, around the mean of Y

5.) Correlation, or connection, between X and Y, which also requires a calculation for covariance

Using the information provided by the toy manufacturer, let's do the math, beginning with the formula for the mean.

1.) Mean of X

Formula for Mean

X = (1 + 2 + 3 + 4 + 5) / 5 = 3.

2.) Mean of Y

Y = (0.9 + 1.6 + 1.9 + 2.6 + 3.0) / 5 = 2.

3.) Standard deviation of X:

Standard Deviation2

{((1 - 3)^2 + (2 - 3)^2 + (3 - 3)^2 + (4 - 3)^2 + (5 - 3)^2)/(5 - 1)}^0.5 = 1.58.

To unlock this lesson you must be a Study.com Member.
Create your account

Register to view this lesson

Are you a student or a teacher?

Unlock Your Education

See for yourself why 30 million people use Study.com

Become a Study.com member and start learning now.
Become a Member  Back
What teachers are saying about Study.com
Try it risk-free for 30 days

Earning College Credit

Did you know… We have over 200 college courses that prepare you to earn credit by exam that is accepted by over 1,500 colleges and universities. You can test out of the first two years of college and save thousands off your degree. Anyone can earn credit-by-exam regardless of age or education level.

To learn more, visit our Earning Credit Page

Transferring credit to the school of your choice

Not sure what college you want to attend yet? Study.com has thousands of articles about every imaginable degree, area of study and career path that can help you find the school that's right for you.

Create an account to start this course today
Try it risk-free for 30 days!
Create an account
Support