What is Parallel Computing? - Performance & Examples

Instructor: David Gloag

David has over 40 years of industry experience in software development and information technology and a bachelor of computer science

In this lesson, we'll take a look at parallel computing. We will learn what this means, its main performance characteristic, and some common examples of its use.

The Inherent Need for Speed

We want things done fast. If we can get it by the end of the week, we actually want it tomorrow. If we can get it tomorrow, we would really like it today. Let's face it, we're a society that doesn't like to wait. If you don't agree, just think about the last time you stood in line at a fast food restaurant and had to wait for more than a couple of minutes for your order. This idea extends to other things like the weather. We routinely check the hourly forecast to see what the weather will be like on our commute to and from work. We expect that there is a computer, behind the scenes, providing this information. But did you know that a single computer is often not up to the task? That is where the idea of parallel computing comes in.

Parallel Computing

In simple terms, parallel computing is breaking up a task into smaller pieces and executing those pieces at the same time, each on their own processor or on a set of computers that have been networked together. Let's look at a simple example. Say we have the following equation:

  • Y = (4 x 5) + (1 x 6) + (5 x 3)

On a single processor, the steps needed to calculate a value for Y might look like:

  • Step 1: Y = 20 + (1 x 6) + (5 x 3)
  • Step 2: Y = 20 + 6 + (5 x 3)
  • Step 3: Y = 20 + 6 + 15
  • Step 4: Y = 41

But in a parallel computing scenario, with three processors or computers, the steps look something like:

  • Step 1: Y = 20 + 6 + 15
  • Step 2: Y = 41

Now, this is a simple example, but the idea is clear. Break the task down into pieces and execute those pieces simultaneously.

Performance Characteristics

The main performance characteristic is an increase in speed. If you use a single computer, and it takes X amount of time to perform a task, then using two of the same computers should cut the time it takes to perform that same task in half. If you use three, then it should take a third of the time for the same task, and so on. This is a pretty easy concept to grasp.

However, in practical terms, this isn't always true. In particular:

  • It doesn't take into account that the task might not be divisible.
  • It doesn't take into account that the divisions may not be equal.
  • It doesn't take into account the overhead associated with splitting the task up.

So, while there is usually a performance increase, that increase doesn't follow any set formula.

To unlock this lesson you must be a Member.
Create your account

Register to view this lesson

Are you a student or a teacher?

Unlock Your Education

See for yourself why 30 million people use

Become a member and start learning now.
Become a Member  Back
What teachers are saying about
Try it risk-free for 30 days

Earning College Credit

Did you know… We have over 200 college courses that prepare you to earn credit by exam that is accepted by over 1,500 colleges and universities. You can test out of the first two years of college and save thousands off your degree. Anyone can earn credit-by-exam regardless of age or education level.

To learn more, visit our Earning Credit Page

Transferring credit to the school of your choice

Not sure what college you want to attend yet? has thousands of articles about every imaginable degree, area of study and career path that can help you find the school that's right for you.

Create an account to start this course today
Try it risk-free for 30 days!
Create an account