Big Data Architecture Patterns

Instructor: David Gloag

David has over 40 years of industry experience in software development and information technology and a bachelor of computer science

More and more, businesses are relying on big data to find answers to their questions. In this lesson, we'll take a look at big data, the architectures it uses, and some patterns associated with those architectures.

The Bigness of Information

Businesses handle a lot of information these days. Accounting firms sift through reams and reams of transactions for their clients, law firms wade through a mountain of supporting evidence and legal precedence protecting their customers, and retailers struggle with the never-ending flow of receipts and returns. It threatens to overwhelm us. But is there a way to get a handle on these situations? A way to gracefully adapt to the rapid influx of information? Some say no. They feel that the amount of information needing attention is simply too big to be handled. Others say yes, putting their faith in a new area of research, an area that deals with these types of scenarios. It's called big data.

What is Big Data?

Big data is the area of science that focuses on large information sets. By large, we mean those that are too big to be handled by conventional means. By today's standards, that means the information can't be handled by applications like Microsoft's Excel, Access, and SQL Server (or their equivalents from companies like Oracle). Even with the power today's machines provide, the volume of information used pushes existing boundaries. And these boundaries are changing. Every year sees more information, faster processors, and cheaper storage. In other words, today's excess will become tomorrow's normal.

What is an Architecture?

An architecture is the layout or structure that describes how something is organized. In a house, this would be the floor plan, which shows the relative size and locations of the rooms. For a computer, it would be a block diagram, which shows the layout of key components. It is important to note that the architecture doesn't show all the details, only the essential ones. It provides an overview of the topic, and allows everyone to immediately understand the basics.

What is a Pattern?

A pattern is a template, a way of encapsulating something so that it can be repeated in multiple places. Think of it like the patterns people use to create clothing. A tissue paper template is laid over their own material to create something with known benefits, but with the creator's touch. Similarly, a pattern is a common term used in computer science to represent a dependable and efficient way to solve a particular computing problem. For example, the Model-View-View Model (MVVM) pattern describes a method for separating the elements of an application's user interface, from the back-end business logic.

How Does an Architecture Apply to Big Data?

An architecture in big data describes the organization of information in the system. Specifically, it identifies the key components used to manipulate the information:

  • Storage - the location of information in the system: where the information is stored (static storage, dynamic storage), and what types of information are stored (long term, short term, new, and frequently used).
  • Access/Retrieval - how the information is added, accessed and/or retrieved.
  • Processing - the type of processing performed, for example, sorting strategies, traversal (moving through), calculations, and business logic.
  • Information Flow - how information moves through the system, and which elements are part of that movement.

What are Some Big Data Architecture Patterns?

There are many big data architecture patterns available. Some are general and meant to address a number of big data computing problems. The rest target more specific applications. Here are a few notable ones:

To unlock this lesson you must be a Member.
Create your account

Register to view this lesson

Are you a student or a teacher?

Unlock Your Education

See for yourself why 30 million people use

Become a member and start learning now.
Become a Member  Back
What teachers are saying about
Try it risk-free for 30 days

Earning College Credit

Did you know… We have over 200 college courses that prepare you to earn credit by exam that is accepted by over 1,500 colleges and universities. You can test out of the first two years of college and save thousands off your degree. Anyone can earn credit-by-exam regardless of age or education level.

To learn more, visit our Earning Credit Page

Transferring credit to the school of your choice

Not sure what college you want to attend yet? has thousands of articles about every imaginable degree, area of study and career path that can help you find the school that's right for you.

Create an account to start this course today
Try it risk-free for 30 days!
Create an account