Paul is a GIS professor at Vancouver Island U, has a PhD from U of British Columbia, and has taught stats and programming for 15 years.
Cache memory is a type of memory used to hold frequently used data. Cache memory is relatively small, but very fast. Most web browsers use a cache to load regularly viewed webpages fast. The most important type of cache memory is the CPU cache.
Cache Memory & Related Definitions
Computer engineers are always looking for ways to make a computer run faster. A multi-core chip, more memory, and a faster video card can all be used to improve the performance of a computer. One very effective method to improve speed is the use of cache memory. Cache is a type of memory that is relatively small, but can be accessed very quickly. It essentially stores information that is likely to be used again.
For example, web browsers typically use a cache to make webpages load faster by storing a copy of the webpage files locally, such as on your local computer. This is referred to as a web cache. Clever protocols are used to make sure that the copy is still current - if not, the web browser will get the new files from the original source. The general purpose of the web cache is to make the web pages load faster. Cache memory is used in other parts of a computer system. The most important type is the CPU cache, which will be the focus of the rest of the lesson.
The central processing unit (CPU) is the brain of the computer. All of the instructions have to run through the CPU for the various parts of a computer to work together. CPU chips have been getting smaller and faster as chip technology has advanced. One of the slower aspects of computer processing is the interaction between the CPU chip and the main memory in the form of random-access memory (RAM). Installing more memory is not always a solution - the bottleneck is often the time it takes to access the memory.
So, what have chip designers come up with? A small form of memory located directly on the chip itself. This is the CPU cache. It is much smaller, but can be accessed much faster than the main memory. The CPU cache stores the most frequently used pieces of information so they can be retrieved more quickly. This information is a duplicate of information stored elsewhere, but it is more readily available.
How the CPU Cache Works
To carry out a particular instruction, the CPU needs a specific piece of information. The CPU will first check to see if this information is available in the CPU cache. If the information is found, this is called a cache hit. If the information is not found, this is called a cache miss, and the CPU goes on looking for the information elsewhere. In the case of a cache miss, the piece of information will be found in the main memory, but it will simply take longer.
A lot of research has gone into how to optimize the design of cache memory. The result has been somewhat counter-intuitive: smaller is faster. What this means is that a relatively small CPU cache improves speed, but as the cache gets really large, it no longer helps as much, and the CPU might as well look for the information in the main memory.
Think of it this way: let's say the main memory is like a 500-page textbook packed with information. It contains everything about the subject, but finding information means you have to look through the table of contents or search the index. You know the information is there, but it takes a bit of time to find the right page. Now, think of the CPU cache as a small notebook with only the most important information.
Over 79,000 lessons in all major subjects
Get access risk-free for 30 days,
just create an account.
Looking through this notebook is very quick, and if you find what you need - bingo! If not, you turn to the 500-page textbook; however, if you end up with a notebook of 100 pages, it no longer has the same benefit in terms of speed as the really small notebook with only a few pages. And, by the way, every page in the notebook costs a lot more than a page in the textbook. So, we want to keep the CPU cache small and efficient.
In a typical computer system, the main memory in terms of RAM will be around two to eight gigabytes. A CPU cache on a modern chip is in the order of several megabytes, which is around 1,000 times smaller. So CPU caches are very small in comparison to main memory, but really fast.
So you've learned that smaller CPU caches are faster, but a small cache can only contain a certain amount of information. When this fills up, the CPU has to use the much slower main memory. One solution to this is the use of multi-level caches. The smallest level 1 (L1) cache is accessed first. In the case of a cache miss, the CPU accesses the much larger (and slightly slower) level 2 (L2) cache. Several additional caches can be used before the main memory is accessed.
In older chip designs, the L1 cache was located on the CPU chip itself, while the larger L2 cache was located on the motherboard directly adjacent to the CPU. As the components got smaller, additional caches were located on the chip itself. This photograph illustrates a chip from around 2006. The left portion is the CPU with a built-in L1 cache. The right portion is the L2 cache, located inside the same chip, but as a separate component. Note that this is normally not visible - for the photograph, the protective cover was removed. Present-day CPU chips can have up to three caches: L1, L2, and L3. They have gotten so small that they are completely integrated into the CPU itself.
CPU chip from 2006
A cache is a type of memory that is relatively small, but really fast. It stores information that is frequently needed for quick access. A CPU cache increases the speed of the computer, since accessing information on the cache is much faster than accessing the main memory. Present-day chips have multi-level caches to run even faster.
By the time you are finished, you should be able to:
Explain what cache memory is and what it is used for
Describe the CPU cache and how it works
Recite how multi-level CPU caches process information
Did you know… We have over 200 college
courses that prepare you to earn
credit by exam that is accepted by over 1,500 colleges and universities. You can test out of the
first two years of college and save thousands off your degree. Anyone can earn
credit-by-exam regardless of age or education level.