Cache memory is a special type of high-speed memory used to synchronize data with the CPU at a very high speed. This memory is more expensive than typical disk memory or main memory but very economical than the CPU registers.
A cache is a super-fast type of memory that acts as a buffer between the CPU and the RAM. This memory holds data and instructions often requested for different CPU operations. As a result, a lot of time is saved when accessing data from the main memory. Before we get into cache memory performance, let’s dive into various cache types.
Types of caches
L1 cache
This is a memory cache that is designed with a microprocessor. It’s used for storing recently accessed information from the microprocessor hence the primary name cache. In terms of speed, it has the highest rate level of all other types of caches.
That’s because it’s built within the microprocessors chip with the zero-state interface. It’s the most expensive type of cache, though with limited size. L1 cache stores critical files that need to be executed immediately, and that’s why it’s accessed and processed immediately. The processor performs computer instruction.
L2 cache
It is also known as the secondary cache or external cache, L2 cache is located outside and separate from the microprocessor chip. However, it’s usually included with the processor package. The earlier versions of the microprocessors fixed the computer motherboard, making them comparatively very slow.
Most modern CPUs have L2 caches in their microprocessors though they are slower than the L1 cache. The advantage that modern CPUs have is that the capacity of the L2 cache can be increased. The first L2 cache was introduced on the Intel Pentium and Pentium Pro-powered computers.
Disk cache
A disk cache is a memory cache used to speed up the process of storing and accessing data from the host hard disk. It’s used to enable faster reading/writing and processing of commands and other input and output processes between the memory and the computing components.
This memory cache is an integrated part of the hard disk, and it’s one of the essential features of most hard drives. The cache size ranges from 128 MB to about 1 GB on solid-state disks. When implemented on the RAM, the disk cache can also be a soft disk rather than the native hard disk.
Split cache
A split cache usually consists of two physically separate parts. One of the parts is called instruction, and it’s dedicated for holding instructions, while the other one is called data cache, and it’s dedicated for holding data. Both two parts are logically considered to be a single cache.
That’s because they are hardware-managed caches designed for the same physical address space in the memory hierarchy. If you come across a cache with no split, it’s called a unified cache.
Now that you have a basic understanding of the types of cache memories, let’s take a look at the performance of the cache memory. The cache memory performance is usually measured in terms of Hit Ratio. If the CPU can successfully refer to the memory and reveal the word in the cache, the hit has successfully occurred. On the other hand, if it fails, then it’s referred to as miss to cache.
Hit Ratio
Hit Ratio = Number of Hits/Total CPU references to the memory
In simple terms, the Hit Ratio is the probability of getting hits out of the total CPU references. Here, you have to remember that the CPU also references an equal Number of Hits plus the Number of Misses. The range is usually 0<=Hit Ratio<=1.
Average Access Time
It’s often abbreviated as “tavg,” and the formula tavg usually gives it = Hit Ratio X Access time+ (1- Hit Ratio) X (Access time + Main Access time)
where letters tc, h, and tm denote the access time, hit Ratio, and the main access time, respectively.
The average access time is given by adding the Hit Time and product of the Miss Rate and Miss Penalty. Miss Rate is the fraction of the accesses not in the cache. On the other hand, Miss Penalty is defined as the additional clock cycles that service the miss. That is the extra time needed to carry the favored data into the primary memory in the case of a miss in cache.
There are various types of Cache Misses, and they include.
- Coherence Miss
- Conflict Miss
- Compulsory Miss
- Capacity Miss
How the CPU handles cache
To understand how the CPU handles cache, you need to be well-versed with the fraction of the responses. The CPU works by dividing time into clock cycles used to execute different programs and wait for the memory system. CPU performance is vital for e-commerce business success as it determines the speed of various operations. Cache Hits are also part of the regular Central Processing Unit cycle.
So, the CPU time is given by adding CPU execution clock cycles and memory stall clock cycles and then multiplying the result by Clock Cycle time.
There are essential expressions that you need to understand for the Memory Stall Clock cycles that are typically used for the write-back cache.
The first one is the Write-Stall Cycle given by the product of Write/Programs, Write Buffer Stalls and Write miss rate. For the case of the assumed write buffer stalls, each access is treated similarly, and thus they are negligible. Sometimes, there might be different penalties for the Instruction and Data accesses, so you need to compute them separately.
Measuring and Improving Cache Performance
Various techniques are used for measuring and improving the cache performance. The first one is used to minimize the average memory access time, which is achieved by reducing the product Miss penalty and Miss Rate. Sometimes decreasing the Hit time or the Miss Rate can minimize the average access time.
If you wish to reduce the Miss Penalty, you can apply the following techniques.
- Give priority to the read misses overwrite
- Make use of the Victim Caches
- Use the Multi-level cache technique
Increased block size, large cache, or compiler optimization techniques are used for reducing Miss Rate.