Memory management is an essential skill that every programmer develops over time. To develop that skill you need to have a sense of how computer memory works. Surprisingly, memory is not a simple thing. It is actually quite complex and only a few experts actually understand all of the details. In this post, I make an analogy between computer and human memory to explain the basics of both. This leads me to dive into the core of both computers and human minds as these are deeply intermingled.
As stated by wikipedia, memory refers to the physical devices used to store programs (sequences of instructions) or data. Both computers and humans have multiple forms of memory. I don’t know whether human memory inspired the design of computers but the similarities are striking as you are going to see.
- RAM or Random Access Memory and the Hippocampus
When someone refers to computer memory, it usually means RAM. RAM or Random Access Memory was designed to provide a quick access to series of numbers. To limit access time, RAM is organized around arrays of electronic components that allow very quick access to any location in the memory. The main advantage of RAM is speed (typical RAM bandwidth today is around 5-10 GBytes/s). The constraints on speed have historically limited the overall size of available RAM on computers (today typical RAM memory is around 8 GBytes). The main disadvantage of this form of memory is its volatility : you turn off your computer and it is gone.
Humans do have a form of RAM and it is called short-term memory. Interestingly, short-term memory is usually associated with a very specific region of the brain called the hippocampus. Short-term memory is the memory you will use to recall immediate actions (like the series of words you are actually reading).
We don’t really understand how the hippocampus does it but I have always been puzzled by this structure homogeneity. It is composed of billions of cells that are very densely packed, like on an array as can be seen on the Golgi staining in the attached image. Reminds you something?
It is traditionally thought that the memory content stored in the hippocampus is transferred to another brain area for long term storage. This process is supposedly happening when you are turned off (your sleep). Again another similarity….
- The hard drive and the cortex.
As for the hippocampus, RAM storage was too volatile so engineers came out with a complimentary form of storage for long term memory in computers : the Hard Drive or HD. Hard drive have low bandwidth (usually 100 MBytes/s) but can store much more (computer usually comes with 500 GBytes of hard drive). This is where all of your data and programs is located.
Human long-term memory is presumably located in the cortex, exactly why and how is a matter of intense research nowadays. I am sure you have experienced that recollecting old memories can take quite some time (sometimes minutes). This process is usually variable as it is when reading from the hard drive. Hard drive can have very different read/write speed depending on where the memory is stored.
As for humans, when you put a computer to sleep, RAM memory is transferred to the hard drive for long term storage. This process is reversed when you turn it on to make the best usage of the fast RAM. I suppose the same thing is happening when we wake up in the morning (in my case, this process is especially slow).
So far so good. Nothing too fancy here as the distinction between RAM and HD is usually known.
There are still 3 forms of computer memory that we need to understand, so hold on.
- CPU Cache and neuronal network attractors
Computers have something called CPU cache. CPU or Central Processing Unit is probably the main thing a computer have that we don’t. This is the center of your system where all calculations are being done. In essence a CPU is just a ton of operators packed in a very tight space, they can combine bits of information in very specific ways. Thanks to decades of work, CPU operates at very very high frequencies, in the GHz range. Most CPUs are now 64 bits which means they can process 64 bits of information at each clock cycle. Because the frequency is so high, it became essential to have a high bandwidth to provide information to the CPU. If you check the numbers, it appears that the RAM is not fast enough to feed these monsters (the order of magnitude is 24 GBytes/s). That’s what the CACHE was designed for. The total memory content of the CACHE is very small as we really want this thing to be dead-fast (by small I mean in the MByte range). The CACHE is a very little memory space packed at the close proximity of your CPU to limit access times. There are multiple levels of cache (L1, L2 as depicted on the associated image but there are more) to provide intermediate levels of storage to make the CPU work smooth and organized.
Our brain is not clocked at such speeds. A typical neuron emits a new spike at about 10-40 Hz. While it can go higher, we can say we have a 40 Hz clock running in our skull. This is why our computer screens are only refreshed at 60 Hz. Why would you go faster, we don’t see it…
But as for more modern computer architecture, things are done in parallel in our little head. With such a low clock speed, information is transferred very slowly from one side of the head to the other so we can’t afford to have a single memory storage.
Indeed, Memory in the brain is distributed all other the place. I see the cortex as a gigantic CACHE storage. Each bit of information that is processed wake up the local neuronal network, reactivate its memory and compare this memory with the provided input. The beauty of neuronal networks is that both the program (what we usually script in Matlab in M-files) and the associated memory (the program parameters or your grandmother face) can be stored in the same place.
- Protected memory and explicit memory
Human memory is further divided into two major classes called explicit and implicit memory. Explicit memory is the memory you are consciously using, like when you remember where you left your doors keys the day before, while implicit memory is not necessarily conscious : you know how to walk but you don’t have to think to walk. Still you had to learn and store it somehow. I have no idea how this separation is done at the neuronal level but I do know that something very similar exists in computers and it’s called memory protection. Memory protection is built-in in any modern OS. When you write a program that needs some memory for its behavior, the OS allocates some space to it. It does so with some protection mechanism to prevent the program from accessing any other memory. This technique obviously prevent bugs and promote overall stability of your system. It does seem to me that explicit and implicit memory fullfill the same purpose : stability of our brain “Operating System” and protection of important memories.
- Memory swapping and writing
It sometimes happens that a program asks for more memory than what is available in the RAM. When this happen, your operating system has no other solution than to use what’s available : your hard drive. In that case, your OS will allocate some space on the hard drive for your program to work. This is usually refers as “swapping”. Since hard drive are several order of magnitude slower, this leads to great loss in performance. This is probably the number one reason things can go slow within Matlab. Your program will not be able to tell if its data is stored on the hard drive or in the RAM as this process is seemingly transparent. I personally believe that if your program start to swap, it is because your program was not designed properly. It is your job to be able to work within the memory limits of your computer.
Swapping does not exactly exists for human mind as our mind was extremely well designed to work within its memory limits. However, in some ways we do swap all the time.
Indeed, what do you think you are doing when you take notes during a class?
The memory content of the class is too high for your brain to take it straight like this. You need an intermediate data storage device to allow you to assimilate the new data set. So you swap the data to a piece of paper using a pen. In some way, we human compensated our lack of a swapping device via writing…