Demystifying Multi-core

All of us have probably seen many of the advertisements from the major PC vendors or PC CPU vendors highlighting the virtues of their latest and greatest Multi-core CPUs and why we need to urgently dump our old PCs and spend the $$ to buy their new products.

Let me try to see if I can demystify this term and also get a peek into the basic principles of multi-core CPUs that are commonly available today.

To put it very simply, multi-core chips have more than 1 Microprocessor in them. Dual-core which means 2 CPUs is very common today and Quad-cores with 4 CPUs are becoming available for desktop configurations. Typically, on a single piece of silicon, the chip makers fabricate 2 or more Microprocessors. They normally have a local L1 cache and also connect to a common L2 cache, and then to the common internal bus structures. Most processor vendors have chosen the multi-core route to add processing capacity as increasing the clock speed beyond 3 GHz became quite difficult and the resulting power densities at such high speeds were hard to manage as well. From a silicon design perspective, I would say it was easier to design a multi-core CPU with connected Caches and buses than to push up the speeds further.

But, are we now tapping all that extra processing power with the added processor cores?

Not really, as the silicon designers have really pushed the problem of getting more and more performance over to the Software designers. You can have two CPUs on a single chip, but one has to partition the software appropriately to take benefit of the extra capacity. Most desktop Operating systems provide support for multiple cores by scheduling processes or threads on the available core. But, for an individual application to benefit, it has to be re-architected with the knowledge that there is more than one CPU available at any time, and things can be done in parallel. Many old applications have been designed for a single processor era and did not exploit features like multi-threading extensively. Let me take a trivial example to explain this. Say we want to add 1 million numbers stored in memory. Our normal monolithic code will have a single loop going through each of those locations and adding them up. This code will not find any speedup when running on a multi-core chip as only 1 CPU is working to add all the numbers. Let us re-write the code to split the addition into two threads. We will have 1 thread adding the first 0.5 million and the other thread the second 0.5 million, each running on 1 core. They finally sync up to add the sub-totals. In this scenario, we can quite easily get double the speed with a Dual-core chip. Of course since they share the bus, memory path etc., the speed up may be little less than double.

So, the real challenge is to split our applications into as many parallel chunks as possible to get the maximum speed-up. The serial portion of our code, which cannot be done in parallel, will effectively limit the speed-up one can get, irrespective of the number of cores you have. A nice analogy can be from cooking. You may have a multi-burner cooking range, but most dish preparation processes have a serial portion after you mix everything together that you cannot avoid, isn’t it? So, you will sometimes have unused burners with your favorite dish simmering in just one of the burners. In practice, many applications do have significant chunks of code which are not very amenable to being parallelized.

Developing programs which have multiple concurrent modules working together is challenging to the developer as well. One needs to be able to think of all the parallel threads running, with possibly multiple interactions between them. The state space required to be explored for such a scenario and the effort required to prove that the code will execute correctly, makes it a challenging area to work. Neurologists trying to understand the human brain say it is hard for our species to remember or process more than 7 things at a time. So, using a large number of processors effectively is going to challenge our grey matter quite a bit.

But, advance in Science and Technology often comes in spite of such challenges and often spurns companies and individuals to innovate. We are now seeing a spate of new tools and debuggers that will help in managing the complexity better. For all of us as engineers, this is the right time to catch up on the basics of concurrent programming to exploit the multi-core CPUs effectively.

Ramesh Raghavan

General Manager, Magnum Opus