Software package can bypass CPU for more efficient computing

Technion Researchers have developed a software package that enables computers to perform processing operations directly in memory, bypassing the CPU. This is a significant step toward developing computers that perform calculations in memory, avoiding time-consuming and energy-intensive data transfers between hardware components.

A new and exciting field has emerged in the hardware domain in recent years: in-memory computing. The in-memory computing approach introduces a significant change from the way computers typically operate.

While traditionally the CPU runs calculations based on information stored in the computer’s memory, with this innovative approach, some operations are performed directly within the memory, reducing data transfers between the memory and the CPU.As transferring data between computer units is time- and energy-intensive, this change leads to significant savings in both.

Recent decades have seen dramatic improvements in the performance of these two components; the calculation speed of processors has skyrocketed, as has the storage capacity of memory units. These developments have only exacerbated the problem, with data transfer becoming a bottleneck that limits the computer’s overall speed.

Professor Shahar Kvatinsky from the Andrew and Erna Viterbi Faculty of Electrical and Computer Engineering has dedicated the past few years to finding solutions to “the memory wall problem”—the problem of computations requiring two separate hardware components.

In papers published in recent years, he has presented hardware technologies that enable some operations to run in memory, mitigating the “traffic jams” created between the processor and memory in conventional computers.

This paradigm shift in computer architecture has groundbreaking applications in many fields, including artificial intelligence, bioinformatics, finance, information systems and more. Unsurprisingly, many research groups in academia and industry are working on this issue: looking into memory architecture, researching the production of innovative memory units in chip factories, and studying the basic computational operations that would take place in a computer designed with an in-memory-computing approach.

However, one crucial aspect of this approach has been almost entirely unexplored until now: software. For decades, computer programs have been written for “classic” computers, the fundamental structure of which has barely changed since the very first computers in the 1940s.

These programs are collections of read and write operations taking place in the computer’s memory, and computational operations performed by the processor. The units of information stored in the memory have addresses that enable software to locate and transfer them to the CPU for processing.

“With some computations now handled by the memory, we need new software,” explains Professor Kvatinsky. “This new software has to be based on new instructions that support in-memory computations. This new computation method is so different from the conventional one that it renders some of the existing building blocks of computer science unusable. Therefore, we need to write new code, which requires a lot of time and effort from software developers.”

A new article by Professor Kvatinsky’s research group, led by Ph.D. student Orian Leitersdorf in collaboration with researcher Ronny Ronen, presents a solution to this problem. Their new platform uses a set of commands that bridges the gap between in-memory computing solutions and popular programming languages like Python.

To build this new platform, the researchers developed a theory for the programming interfaces of in-memory computing architecture and created software development libraries that convert Python commands into machine commands executed directly in the computer’s memory.

They call this new concept PyPIM—a combination of the abbreviation for Python and the acronym for Processing-in-Memory. With this new platform, software developers will be able to write software for PIM computers with ease.

The researchers have also created a simulation tool for developing hardware and measuring performance, allowing developers to estimate the improvement in code runtime relative to a regular computer. In their paper, the researchers demonstrate various mathematical and algorithmic computations performed using the new platform, with short and simple code, resulting in significant performance improvements.

The new research was presented at the IEEE/ACM International Symposium on Microarchitecture, which took place in Austin, Texas. The paper is also available on the arXiv preprint server.

Orian Leitersdorf, 21, is soon to be the Technion’s youngest-ever Ph.D. graduate. Ronny Ronen is a senior researcher in the faculty and is a faculty member and head of the Architectures and Circuits Research Center (ACRC).