When the memory industry can no longer leverage Moore's Law to create new value, it has to find new technologies to establish a new business model that can create new economic value.
To have a good picture of how the memory industry could leverage new technologies to create new economic value, we must consider the fundamental function of memory. According to the von Neumann architecture, the processor and memory are two independent core modules. However, since the operating speeds of the processor and memory do not match, memories could be categorized into three types - SRAM cache, DRAM, and NAND flash SSD - based on their operating speeds. During data processing, messages are repeatedly transmitted and converted between processors and NAND flash SSD. The present computing semiconductor technology sees most of the power needed for data processing used on this repeated message transmission and conversion, and real data processing only accounts for a small portion of power consumption. This is the biggest challenge for modern semiconductor design and manufacturing.
One solution that does not rely on scaling and can continue to increase economic value is to have the entire computing system gradually depart from the von Neumann architecture by starting with semiconductor components. Guided by this logic, the memory industry can explore three models to create new economic value: emerging memories; memory-centric computing; and integration of application-oriented memory and logic IC.
The path for developing emerging memories is clear. Basically, emerging memories should have the same data read/write speed and durability as DRAM, and the same large storage volume and persistent data storage capability as NAND flash. Emerging memories should have the advantages of both memory types and function as storage-class memory. Strictly speaking, memory is considered volatile. SRAM and DRAM, for example, have fast operating speeds but can only store data temporarily. Storage, on the other hand, is considered permanent, such as NAND flash. Storage-class memory refers to new components that can serve as both volatile memory and permanent storage. MRAM and FeRAM (or FeFET) are potential candidates for this kind of emerging memories, and they can significantly reduce power consumption by eliminating the repeated data transmission between DRAM and NAND flash. There is great growth potential for emerging memory development, with SOT MRAM and FeFET possibly being upgraded with 3D structure.
There are short-term, mid-term, and long-term targets for memory-centric computing development. The first is near-memory computing, which reduces transmission latency and power consumption by moving the processor closer to DRAM. The technology stacks up multiple layers of DRAM chips and combine them with processors to form a module using high-end packaging, and it is already being used in the industry. HPC development is now moving toward this direction. The next step might be switching all electrical connection between DRAM and processors to optical signal connection to enhance transmission speed and lower power consumption.
The mid-term target is in-memory computing, which has been a hot topic often discussed in semiconductor industry meetings in recent years. The method is to have emerging memories simultaneously perform functions of both processors and memories. This computing structure has essentially departed from the von Neumann architecture and no longer treats processors and memories as separate entities, and therefore it can eliminate latency and power consumption caused by repeated data transmission between processors and memories.
The long-term target is the neuromorphic chip, which is learning from natural progression and imitating how human brains work. By using memristor to imitate neurons and synapses in a human brain, the chip learns to establish and enhance connection and growth of synapses. Intel's Loihi I and Loihi II are examples of initial attempts toward this direction.
Developing an application-oriented memory-logic IC integration is a model that can create new economic value, and it is a heterogenous integration already adopted in the industry. Integrating DRAM, ISP (image signal processor), filter, and micro lens in a CIS (CMOS image sensor) using advanced packaging technology is an example of near-computing application. Integrating DRAM and AI chip using WoW (wafer-on-wafer) or CoW (chip-on-wafer) packaging technologies, such as Graphcore's IPU-2 that recently entered volume production, is another example. Although this model has a looser development direction, it is expected to be a popular method for integrating processors and memories for automotive semiconductor and AIoT applications. The adoption of this model is expected to grow rapidly.
This application method is special in that memories used do not need to have standardized specifications. Its important implication is that DRAM will no longer be considered a commodity, but that it will serve as a product for specific application-oriented design instead. The characteristics of the market have also changed.
(Editor's note: This is part of a series about the memory industry written by DIGITIMES adviser Albert Lin. Lin received his PhD in Physics in 1988, taught at National Central University, and then move to the technology industry. Lin used to serve as director and vice president of ProMOS Technologies, and president of ConDel International Technologies. He was an advisory member of Taiwan Semicon, chaired the Huangguang Forum, and was chairman of the Supervisory Board of Taiwan Semiconductor Industry Association. Now, Lin is visiting research fellow in the Department of Physics, National Taiwan University. His main research fields are new materials, new mechanisms, and basic research on quantum information. He is standing supervisor of the Taiwan Association of Quantum Computing and Information Technology.)
