Moore's Law Dead, Computational Storage Solution
Key Points
- Moore’s Law has effectively ended: after five decades of regular, cheaper, low‑power CPU upgrades, new processors now arrive less frequently, cost more, and consume 200‑400 W, challenging sustainability goals.
- As raw transistor scaling stalls, manufacturers are packing more functionality (compression, encryption, database, ML/DL workloads) into CPUs, but this adds complexity and power draw without solving performance bottlenecks.
- The prevailing architecture still forces massive data movement—entire datasets must be fetched from storage, streamed over the network, and scanned by the CPU even when only a tiny fraction is relevant, wasting bandwidth, memory, and CPU cycles.
- The “computational storage” paradigm proposes moving analytics and filtering logic into the storage layer so that irrelevant records are discarded before they reach the server, dramatically cutting I/O and processing overhead.
- By offloading tasks like search, compression, and encryption to storage devices, organizations can mitigate the limits imposed by the end of Moore’s Law and achieve more efficient, sustainable compute workloads.
Sections
- Moore's Law Declared Dead - The presenter explains that although CPUs still evolve, escalating costs and power demands have ended the historic 18‑to‑24‑month performance gains that defined Moore's Law.
- Computational Storage Offloads CPU Work - The speaker describes how moving tasks like compression, encryption, RAID, and erasure coding from the main processor to smart NICs/DPUs (computational storage) frees memory and CPU cycles, reducing bandwidth and latency overhead.
- SSD Compute Offloads Data Processing - The speaker explains that modern SSDs contain powerful compute resources (ARM cores, ASICs, FPGAs) that can use excess internal bandwidth to compress, filter, scan, and analyze data—offloading tasks like ransomware detection and targeted queries from the CPU.
- Optimism Beyond Slowing Moore's Law - The speaker acknowledges Moore's Law is decelerating yet asserts that ongoing optimization and innovation will sustain a brilliant future of technical advancement, ending with a typical video call‑to‑action.
Full Transcript
# Moore's Law Dead, Computational Storage Solution **Source:** [https://www.youtube.com/watch?v=lCuJnhZaTv8](https://www.youtube.com/watch?v=lCuJnhZaTv8) **Duration:** 00:09:57 ## Summary - Moore’s Law has effectively ended: after five decades of regular, cheaper, low‑power CPU upgrades, new processors now arrive less frequently, cost more, and consume 200‑400 W, challenging sustainability goals. - As raw transistor scaling stalls, manufacturers are packing more functionality (compression, encryption, database, ML/DL workloads) into CPUs, but this adds complexity and power draw without solving performance bottlenecks. - The prevailing architecture still forces massive data movement—entire datasets must be fetched from storage, streamed over the network, and scanned by the CPU even when only a tiny fraction is relevant, wasting bandwidth, memory, and CPU cycles. - The “computational storage” paradigm proposes moving analytics and filtering logic into the storage layer so that irrelevant records are discarded before they reach the server, dramatically cutting I/O and processing overhead. - By offloading tasks like search, compression, and encryption to storage devices, organizations can mitigate the limits imposed by the end of Moore’s Law and achieve more efficient, sustainable compute workloads. ## Sections - [00:00:00](https://www.youtube.com/watch?v=lCuJnhZaTv8&t=0s) **Moore's Law Declared Dead** - The presenter explains that although CPUs still evolve, escalating costs and power demands have ended the historic 18‑to‑24‑month performance gains that defined Moore's Law. - [00:03:05](https://www.youtube.com/watch?v=lCuJnhZaTv8&t=185s) **Computational Storage Offloads CPU Work** - The speaker describes how moving tasks like compression, encryption, RAID, and erasure coding from the main processor to smart NICs/DPUs (computational storage) frees memory and CPU cycles, reducing bandwidth and latency overhead. - [00:06:11](https://www.youtube.com/watch?v=lCuJnhZaTv8&t=371s) **SSD Compute Offloads Data Processing** - The speaker explains that modern SSDs contain powerful compute resources (ARM cores, ASICs, FPGAs) that can use excess internal bandwidth to compress, filter, scan, and analyze data—offloading tasks like ransomware detection and targeted queries from the CPU. - [00:09:20](https://www.youtube.com/watch?v=lCuJnhZaTv8&t=560s) **Optimism Beyond Slowing Moore's Law** - The speaker acknowledges Moore's Law is decelerating yet asserts that ongoing optimization and innovation will sustain a brilliant future of technical advancement, ending with a typical video call‑to‑action. ## Full Transcript
Moore's Law is dead.
Now, I know some of you are thinking, well, is it really dead?
We've had a tremendous run in the last 50 years.
And so are we really saying goodbye to Moore's Law?
Well, we have had a tremendous run.
Let's just draw a compute system over here with memory.
With the CPU with storage down here, and the CPU is connected to the storage device over a network.
And I'm very sorry, network people.
I have reduced you to a single line.
And although the network is a critical part of the system, it sometimes seems like it's too narrow for us.
And even with all of the growth in the network and the storage, we still are at a point where things are slowing down.
Let's look at the CPU, for example.
For 50 years, we could count on a new release every 18 to 24 months,
and we would get more function, we would get more cores, we would get more offload engines inside the CPU.
And not only that, we would pay less for it.
And the power, the wattage, would be about the same.
So every two years we could have more functionality and pay about the same or less for the same server.
That is what is dead.
We may still get releases of new CPU's, but
what's happening now is that they tend to cost more each generation to get that extra power.
And powerful is starting to take on new meaning as CPUs are starting to take 200, 300, 400 or more watts.
It is making it difficult in a world where sustainability is becoming critical.
And so as this becomes more constrained, what we have also done is to put more and more function inside.
So now we have the compression, we have encryption, we do all of the database work.
We have machine learning and deep learning.
So inside this storage box, all the data has to come up here.
It has to be processed by the CPU, it has to be analyzed,
and only the data that is needed is, is what the CPU is going to work on. But everything has to be brought up.
Take, for example,
that you want to look in your your huge amount of data for all the customers who bought a handbag in 2022
so that you can put ads for handbags as they do their Christmas shopping.
Well, to do that, you're going to have to bring my record in.
And I have never bought a handbag in all my life.
And so I'm sitting in memory taking up space along with all of the other people who have never bought handbags.
And you're having to look at each one of these saying, Nope, not him, not him, not him aah yes, him or her..
and that then gets processed.
But all of that data taking up bandwidth, speed, taking up memory and taking up precious CPU cycles.
That is the problem that we are facing, that although we have beautiful processing systems,
we are spending a lot of time doing things which really could be done other places,
and that is the nature of computational storage.
Let's see where computational storage can help.
Inside the CPU system here - this server, are things that we call HBAs.
Here is a NIC [network interface card] for Ethernet, and here is fiber channel or some other protocol.
If we expand that NIC and
itself has ARM processing, memory and ASIC [application-specific integrated circuits].
And in that ASIC, there could be offload like compression or encryption or other things.
And so now think about this.
We could take functionality that we're running here,
like a storage stack, like the raid or the erasure coding,
and we could move that into this, and it is in line here.
So data coming from the storage can be processed
by the sNIC, they're today called DPUs as well, Data Processing Units.
So this is an example of where computational storage offload can exist.
But let's go further.
Let's look at a storage box.
A external storage box can have hard disk drives in it and it can have tape.
Tape has made a beautiful ride over the past 50 years.
But in order to really process this quickly in these days, it's SSDs.
And you can have dozens of them in one of these storage arrays.
There's also controllers.
And these controllers are full blown servers in their own right with Intel or AMD processors in them.
And so imagine taking some of that function, which today resides all in the CPU and moving some of it into here.
Today, the job of storage is to store.
I know.
That's amazing, isn't it?
It stores data.
It manages that data.
It does data reduction on that data.
It does encryption, but it largely just stores the data.
But it's got these large processors in it.
And what's more, if you looked at all the bandwidth inside for these systems, it can be 168 gigabytes per second.
But the maximum you're going to get here is on the order of 50 gigabytes per second.
What does that mean?
There's extra bandwidth to use to scan things, to look for customers who bought handbags in 2022
and only report to you those who are relevant for what you want to do with the data.
Now we can go one step further.
So we could do computation here by running a VM or a container and do some of that offload that today is here.
But let's also look at one of these SSDs.
In a modern SSD, there is also a lot of compute power.
There are ARM cores.
There's an ASIC or an FPGA.
And inside here, like in the IBM FlashCore module, there's a compressor
and that compressor is transparently doing data reduction,
offloading what is done here so that the CPU can do other valuable analytics.
Now there's also, of course, a bank of NAND flash, which is where the data is stored.
And today the SSD job, you guessed it, it's to store data and to retrieve data.
But what about using some of the extra bandwidth?
This can have 16 gigabytes per second,
and yet you're not going to get much more out of that than six gigabytes per second.
So imagine all of the filtering, the scanning...
You could do things like looking for ransomware indications,
looking for filtering again and sorting and searching,
offloading some of that so that each of these units is doing and distributing the load
so that it doesn't all have to fall on the CPU.
That's computational storage at its finest.
I'm often asked, "Well, where is computational storage going to happen?
Is it going to be in the smart NIC?
Is it going to be in the storage box?
Is it going to be in the SSD?"
My brilliant answer is "yes".
It's going to be in all of those because each one has a unique benefit for certain offload.
So what is happening now to move computational storage to the next level
is that OCP, SNIA and NVMe are working on standards
so that you can do this offload in a in a standardized way
so that it can be brought then to the applications, to the database, to the search engines
so that we can get this across a wide variety of applications.
What we're going to see then is that although Moore's Law is slowing down,
we are still going to have a brilliant future of being able to optimize what we have, take advantage of it,
and still further the fantastic revolution that we've had in technical advancements.
Thank you so much for watching.
If you like this video and want to see more like it, please like and subscribe.
If you have questions, please drop them in the comments below.