AMD Jumps into AI Memory Challenge
With the memory crisis showing no signs of easing, AMD believes that AI, the very culprit behind the shortage, could help unravel this issue. This week, our friends at the House of Zen acquired predictive memory startup Mext for an undisclosed amount, paving the way for a future where bots manage which data gets comfy in RAM and which gets tossed into more affordable flash.
Introducing Mext, the Memory Guru
Established in 2023, the Mext proactive memory platform focuses on machine learning algorithms and smart heuristics. It’s akin to having a personal assistant that relocates “cold” memory to flash storage, then retrieves it just before you realize you need it. Contemporary flash arrays are approaching main memory in speed, yet switching to disk still incurs significant latency. Mext claims it can amplify the memory value by 2 to 4 times using flash, which is a real bargain compared to DRAM. All you have to do is execute the Mextd daemon, and your OS will perceive it as conventional memory.
The Old and New of Memory Tiering
Memory tiering isn’t exactly groundbreaking; it has seen more iterations than Doctor Who, boasting some software-based varieties and others like Intel’s Optane utilizing that advanced 3D XPoint memory. What elevates Mext as the standout contender is its ingenious application of machine learning to rearrange data between active memory and cold storage, almost like a savvy branch predictor — perfectly aligned with AMD’s vision.
Mext’s Intelligent Strategy
Mext doesn’t depend on a single model to determine when to shuffle data. It balances a series of heuristics, long short term memory, and the latest transformer architectures to identify which combination performs best. Dan McNamara, SVP of AMD’s compute and enterprise AI division, suggests this could significantly reduce infrastructure expenses, enhance resource utilization, and assist customers in scaling general-purpose and AI workloads more efficiently.
Consequences for AI Serving
Apart from the mundane enterprise aspects, this technology could create ripples in AI serving. Modern mixture of experts (MoE) models operate like a clever ensemble of sub-models. For each token it predicts, a different member of the team may take the stage. In practical terms, a large language model might leverage some models more frequently than others. We wouldn’t be surprised if AMD employed Mext’s intelligent prediction algorithms to transfer less-active experts from HBM to slower system memory, enabling large enterprises to operate more substantial models with reduced resources. That’s just our playful speculation, but we’ve reached out to AMD for updates; we’ll let you know if we hear anything back.
Summary
What’s the Buzz? – So, AMD is putting its AI efforts into resolving the RAM crisis, utilizing Mext’s innovative algorithms for memory sorcery. They’re aiming to transform this technological twist into a substantial game-changer. Stay tuned, everyone!
