TCP’s No Longer the Top Choice for AI
TCP, that reliable old data transport protocol that’s managed the web and cloud like a pro, isn’t quite up to snuff for the emerging AI workloads, according to a former Stanford professor. He’s advocating for a new protocol named Homa. This John Ousterhout fellow claims, “TCP’s been solid, but it’s a hassle in datacenters.” Transitioning away from TCP is no small task, but Ousterhout mentioned to GadgetLad that it’s simpler than it looks. Just grab Homa from GitHub, integrate it into your Linux kernels, and you’re good to go, no reboot required. “Homa works alongside TCP, so you can transfer data incrementally,” he wrote. Reportedly, TCP even operates more efficiently with Homa included.
Classic Ideas, Updated Tools
Homa’s a novel approach to traffic management, states Ousterhout. The endeavor began as a PhD project in 2019 by Behnam Montazeri, who is now innovating at Google. Ousterhout, having moved on from teaching, has dedicated his career to promoting Homa. What sets it apart? Homa’s message-based design allows the receiver to manage congestion control. The initial packet indicates how much more data is forthcoming, and the receiver determines when the packets should move, prioritizing shorter messages with SRPT. Ousterhout believes this significantly reduces latency for short messages, achieving 92 microseconds with Homa compared to 1.2 milliseconds with TCP. Even for larger transfers, Homa is reportedly twice as fast.
List of TCP’s Limitations
Ousterhout is actively preparing standard documentation for Homa with the IETF and working to integrate it with the Linux kernel. Back in March, he successfully implemented Homa on Red Hat Enterprise Linux versions 8 and 9.5. He’s also assisting major corporations in evaluating Homa’s potential – currently testing with a large financial institution. However, not everyone is abandoning TCP. Ivan Pepelnjak, a prominent network architect, published a paper in 2023 sharply critiquing Homa, questioning whether it’s a solution in search of a problem. Yet TCP faces other critics too; high-performance database enthusiasts are leveraging DPDK to bypass the TCP stack for faster querying, NVMe-oF is making strides in storage networks, and Google’s QUIC aims to eliminate TCP’s head-of-line blocking for quicker web browsing.
TCP’s Control Delays
Vint Cerf and his colleagues developed TCP to impose some order on the disarray of message packets within networks, featuring flow control, guaranteed delivery, and more. Congestion control prevents overload along the path, but TCP’s byte streams provide no prioritization, maintaining a steady flow of data. For receivers, lengthy and brief byte streams are treated equally. A server overwhelmed with traffic may slow down senders, yet it cannot monitor the incoming traffic. Latency can be problematic, particularly for AI, where even milliseconds can make a difference.
AI Can’t Afford Latency
The experts behind extensive language models require high-performance networks for tasks like weight gradients and cache entries. Large data transfers must now coexist with rapid bursts from control operations. “Latency is crucial,” asserts Ousterhout. Even a millisecond delay leaves costly GPUs idle. “Outdated protocols just aren’t suitable,” he adds. Has Homa finally secured its niche? Or was it merely ahead of its time? For the moment, TCP remains dominant.®
Conclusion — TCP’s Backup
Well, folks, TCP may still reign supreme, but it’s starting to feel like lugging around an old television while trying to surf the web via dial-up. Homa could very well be the sleek new ride we need, but until it gains traction, TCP isn’t going anywhere. Stay alert; things could get exciting!
