Rust Generics: All Aboard the Cross-Language Express
The generics of programming languages can now travel in comfort between compilers of different languages, thanks to experimental efforts by Mojo core contributor Evan Ovadia. This ex-Google engineer aims to transform Rust into a library that can be leveraged by other languages, showcasing a method to exchange generic data types and functions between Rust and another language, a feat that the C ABI does not facilitate. “From a technical standpoint, this is the most intriguing thing I’ve encountered in months,” the host of the Let’s Get Rusty YouTube channel exclaimed, referring to the project, which he dubbed “Rust++.” It’s undoubtedly a geeky yet hard-earned achievement. Ovadia interprets his work in a detailed blog entry as the “Golden Spike” connecting two distinct languages, alluding to the last spike driven into the first transcontinental railroad in the United States. Rather than being reduced to basic raw elements merely to fit into the clumsy carriages of C ABI, rich-featured libraries can be swiftly conveyed to external environments on robust, rapid transit lines.
Farewell C ABI: The Fresh Face on the Scene
This initiative emerged from Ovadia’s ambition to develop his own language, Valen (a derivative of a prior attempt named Vale), which he imagines as a faster, more developer-friendly alternative to Rust. He considered that the challenge of constructing a new language from the ground up was his inability to utilize his beloved Rust libraries (especially the impressive wgpu graphics library from Rust). Currently, a typical method for employing a library from one language in another language’s application is via the C ABI (application binary interface), a pragmatic interoperability layer defining standards for transferring data and function calls between compiled programs. (While Rust has its unique ABI, it does not guarantee stability.) With the C ABI, if a Rust library presents its functions as extern “C” and its types as repr(C), they can be accessed from a different language using a Foreign Function Interface (FFI). All is well in theory, yet the challenge Ovadia faced was that the C ABI had no comprehension of generics (or placeholders for data types supplied later) since C doesn’t incorporate the concept of generics. However, Rust utilizes generics extensively, as does Valen, albeit differently. “Thus, the core inquiry—the pivotal enigma to solve—is how can we implement cross-language generics?” Ovadia elaborated.
Cross-Language Generics: The Grand Scheme
Coordinating two compilers is exceptionally challenging, as compilers operate as self-contained entities, rulers of their respective domains. Therefore, Ovadia needed to devise a mechanism for Valen to invoke a Rust generic function, while also enabling a Rust generic function to call back into Valen for required values. After considerable contemplation, Ovadia formulated a two-part approach to achieve cross-language generics: firstly, execute the Rust compiler as a library. Remarkably, Rust already has this functionality (rustc_driver), which would permit Valen to integrate rustc directly within its memory environment. The second step involves adjusting Rust so that when it encounters non-Rust Valen types, it transfers control to the Valen compiler to manage its section and awaits the compiled output. Ovadia concedes that modifying the Rust compiler was the “nuclear option.” “Truthfully, this represented the most perilous component of the entire project,” he noted. The modification spans only about 100 lines of code, yet is somewhat hacky and presumes Rust is utilizing LLVM for machine code generation, which is not universally applicable. “It functions, but it’s definitely not suitable for upstream integration,” Ovadia remarked. Nevertheless, Ovadia’s endeavor towards cross-language bidirectional generics marks a significant advancement in programming language interoperability.
Community Buzz: Cheers and Jeers
Ovadia is currently occupied with Valen, although his work on cross-language generics paves the way for Rust to facilitate the genesis of new languages founded on Rust libraries. With some adjustments to Rust itself, the old reliance on the cumbersome C ABI might one day become obsolete. On the Reddit and Lobste.rs programming forums, spectators commended Ovadia for addressing such a challenging issue, even as they expressed concerns over various aspects. “I admire the intention to supplant C as the ‘universal FFI language,’” one commentator remarked. “It’s a daring choice and would necessitate significant effort.” Another reader suggested that Rust might not serve as the optimal language for interoperability, given the constraints of its type system. “However, just imagine constructing bindings on top of Ada! Now that would be extraordinary,” they stated.
Summary: Rusty Rails to the Rescue!
Ovadia’s initiative is a courageous leap for the programming community: steering away from the rusty tracks of the C ABI towards the gleaming routes of Rust libraries. If Valen and Rust ever become allies, we may be well on our way to leaving behind the outdated methods for good. Cheers, Ovadia! 🍻
