wtransport-elixir – Elixir bindings for WebTransport

In this developer showcase article, TalkJS developer Franco Bugnano discusses their recent side project wtransport-elixir, a library for writing WebTransport servers in Elixir.

Hi, I’m Franco, and outside of my work at TalkJS I am a hobbyist game developer. As part of my journey into game development, I recently built wtransport-elixir, a library for writing WebTransport servers in Elixir. In this article I’ll detail how this side project got started, some of the core choices I made, and what’s next for the project.

Performance

In game development, performance is everything. Gaming applications will need to be able to send many messages as quickly as possible, where those messages could come out of order, and where the data transmission may not always be reliable. Especially when you’re working on real-time, online multiplayer games, that means that you will require low-latency network protocols. This requirement got me looking into WebTransport.

The WebTransport API allows you to transmit data between client and server using the HTTP/3 protocol. The ability to be able to use HTTP/3, which is itself based on Google's QUIC protocol, can be quite an advantage. Due to better transport efficiency, HTTP/3 provides faster performance than the classic TCP protocol. WebTransport provides low-level access to two-way communication, and also supports both reliable data transmission (via streams) and unreliable data transmission (via datagrams). Given my focus on the performance required by real-time game servers, WebTransport provided exactly what I needed.

Joys of Elixir

Once I had decided to work with WebTransport, I also realized I wanted to do so using the Elixir programming language. At TalkJS, we use Elixir in production. Through my work at TalkJS, I’ve fallen in love with the language.

Elixir is very different from most other languages I know. First of all, it's a functional language, and this implies a very different way of thinking. All the data is immutable. There is no ‘else if’-construct. Nor is there a ‘return’ keyword—the return value is simply the result of the last expression of the function, and there's no early return. All these might seem like limitations, but in reality they force you to think in a specific way, and from my experience, it leads to code that's easy to maintain.

Further, Elixir has pattern matching and the pipe operator |>, the latter of which allows you to take the result of one expression and pass it as the first argument to a function. Together, these make programming in Elixir really fun.

Lastly, with Elixir you have the Erlang virtual machine (VM), also known as the ‘BEAM’, and the philosophy of creating multiple independent lightweight processes that communicate via message passing and are preemptively scheduled. I think that this is by far the best way to handle concurrency. Not to mention that with the BEAM you also get distributed systems and great debugging tools for free. Because of these attractive characteristics of Elixir, I decided that I would use Elixir on my next side project.

Building a user-friendly API

Let’s now turn to the actual library itself. The wtransport-elixir library uses the WTransport Rust library as a base (I'm a big fan of the Rust programming language as well!), as I did not feel like building everything entirely from scratch. WTransport is a pure-rust implementation of the WebTransport protocol. Using a cool project called Rustler, which allows you to bridge between Rust and Elixir code, I then wrote Elixir bindings.

Creating a good API is hard. For this reason, the wtransport-elixir bindings implement the server part of WTransport with an API heavily inspired by Thousand Island, a pure Elixir socket server. Because if somebody else already successfully created a good API, why not take inspiration from that? Taking inspiration from Thousand Island got me two benefits. First, Thousand Island is already used by Bandit, an HTTP server for Plug and WebSock apps written in Elixir, and by proxy, optionally by Phoenix, the web development framework written in Elixir. This means that the Thousand Island API has been proven to be adequate at the very least. Second, if someone is already familiar with Thousand Island, they will learn the API of wtransport-elixir very easily, as they already know all the concepts.

The wtransport-elixir project is open source, and all the code can be found on GitHub, so you can just check it out for yourself. If you would like to see the wtransport-elixir library in action, then there is a demo app that will allow you to do so. Go to the examples/wtransport_echo directory of the Github repository, where you’ll find the demo app. The project README provides the full instructions on how to try it out.

Bugs and documentation challenges

When building wtransport-elixir, I did run into a couple of challenges. For one, Rustler, which I used to bridge between Rust and Elixir code, while super cool, could be better documented. The Rustler documentation shows you how to make a function that computes a value, and immediately returns the result. But what if I need to start an async runtime in Rust, and communicate results asynchronously to Elixir? How do I pass some Rust data types to Elixir? That wasn’t described in the Rustler documentation. Such gaps meant that I had to look online at various unrelated, unofficial projects and blog posts, to understand how to use Rustler’s more advanced features.

Another challenge was that WebTransport has been developed only fairly recently. (The W3C’s first public working draft for WebTransport dates from May 2021.) As a result, when something didn’t work, it could be tricky to figure out whether this was due to a bug in the underlying Rust library, a bug in my code, or a bug in my Firefox browser. Luckily we got there in the end.

Next steps

At the moment the wtransport-elixir code is fairly complete, so I don't expect to be adding or changing anything—except for bug fixes, of course. Once I’ve properly documented the library, then the project is ready for being published on Hex, the package manager for the Erlang ecosystem. I'm planning to do this in the next few weeks.

All in all, wtransport-elixir was a really interesting project to work on. Moreover, writing a side project like this in Elixir has also benefited my work at TalkJS, because I'm much more familiar with the language because of this project. I learned a lot from it. And of course, if you’re into game development, I hope that wtransport-elixir can make your development life just that bit easier.