Introducing Elixir WebRTC
For the last few months, we’ve been working really hard on an Elixir implementation of the W3C WebRTC specification and we are glad to officially announce its very first release ― the 0.1 version of ex_webrtc!
But why 🤔? Well, we’ve been using WebRTC in Elixir for a few years now, but it was mostly some C and Erlang libraries glued together using the Membrane Framework, and it was fine for the most part, except for some issues:
- sometimes we needed to contribute to the underlying libraries, or even worse, maintain our own forks ― that was a struggle,
- it was tough to introduce engineers from outside to our code, as it was completely different from the usual WebRTC API,
- Membrane is amazing for what it’s designed for, but we realized that the WebRTC API doesn’t really map well to the pipeline model, which made some things unnecessarily difficult.
With the previous experiences and these issues in mind, we decided to write an (almost) pure Elixir, W3C-compliant, open-source WebRTC implementation from the ground up, with the whole networking stack. Our goal is to create a solution similar to projects like Pion or WebRTC.rs, but for the Elixir ecosystem. So far, we’re on the right track!
Before getting into the nitty-gritty of what’s in version 0.1, check out this quick tutorial on how to use the Phoenix Framework, Elixir Nx, and Elixir WebRTC to create a web app that performs image recognition based on the web-cam video feed from a browser!
The app will be publicly hosted for some time 👉here👈.
What’s in the 0.1?
The first release focuses on three aspects:
- API compliant with the standard,
- ease of integration with the Elixir ecosystem,
- learning resources.
API that you’re familiar with
ExWebRTC tries to be compliant with the well-known JavaScript API while still being idiomatic to Elixir. The goal here is to make it easy for people familiar with WebRTC to jump in, but not compromise on the amazing features that make Elixir so powerful and unique. The version 0.1 already brings you most of the functions available in the JavaScript API.
For example, this snippet in JS:
const pc = new RTCPeerConnection();
const tr = pc.addTransceiver("audio");
const offer = await pc.createOffer();
await pc.setLocalDescription(offer);maps to the following Elixir code:
{:ok, pc} = PeerConnection.start_link()
{:ok, tr} = PeerConnection.add_transceiver(pc, :audio)
{:ok, offer} = PeerConnection.create_offer(pc)
:ok = PeerConnection.set_local_description(pc, offer)See our API reference and examples for more.
Simple integration
One of the goals of Elixir WebRTC is to be appealing to the Elixir folks and easy to integrate into existing Elixir projects! With that in mind, we wrote the ExWebRTC Dashboard ― statistics visualization for the Phoenix Framework. Just add ExWebRTCDashboard as an additional page for the Phoenix’s LiveDashboard and you will be provided with something similar to Chrome’s chrome://webrtc-internals(but good looking 😎)!
# add this to your Phoenix app's router.ex
live_dashboard "/dashboard",
additional_pages: [exwebrtc: ExWebRTCDashboard]
Learn some WebRTC
Understanding the W3C spec and all of its quirks is not a trivial task. We spent days reading the RFCs (my personal favorite has to be the RFC 8829, a great read for a winter evening with a cup of hot chocolate ☕), reverse-engineering Chrome’s implementation, and making issues with (sometimes dumb) questions in the webrtc-pc repo.
We wanted to gather all of that knowledge somewhere, and the first step in that direction is the Mastering Transceivers guide. It consists of a series of examples explaining some tips and tricks for WebRTC’s Transceivers. Each example contains both JS (you can quickly run it in the browser and experiment with it, all thanks to the JS Fiddle!) and Elixir snippet, which makes it easy to understand how the browser API maps to ours. We hope to add Livebooks for the Elixir code in the future!
What’s next?
We are extremely excited about what we have accomplished so far, and even more excited about what lies ahead of us! Our plan for the 0.2 is to:
- add support for Quality of Service features, like Transport-Wide Congestion Control (WebRTC sources, like browsers, use it to estimate how much network bandwidth they can use, and thus, how good the quality of the media may be) or Retransmissions (remember, WebRTC usually uses UDP, which is unreliable),
- add support for some ICE features, like mDNS ICE candidates (to prevent leakage of private IP addresses) or TURN servers (in some cases, P2P connection is not possible and TURN server needs to relay the network traffic),
- make QoL and performance improvements, and fix bugs (which are surely there).
We would love to hear your feedback and suggestions on what you need or what would be cool to add. Don’t hesitate to leave a comment on our Roadmap issue on GitHub! 🎉
Big thanks to Sean DuBois for creating the awesome Pion project and Philipp Hancke with the WebRTC community for answering our questions on webrtc-pc and the discuss-webrtc group!
Happy streaming!
