Existing WebRTC is not great for broadcasting use cases
WebRTC was originally designed for real-time communication with a small number of participants, where latency requirements are extremely strict (typically <250ms). However, it has also been utilized for broadcasting use cases, such as YouTube Studio or Cloudflare CDN, where protocols used in the past have been different, typically Adobe’s RTMP and protocols based on HTTP.
Why do we have lower quality when using WebRTC?
- Smaller buffers: WebRTC implementations prioritize latency over quality by using small buffers, which can sometimes be too small to receive retransmissions in time. TCP-based protocols with large buffers prioritize quality over latency and do not face this issue.
- Aggressive congestion control: WebRTC employs aggressive congestion control that quickly reduces video transmission when congestion is detected to prevent increased latency and potential packet loss. This helps to have more predictive low latency but in many cases being less aggressive would maintain a higher quality at the expense of a higher latency.
- Only real time reliability: If you lose connectivity for some seconds WebRTC is not going to queue the media to deliver it whenever you recover. Same way it is not going to recover a previous frame after a newer one has been decoded. The most obvious impact on this is in case of server side recordings.
- Fast Encoding: WebRTC codecs configuration are optimized for speed and do not use two-pass encodings or bidirectional predicted frames, instead using conservative settings to ensure fast encoding. However, this means that the encoding efficiency is not as good as that of broadcasting applications, resulting in slightly worse encoding for a given bitrate.
- Unpredictable Bitrate: The fact that WebRTC cannot use 2 pass encoding makes the quality/bitrate decisions by the codecs less optimal and unstable.
What should we do for interactive broadcasting?
- More tunneable WebRTC. With some more flexibility in WebRTC APIs it should be possible to overcome the limitations described in the previous section. For example things like configurable congestion control with settings like RTCPeerConnection.congestionControl = "low latency" or some lower level APIs in the new WebCodecs API.
- New protocols working over UDP and reusing some of the WebRTC techniques but providing more flexibility (f.e. new QUIC protocols like RUSH). Implementations using those protocols created from scratch can overcome most of those limitations because they leave part of the problem (buffering or encoding) to the application. Although they would also need the ability to control the congestion control behavior.