Different WebRTC flavours from a server/platform point of view

WebRTC is a complex component with many optional features.  Because of that there are different ways to use it, specially when your solution includes a media server in the middle, and not everybody is using it the same way.

In some cases because of the use case requires some specific features, in others because of the background/history of the solution and in others maybe because of lack of development resources the fact is that today it is difficult to find two platforms/servers using WebRTC exactly with the same configuration/features.

In this post we are trying to present how some of the most popular platforms/servers are using it.  The goal is not to compare them or decide who is the best one (because as we said it depends on many factors) but to give an overview of the different options you have.

This table has been made with help from many contributors, mostly the main developers of some of those media servers / platforms and the idea is to keep it updated and extend it.  Feel f…

HOWTO Use temporal scalability to adapt video bitrates

We have talked many times about video scalability (VP9 SVC, Simulcast) and how it enables modern RTC infrastructures to provide high quality multiparty experiences while maintaining low infrastructure costs (re-encoding the video is more expensive and also degrades the media quality).  This approach was popularised by Vidyo and can be seen in modern platforms like OpenTok and services like Hangouts/Meet these days.

Typically we talk about temporal and spatial scalability although you can also have quality scalability.   With spatial scalability you generate multiple versions of the video with different resolutions.  In case of temporal scalability the sequence of frames is encoded in a way so that some frames can be dropped in the server and the resulting frames sequence can be still decoded in the receiver side.

This is an example of a typical VP8 encoding with 2 temporal layers where the base layer (blue frames) don't depend on the higher layer (yellow frames):

This type of enco…

How are retransmissions implemented in WebRTC

Executive Summary: This ended up being more complex than expected, if you are building an SFU just take the & co files and use them.  Or even better, just use OpenTok or any other existing platform.   Anyway it is fun stuff to play with.

Introduction When your network is congested and some of the packets are lost you need a mechanism to recover them.   If we ignore signal processing tricks (like stretching the previous and next audio packets in the jitter buffer) there are two alternatives:
Forward error correction: in each packet you add some extra information about the previous ones in case they are lost and you need to reconstruct them (flexfec is the new format to do this in WebRTC [1]).Retransmissions: the sender will resend the packets usually after the receiver requests that with a negative acknowledgement message (NACK) indicating which packets were lost. These mechanisms can be combined depending on the network conditions and can also be tuned for specific c…

Bandwidth Estimation in WebRTC (and the new Sender Side BWE)

Bandwidth estimation is probably the most critical component in the video engine of WebRTC. The bandwidth estimation (BWE) module is responsible for deciding how much video* traffic you can send without congesting the network to prevent degradation of the video quality.

In the past bandwidth estimation algorithms used to be very rudimentary and based mostly on packet loss. Basically we used to start increasing slowly the video bitrate until we detected packets being lost.  To detect the packet loss you use the standard RTCP feedback mechanisms where the receiver side reports packet loss periodically using RTCP Receiver Report (RR) messages.

Modern bandwidth estimation algorithms are more advanced and try to detect congestion before it is bad enough to make routers discard packets. These algorithms predicts congestion analyzing the delay between packets. The idea is that when you start having some congestion, the buffers in the routers will start filling and the delay will be more va…

Using DSCP for WebRTC packet marking and prioritization

It is a common request from WebRTC developers and customers to know how they can differentiate WebRTC traffic from other type in their networks.  Usually the goal is to be able to prioritize RTC traffic over other types of less important traffic.

By prioritizing the WebRTC traffic in the edge routers it is possible to improve the quality in some network scenarios.  The most common cases where this marking may help are:

Congested broadband uplink where the router can discard other type of traffic instead of WebRTC traffic when queues get full.Congested local wireless network One obvious way to do this is forcing all the traffic to be relayed through a TURN or SFU server and se the priority based on IP addresses.   That's easy and always work, but the problem is that it is very difficult to maintain when your infrastructure changes often (scaling elastically, new datacenters...).

Other way to accomplish this is to use Differentiated Services Code Point (DSCP) marking. With DSCP you c…

Controlling bandwidth usage in WebRTC (and how googSuspendBelowMinBitrate works)

There are cases when we would like to limit the maximum bitrate being transmitted by WebRTC to avoid wasting resources in the user endpoints or save money reducing the bandwidth usage in our servers.   This is because the maximum bitrate by default in Chrome is around 2Mbps and for many use cases a much lower bitrate provides still pretty good quality.   BTW using a lower bitrate can also help with stability of quality in multiparty scenarios by reducing the amount of competition among different streams.

There is no simple API to configure the maximum bitrate in WebRTC (although there is one in ORTC) but there are 3 ways to do this by mangling the SDP.

1. Use the standard b=AS:BITRATE (Chrome) or b=TIAS:BITRATE (Firefox) attributes in the SDP for the audio or video channel[1]
2. Use codec specific attributes (this work at least for opus audio codec with maxaveragebitrate property) [2]
3. Use the proprietary x-max-bitrate attribute in the video channel of the SDP answer.  For example w…

Using Native WebRTC simulcast support in Chrome (or how to be as good as Hangouts) [WiP]

Some weeks ago Philipp Hancke presented an awesome analysis of how Google Hangouts uses WebRTC in Chrome.  In that blog post he explained that based on the SDP contents Google is using simulcast but didn't entered in the details of how to activate it.   So I had a lot of curiosity and thought that it could be great if people (beyond Google) could use this feature so I tried to replicate their behavior.

Step one: Add simulcast ssrcs and "SIM" group to the offer SDP

The first thing I tried is to implement some SDP mangling to make my SDP look like the Hangouts SDP.   That means adding 3 substreams grouped by simulcast semantics. This is the code of my quick and dirty implementation:

Result: no simulcast :(

Step two?: Add a renegotiation

I saw a a renegotiation in the initial connection from Google Hangouts (when there are no other participants) and I thought this was needed to enable simulcast and implemented it but this…