How are retransmissions implemented in WebRTC

Executive Summary: This ended up being more complex than expected, if you are building an SFU just take the & co files and use them.  Or even better, just use OpenTok or any other existing platform.   Anyway it is fun stuff to play with.

Introduction When your network is congested and some of the packets are lost you need a mechanism to recover them.   If we ignore signal processing tricks (like stretching the previous and next audio packets in the jitter buffer) there are two alternatives:
Forward error correction: in each packet you add some extra information about the previous ones in case they are lost and you need to reconstruct them (flexfec is the new format to do this in WebRTC [1]).Retransmissions: the sender will resend the packets usually after the receiver requests that with a negative acknowledgement message (NACK) indicating which packets were lost. These mechanisms can be combined depending on the network conditions and can also be tuned for specific c…

Bandwidth Estimation in WebRTC (and the new Sender Side BWE)

Bandwidth estimation is probably the most critical component in the video engine of WebRTC. The bandwidth estimation (BWE) module is responsible for deciding how much video* traffic you can send without congesting the network to prevent degradation of the video quality.

In the past bandwidth estimation algorithms used to be very rudimentary and based mostly on packet loss. Basically we used to start increasing slowly the video bitrate until we detected packets being lost.  To detect the packet loss you use the standard RTCP feedback mechanisms where the receiver side reports packet loss periodically using RTCP Receiver Report (RR) messages.

Modern bandwidth estimation algorithms are more advanced and try to detect congestion before it is bad enough to make routers discard packets. These algorithms predicts congestion analyzing the delay between packets. The idea is that when you start having some congestion, the buffers in the routers will start filling and the delay will be more va…

Using DSCP for WebRTC packet marking and prioritization

It is a common request from WebRTC developers and customers to know how they can differentiate WebRTC traffic from other type in their networks.  Usually the goal is to be able to prioritize RTC traffic over other types of less important traffic.

By prioritizing the WebRTC traffic in the edge routers it is possible to improve the quality in some network scenarios.  The most common cases where this marking may help are:

Congested broadband uplink where the router can discard other type of traffic instead of WebRTC traffic when queues get full.Congested local wireless network One obvious way to do this is forcing all the traffic to be relayed through a TURN or SFU server and se the priority based on IP addresses.   That's easy and always work, but the problem is that it is very difficult to maintain when your infrastructure changes often (scaling elastically, new datacenters...).

Other way to accomplish this is to use Differentiated Services Code Point (DSCP) marking. With DSCP you c…

Controlling bandwidth usage in WebRTC (and how googSuspendBelowMinBitrate works)

There are cases when we would like to limit the maximum bitrate being transmitted by WebRTC to avoid wasting resources in the user endpoints or save money reducing the bandwidth usage in our servers.   This is because the maximum bitrate by default in Chrome is around 2Mbps and for many use cases a much lower bitrate provides still pretty good quality.   BTW using a lower bitrate can also help with stability of quality in multiparty scenarios by reducing the amount of competition among different streams.

There is no simple API to configure the maximum bitrate in WebRTC (although there is one in ORTC) but there are 3 ways to do this by mangling the SDP.

1. Use the standard b=AS:BITRATE (Chrome) or b=TIAS:BITRATE (Firefox) attributes in the SDP for the audio or video channel[1]
2. Use codec specific attributes (this work at least for opus audio codec with maxaveragebitrate property) [2]
3. Use the proprietary x-max-bitrate attribute in the video channel of the SDP answer.  For example w…

Using Native WebRTC simulcast support in Chrome (or how to be as good as Hangouts) [WiP]

Some weeks ago Philipp Hancke presented an awesome analysis of how Google Hangouts uses WebRTC in Chrome.  In that blog post he explained that based on the SDP contents Google is using simulcast but didn't entered in the details of how to activate it.   So I had a lot of curiosity and thought that it could be great if people (beyond Google) could use this feature so I tried to replicate their behavior.

Step one: Add simulcast ssrcs and "SIM" group to the offer SDP

The first thing I tried is to implement some SDP mangling to make my SDP look like the Hangouts SDP.   That means adding 3 substreams grouped by simulcast semantics. This is the code of my quick and dirty implementation:

Result: no simulcast :(

Step two?: Add a renegotiation

I saw a a renegotiation in the initial connection from Google Hangouts (when there are no other participants) and I thought this was needed to enable simulcast and implemented it but this…

What does the MSID attribute in the SDP mean?

There is a work in progress in the IETF [1] to define a new MSID attribute to be used in the SDP. This attribute has been defined in the context of WebRTC and it is already being sent by WebRTC endpoints (Firefox and Chrome). This is a typical SDP generated by Chrome:

v=0 o=- 658899108507703479 2 IN IP4 s=- t=0 0 a=group:BUNDLE audio video a=msid-semantic: WMS FmJZgkjbR2gBS25U0hP3qoUSjvfDddQl4UjJ m=audio 1 RTP/SAVPF 111 103 104 0 8 106 105 13 126 ... a=ssrc:1400694016 cname:Yk4LvPXyWNZKkW6S a=ssrc:1400694016 msid:FmJZgkjbR2gBS25U0hP3qoUSjvfDddQl4UjJ a228df45 a=ssrc:1400694016 mslabel:FmJZgkjbR2gBS25U0hP3qoUSjvfDddQl4UjJ a=ssrc:1400694016 label:a228df45-515e-49b6-be8f-0d044b65de64 m=video 1 RTP/SAVPF 100 116 117 ... a=ssrc:1484622160 cname:Yk4LvPXyWNZKkW6S a=ssrc:1484622160 msid:FmJZgkjbR2gBS25U0hP3qoUSjvfDddQl4UjJ 00f2f3d2 a=ssrc:1484622160 mslabel:FmJZgkjbR2gBS25U0hP3qoUSjvfDddQl4UjJ a=ssrc:1484622160 label:00f2f3d2-974e-48a6-b8dc-0f70194949d5 But what is it for and why…

My professional life in RTC stacks ...

I was wondering today what RTC stacks I've used in all these years and I though it would be interesting to remember them because somehow you can see the evolution of communications through them.   This is the approximate timeline (I'm sure I'm forgetting some):

2013 python-sip
2012 pjsip
2011 asmack
2008 sofia-sip
2006 NIST SIP Stack
2004 In house developed SIP Stack
2002 OpenH323
2000 NetMeeting ActiveX

 With a couple of exceptions I have always favored the usage of opensource stacks because they have always been very compelling and mostly because it is a lot of fun the possibility to hack them.

There are much more stacks that I've used for prototypes or Hello Worlds and were also great but this list only includes the ones used in commercial projects.