Sending Packet Loss Feedback in WebRTC SFUs
One of the responsibilities of WebRTC SFUs is to receive and send RTCP packets. RTCP packets include different types of feedback about audio and video streams and one of the most important RTCP packets is the Receiver Report (RR).
RR packets are sent from the receiver of the media stream towards the sender of that media stream. In case of an SFU the RR are generated and sent from the SFU to the media stream Sender and also from every stream Receiver to the SFU (Figure 1).
The feedback sent inside RR packets include fields to calculate the round-trip-time delay, the jitter and the packet loss introduced by the network.
The packet loss reported in these RR packets is important because the audio and video being sent will be adjusted based on that parameter:
In the next sections you can find a suggestion for how to handle this in three different types of streams: Audio, Video with simulcast and Video without simulcast.
If we want audio FEC to work well for the participant with the weakest link then we have to make sure we report the worse packet loss to the sender.
Because of that the packet loss reported from the SFU to the sender should be the worse packet of the receivers (max(PL1, PL2)).
For example the packet loss reported to the sender will be 20% if one of the receiving participants is experiencing a 20% packet loss in the downlink even if the sender and rest of receivers are in perfect network conditions.
Because of that the packet loss reported from the SFU to the sender should be the packet loss in that link (Sender-SFU) no matter what the packet loss received the Receiver 1 and 2 is.
For example the packet loss reported to the sender will be 0 if the sender uplink is good even if the receivers are experiencing big packet loss in the downlink. That will be corrected with retransmissions and bitrate adaptation done in the SFU by selection a lower resolution/layer to be forwarded to those participants.
This bitrate adaptation is mostly controlled by REMB RTCP packets where the SFU includes the bandwith available in the worse receiver. But packet loss has also an impact on top of the bitrate reported in the REMB packets so we need to decide what packet loss to include in the RR packets.
In this case I think both approaches described in the previous sections can work reasonably well. Either a) the SFU reports back the worse packet loss of the receivers or b) use the packet loss of each receiver to estimate its bandwidth and report only the sender side packet loss in the RR packets.
Note: If you use P2P connections when there are 2 participants in the room and switch to the SFU with Simulcast when there are 3 or more participants in the room you should be almost never in this Video with No-Simulcast and multiple participants case.
You can follow me in Twitter if you are interested in Real Time Communications.
RR packets are sent from the receiver of the media stream towards the sender of that media stream. In case of an SFU the RR are generated and sent from the SFU to the media stream Sender and also from every stream Receiver to the SFU (Figure 1).
The feedback sent inside RR packets include fields to calculate the round-trip-time delay, the jitter and the packet loss introduced by the network.
The packet loss reported in these RR packets is important because the audio and video being sent will be adjusted based on that parameter:
- In case of audio streams the packet loss in the network modifies the level of robustness of the OPUS codec. In presence of high packet loss the sender increases the level of redundancy of the forward error correction (FEC) included in the audio packets.
- In case of video streams the packet loss modifies the video bitrate encoded and sent. In presence of high packet loss the sender decreases the bitrate sent to reduce the possible congestion in the network.
In the next sections you can find a suggestion for how to handle this in three different types of streams: Audio, Video with simulcast and Video without simulcast.
Audio
Opus FEC is sent inband and because of that FEC is end to end (cannot be added/updated in an SFU) and everybody in the room will receive the same level of FEC.If we want audio FEC to work well for the participant with the weakest link then we have to make sure we report the worse packet loss to the sender.
Because of that the packet loss reported from the SFU to the sender should be the worse packet of the receivers (max(PL1, PL2)).
For example the packet loss reported to the sender will be 20% if one of the receiving participants is experiencing a 20% packet loss in the downlink even if the sender and rest of receivers are in perfect network conditions.
Video with Simulcast
When using Simulcast the SFU has the ability to send a different video quality to each participant so there is no need to adapt the sender stream to any specific participant.Because of that the packet loss reported from the SFU to the sender should be the packet loss in that link (Sender-SFU) no matter what the packet loss received the Receiver 1 and 2 is.
For example the packet loss reported to the sender will be 0 if the sender uplink is good even if the receivers are experiencing big packet loss in the downlink. That will be corrected with retransmissions and bitrate adaptation done in the SFU by selection a lower resolution/layer to be forwarded to those participants.
Video with No-Simulcast
When there is no Simulcast the SFU has to send the same video quality to every participant. Because of that it is important that the sender adapts the sending bitrate to the participant with the weakest link (and/or disable the video for some participants).This bitrate adaptation is mostly controlled by REMB RTCP packets where the SFU includes the bandwith available in the worse receiver. But packet loss has also an impact on top of the bitrate reported in the REMB packets so we need to decide what packet loss to include in the RR packets.
In this case I think both approaches described in the previous sections can work reasonably well. Either a) the SFU reports back the worse packet loss of the receivers or b) use the packet loss of each receiver to estimate its bandwidth and report only the sender side packet loss in the RR packets.
Note: If you use P2P connections when there are 2 participants in the room and switch to the SFU with Simulcast when there are 3 or more participants in the room you should be almost never in this Video with No-Simulcast and multiple participants case.
You can follow me in Twitter if you are interested in Real Time Communications.
The correct thing to do is highly dependant n the context. If your SFU has 20 clients with great connections and only 1 with a very bad connection. Do you really want to reduce image quality for everybody?
ReplyDelete
ReplyDeleteNice blog..! I really loved reading through this article. Thanks for sharing such a
amazing post with us and keep blogging... iot training in chennai | iot training in chennai quora | iot training and placement in chennai | iot training center in chennai | best iot training centre in chennai
Thanks for your great and helpful presentation I like your good service. I always appreciate your post. That is very interesting I love reading and I am always searching for informative information like this.angular 4 training in chennai | angularjs training in chennai | .angularjs best training center in chennai | angularjs training in velachery |
ReplyDeleteVery good information provided, Thanks a lot for sharing such useful information.
ReplyDeletebest aviation academy in Chennai
Air Hostess Training Institute in Chennai
Airline Courses in Chennai
airport ground staff training in Chennai
Aviation Academy in Chennai
air hostess training in Chennai
airport management courses in Chennai
ground staff training in Chennai