Sending Packet Loss Feedback in WebRTC SFUs
One of the responsibilities of WebRTC SFUs is to receive and send RTCP packets. RTCP packets include different types of feedback about audio and video streams and one of the most important RTCP packets is the Receiver Report (RR).
RR packets are sent from the receiver of the media stream towards the sender of that media stream. In case of an SFU the RR are generated and sent from the SFU to the media stream Sender and also from every stream Receiver to the SFU (Figure 1).
The feedback sent inside RR packets include fields to calculate the round-trip-time delay, the jitter and the packet loss introduced by the network.
The packet loss reported in these RR packets is important because the audio and video being sent will be adjusted based on that parameter:
In the next sections you can find a suggestion for how to handle this in three different types of streams: Audio, Video with simulcast and Video without simulcast.
If we want audio FEC to work well for the participant with the weakest link then we have to make sure we report the worse packet loss to the sender.
Because of that the packet loss reported from the SFU to the sender should be the worse packet of the receivers (max(PL1, PL2)).
For example the packet loss reported to the sender will be 20% if one of the receiving participants is experiencing a 20% packet loss in the downlink even if the sender and rest of receivers are in perfect network conditions.
Because of that the packet loss reported from the SFU to the sender should be the packet loss in that link (Sender-SFU) no matter what the packet loss received the Receiver 1 and 2 is.
For example the packet loss reported to the sender will be 0 if the sender uplink is good even if the receivers are experiencing big packet loss in the downlink. That will be corrected with retransmissions and bitrate adaptation done in the SFU by selection a lower resolution/layer to be forwarded to those participants.
This bitrate adaptation is mostly controlled by REMB RTCP packets where the SFU includes the bandwith available in the worse receiver. But packet loss has also an impact on top of the bitrate reported in the REMB packets so we need to decide what packet loss to include in the RR packets.
In this case I think both approaches described in the previous sections can work reasonably well. Either a) the SFU reports back the worse packet loss of the receivers or b) use the packet loss of each receiver to estimate its bandwidth and report only the sender side packet loss in the RR packets.
Note: If you use P2P connections when there are 2 participants in the room and switch to the SFU with Simulcast when there are 3 or more participants in the room you should be almost never in this Video with No-Simulcast and multiple participants case.
You can follow me in Twitter if you are interested in Real Time Communications.
RR packets are sent from the receiver of the media stream towards the sender of that media stream. In case of an SFU the RR are generated and sent from the SFU to the media stream Sender and also from every stream Receiver to the SFU (Figure 1).
The feedback sent inside RR packets include fields to calculate the round-trip-time delay, the jitter and the packet loss introduced by the network.
The packet loss reported in these RR packets is important because the audio and video being sent will be adjusted based on that parameter:
- In case of audio streams the packet loss in the network modifies the level of robustness of the OPUS codec. In presence of high packet loss the sender increases the level of redundancy of the forward error correction (FEC) included in the audio packets.
- In case of video streams the packet loss modifies the video bitrate encoded and sent. In presence of high packet loss the sender decreases the bitrate sent to reduce the possible congestion in the network.
In the next sections you can find a suggestion for how to handle this in three different types of streams: Audio, Video with simulcast and Video without simulcast.
Audio
Opus FEC is sent inband and because of that FEC is end to end (cannot be added/updated in an SFU) and everybody in the room will receive the same level of FEC.If we want audio FEC to work well for the participant with the weakest link then we have to make sure we report the worse packet loss to the sender.
Because of that the packet loss reported from the SFU to the sender should be the worse packet of the receivers (max(PL1, PL2)).
For example the packet loss reported to the sender will be 20% if one of the receiving participants is experiencing a 20% packet loss in the downlink even if the sender and rest of receivers are in perfect network conditions.
Video with Simulcast
When using Simulcast the SFU has the ability to send a different video quality to each participant so there is no need to adapt the sender stream to any specific participant.Because of that the packet loss reported from the SFU to the sender should be the packet loss in that link (Sender-SFU) no matter what the packet loss received the Receiver 1 and 2 is.
For example the packet loss reported to the sender will be 0 if the sender uplink is good even if the receivers are experiencing big packet loss in the downlink. That will be corrected with retransmissions and bitrate adaptation done in the SFU by selection a lower resolution/layer to be forwarded to those participants.
Video with No-Simulcast
When there is no Simulcast the SFU has to send the same video quality to every participant. Because of that it is important that the sender adapts the sending bitrate to the participant with the weakest link (and/or disable the video for some participants).This bitrate adaptation is mostly controlled by REMB RTCP packets where the SFU includes the bandwith available in the worse receiver. But packet loss has also an impact on top of the bitrate reported in the REMB packets so we need to decide what packet loss to include in the RR packets.
In this case I think both approaches described in the previous sections can work reasonably well. Either a) the SFU reports back the worse packet loss of the receivers or b) use the packet loss of each receiver to estimate its bandwidth and report only the sender side packet loss in the RR packets.
Note: If you use P2P connections when there are 2 participants in the room and switch to the SFU with Simulcast when there are 3 or more participants in the room you should be almost never in this Video with No-Simulcast and multiple participants case.
You can follow me in Twitter if you are interested in Real Time Communications.
The correct thing to do is highly dependant n the context. If your SFU has 20 clients with great connections and only 1 with a very bad connection. Do you really want to reduce image quality for everybody?
ReplyDeleteThe development of artificial intelligence (AI) has propelled more programming architects, information scientists, and different experts to investigate the plausibility of a vocation in machine learning. Notwithstanding, a few newcomers will in general spotlight a lot on hypothesis and insufficient on commonsense application. machine learning projects for final year In case you will succeed, you have to begin building machine learning projects in the near future.
DeleteProjects assist you with improving your applied ML skills rapidly while allowing you to investigate an intriguing point. Furthermore, you can include projects into your portfolio, making it simpler to get a vocation, discover cool profession openings, and Final Year Project Centers in Chennai even arrange a more significant compensation.
Data analytics is the study of dissecting crude data so as to make decisions about that data. Data analytics advances and procedures are generally utilized in business ventures to empower associations to settle on progressively Python Training in Chennai educated business choices. In the present worldwide commercial center, it isn't sufficient to assemble data and do the math; you should realize how to apply that data to genuine situations such that will affect conduct. In the program you will initially gain proficiency with the specialized skills, including R and Python dialects most usually utilized in data analytics programming and usage; Python Training in Chennai at that point center around the commonsense application, in view of genuine business issues in a scope of industry segments, for example, wellbeing, promoting and account.
ReplyDeleteNice blog..! I really loved reading through this article. Thanks for sharing such a
amazing post with us and keep blogging... iot training in chennai | iot training in chennai quora | iot training and placement in chennai | iot training center in chennai | best iot training centre in chennai
Thanks for your great and helpful presentation I like your good service. I always appreciate your post. That is very interesting I love reading and I am always searching for informative information like this.angular 4 training in chennai | angularjs training in chennai | .angularjs best training center in chennai | angularjs training in velachery |
ReplyDeleteVery good information provided, Thanks a lot for sharing such useful information.
ReplyDeletebest aviation academy in Chennai
Air Hostess Training Institute in Chennai
Airline Courses in Chennai
airport ground staff training in Chennai
Aviation Academy in Chennai
air hostess training in Chennai
airport management courses in Chennai
ground staff training in Chennai
This is an awesome post.Really very informative and creative contents. These concept is a good way to enhance the knowledge.I like it and help me to article very well.Thank you for this brief explanation and very nice information.Well, got a good knowledge.
ReplyDeleteDedicatedHosting4u.com
Consistently learning the new skills and using them to improve myself in the form of performance for betterment.
ReplyDeleteThanks
Cpa offers
ReplyDeleteThis is a fantastic idea! I like it a lot because it's super easy for the audience to see the value of opting in. wonderful and amazing post very use full your post thanks for sharing your article
Android Application development
Web application
ReplyDeleteYour post is very good. I got to learn a lot from your post. Thank you for sharing your article for us. it is amazing post
what is seo
types of seo
Mua vé máy bay tại đại lý vé Aivivu, tham khảo
ReplyDeleteve may bay di my gia re
vé máy bay từ mỹ về việt nam giá bao nhiêu
vé máy bay từ đức về việt nam
dịch vụ vé máy bay tại nga