WebRTC Video Codecs performance evaluation

The standard and most popular codecs in WebRTC are VP8 and H.264 but those are not the only options we have.   VP9 has been available for a while and some big services are using it and AV1 support has been recently added to Chrome.

When comparing codecs there are interesting considerations like interoperability and licensing but probably the most important factors are how good the codec is in terms of compression and how cheap the codec is in terms of cpu&memory usage.

The compression ratio is usually the first thing we look at and there are many comparisons available for that, but the resource consumption is equally important if we want to be able to use the codecs for real time use cases.

Given that AV1 is available in the Chrome Canary versions I decided to run some tests to get estimation of where we stand in terms of cpu usage for the 4 available codecs in WebRTC ecosystem.   The idea of the test is to compare the whole video pipeline with those 4 codecs and not just the codecs in isolation.

Testing environment

The tests were done with a simple web page establishing a connection between 2 PeerConnections (one sending and the other receiving).  The test page is here in case you want to run the tests yourself:

https://jsfiddle.net/tvo7czxs/

The tests with that page were performed changing 3 variables:  

  • Codec: VP8, VP9, H264, AV1
  • Resolution: HD, VGA, QVGA
  • Bitrate: 200Kbps, 800Kbps, 2Mbps
If you look at the test page it is easy to change those 3 parameters to run the tests with other configurations or in other devices.

The Chrome version used was the latest one synced from git this week (1/2/21) and the tests were ran in a MacBook Pro (2.4 GHz 8-Core Intel Core i9).   

To check the CPU usage I just looked at the average CPU consumed by the Chrome process in the system Activity Monitor after waiting 30 seconds to give time for the stabilisation of bandwidth estimation and resolution/framerate adaption inside WebRTC.  When the results below say 100% that means 1 full core of the machine.

Nothing too fancy but hopefully easy enough to understand.

In that environment I ran the 36 combinations of the parameters a couple of times, averaged the results and summarised them in the following sections: 

QVGA results

For QVGA resolution the results were as expected, with VP9 requiring a bit more CPU than VP8 and AV1 requiring almost 2x CPU as VP8.   H.264 was the one requiring less CPU usage as it is using hardware acceleration for that.

 
% of CPU usage with different codecs
QVGA200Kbps800Kbps2Mbps
VP8182228
VP9202833
H264101415
AV1364650

VGA results

For VGA the results were not very different but at low bitrates only VP9 was able to maintain the resolution and AV1 was using more than 1 core when increasing the bitrate limit to 2 Mbps.   Quality of H.264 at 200Kbps was really bad with very low framerate and lot of blocking as adaptation is apparently not working well in this case in Chrome.

% of CPU usage with different codecs
VGA200Kbps800Kbps2Mbps
VP8183046
VP9263840
H26481316
AV13380150

HD (1280x720) results

HD results were similar to VGA but AV1 was not able to encode the original resolution and downscaled it in the tests with all the bitrates.  H264 also behaved very poorly at low bitrate and the difference between VP8 and VP9 cost was much higher than with VGA.

% of CPU usage with different codecs
HD200Kbps800Kbps2Mbps
VP8173065
VP9305080
H26481216
AV1354890

(In addition AV1 with HD resolution crashed often with a memory issue in a Mac related code, but maybe the bug is fixed at the time you read this)

Encoding vs Decoding cost

I did one more test to split the cost between encoding (Sender side) and decoding (Receiver side).   That test was performed for VGA at 800 Kbps and the results were the next ones for the four codecs under consideration.
The results are not very different but the decoding in case of VP9 and AV1X was relatively cheaper compared with the encoding.

Comparing just the decoding cost with different codecs it looks like AV1 is around 2x more expensive than the others.   VP9 is slightly more expensive than VP8 and VP8 slightly more expensive than H264 but without big differences between those three.

Conclusions

Having new codecs is amazing and there is no doubt AV1 is the future of real-time video communications but it looks like we should wait a bit to be able to use it for general purpose videoconferencing applications.   In the mean time we can probably use it for specific use cases like broadcasting, dedicated powerful devices or to encode the low resolution versions of the video streams when using simulcast.

For other use cases VP8 and VP9 look like still the best options unless you don't worry that much about low bitrate cases or you are using high resolutions and battery/cpu consumption is a big concern and you can consider H264.

Also there is apparently a new libaom patch coming that could improve the performance by a 15% so it would be good to repeat these tests with future versions of Chrome and different devices (AV1 could be potentially more optimized for ARM CPUs).

If you want to replicate the test and share your data I will update these tables/graphs with the average of the measurements. And as usual feedback is more than welcomed either here or in Twitter.

Thx a lot for the early feedback from @murillo, @elminiero, @dan_jenkins, @maniksach.

Comments

  1. Excellent and very cool idea and great content of different kinds of the valuable information’s.
    https://www.kellytechno.com/Hyderabad/Course/amazon-web-services-training

    ReplyDelete
  2. Thanks for sharing this informative content, Great work.

    To crack Scrum master interview: Scrum master interview questions

    ReplyDelete
  3. This comment has been removed by the author.

    ReplyDelete
  4. Best Essay Writing Service Students Our company is among the leading UK custom essays services because we've already gained our clients' trust for many years.

    ReplyDelete
  5. Hello, I would like to thank you for sharing this blog post with others. The info related to video codec performance is interesting, nevertheless, it's quite hard for me to understand. One of my friends who is actually works as a programmer told me about the source which can help to do my discussion board https://mid-terms.com/buy-discussion-board-post/ and he explained me the connection between the codec.

    ReplyDelete
  6. Emblix Academy – Digital marketing institute in KPHB, we address all major and minor aspects required for any student’s advancement in digital marketing. Clutch USA named our Digital Marketing Institute the best SEO firm. for more info visit here :- https://emblixacademy.com/

    ReplyDelete
  7. Thanks for posting this info i feel same as like i read about your Buy Coursework post. I just want to let you know that I just check out your site and I find it very interesting and informative. I can't wait to read lots of your posts.

    ReplyDelete

Post a Comment

Popular posts from this blog

Bandwidth Estimation in WebRTC (and the new Sender Side BWE)

Improving Real Time Communications with Machine Learning

Controlling bandwidth usage in WebRTC (and how googSuspendBelowMinBitrate works)