Posts

How are Images/Videos sent in WhatsApp?

Image
I've been involved in the development on different IM&P services in the past and always one of the core features was the ability to share media files (audio, video, pdf...) with other users or groups.


Probably the most popular of these services was TU Me that was based on SIP but used HTTP to upload and download the files from a central storage. SIP was used to send the message with the url to the other end so that he can download the media file.

In cases where I had to use XMPP there were typically two approaches at least if you need to provide offline messaging:
Some inefficient inband alternatives where the media files are sent using the XMPP connection.Custom (or not widely supported like xep-0363) extensions based on HTTP Uploads. But today I was curious on how this is done in WhatsApp.  I found this opensource python implementation (yowsup) that looks great and got an idea on how it could be done but I wanted to confirm what is the protocol used by the most recent WA clie…

Improving Real Time Communications with Machine Learning

Image
When we talk about the applications of Artificial Intelligence / Machine Learning (AI/ML) for Real Time Communications (RTC) we can group them in two different planes:
Service Level: There are many features that can be added to a videoconference service, for example identification of the participants, augmented reality, emotion detection, speech transcription or audio translation.  These features are usually based on image and speech recognition and language processing.Infrastructure Level: There are many ways to apply ML that do not provide new features but improve the quality and/or reliability of the audio/video transmission.