Ravindhran Sankar (Ravi)

Everyone has experienced video call issues over the past 20months - robotic voice, video glitches, low quality video.. What is actually going on? 
In this talk, we'll intentionally deteriorate a call and observe how these issues appear to the user and explore how they manifest in WebRTC metrics.

Debugging WebRTC call issues seems like a dark art. Everyone experiences video call issues, but few people can pinpoint what causes them. With dozens to thousands of call participants distributed across different geographies, environments, machines, and platforms, the range of possible reported issues is infinitely complex and varied. Whether you are a developer integrating real-time video experiences in your product or a user who simply wants to join video calls without interruptions - what are the things to think about & what can you do about them?

At Daily, we have actively debugged 1000s of call issues. We have learned that a small number of constraints are responsible for the majority of user frustrations.

In this talk, we'll play with a video call. We'll deteriorate a call on purpose. We will recreate some of the common issues users experience every day, understand why they happen and how they manifest themselves in WebRTC metrics. 

Hopefully, this talk will give the casual video call user some things to try and the video developer tools to debug these common issues.

The dark arts of debugging WebRTC calls

Finally, machine learning models have made it to the browser. Virtual backgrounds and blurs are everywhere!

In this talk, let's explore how a simple background blur works, how you can code your own blur for a WebRTC call, and most interestingly - what more can we build from here?

Machine learning models and inferences are being run on the browser and have become very performant. Libraries and frameworks like mediapipe, bodypix have made it easy to integrate ML-based experiences into a WebRTC call with relative ease. We're still in the early days of discovering their possibilities in web applications.

In this talk, we will walk through how Daily integrated mediapipe in its WebRTC library. We will explore:

1. How segmentation models, like the ones used blur backgrounds, work.
2. How one can integrate an ML segmentation library (like mediapipe) in a WebRTC video call.
3. What other sorts of custom video applications and experiences one can build using a tech stack like these.

Hopefully, this leaves the audience with ideas for writing custom video processors that might enable the next generation of video experiences on WebRTC.

Ravindhran Sankar (Ravi)

Bio

Talks