Video backgrounds are becoming blurry, but the future is clear: more machine learning models in WebRTC applications

Going live on at

• Ravindhran Sankar (Ravi)

Machine learning models and inferences are being run on the browser and have become very performant. Libraries and frameworks like mediapipe, bodypix have made it easy to integrate ML-based experiences into a WebRTC call with relative ease. We're still in the early days of discovering their possibilities in web applications.

In this talk, we will walk through how Daily integrated mediapipe in its WebRTC library. We will explore:

1. How segmentation models, like the ones used blur backgrounds, work.
2. How one can integrate an ML segmentation library (like mediapipe) in a WebRTC video call.
3. What other sorts of custom video applications and experiences one can build using a tech stack like these.

Hopefully, this leaves the audience with ideas for writing custom video processors that might enable the next generation of video experiences on WebRTC.