WebRTC Video Conferencing (Zoom)

A real-time video conferencing system like Zoom must move audio and video between participants with sub-second latency over unpredictable home networks. It does this with a signaling layer to set up the call, NAT traversal (STUN/TURN) to punch through home routers, Selective Forwarding Units (SFUs) to route media streams, and simulcast plus bandwidth adaptation so each receiver gets the best resolution its connection can sustain.

Mental Model

Think of WebRTC video conferencing like setting up a multi-person phone call. First, you need an operator (Signaling Server) to exchange phone numbers (SDP/ICE) so everyone knows how to reach each other. If peers are behind home firewalls (NAT), they ask a public directory (STUN) for their public address. If they are behind a strict corporate security wall, they can’t connect directly; they must route calls through a paid relay operator (TURN). For the meeting layout, you can either call everyone directly (Mesh), send your feed to a central forwarder who passes copies around (SFU), or send your feed to a TV studio that mixes everyone into one split-screen video before sending it back (MCU).

Concept Flow

graph TD
    A[WebSocket Signaling Server] -->|Exchange SDP & ICE| B[Client A]
    A -->|Exchange SDP & ICE| C[Client B]
    B -->|Binding Query| D[STUN Server]
    C -->|Binding Query| D
    B -->|Fallback Media Relay| E[TURN Server]
    C -->|Fallback Media Relay| E
    B -->|Media Uplink| F[SFU Media Server]
    F -->|Forwarded Downlinks| C

Components and Why They Exist

WebSocket Signaling Server

The WebSocket signaling server coordinates initial peer discovery and session negotiation out-of-band before media streaming begins. During call setup, connection metadata packets route through this signaling server to exchange Session Description Protocol (SDP) offers and answers along with ICE network candidates between participant devices.

STUN Server

The STUN server helps client devices discover their public-facing IP addresses and ports behind home NAT routers. Each client queries the STUN server with a UDP binding request to learn its reflexive public NAT mapping before attempting direct peer-to-peer connectivity.

TURN Server

The TURN server acts as a fallback media relay when strict Symmetric NAT firewalls block direct UDP connectivity. Whenever direct peer paths are blocked, encrypted UDP/SRTP media packets route through the TURN relay server so isolated participants can still join the call.

SFU (Selective Forwarding Unit)

The Selective Forwarding Unit (SFU) routes multi-party conference streams efficiently with low server CPU overhead and a single upload per publisher. It ingests single simulcast uplinks from each participant and selectively forwards the appropriate quality layer down to each subscriber according to their available bandwidth.

MCU (Multipoint Control Unit)

The Multipoint Control Unit (MCU) minimizes subscriber download bandwidth and client decoding load by mixing all video feeds on the server. It decodes every incoming participant stream, merges them into a single composite grid layout, and sends one re-encoded stream out to each viewer.

Edge Cases and Misconceptions

• P2P is Not Always Free: A common misconception is that WebRTC is entirely peer-to-peer and incurs no server hosting costs. In real-world environments, up to 20% of calls fail direct connection checks and must route through TURN servers, creating substantial network bandwidth fees.
• SFU vs. MCU Client Constraints: While an SFU is cheap to host, it forces the viewer client’s device to decode $N-1$ separate incoming video streams simultaneously. For large calls, this heats up mobile devices and drains batteries rapidly. MCUs keep client CPU cool by doing the compositing work on the server side.
• Encryption Limits Server Features: When End-to-End Encryption (E2EE) is enabled, media packets are encrypted using SRTP. Since SFUs only route encrypted ciphertext without decrypting it, the server cannot perform advanced features like active speaker detection (via audio volume analysis), server-side virtual backgrounds, or composite recording.