TikTok Recommendation and Delivery System
Walk through a TikTok-style short-video platform from creator upload through encoder output, Object Storage arrival, item embeddings, Two-Tower ranking, CDN pre-fetching, viewer playback, and feedback recovery.
Continue with the guided workspace
Use the paired workspace to connect this guide's mental model with live component state, visible signals, and the outcome each step or scenario is meant to prove.
A short-video platform like TikTok combines a high-throughput ingestion pipeline, a recommendation engine, and a global CDN. Creators upload raw video, a transcoding fanout cuts it into streamable segments, a Two-Tower recommendation model scores candidate videos against the viewer’s engagement profile, and CDN edges cache the hottest segments so playback starts in under a second.
For parameter experimentation, the TikTok System Simulator is also available.
Mental Model
Think of TikTok’s architecture like a high-speed library combined with a personal book matcher. Creators upload raw drafts (uploads) that are automatically formatted, cut into short chapters, and placed in a central warehouse (transcoding & object storage). The head librarian (CDN edge) keeps copies of popular chapters nearby for immediate reading. To recommend books, the librarian uses two matching boards (Two-Tower model): one representing book subjects (Item Tower) and one representing the reader’s mood (User Tower). As the reader flips pages (user engagement), the librarian updates their mood profile in real-time to hand them better matched chapters next.
Components and Why They Exist
Creator App
The Creator App gives content creators an interface to record and publish raw video assets to the platform. When a creator submits a clip, the client streams the raw video payload and initial post attributes toward the ingestion gateway.
Video Upload API
The upload API acts as the ingestion gateway handling incoming video uploads, metadata creation, and worker triggering. It validates authentication and media format, persists the initial publish record, and enqueues an asynchronous transcoding job so the upload call can return without waiting for video encoding.
Video Transcoding Service
The transcoding service prepares video streams for varied network bitrates and smooth playback. It slices uploaded videos into uniform 3-second segments and encodes multiple bitrate renditions before pushing the completed segment files and manifest to Object Storage.
Object Storage (Origin)
Object storage serves as the central, highly durable persistent store for all static chunk assets. Once the transcoder writes the encoded segments and manifest to the origin node, the segment count updates and the video becomes ready for item embedding and CDN edge caching.
Metadata DB
The metadata database stores the durable publish-metadata record alongside engagement analytics counters. Its state transitions to video_482 persisted on the upload path and increments the analytics counter as viewer watch events arrive, while keeping user embedding vectors separated in the User Tower.
Two-Tower Recommendation Engine
The recommendation engine pairs the Item Tower candidate embeddings with the live User Tower vector to produce a ranked feed. Streaming feedback from the Event Stream refreshes the User Tower state from baseline to fresh: dance + diy, which then updates the Ranker score order and drives the downstream CDN pre-fetch plan.
CDN Edge Node
The CDN edge node caches transcoded video segments near users to reduce playback latency and shield the origin from read congestion. Hot chunk files are held in edge cache slots and served locally in ~15ms, falling back to Object Storage only on a cache miss.
Viewer Client
The viewer client is the player application that renders video streams and captures watch telemetry. By buffering the first chunks of upcoming ranked videos in advance, the player delivers a near-zero-latency swipe transition while emitting watch duration events.
User Engagement Queue
The engagement event stream captures real-time viewer watch and swipe telemetry. Each playback event appended by the viewer client fans out to increment the Metadata + Analytics counter and refresh the User Tower vector so subsequent feed rankings reflect current session behavior.
Edge Cases and Misconceptions
- Two-Tower Model Latency: A common misconception is that the recommendation system runs heavy deep neural network inference for every viewer swipe in real-time. In reality, the Item Tower embeddings are pre-computed offline. The User Tower computes a user vector in real-time, and candidate selection is performed using fast Approximate Nearest Neighbors (ANN) search (e.g. Hierarchical Navigable Small World graphs) on the pre-computed item vectors.
- Pre-fetching Trade-offs: Pre-fetching the first few chunks of the next recommended videos creates a seamless, zero-latency swiping experience. However, if a user swipes rapidly (low watch time), pre-fetched chunks are discarded, wasting massive network egress bandwidth. Recommenders must dynamically scale the pre-fetch window based on the viewer’s swipe speed and network conditions.
- Thundering Herd on Cache Eviction: If a highly viral video’s chunks expire or are manually flushed from the CDN simultaneously, thousands of viewer clients will query the Object Storage origin at once. This thundering herd can overwhelm database metadata services and origin storage throughput.
Common Questions
How does the recommendation feedback loop update the user vector in real-time?
As a viewer watches, skips, likes, or replays videos, the client emits real-time interaction events into a streaming pipeline (such as Kafka). Stream processors aggregate these signals—weighting watch duration, completion rate, and explicit interactions—and update the user's embedding in the User Tower feature store within seconds, allowing the candidate retrieval and ranking stages to adapt the very next feed batch to the viewer's current session interests.
Why are videos sliced into short 3-second chunks rather than streamed as whole files?
Slicing videos into small chunks allows the player to start playback instantly with minimal buffering. It also enables dynamic adaptive streaming (bitrate switching), where the player can download a lower-resolution chunk if network bandwidth drops, preventing playback stalls.
What is the difference between candidate generation and ranking in recommendation systems?
Candidate generation (retrieval) uses lightweight models (like the Two-Tower model) to narrow down a catalog of millions of videos to a few hundred candidates. The ranking phase then runs a more complex, feature-rich neural network (incorporating user demographics, historical interactions, and freshness) on those few hundred candidates to generate the final sorted feed.
How does the CDN edge cache mitigate origin load?
A larger CDN cache capacity stores more transcoded segments, increasing cache hit rates (~15ms response) and protecting the Object Storage origin from direct read spikes (~220ms response).
What happens if the recommendation feedback updates are disabled or slow?
If user vector updates are delayed or disabled, the recommendations become stagnant ('recommendation drift'). The system continues serving videos matching outdated interests, leading to lower engagement, rapid swiping, and higher session abandonment rates.
How does the system handle video transcoding failures?
Transcoding is treated as an asynchronous, retryable background job. If transcoding fails, the upload remains hidden from the public catalog. Legitimate clients only see videos once their metadata status is updated to 'ready' after successful transcoding and chunk upload.
How does CDN cache capacity affect origin scalability?
Low cache capacity forces the CDN edge to evict active segments frequently (cache thrashing). When a viewer requests evicted segments, the CDN must pull them from the Object Storage origin, increasing request latency and compounding the load on origin servers.