System Design design youtube
▶️ Part III · Classic Case Studies · chapter 17 / 24

Upload once,
stream everywhere

A user drops in one big video file and expects it to play smoothly on a phone in a tunnel and a TV on fibre, anywhere on Earth. This chapter follows that one file through upload, a transcoding pipeline, adaptive streaming, and a global CDN.

A video platform looks like a video problem. It's really a pipeline-and-delivery problem: take one giant file someone uploaded, grind it into dozens of smaller files, and get the right one to the right screen with the least waiting.

Think about what the system actually has to juggle. The original upload might be a 4K, multi-gigabyte file. The viewer might be on a 3G phone, a laptop on hotel wifi, or a smart TV on gigabit fibre — and their connection speed can change mid-video. They could be in Tokyo, São Paulo, or Lagos. And there could be billions of these files, watched billions of times, where a tiny fraction of videos get nearly all the views.

So the design splits cleanly into two halves. The ingest & processing half happens once per upload: store the original, then run a pipeline that produces many versions (renditions) at different qualities, each chopped into bite-sized segments. The delivery half happens billions of times: a player picks the quality that matches the viewer's current bandwidth and streams segments from the nearest edge cache. We'll walk both, then squeeze cost out with deduplication and make the whole thing survive failure.

1The upload flow — get the bytes in safely

The first job is dull but critical: receive a possibly-enormous file without melting your servers and without making the user re-upload from scratch if their wifi blips at 98%. The trick is to keep the heavy bytes away from your application servers entirely.

Here's the flow when someone clicks "upload":

  • Client asks for permission, not a pipe. The browser tells your API "I want to upload cat.mov, 2.1 GB." Your API doesn't accept the file itself — it hands back a presigned URL: a temporary, signed link that lets the client upload directly to blob storage (S3, GCS) for a short window. Your app servers never touch the raw bytes.
  • The original lands in blob storage. Blob ("binary large object") storage is built for exactly this: cheap, durable, effectively unlimited storage for big opaque files. This copy is the master — the source of truth we'll transcode from later.
  • Resumable uploads survive flaky networks. The file is sent in chunks; if the connection drops, the client resumes from the last acknowledged chunk instead of starting over. Essential when uploads take minutes or hours.
  • Metadata goes to a database. Title, description, owner, visibility, and a status of uploaded → processing go into a metadata DB — small structured rows, totally separate from the giant file.
  • Enqueue the work. Once the original is safely stored, drop a message onto a queue: "transcode video X." Your API returns instantly; the user sees "your video is processing." The slow work happens in the background.

Notice the separation: structured metadata in a database, giant opaque bytes in blob storage, and the slow processing handed off via a queue. That split repeats everywhere in this design.

λ
The FP framing: uploading and processing are decoupled by a queue — exactly the producer/consumer pattern from Chapter 9. The upload endpoint is a fast pure-ish function that returns a receipt; the heavy, effectful transcoding runs later, at its own pace, with retries.

2The transcoding pipeline as a DAG

"Transcoding" means rewriting the video into other formats and qualities. But it's not one step — it's many small steps with dependencies. Some must happen before others; many can happen at the same time. The clean way to model "steps with dependencies" is a directed acyclic graph (DAG): boxes (tasks) with arrows (this must finish before that can start), and no loops.

A typical pipeline for one video:

  • Inspect — probe the original: resolution, frame rate, codecs, duration. Everything downstream depends on this, so it goes first.
  • Encode video renditions — produce 240p, 480p, 720p, 1080p versions. These are independent of each other, so they run in parallel.
  • Encode audio — separate audio tracks, also in parallel.
  • Thumbnails — grab frames for the preview image and the hover-scrub strip.
  • Watermark (optional) — overlay a logo, if required.
  • Assemble & package — stitch the encoded pieces into the streaming format (segments + manifest) and write them to encoded storage. This depends on all the encode steps finishing.

To run this, you need a few moving parts. A preprocessor turns the upload into a concrete DAG (which renditions, which presets). A DAG scheduler walks the graph and, whenever a task's dependencies are all done, marks it ready. A resource manager assigns ready tasks to free machines. Task workers do the actual encoding. The results land in encoded storage. The scheduler's whole job is one tiny rule, which we'll write in code at the end: a task is ready when all of its dependencies are done.

Interactive · transcoding DAG runner Press play — watch independent tasks light up in parallel
3
idle
tasks done
0 / 8
running now
0
total time
0 s
🔀
The DAG is the heart of the system. Because the encode steps don't depend on each other, more workers means they finish sooner — but assemble can't start until the slowest encode is done. That's the parallelism story; we'll push it further in Section 6.

3Adaptive bitrate streaming — let the player choose

Why make all those renditions? Because the viewer's connection isn't fixed. If you only shipped a single 1080p file, a phone on a weak signal would stall constantly, buffering. If you only shipped 240p, the fibre user would get a blurry mess. The answer is to let the player decide, second by second, which quality it can sustain. That's adaptive bitrate streaming (ABR).

It works because of how the video is packaged:

  • Each rendition is chopped into short segments — typically 2–6 seconds each. So 1080p is a sequence of little chunks, and so is 480p, and so is 240p, all time-aligned.
  • A manifest lists everything. A small text file (a .m3u8 for HLS, or a .mpd for DASH) tells the player which renditions exist, their bitrates, and the URLs of every segment.
  • The player measures and switches. It downloads a segment, times how fast it arrived, estimates available bandwidth, and picks the rendition for the next segment accordingly. Bandwidth drops? It quietly steps down to 480p. Recovers? It steps back up. Because switches happen at segment boundaries, the change is seamless.

HLS (Apple) and DASH (the open standard) are the two dominant formats; both express the same idea — segmented renditions plus a manifest. The buffer is the safety margin: the player keeps a few seconds queued ahead, and as long as it refills the buffer faster than playback drains it, you never see the spinner.

Interactive · adaptive bitrate simulator Watch the player chase the fluctuating bandwidth line
240p · 480p · 1080p
current rendition
buffer health
rebuffering
0%
λ
The FP framing: the player is a feedback loop — observe bandwidth, choose a rendition, observe again. The control is on the client, not the server. The server's job was finished at packaging time: it just exposes a menu (the manifest) and lets every viewer self-select.

4CDN & delivery — bytes near the eyeballs

Video is mostly bytes leaving your system, not requests hitting your logic. At scale, the bandwidth bill dwarfs everything else, and serving a viewer in Lagos from a data center in Virginia is slow and expensive. So you push the segments out to a content delivery network: a global mesh of edge caches near the viewers (the same idea as Chapter 1's CDN, now carrying the heaviest payload in the whole book).

  • Geo-routing. A viewer's request is steered to the nearest edge node, so the segments travel across town instead of across an ocean.
  • Edge cache hit vs. origin fetch. If the edge already has the segment, it's served instantly — a cache hit. If not, the edge does an origin pull from your encoded storage, caches it, then serves it. The first viewer in a region pays the slow price; everyone after gets the fast copy.
  • Only popular content belongs on the edge. Caching every segment of every video everywhere would cost a fortune. Video views follow a brutal power law: a small set of hot videos gets the vast majority of watches. So you push popular content to the CDN and serve the long tail (rarely-watched videos) straight from origin. The CDN earns its keep on the videos that are watched a million times; the one watched twice a year isn't worth pre-positioning.

The cost lever is the hit ratio. A high cache-hit ratio means most bytes leave from cheap edge bandwidth and your origin stays quiet; a low ratio means expensive origin pulls. Popularity drives the ratio, which is exactly what the next widget lets you feel.

Interactive · CDN cache-hit map Play videos from regions — green = edge hit, red = origin fetch
70%
edge nodes warming…
cache-hit ratio
origin load
avg latency

5Blob storage & the data model

Step back and look at where everything lives. There are three very different kinds of data, and each gets a home tuned for its shape.

  • Original (master) blob storage. The raw uploaded files. Large, write-once, read-rarely (only the transcoder reads them). You keep them so you can re-transcode if you add a new rendition or codec later, but they're cold — perfect for cheap archival tiers (think S3 Glacier).
  • Transcoded blob storage. The segments and manifests viewers actually stream. Far more numerous (one original → dozens of renditions × hundreds of segments), read constantly, and the source the CDN pulls from. The hot popular ones live close and on fast tiers; cold long-tail ones can move to cheaper storage.
  • Metadata DB. Small structured rows: video id, title, owner, status, duration, view count, list of available renditions, segment URLs. This is read on every page load, so it's sharded (Chapter 1) — typically by video_id — so no single database holds every video.

On top sits the CDN cache layer for the hottest segments. The whole stack is a tiering story: hot segments at the edge, warm in fast transcoded storage, cold originals archived. The art is moving data down the tiers as it cools and up when something suddenly goes viral, so you pay for speed only where speed is being watched.

🗄️
Never store big bytes in your metadata database. The DB holds pointers (URLs, keys) to blobs; the blobs live in object storage; the hottest blobs are mirrored at the CDN edge. Each layer is sized and priced for the access pattern it actually serves.

6Parallelism — how transcoding gets fast

A two-hour 4K film could take many hours to encode on one machine. Viewers won't wait, and creators won't either. The way out is parallelism — and there are two levels of it.

Across renditions (Section 2): the 240p, 480p, 720p, and 1080p encodes are independent, so they run on different workers at once. But that only gives you a handful of parallel tasks. The bigger win is parallelism within a single rendition:

  • Split the video into GOP segments. A GOP ("group of pictures") is a self-contained chunk that starts with a full keyframe and can be decoded without needing any other chunk. Because each GOP is independent, you can hand a hundred GOPs to a hundred workers, encode them all simultaneously, then concatenate the results. A film that took hours on one box finishes in minutes across a fleet.
  • Message queues glue the DAG stages. Between stages — inspect → split → encode → assemble — work flows through queues (the callback to Chapter 9). A stage's output messages become the next stage's input. Queues absorb bursts, let each stage scale its worker pool independently, and give you retries for free.
  • Presets drive the DAG. Rather than hand-coding a pipeline per video, you pick a preset — "standard 16:9 video" or "vertical short" or "4K HDR" — and the preprocessor expands it into the concrete DAG of tasks. New device or codec? Add a preset, not new plumbing.

So the speed comes from two splits: split across renditions, and split each rendition across GOPs, fanning the work out over a large worker fleet and reassembling at the end.

λ
The FP framing: GOP-parallel encoding is map-reduce. Split the video into independent chunks, map the encode function over them in parallel (because each chunk is pure — its output depends only on its own bytes), then reduce by concatenating. Independence is what makes parallelism safe.

7Dedup & cost — don't transcode the same thing twice

Transcoding is expensive: it burns CPU/GPU time and produces storage you then pay to keep and serve. The cheapest work is the work you never do — so before kicking off a pipeline, ask: have we seen this exact video before?

  • Fingerprint the upload. Compute a content hash (and/or a perceptual fingerprint) of the original. If a new upload's fingerprint matches one you've already fully transcoded, you can skip the pipeline entirely and just point the new video's metadata at the existing renditions. Same bytes, computed once.
  • This happens constantly at scale. People re-upload the same clips, memes, and trailers. Exact-match dedup on the master blob saves a surprising amount of compute and storage; perceptual fingerprints catch near-duplicates (re-encoded or lightly edited copies).
  • Storage-cost optimization. Beyond dedup: don't keep renditions nobody watches (you can re-generate from the master on demand), tier cold content to cheaper storage, and only pre-position popular videos on the CDN. Every stored-and-served byte has a recurring cost, so the system constantly trims renditions and tiers that aren't earning their keep.

The mindset is the opposite of "encode everything to everything, everywhere." It's "compute the minimum, store the minimum, and serve the minimum from the most expensive places."

8Resilience — surviving a long, flaky pipeline

A transcoding job touches many machines over a long time. Workers crash, encodes fail, storage hiccups. The pipeline has to expect failure and shrug it off.

  • Retry failed tasks. A DAG task that fails (a worker died mid-encode) is simply re-queued and retried, often on a different machine. Because tasks are small, a retry is cheap — you re-do one GOP, not the whole film.
  • Make stages idempotent. Since a task might run more than once (retries, at-least-once queues from Chapter 9), running it twice must be safe — write to a deterministic output key so a re-run overwrites rather than duplicates. Idempotency is what makes blind retries safe.
  • Monitor transcoding latency. Track how long jobs take end-to-end and alert when the pipeline slows down or a stage's queue backs up. Processing latency (upload-to-watchable) is a real user-facing SLA, not just an internal metric.
  • Copyright hooks. Before publishing, run the fingerprint against a reference database (a Content-ID-style check) to flag matches with claimed content. This rides on the same fingerprinting machinery you built for dedup — one fingerprint, two jobs.

Here's the whole platform as a checklist — the path one uploaded file takes from raw bytes to a smooth stream on the other side of the world:

  1. 1Upload to blob storage via a presigned, resumable upload; write metadata; enqueue the job.
  2. 2Build a transcoding DAG from a preset — inspect, encode renditions, thumbnails, assemble.
  3. 3Run the DAG in parallel across workers; split each rendition into GOPs for more parallelism.
  4. 4Package as ABR — segmented renditions plus a manifest (HLS/DASH).
  5. 5Store transcoded output in hot blob storage; archive the cold master.
  6. 6Push popular content to the CDN; serve the long tail from origin.
  7. 7Dedup & trim — fingerprint to skip re-transcodes; tier cold data down.
  8. 8Retry, stay idempotent, monitor, and run copyright checks on the way out.

9Modeling the transcoding DAG

Section 2 said the scheduler's whole job is one rule: a task is ready when all of its dependencies are done. Here it is in code. A Stage is just a name plus the names it depends on; ready checks whether every dependency is in the "done" set. The scheduler loops: find every ready, not-yet-done stage, run them (in parallel — they don't depend on each other), add them to done, repeat until the graph is finished. Flip between the two languages; the shape is identical.

λ
The pipeline is the DAG — the scheduler never hardcodes an order. It only knows the rule "run any stage whose deps are done," and the dependency arrows do the rest. Add a new stage by declaring its deps; the scheduler figures out when to run it. That's data (the graph) driving control (the schedule).