From 04721a78eac9a10efa4fdd30f2006b4b1e459766 Mon Sep 17 00:00:00 2001 From: Claude Date: Thu, 13 Aug 2026 12:08:33 +0100 Subject: [PATCH] Measure parallel fetching: it does not help The ceiling is the internet connection at ~8 MB/s. Parallel chunks buy 10-30%, concurrency across different videos buys nothing, and --concurrent-fragments is inapplicable because these formats carry no fragments at all. yt-dlp is already doing the thing that matters: the formats advertise http_chunk_size=10485760 and that chunking is worth 13x, because a single long range request gets throttled to 0.60 MB/s. Also tested option 3 while here: NFO is ignored for episodes -- RunTimeTicks stays null and MediaStreams empty -- so Jellyfin cannot be talked out of transcoding that way either. The useful number: download is 8 MB/s against a 0.39 MB/s playback bitrate, 20x headroom. Streaming while downloading was never bandwidth bound; it failed purely on Jellyfin's probe decision. Co-Authored-By: Claude Opus 5 --- plan.md | 64 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 64 insertions(+) diff --git a/plan.md b/plan.md index 55643a5..4f10e2c 100644 --- a/plan.md +++ b/plan.md @@ -1346,3 +1346,67 @@ fix for it, and the right one is not in yet. The options, none free: Option 2 is the one to measure first: it is the only one that costs nothing and helps every case. + +## 23. Option 2 tested: parallel fetching does not help — 2026-08-13 + +Measured against the real format-298 URLs, 10 MB chunks throughout except where +stated: + +| approach | throughput | +|---|---| +| one 120 MB range request | **0.60 MB/s** | +| 10 MB chunks, sequential (what yt-dlp already does) | 5.9–7.8 MB/s | +| 10 MB chunks, 4 parallel | 8.59 MB/s | +| 10 MB chunks, 8 parallel | 8.51 MB/s | +| 3 *different* videos, chunked, concurrently | 7.65 MB/s aggregate | + +**No.** Parallelism buys 10–30% at most, and concurrency across different videos +buys nothing — the ceiling is the internet connection at roughly 8 MB/s (~65 Mbps). + +Two things worth keeping: + +* `--concurrent-fragments` is **inapplicable**: these formats carry no fragments + (`fragments=None`, no `manifest_url`), so the flag has nothing to parallelise. +* yt-dlp is **already** doing the thing that matters. The formats advertise + `downloader_options.http_chunk_size = 10485760`, and that chunking is worth + **13x**: a single long range request collapses to 0.60 MB/s because YouTube + throttles it. Do not "simplify" it away. + +So fetch time is bandwidth-bound and cannot be reduced: ~70 s for a 525 MB upload, +**~5 minutes for a 2-hour one**. That is the floor. + +### The number that actually matters + +Download runs at **8 MB/s**; a 720p episode plays at **0.39 MB/s** (3.29 Mbps). +That is **20x of headroom**. Streaming while downloading was never a bandwidth +problem — §22's failure was entirely Jellyfin's probe-and-transcode decision. + +### Option 3 tested too, and it also fails + +`` was added to a real episode's NFO (h264, 1280x720, +734 s, aac) and the item refreshed with `replaceAllMetadata=false`: + +``` +RunTimeTicks: None MediaStreams: [] proxy requests: 0 +``` + +Jellyfin **ignores** it for episodes — it does not populate `MediaStreams` or +`RunTimeTicks`, so it cannot be talked out of transcoding this way. This confirms +`test_nfo_has_no_streamdetails` from a second direction: not only does pre-seeding +not change probing, it does not reach `PlaybackInfo` either. + +Progressive (already-muxed, seekable, known-size) formats would sidestep the whole +problem, but the only one YouTube still offers is **format 18 at 360p**. Trading +720p for it is not worth it. + +### What is left + +**Option 1, pre-warming, is the only remaining lever** — and the measurements +size it. Warming the newest episode of all 119 channels would be ~60 GB and two +hours of solid downloading; that is not it. Warming the **N most recently +published episodes across all channels** is tractable: 5 videos ≈ 2.5 GB, ≈ 5 +minutes of background fetch per hour against an 8 GB cache, and it covers the +dominant case for a subscription feed — watching something that has just landed. + +It does contradict the premise that nothing is fetched until someone presses play, +so it is a decision, not a fix to apply quietly.