ytstream/plan.md

ytstream — implementation plan

Target machine: susan Status: streaming PoC verified end to end against real videos and real Jellyfin — every measurement behind this plan is written up in FINDINGS.md alongside this file. Nothing is installed as a service yet. This document is the plan for turning it into one.

Repo: /opt/ytstream, pushed to /disks/git-repos/ytstream.git, branch main. The PoC code still lives in /home/susan/ytstream and is not under version control; Phase 2 moves it in and retires that directory.

Relationship to youtube-automate: ytstream replaces it. The two are entirely separate trees, databases, services and Jellyfin libraries, and they will run side by side only for as long as it takes to satisfy §13. youtube-automate is then decommissioned (§12). Nothing in /opt/youtube-automate is modified by this work.

---

1. What we're building

A DVR-shaped YouTube library for Jellyfin that stores no video bytes.

youtube-automate downloads each video to disk and points Jellyfin at the file. ytstream writes a ~50-byte .strm file containing a URL, and materialises the actual video only when somebody presses play — fetched on demand by a local proxy, held on tmpfs, dropped when the cache fills.

Explicitly in scope

• Mirroring one YouTube account's subscriptions as Jellyfin TV series, one episode per video — that account is the source of truth for what exists (§4)
• Automatic subscription pickup from a YouTube account's public subscription list (§4) — the new requirement
• A bounded back catalogue per channel — 3 months or 300 videos, whichever comes first — that then grows forward indefinitely (§3, §5)
• Just-in-time streaming via the proxy, with per-video caching
• NFO metadata, episode thumbnails, channel poster/fanart
• An admin UI for subscriptions, status and settings
• Running as real systemd services with real logs and real alerting

Explicitly out of scope — do not build these

• Any form of transcoding. susan is a dual Westmere Xeon with no AVX; software transcode is off the table. The proxy produces -c copy fMP4 that Jellyfin direct-plays, and that is the only supported path.
• Downloading and keeping video files. If we want a permanent copy of something, that is a different tool.
• Playlists, Shorts, livestreams, comments, community posts, memberships.
• Anything that runs on victoria (the Linode). susan's residential IP is a load-bearing part of not getting flagged by YouTube.
• A YouTube account login / cookies. PO tokens only, exactly as youtube-automate does it.

---

2. This is a fork, not a green-field rewrite

youtube-automate is 3,543 lines and most of it is correct, verified, and has nothing to do with downloading. Rewriting it from scratch would mean re-deriving the RSS filtering, the episode numbering, the NFO schema and the auth code — all of which were validated on the live machine on 2026-08-11 and are documented in /opt/youtube-automate/specs.md.

So: copy the tree, then add, delete and replace.

youtube_automate module	Fate in ytstream	Note
discovery.py	Lift, then extend	UULF-feed polling is the cheapest correct incremental source and stays. Gains an API-backed full-catalogue backfill (§3).
naming.py	Lift unchanged	Season = upload year, episode = MMDD*10 + ordinal. Keep byte-identical so both trees sort the same during the overlap.
nfo.py	Lift, small change	Drop <fileinfo><streamdetails> — measured to accomplish nothing (FINDINGS §6). Keep <durationinseconds>.
channels.py	Lift, minus artwork fetch	Artwork moves to the API/i.ytimg.com path already built in add_thumbnails.py.
db.py, settings.py, util.py, config.py	Lift	New DB file and new schema version (§7).
web/ (auth, server, templates)	Lift	838 lines of working scrypt auth + admin UI. New routes for subscription sources.
jellyfin.py	Lift, harden	Library refresh must never be replaceAllMetadata (§5).
doctor.py	Lift, extend	Add checks for API key validity, proxy health, .strm orphan count.
ytdlp.py	Lift, narrow	Only the proxy calls yt-dlp now.
download.py (330 lines)	Delete	Replaced by strm.py, which writes a text file.
reap.py (150 lines)	Delete, replace	Reaping existed to reclaim disk. There is no disk to reclaim (§3). Replaced by a much smaller "retire a video whose channel is gone" path.
runner.py	Rewrite	The run loop changes shape: sync subscriptions → poll → materialise → refresh.
—	New: strm.py	Writes .strm + .nfo + thumbnail for one video.
—	New: subsync.py	The brother-subscription puller (§4).
—	New: api.py	YouTube Data API v3 client (§3).
—	New: proxy/	/home/susan/ytstream/ytstream.py, moved in and split up.

Everything lifted keeps its tests. youtube-automate has 12 test modules; they come across too.

---

3. The pivot: the YouTube Data API becomes the metadata source

The subscription feature (§4) forces us to get a Google API key. Once we have one, it is worth noticing what else it buys, because it resolves the two problems that would otherwise make the full back catalogue impractical.

Verified quota costs (developers.google.com/youtube/v3/determine_quota_cost): a new project gets 10,000 units/day shared across everything except search.list and videos.insert, which have their own 100-call/day buckets. subscriptions.list, playlistItems.list, videos.list and channels.list are 1 unit each, and each returns up to 50 items.

That changes the arithmetic completely:

Job	Method	Calls	Units
Sync one account's subscriptions, hourly	subscriptions.list	24/day (1 page)	24
Backfill one channel to the §5 bound (≤300 videos)	playlistItems.list @50	≤6	≤6
Durations for those videos	videos.list @50 ids	≤6	≤6
Steady-state incremental discovery	RSS feed	—	0

A 20-channel initial build costs at most 240 units — under 3% of one day's quota, once. Steady state costs under 100 units/day. We will never come close to the limit, provided we never touch search.list (100 calls/day, and we have no use for it).

Why this matters more than the quota

It takes yt-dlp out of the cataloguing path entirely. Today, youtube-automate needs a yt-dlp extraction per video to learn an upload date during backfill, at roughly 2.3s each. For a 20-channel build at the §5 bound that is 6,000 videos ≈ 4 hours of continuous requests to YouTube from a residential IP — the "scan storm" failure mode the original handover warned about, and it recurs every time a channel is added. The API is a documented, keyed, quota-metered endpoint that does not care about PO tokens, SABR, or client selection, and cannot be rate-limited by YouTube's anti-bot heuristics.

So the split becomes:

• Cataloguing (what exists, when, how long, what it's called) → YouTube Data API + RSS. No yt-dlp, no PO token, no IP-flagging risk.
• Playback (actual media bytes) → yt-dlp inside the proxy, one video at a time, only when a human pressed play.

That is a much better boundary than the current one, and it is the main reason to build ytstream as a new service rather than patch youtube-automate.

It also fixes exact upload dates

Measured during the PoC: --flat-playlist reports timestamp: None, and youtubetab:approximate_date is wrong by up to 2 days. Since season/episode is derived from the upload date, an approximate date means episodes numbered into the wrong day — and Jellyfin caches episode numbers, so fixing it later is a metadata-wipe operation. playlistItems.list returns contentDetails.videoPublishedAt as an exact RFC-3339 timestamp. Use it.

One thing to verify before relying on it

youtube-automate polls the UULF playlist (UU with LF spliced in), which is undocumented but excludes Shorts and livestreams at the cheapest possible point — verified in specs.md §4. Whether playlistItems.list accepts a UULF id is unverified; only UU is documented.

• If UULF works: use it, and the existing filtering carries over unchanged.
• If it 404s: fall back to UU (definitely works, includes Shorts and livestreams) and filter with the videos.list call we are making anyway — contentDetails.duration < min_duration_seconds drops Shorts, and the presence of liveStreamingDetails drops streams.

Either way it is one extra unit per 50 videos. Verify with a single curl on day one:

curl -s "https://www.googleapis.com/youtube/v3/playlistItems?part=contentDetails\
&playlistId=UULF2EvK7nHUOEw1IvWFpTourQ&maxResults=5&key=$KEY" | head -40

---

4. Subscription mirroring — the new requirement

Goal: the brother subscribes to a channel on YouTube, and it appears in Jellyfin without anyone touching an admin page.

The account to mirror — resolved

Handle	@cflux1030
Channel id	UCPcTWaLV8zwx4WP4QExHj4Q
Display name	C Flux

Resolved via yt-dlp on 2026-08-12, so no channels.list call is needed at build time — seed it as the single source row.

This account is the sole source of truth. There is no second source and no parallel manual subscription workflow: what he follows on YouTube is what exists in Jellyfin. Tom's own subscriptions are explicitly not part of this. That is a simplification (§4.3) but it moves all the weight onto the removal safeguards (§4.4), because nothing else protects a channel any more.

It works, with one condition on his side

Confirmed against the API reference: subscriptions.list accepts a channelId filter — "The API will only return that channel's subscriptions" — and unlike mine, mySubscribers and myRecentSubscribers it is not documented as requiring an authorized request. So a plain API key is enough. The condition is that his subscriptions must be public: the implementation guide states the API returns 403 if the channel "does not publicly expose its subscriptions and the request is not authorized by the channel's owner", and the errors table lists subscriptionForbidden (403) — "The requester is not allowed to access the requested subscriptions."

This is a good failure mode: 403, not an empty list. We can tell "he made his subscriptions private again" apart from "he has no subscriptions", which matters a lot for §4.4.

There is no scraping fallback — verified. https://www.youtube.com/@cflux1030/channels returns 200 but silently serves the Home tab: the rendered ytInitialData lists exactly four tabs (Home, Videos, Playlists, Search) and contains zero channel ids. YouTube retired the public subscriptions tab, so the API is the only route to this list. That makes the privacy checkbox genuinely mandatory rather than merely the convenient path, and it means a 403 has no workaround short of §4.2.

4.1 Setup — what has to happen once

Google side (Tom, ~5 minutes, free, no billing account required):

1. console.cloud.google.com → new project, e.g. ytstream.
2. APIs & Services → Library → YouTube Data API v3 → Enable.
3. Credentials → Create credentials → API key.
4. Restrict the key: Application restrictions → None (it is called from a server, so referrer and Android/iOS restrictions do not apply; an IP restriction is optional and breaks if susan's residential IP rotates). API restrictions → YouTube Data API v3 only.
5. Paste it into the ytstream admin UI. It is stored in the setting table like jellyfin_api_key already is — never in the repo, never in a systemd unit.

No OAuth consent screen. No app verification. No user-facing consent flow.

Brother's side (one checkbox, and it is the only thing he has to do):

• youtube.com → Settings → Privacy → uncheck "Keep all my subscriptions private".

Verify immediately — before writing any of subsync.py — that his account actually returns data:

curl -s "https://www.googleapis.com/youtube/v3/subscriptions?part=snippet\
&channelId=UCPcTWaLV8zwx4WP4QExHj4Q&maxResults=50&key=$KEY" \
  | python3 -c 'import json,sys; d=json.load(sys.stdin); \
      print(d.get("pageInfo")); \
      [print(i["snippet"]["resourceId"]["channelId"], i["snippet"]["title"]) \
       for i in d.get("items",[])]'

A 200 with items[] confirms the design on the real account. A 403 means the privacy checkbox has not taken effect — and per the finding above, there is no way around it.

This call also produces the number that sets subsync_max_new (§4.4): pageInfo.totalResults is how many channels he is subscribed to today, which is the one input the cap needs and which cannot be discovered any other way. Record it here when known:

totalResults = TBD — Phase 0

4.2 Fallback if channelId turns out not to work

Ranked, if and only if the day-one curl fails. Note that all three require him to do something — there is no silent workaround, because the public HTML route is gone.

1. OAuth with mine=true. Robust and privacy-setting-independent, but carries a trap: confirmed in Google's OAuth 2.0 docs, a project whose consent screen publishing status is "Testing" is "issued a refresh token expiring in 7 days" unless the only scopes are basic profile ones. youtube.readonly is not, so the token would die weekly. Avoiding that means moving the app to "In production", which for a sensitive scope means Google verification — a disproportionate amount of process for one brother. Only do this if option 3 is unacceptable.
2. Google Takeout subscription CSV, dropped into the admin UI. Manual, but a 30-second job whenever he adds channels, and zero moving parts.
3. Just tell him to use the admin UI. It already exists and it is one text box.

Design subsync.py so the source of a channel list is pluggable (§4.3 makes this fall out naturally), so swapping between these is a small change rather than a rewrite.

4.3 One source of truth — the sync is authoritative

His subscription list is the subscription list. So the model is simply "make the DB match the API response", and it needs no reconciliation machinery:

• A channel in his list is subscribed. A channel that leaves it is unsubscribed, subject to §4.4.
• Two columns on channel carry what the sync needs: source (provenance) and missing_syncs (the §4.4 counter). No join table.

An earlier draft modelled this as multi-source claims, so that a manually-added channel could not be deleted by someone else's list. With one source that machinery is dead weight, and speculative generality in the part of the system that deletes things is the wrong place to spend it. If a second account is ever mirrored, source is already there to key on and the claims model can come back then.

One escape hatch survives: source = 'manual' marks a channel the sync will never remove. It is for pinning something during debugging, not a workflow, and it is not exposed as "subscribe to a channel" in the UI — the way to add a channel is to subscribe to it on YouTube.

The cost of this choice, stated plainly: if he unsubscribes from a channel, its series stops updating and disappears from the library view. That is the correct behaviour for a mirror, and it is exactly why §4.4 refuses to act on a single bad response.

4.4 Removals must be slow and loud; additions must be capped

Two failure modes here are genuinely destructive, and both are cheap to defend against.

Runaway additions. If he has 400 subscriptions, the first sync queues 400 channels — at the §5 bound that is 120,000 Jellyfin episodes, which will not end well. So:

• A sync that would add more than subsync_max_new channels adds none of them. It records them as pending_approval, alerts, and waits for a click in the admin UI.
• The cap applies per sync run, so ordinary drip-feed additions never trip it.

Separate the first sync from steady state. Otherwise the cap always trips on day one, whatever it is set to, and the guard trains everyone to ignore it. So a source's first sync is an explicit bulk import: the admin UI shows the whole list with per-channel checkboxes and a count of the episodes it implies, and nothing is subscribed until someone confirms. From the second sync onwards the cap is a runaway guard, and subsync_max_new is set from the day-one totalResults (§4.1) — a sensible rule is max(10, ceil(totalResults × 0.2)), so a genuine burst of activity gets through but an order-of-magnitude jump does not. Provisional default 25 until Phase 0 produces the real number.

Runaway removals. A transient 403, a network blip, or him re-ticking the privacy box all look like "he unsubscribed from everything". Deleting 20 channels' worth of Jellyfin metadata on that basis would be unrecoverable in any pleasant way. So:

• A 403, a 5xx, a timeout, or a zero-item 200 is never treated as a removal. It increments the source's failure counter, alerts, and changes nothing. A genuinely empty list is indistinguishable from a broken one in consequence, and we prefer the harmless reading.
• A channel missing from an otherwise-healthy response increments missing_syncs. Only at subsync_missing_threshold (default 3 consecutive syncs, so ~3 hours) is it unsubscribed.
• Unsubscribing does not delete anything on disk. It sets the channel inactive: polling stops, the tree stays, and re-subscribing on YouTube restores it with no refetching. Actual deletion is a separate, explicit, human-initiated admin action. .strm files cost ~50 bytes each; there is no pressure to reclaim anything, so there is no reason to ever delete automatically.
• source = 'manual' channels are exempt from all of the above.

Ordering: the sync runs before the poll in the same run, so a channel added at 14:00 has its catalogue built in the same pass.

4.5 Alerting

Existing convention on susan is runitor + healthchecks.io at hc.jihakuz.xyz, one UUID per job. Follow it. A sync that hits 403, trips the add cap, or accumulates failures must exit non-zero so the check goes red — a silently-broken subscription mirror is the worst possible outcome, because nothing appears to be wrong until someone asks why a channel never showed up.

4.6 The brother gets admin access, so auth becomes multi-user

This is the right call — the approval queue (§4.4) and the "channel went missing" alerts are about his subscriptions, so they should be his to action rather than landing on Tom every time. But it does mean a change: youtube-automate has exactly one credential, admin_password_hash in the setting table. Sharing it would mean neither party can have their access revoked or their password changed independently, and the approval log would not record who approved what.

So replace the single setting with a minimal user table (§7): username, password_hash (scrypt, same parameters), created_at, last_login_at. Two rows. No roles, no permissions system — both accounts can do everything, which is correct for two brothers and one media server, and the alternative is a permission model nobody will maintain.

Everything else in web/auth.py carries over unchanged, and it is adequate for an internet-facing login: scrypt hashing and per-address failed-login throttling that reads X-Forwarded-For (which nginx must set — it already does, see §12 step 4).

Record approved_by on pending_approval resolutions. It costs one column and it answers "why is this channel here" six months later.

---

5. Scale — bounded backfill, unbounded growth

Measured during the PoC: Pitch Side has 1,249 videos. The youtube-automate DB holds 15 of them, because a 9-day retention window and 1.3 TB of free disk is what bounded it. .strm files remove that bound entirely, so the bound is now a policy choice, and the choice is:

Backfill 3 months or 300 videos per channel, whichever comes first.

Both settings exist (backfill_max_days = 90, backfill_max_videos = 300) and whichever is reached first stops the walk. The two are well matched in practice: a channel uploading 2–3 times a day — which is what Pitch Side's episode numbering implies — produces roughly 270 videos in 90 days, so neither bound dominates. A weekly uploader gets ~13 videos and is bounded by time; a daily-podcast firehose gets 300 and is bounded by count. That is the right behaviour in both directions.

This bounds the initial build, not the library. Nothing ages out — .strm files cost ~50 bytes and there is no disk to reclaim, so aging out would destroy metadata for no gain. The consequence is honest and worth stating: the library grows forever from the subscribe date. At 20 channels averaging 2 uploads/day that is ~15,000 new episodes a year, so the 20,000-episode question in point 1 below is deferred by about eighteen months, not answered. The lever, if it ever bites: a hidden video state that removes the .strm and .nfo from disk while keeping the DB row, so the catalogue stays re-materialisable. Do not build it now — build the measurement that tells us when we need it.

Initial build: ~20 channels × ≤300 = ≤6,000 episodes, ≤12,000 files. Comfortable.

1. Jellyfin scan cost is still unmeasured, and is the number that decides everything above. Every episode is a .strm + a .nfo + a thumbnail. Normal scans were measured to make 0 media probes (FINDINGS §6), which is the thing that makes this viable at all — but stat-ing and NFO-parsing 12,000 files on a Westmere with the library DB on spinning disk is its own cost. Measure in Phase 3 by building one channel to the bound, timing a full scan, and recording seconds-per-1,000-episodes here. That single figure sizes both the initial build and the growth runway.
2. replaceAllMetadata remains a catastrophe rather than an annoyance. It is the one operation verified to probe media, and at 6,000 items it means 6,000 cold starts. The cold-start rate limiter (20/hour) contains the damage to YouTube's side, but the library-side result is 6,000 items whose metadata got wiped and not re-derived. Defences, all of them:
   • the proxy 503s past the budget (built, tested — test_limits.py)
   • ytstream never itself issues a refresh with replaceAllMetadata=true
   • the admin UI documents "do not click Replace all metadata" in the place where someone would be tempted to
   • NFOs are the source of truth, so recovery is a re-materialise pass over the tree, not a re-fetch from YouTube. Make sure that pass exists and is one CLI command.
3. Episode numbering holds. MMDD*10 + ordinal clamps at 10 uploads/channel/day, computed against the DB rather than the batch, so it is stable across a 300-video backfill. Season = upload year means a 3-month window usually spans one season and occasionally two (a January subscribe reaches back into the previous year), which is correct and needs no special handling.
4. The .strm URL is baked into every file. Changing the proxy's host or port means rewriting all of them. Cheap (it is a tree walk) but it must be a supported CLI command, not a sed someone invents under pressure. Settle the URL now: http://127.0.0.1:8099/watch/<id>. Jellyfin is the only client that ever reads it, it runs on the same host, and 127.0.0.1 means the proxy is unreachable from the network by construction.
5. Backfill must be resumable. Quota is a non-issue but a crash halfway through 12,000 files needs to resume, not restart. Track progress per channel (channel.backfill_cursor), commit per page of 50.

---

6. Components

Four processes. Two are long-running services, two are cron jobs.

6.1 ytstream-proxy — systemd service

The PoC, productionised. /home/susan/ytstream/ytstream.py, 31 KB, currently running by hand.

Already built and verified: /watch/<id> and /healthz, single extraction reused by both yt-dlp legs via --load-info-json, FIFO→ffmpeg -c copy fMP4 on tmpfs, correct HTTP range handling (25 assertions), LRU cache, concurrency cap, retry-on-403, cold-start rate limiter (18 assertions), counters.

Outstanding before it is a service:

• Install the unit. ytstream.service is written but not installed; susan has no passwordless sudo, so this goes in deploy/deploy.sh for the operator to run once. The unit pins Environment=PATH=/var/lib/ytstream/venv/bin:... (load-bearing — the venv yt-dlp is the only one with the POT plugin), User=susan Group=mediaserver UMask=0002, and Requires=docker.service for the POT provider container.
• Startup sweep of the work dir. Sessions live in memory; a restart orphans whatever is in /dev/shm/ytstream. Wipe it on start.
• Prove LRU eviction. It has never fired in testing. Add a test with a tiny --cache-gb.
• Decide the tmpfs budget. /dev/shm is 24 GB of 47 GB RAM; throughput measured at ~480 MB/hour of video. --cache-gb 8 holds ~16 hours of content, which is ample. Do not raise it without thinking about what else on susan wants RAM.
• Log to journald rather than a scratch file, and keep the access log behind a flag.

Unresolved, and staying that way: the intermittent 403 has no known cause. Ruled out during the hardening pass: rate limiting, --load-info-json round-tripping, concurrency, and — the plausible-sounding one — missing PO tokens. Pinning player_client=mweb to get token-bearing URLs measured 0 ok / 6 fail, while token-less ANDROID_VR URLs work; had that "fix" shipped it would have broken every playback. Retry with a fresh extraction is the mitigation, it works, and it fires roughly once per dozen cold starts. Keep the retried counter visible so a change in that rate is noticeable.

6.2 ytstream-admin — systemd service

Lifted from youtube_automate/web/. scrypt password hashes, now per user in the user table (§4.6) rather than one shared setting, per-address failed-login throttling reading X-Forwarded-For, behind nginx on 127.0.0.1:8086 (8085 is youtube-automate's during the overlap).

New routes beyond what exists: subscription sources (add/remove a mirrored YouTube account, show its last sync, its failure count, its pending_approval queue with an approve/reject action), the first-sync bulk-import screen, per-channel backfill-depth controls, account management for the two users, and the API key field.

Two people use this, so the copy matters more than it did with a single operator: the "do not click Replace all metadata" warning (§5) has to be where someone would be tempted, and the approval screen has to show what a channel implies — an episode count — before it is approved.

6.3 ytstream run — hourly cron

sync sources → poll feeds → materialise → refresh Jellyfin. Under flock on /var/lib/ytstream/run.lock, under runitor with its own healthchecks UUID, via sg mediaserver "..." to match the existing convention.

6.4 ytstream backfill — manual / one-off per channel

The expensive path (§3, §5). Deliberately not on a timer: it runs when a channel is first subscribed, and when someone explicitly asks for a deeper catalogue.

---

7. Data model

/var/lib/ytstream/ytstream.db. Start from youtube-automate's schema — channel, video, setting are the right shape — with these changes:

channel — add backfill_cursor TEXT (resumable backfill), backfill_max_videos INTEGER and backfill_max_days INTEGER (per-channel overrides of the §5 bounds; NULL = use the global setting), active INTEGER NOT NULL DEFAULT 1 (§4.4 soft-delete), uploads_playlist TEXT (whether UULF or UU won, §3). Drop retention_days.

video — drop size_bytes, downloaded_at, deleted_at, attempts, last_error; none of them mean anything when there is no download. Keep duration (now from videos.list, and it feeds <durationinseconds>). state collapses to {listed, materialised, skipped_short, skipped_live, hidden} — no pending/downloading/failed, because materialising a .strm cannot meaningfully fail. Add published_at TEXT (exact RFC-3339 from the API) alongside the existing date-only upload_date that naming uses.

channel, continued — two more columns instead of the join table an earlier draft had (§4.3): source TEXT NOT NULL ('youtube' for synced, 'manual' for the pinning escape hatch) and missing_syncs INTEGER NOT NULL DEFAULT 0 (the §4.4 counter).

source — new, and in practice exactly one row: key, label, channel_id, enabled, last_sync_at, last_sync_ok, consecutive_failures, last_error. A one-row table rather than six setting keys, because these are fields of one thing and they change together.

pending_approval — new, the §4.4 add-cap and first-sync import queue: source, channel_id, title, seen_at, resolved_at, resolution (approved/rejected), approved_by.

user — new (§4.6), replacing the single admin_password_hash setting: username (PK), password_hash (scrypt, same parameters as youtube-automate), created_at, last_login_at. Two rows, no roles.

setting — carry over what still applies (jellyfin_url, jellyfin_api_key, pot_provider_url, max_height, min_duration_seconds, session_secret, last_run_at). Drop retention_days, disk_cap_gb, write_subs, sub_langs, sponsorblock_mark, max_attempts, backfill_days, admin_password_hash — download-era concepts plus the credential that moved to user. Add youtube_api_key, subsync_max_new (25, provisional — §4.4), subsync_missing_threshold (3), proxy_base_url (http://127.0.0.1:8099), backfill_max_videos (300), backfill_max_days (90).

No migration from subs.db. It holds 2 channels and 18 videos. Re-subscribe by hand and let the backfill do the rest; a migration script would be more code than the data is worth.

---

8. On-disk layout and the .strm contract

Media root: /disks/Plex/_ytstream/ — already created and already wired to the YouTube (stream) Jellyfin library, with 2 channels in it from the PoC. Keep it. (The name has a leading underscore matching _ingest/_cache on that volume; /disks/Plex/YouTube stays with youtube-automate until §12 retires it.)

/disks/Plex/_ytstream/
└── Pitch Side/
    ├── tvshow.nfo
    ├── poster.jpg                                  # channel avatar
    ├── fanart.jpg                                  # channel banner
    └── Season 2026/
        ├── Pitch Side - S2026E8120 - Title [dQw4w9WgXcQ].strm
        ├── Pitch Side - S2026E8120 - Title [dQw4w9WgXcQ].nfo
        └── Pitch Side - S2026E8120 - Title [dQw4w9WgXcQ]-thumb.jpg

Naming, sanitisation and dir_name stability rules are youtube-automate's, unchanged (specs.md §5) — identical rules in both trees is what makes the overlap period sane.

.strm contents: one line, no trailing newline required:

http://127.0.0.1:8099/watch/dQw4w9WgXcQ

NFO: episodedetails with title, season, episode, aired, plot, runtime, durationinseconds, uniqueid type="youtube". No <fileinfo><streamdetails> — pre-seeding it was measured to change nothing about whether Jellyfin probes (FINDINGS §6), so it is dead weight.

No .work/ dir under the media root. The proxy's scratch space is /dev/shm/ytstream, outside the library entirely, which is strictly better than the dot-dir-plus-.ignore belt-and-braces that youtube-automate needs.

---

9. Environment and paths

Verified on the machine, 2026-08-12.

Purpose	Path
Source (bare repo)	/disks/git-repos/ytstream.git — exists but is not yet git init --bare
Checkout	/opt/ytstream — exists, susan:automation 0770
Entry point	/usr/local/bin/ytstream (needs root, → deploy.sh)
Virtualenv	/var/lib/ytstream/venv
State DB	/var/lib/ytstream/ytstream.db
Lock file	/var/lib/ytstream/run.lock
Media root	/disks/Plex/_ytstream/
Proxy scratch	/dev/shm/ytstream (tmpfs, 24 GB available)
Admin UI	127.0.0.1:8086, nginx → tube.jihakuz.xyz (§12)
Proxy	127.0.0.1:8099, not exposed

Facts that constrain the design:

• Ownership: the media tree convention is susan:mediaserver, dirs 0770, files 0664. /disks/Plex is 0770 with no setgid; Jellyfin reaches the tree only through its mediaserver supplementary group. Both units run User=susan Group=mediaserver UMask=0002; cron goes through sg mediaserver. /disks/Plex/_ytstream already has the setgid bit — keep it.
• No passwordless sudo. Everything touching /usr/local/bin, /etc/systemd/system or /etc/nginx belongs in deploy/deploy.sh.
• yt-dlp: /usr/local/bin/yt-dlp is a 2023.11.16 binary, far too old to work. Nothing references it; leave it alone. ytstream gets its own venv with yt-dlp[default] (the extra that ships yt-dlp-ejs, mandatory for JS challenge solving), bgutil-ytdlp-pot-provider==1.3.1 matching the container tag, and curl-cffi<0.16. Note that during the overlap two venvs will hold yt-dlp; keep both current, and do not let a bare yt-dlp on PATH be what either service resolves.
• POT provider: container on 127.0.0.1:4416, --restart unless-stopped, Docker enabled at boot. Shared with youtube-automate during the overlap and inherited afterwards. It is the reason the proxy unit declares Requires=docker.service.
• Client selection is settled — do not re-litigate it. CLIENT_ARGS = "youtube:player_client=default". mweb is deliberately absent: its formats 403 on every attempt, and it was the sole source of the DRC and dubbed-language variants behind both original format-picker bugs. web/ios/web_safari/tv are SABR-only and yield nothing usable. Left as default rather than pinned to android_vr so a yt-dlp update can follow YouTube.
• Jellyfin 10.11.4, native systemd, 0.0.0.0:8096, API key already in the DB.

---

10. Playback UX — state it plainly

Measured: wait-for-complete is ready in 20 s for a 21-minute video and ~55 s for a 46-minute one, at ~8 MB/s (~60× realtime). --growing mode gives TTFB of 6.5–7.1 s but ffmpeg writes mvhd.duration=0 and no mehd box regardless of what it knows about the input — verified, including an attempt to patch the boxes afterwards, which ffmpeg ignores — so the player sees a video of unknown, growing length. Seeking and progress bars misbehave.

Ship wait-for-complete. A ~20–55 s spinner that then behaves like a normal file beats instant playback with a broken timeline. Keep --growing as a flag for experimentation. Revisit only if someone complains, and if they do, the fix is a pre-warm on library browse rather than a change to the mux.

---

11. Configuration and secrets

Secrets live in /var/lib/ytstream/ytstream.db, mode 0640 susan:automation, and are entered through the admin UI. Never in the repo, never in a systemd unit, never in a cron line. This matches youtube-automate, where jellyfin_api_key and session_secret already live that way.

Three kinds: youtube_api_key (new) and jellyfin_api_key in setting; session_secret in setting; and two scrypt password hashes in user (§4.6).

The YouTube API key is restricted to the YouTube Data API v3 and has read-only reach over public data. Worst case on leak is quota exhaustion; rotation is a two-minute job in the console.

---

12. Decommissioning youtube-automate

Not until §13 phase 5 passes. Order matters — the point is that every step is reversible until the last one.

1. Stop new work. Comment out the 17 * * * * cron entry. Leave the service running so the admin UI still answers.
2. Watch for a week with both libraries live in Jellyfin. This is the real acceptance test: does anyone reach for the old library?
3. Retire the Jellyfin library. Remove YouTube (/disks/Plex/YouTube) via /Library/VirtualFolders. Rename YouTube (stream) → YouTube. Note from the PoC: deleting and recreating a library at the same path returns the same ItemId and reuses the old items — if a clean slate is ever needed, use a fresh path, as _ytstream already is.
4. Free the hostname. tube.jihakuz.xyz is served by a leftover TubeArchivist server block inside sites-available/jihakuz.xyz, which owns the Let's Encrypt cert and wins because nginx takes the first matching block. Repoint it at 8086 the same way deploy/fix-nginx-tube.sh did for 8085; do not install a competing vhost file.
5. Disable the service. systemctl disable --now youtube-automate.service; remove the unit.
6. Reclaim the bytes. Estimated 5–10 GB under /disks/Plex/YouTube. Delete only after step 2 has actually elapsed.
7. Keep, do not delete: /var/lib/youtube-automate/subs.db (copy it aside — it is the only record of what was subscribed and when), /opt/youtube-automate and its bare repo, and specs.md, which remains the reference for every rule ytstream inherited.

update-ytdlp.sh (Mondays 04:40) must be repointed, not removed — it becomes ytstream's, and it is the thing that keeps playback working as YouTube changes.

---

13. Build order

Each phase ends in something checkable. Do not start the next one until it does.

Phase 0 — verify the assumptions the plan rests on, and get the one number it is missing. ~15 minutes, before any code. git init --bare /disks/git-repos/ytstream.git. Get the API key (§4.1). Then three curls:

1. subscriptions.list on UCPcTWaLV8zwx4WP4QExHj4Q returns 200 with items — and record pageInfo.totalResults, which is what sets subsync_max_new (§4.4). This is a hard blocker: there is no fallback that does not involve him doing something (§4.2).
2. playlistItems.list accepts a UULF playlist id — or does not, and we take the documented UU fallback plus duration filtering (§3).
3. videos.list returns contentDetails.duration for a batch of 50 ids.

→ Done when: all three answers are written back into this file as verified facts or corrections, and subsync_max_new has a real value.

Phase 1 — skeleton and lift. Fork the tree, new package name, new DB path, new schema (§7), lifted modules and their tests passing. No new behaviour. → Done when: pytest is green and ytstream doctor reports a healthy environment.

Phase 2 — the proxy as a service. Move it in, split it up, add the startup sweep and the LRU test, write deploy/deploy.sh, operator runs it. → Done when: systemctl status ytstream-proxy is active after a reboot, /healthz answers, and Jellyfin direct-plays a cold video end to end.

Phase 3 — catalogue. api.py, strm.py, the 3-month/300-video bounded resumable backfill, the hourly run. Build Pitch Side alone and time a Jellyfin scan — the §5 measurement. Worth doing twice, once at the 300-video bound and once unbounded at 1,249, since the second gives the seconds-per-1,000-episodes figure that sizes the growth runway for free. → Done when: the bounded episode count is visible with correct titles, dates, durations and thumbnails; scan time per 1,000 episodes is recorded in §5; and one of them plays.

Phase 4 — subscription sync. subsync.py, the first-sync bulk import, the add cap, the missing-threshold, the user table and two accounts (§4.6), the admin routes, the healthchecks UUID. → Done when: the brother logs in with his own credentials and approves the first import; then he subscribes to a new channel on YouTube and within an hour it is a series in Jellyfin with episodes that play, nobody having touched the admin UI. Plus: a forced 403 and a forced empty response both leave the DB untouched and turn the check red.

Phase 5 — cut over. §12 steps 1–2, run for a week. → Done when: nothing has broken and nobody has used the old library.

Phase 6 — decommission. §12 steps 3–7.

---

14. Decisions — answered 2026-08-12

1. Account to mirror: @cflux1030 → UCPcTWaLV8zwx4WP4QExHj4Q ("C Flux"), resolved and recorded in §4. Sole source of truth — Tom's own subscriptions are not part of this at all, which removed the multi-source claim model from §4.3 and put the weight on §4.4 instead.
2. Catalogue depth: 3 months or 300 videos, whichever comes first. §5 rewritten around it, and both bounds are settings with per-channel overrides. Note the consequence recorded there: this bounds the initial build, not the library, which grows forward indefinitely.
3. subsync_max_new: to be set from pageInfo.totalResults on the day-one subscriptions.list call, since that is the only way to learn how many channels he follows. Provisional 25; rule of thumb max(10, ceil(totalResults × 0.2)) (§4.4). The one open number in this plan — Phase 0 closes it.
4. Brother gets admin access: yes, which turns the single shared password into a two-row user table (§4.6) and adds approved_by to the approval log.

---

15. Gotchas carried forward

Things already paid for once. All of these are verified.

• A normal Jellyfin scan makes 0 media probes, which is the single fact this design depends on. replaceAllMetadata=true does probe. Pre-seeded streamdetails do not prevent it.
• --load-info-json reuses the extraction — confirmed twice, once during the original PoC and again while hunting the 403. Both yt-dlp legs run from one -J.
• ffmpeg will not write a duration into a fragmented MP4, and will not accept one patched in afterwards. Hence wait-for-complete (§10).
• Two picker bugs, both silent, both from mweb: 140-drc was chosen over 140 on an abr tie and 403'd; 140-0 was chosen on a 9-language video and produced German audio. The audio sort key ranks language_preference first, deliberately above the DRC check — a 403 is a loud failure, wrong-language audio is a silent one that would have shipped.
• row["title"] ambiguity renamed every series once. The channel/video join has title on both sides. Pass show fields explicitly, never row["title"]. Caught only by accident.
• Jellyfin's /Items/{id} needs user context — use the /Items?ids= form.
• youtubetab:approximate_date is wrong by up to 2 days. Do not derive episode numbers from it; use playlistItems.list (§3).
• YouTube's public subscriptions tab is gone. /@handle/channels returns 200 and silently serves Home — four tabs, zero channel ids in ytInitialData. Verified 2026-08-12. Nobody should spend an afternoon trying to scrape it; the API is the only route (§4).
• pkill -f 'ytstream.py' kills the shell that runs it, because the command string contains its own pattern. Bracket it: pkill -f 'ytstrea[m].py'.