Build ytstream: catalogue, retention, subscription mirror, proxy

Phases 1-4 of plan.md §13. Forked from youtube-automate as planned rather than
written from scratch: naming, NFO, auth, the admin UI, settings and the DB layer
came across largely unchanged, download.py is gone, and the pieces that only make
sense for a streaming library are new.

  api.py       YouTube Data API v3 client. The whole metadata path.
  strm.py      Materialising: a .strm, an .nfo and a thumbnail. Replaces the
               330-line download.py, because the job is writing a URL to a file.
  subsync.py   The subscription mirror, most of which is refusals.
  reap.py      Retention, rewritten around the 30-day window and min_keep_videos.
  discovery.py RSS polling plus an API-backed, resumable, bounded backfill.
  proxy/       The verified PoC, moved in with a systemd unit.

330 tests, all passing, no network and no yt-dlp in any of them. The suite leans
towards the failure paths, because that is where this design can actually hurt
someone: a 403 that looks like an unsubscribe, a video that ages out and comes
back, a title that never arrives. tests/test_proxy.py replaces the two standalone
scripts under proxy/ and now drives the real make_handler(mgr, ...) rather than the
PoC's make_handler(path, done), so routing and video-id validation are covered too.

Ran it end to end against the live API and it found three real bugs.

The first was mine and the tests caught it: strm.remove pruned empty directories
up to the media root, so a channel directory whose tvshow.nfo happened to be
missing would be deleted along with the season. It only looked safe because
tvshow.nfo normally stops the walk. The prune boundary is now the channel
directory explicitly.

The other two only showed up against real data, and they compounded. The backfill
inserted rows with no title and left the RSS poll to fill them in — but RSS returns
15 entries, which for Pitch Side spans 23 days against a 30-day window, so five of
twenty episodes were named after their video ids. Worse, strm.materialise wrote
that fallback back to the database as the title, which made the row look titled and
permanently disabled the repair path. Both fixed: playlistItems.list now requests
snippet as well as contentDetails, which costs the same single quota unit and
carries the title alongside the exact publish date, and the fallback is used for the
filename without being persisted. A title that does arrive late now also removes the
badly-named files and re-queues, so the episode is rewritten rather than keeping its
video-id name forever. Verified against the live API: all twenty Pitch Side episodes
now carry real titles.

Measured on the real account: 119 subscriptions queued for approval and none added
on the first sync, then a two-channel run backfilled and materialised 26 episodes in
under seven seconds.

Two deliberate departures from plan.md, both recorded there:

min_keep_videos defaults to 5 rather than being left as an open question. Without
it 52 of 117 measured channels are empty Jellyfin series that flicker in and out as
their single video crosses the retention line, and the plan already recommended it.

The Jellyfin refresh is a bare /Library/Refresh with a comment explaining why it
must stay that way. A normal scan makes zero media probes; FullRefresh does probe,
and at 400 episodes that is 400 cold starts.

Not yet done: no systemd units are installed (needs root — deploy/deploy.sh), the
admin UI has no routes for sources or the approval queue yet, and nothing has been
pointed at the real media root.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

deploy/bootstrap.sh

@@ -0,0 +1,46 @@
+#!/bin/bash
+# Everything that does NOT need root. Run as susan, before deploy.sh.
+#
+#     /opt/ytstream/deploy/bootstrap.sh
+set -euo pipefail
+
+STATE=/var/lib/ytstream
+VENV=$STATE/venv
+REPO=/opt/ytstream
+
+say() { printf '\n\033[1m==> %s\033[0m\n' "$1"; }
+
+if [[ ! -d $STATE ]]; then
+    echo "$STATE does not exist yet — run 'sudo $REPO/deploy/deploy.sh' first," >&2
+    echo "or create it with: sudo install -d -o susan -g automation -m 0770 $STATE" >&2
+    exit 1
+fi
+
+say "Creating the virtualenv"
+python3 -m venv "$VENV"
+"$VENV/bin/pip" install --quiet --upgrade pip
+"$VENV/bin/pip" install --quiet -e "$REPO"
+
+say "Verifying yt-dlp and the POT plugin"
+# The plugin is what makes YouTube reachable at all. A venv without it looks fine
+# until the first playback fails with a bare 403.
+"$VENV/bin/yt-dlp" --version
+"$VENV/bin/python3" - <<'PY'
+import importlib.util
+missing = [name for name in ("yt_dlp_plugins",) if importlib.util.find_spec(name) is None]
+print("POT plugin:", "MISSING" if missing else "present")
+raise SystemExit(1 if missing else 0)
+PY
+
+say "Starting the POT provider container if it is not already up"
+if ! curl -sf --max-time 5 http://127.0.0.1:4416/ping >/dev/null 2>&1; then
+    docker run -d --name bgutil-provider --restart unless-stopped \
+        -p 4416:4416 brainicism/bgutil-ytdlp-pot-provider || true
+else
+    echo "    already responding on 127.0.0.1:4416"
+fi
+
+say "Initialising the database"
+"$VENV/bin/ytstream" status || true
+
+say "Done — now run 'sudo $REPO/deploy/deploy.sh'"

deploy/crontab.fragment

@@ -0,0 +1,15 @@
+# ytstream — add these to susan's crontab (`crontab -e`).
+#
+# HC_API_URL is already set at the top of susan's crontab; these follow the
+# existing one-UUID-per-job convention. `sg mediaserver` guarantees new files are
+# group-owned by mediaserver even if the invoking shell's primary group differs.
+#
+# Get fresh UUIDs from hc.jihakuz.xyz before enabling these.
+
+# Sync subscriptions, poll, materialise, reap. Hourly. Runs at :23 to stay clear
+# of youtube-automate's :17 entry during the overlap period.
+23 * * * * runitor -uuid REPLACE-WITH-UUID -- sg mediaserver "/usr/local/bin/ytstream run"
+
+# Keep yt-dlp current — this is the thing that keeps playback working as YouTube
+# changes. Mondays 04:50, after youtube-automate's 04:40 slot.
+50 4 * * 1 runitor -uuid REPLACE-WITH-UUID -- /opt/ytstream/deploy/update-ytdlp.sh

deploy/deploy.sh

@@ -0,0 +1,89 @@
+#!/bin/bash
+# Root-requiring installation steps for ytstream.
+#
+# susan has no passwordless sudo, so everything needing root is collected here for
+# the operator to run in one go:
+#
+#     sudo /opt/ytstream/deploy/deploy.sh
+#
+# Everything that does NOT need root — the venv, the database, the POT provider
+# container, subscriptions, the API key — is handled by `bootstrap.sh` and the
+# application itself. Run bootstrap.sh (as susan) first.
+set -euo pipefail
+
+REPO=/opt/ytstream
+STATE=/var/lib/ytstream
+VENV=$STATE/venv
+HOSTNAME_=tube.jihakuz.xyz
+
+if [[ $EUID -ne 0 ]]; then
+    echo "This script needs root. Run: sudo $0" >&2
+    exit 1
+fi
+
+say() { printf '\n\033[1m==> %s\033[0m\n' "$1"; }
+
+say "Creating $STATE"
+# root-owned directory, group-writable by `automation` so susan's cron job and the
+# admin server can both write the database.
+install -d -o susan -g automation -m 0770 "$STATE"
+
+say "Installing the /usr/local/bin shim"
+cat > /usr/local/bin/ytstream <<EOF
+#!/bin/sh
+# Thin shim onto the venv entry point.
+exec $VENV/bin/ytstream "\$@"
+EOF
+chmod 0755 /usr/local/bin/ytstream
+chown root:automation /usr/local/bin/ytstream
+echo "    /usr/local/bin/ytstream"
+
+say "Preparing the media root"
+# setgid so new directories inherit `mediaserver` — without it Jellyfin loses
+# access to anything created after the fact. See plan.md §9.
+install -d -o susan -g mediaserver -m 2770 /disks/Plex/_ytstream
+
+say "Installing systemd units"
+for unit in ytstream-proxy ytstream-admin; do
+    install -m 0644 "$REPO/deploy/$unit.service" "/etc/systemd/system/$unit.service"
+    echo "    $unit.service"
+done
+systemctl daemon-reload
+systemctl enable --now ytstream-proxy.service ytstream-admin.service
+for unit in ytstream-proxy ytstream-admin; do
+    systemctl --no-pager --lines=5 status "$unit.service" || true
+done
+
+say "nginx vhost for $HOSTNAME_"
+# tube.jihakuz.xyz is served by a leftover TubeArchivist server block inside
+# sites-available/jihakuz.xyz, which owns the Let's Encrypt certificate and wins
+# because nginx uses the FIRST server block matching a name. Installing a second
+# vhost for the same name silently does nothing. youtube-automate repointed that
+# block at 8085; ytstream needs it on 8086.
+if grep -rql "server_name $HOSTNAME_" /etc/nginx/sites-enabled/ 2>/dev/null; then
+    echo "    $HOSTNAME_ is already served by an existing vhost."
+    echo "    Repoint its proxy_pass to http://127.0.0.1:8086 by hand, then:"
+    echo "        nginx -t && systemctl reload nginx"
+    echo "    (Deliberately not edited automatically — that block also serves"
+    echo "     other names and owns the TLS certificate.)"
+else
+    echo "    No existing vhost found. Install one proxying to 127.0.0.1:8086"
+    echo "    and run: certbot --nginx -d $HOSTNAME_"
+fi
+
+say "Done"
+cat <<'EOF'
+Remaining steps, all as susan and none needing root:
+
+    ytstream set-password                 # admin UI login
+    ytstream set-jellyfin-key             # verified against the live server
+    ytstream set-youtube-key              # verified against the live API
+    ytstream add-source @cflux1030        # the mirrored account
+    ytstream sync                         # queues the subscriptions
+    ytstream pending                      # review them
+    ytstream approve --all                # or approve a subset by id
+    ytstream run                          # first real cycle
+    ytstream doctor                       # confirm everything is wired up
+
+Then add the cron entries from deploy/crontab.fragment to susan's crontab.
+EOF

deploy/update-ytdlp.sh

@@ -0,0 +1,23 @@
+#!/bin/bash
+# Keep yt-dlp current inside ytstream's venv, then confirm the install still works.
+#
+# This is the single most important recurring job: YouTube changes its player and
+# streaming protocol frequently, and a stale yt-dlp means every playback fails.
+set -euo pipefail
+
+VENV=/var/lib/ytstream/venv
+
+before=$("$VENV/bin/yt-dlp" --version 2>/dev/null || echo "none")
+"$VENV/bin/pip" install --quiet --upgrade "yt-dlp[default]"
+after=$("$VENV/bin/yt-dlp" --version)
+
+echo "yt-dlp: $before -> $after"
+
+if [[ "$before" != "$after" ]]; then
+    # A new yt-dlp can change format availability, so restart the proxy to drop
+    # any cached extraction state and re-run the checks.
+    systemctl restart ytstream-proxy.service 2>/dev/null \
+        || echo "could not restart the proxy (needs root); do it by hand"
+fi
+
+exec "$VENV/bin/ytstream" doctor

deploy/ytstream-admin.service

@@ -0,0 +1,34 @@
+[Unit]
+Description=ytstream admin server
+Documentation=file:///opt/ytstream/plan.md
+After=network-online.target
+Wants=network-online.target
+
+[Service]
+Type=simple
+# See plan.md §9 — the same ownership rules as the proxy, because subscribing a
+# channel through the UI writes into the media tree.
+User=susan
+Group=mediaserver
+UMask=0002
+
+Environment=PATH=/var/lib/ytstream/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+Environment=PYTHONUNBUFFERED=1
+
+WorkingDirectory=/opt/ytstream
+# Port 8086: youtube-automate holds 8085 for as long as the two run side by side.
+ExecStart=/var/lib/ytstream/venv/bin/ytstream serve --host 127.0.0.1 --port 8086
+
+Restart=always
+RestartSec=5
+
+NoNewPrivileges=true
+PrivateTmp=true
+ProtectSystem=full
+ProtectControlGroups=true
+ProtectKernelTunables=true
+RestrictSUIDSGID=true
+ReadWritePaths=/var/lib/ytstream /disks/Plex/_ytstream
+
+[Install]
+WantedBy=multi-user.target

deploy/ytstream-proxy.service

@@ -0,0 +1,46 @@
+[Unit]
+Description=ytstream just-in-time YouTube streaming proxy
+Documentation=file:///opt/ytstream/plan.md
+After=network-online.target docker.service
+Wants=network-online.target
+# The bgutil POT provider runs in Docker on 127.0.0.1:4416 and yt-dlp cannot
+# fetch anything without it.
+Requires=docker.service
+
+[Service]
+Type=simple
+# Group=mediaserver and UMask=0002 are load-bearing: Jellyfin reaches the media
+# tree only through the mediaserver group. See plan.md §9.
+User=susan
+Group=mediaserver
+UMask=0002
+
+# LOAD-BEARING. The only yt-dlp with the bgutil POT plugin is the one in this
+# venv. /usr/local/bin/yt-dlp is a 2023.11.16 binary that cannot talk to YouTube
+# at all, and if it wins the PATH race every playback fails with no clear reason.
+Environment=PATH=/var/lib/ytstream/venv/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
+Environment=PYTHONUNBUFFERED=1
+
+WorkingDirectory=/opt/ytstream
+ExecStart=/var/lib/ytstream/venv/bin/python3 /opt/ytstream/proxy/ytstream_proxy.py \
+    --host 127.0.0.1 --port 8099 \
+    --work /dev/shm/ytstream \
+    --cache-gb 8 \
+    --max-pipelines 2 \
+    --max-retries 2 \
+    --max-starts 20 --starts-window 3600
+
+Restart=always
+RestartSec=5
+
+NoNewPrivileges=true
+PrivateTmp=true
+ProtectSystem=full
+ProtectControlGroups=true
+ProtectKernelTunables=true
+RestrictSUIDSGID=true
+# /dev/shm is the cache; nothing else needs to be writable.
+ReadWritePaths=/dev/shm
+
+[Install]
+WantedBy=multi-user.target

proxy/ytstream_proxy.py

@@ -0,0 +1,755 @@
+#!/usr/bin/env python3
+"""
+ytstream -- just-in-time YouTube streaming proxy for Jellyfin.
+
+Serves one endpoint per video:
+
+    GET /watch/<video_id>       muxed avc1+mp4a MP4, range-capable
+    GET /healthz                JSON status (sessions, cache use, counters)
+
+A `.strm` file whose contents are `http://127.0.0.1:8099/watch/<video_id>` plays
+that video without any of its bytes ever having been stored on disk beforehand.
+
+ffmpeg reads only local FIFOs; every piece of YouTube protocol handling stays
+inside yt-dlp. See FINDINGS.md for what has and has not been verified.
+
+Safety rails, because a Jellyfin library scan can ask for every episode at once:
+
+  --no-fetch        never start a pipeline. Log the request and return 503.
+                    Use this for a first library scan: it reveals whether
+                    Jellyfin probes .strm targets with zero YouTube traffic.
+  --max-pipelines   cap on CONCURRENT pipelines (default 2). Excess -> 503.
+  --max-starts      cap on TOTAL cold starts per window (default 20/hour). This
+                    is the one that bounds a runaway metadata refresh: concurrency
+                    alone only slows a 1249-episode churn down, it does not stop
+                    it. A person watching podcasts starts a handful of videos an
+                    hour; a refresh storm hits the cap in seconds.
+  --cache-gb        tmpfs budget; least-recently-used complete files are evicted.
+
+Two serving modes, as in the PoC:
+
+  default     wait for the mux to finish, then serve. Correct duration, ranges
+              and seeking. Costs time-to-first-byte (~60x realtime pull, so a
+              46-minute video is ready in about 50s).
+  --growing   serve while writing. Low TTFB, but a probe of a partially written
+              fragmented MP4 reports only the duration written so far. ffmpeg
+              ignores both mvhd.duration and an injected mehd box, so this
+              cannot be fixed in the container.
+"""
+
+import argparse
+import collections
+import json
+import os
+import re
+import shutil
+import subprocess
+import sys
+import threading
+import time
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+
+POT_ARGS = "youtubepot-bgutilhttp:base_url=http://127.0.0.1:4416"
+# player_client: `mweb` is deliberately NOT included. Measured 2026-08-12:
+#   * mweb formats 403 on every attempt, both directly and via --load-info-json,
+#     even though they are the only ones carrying a PO token in the URL.
+#   * mweb is also the ONLY source of the DRC and dubbed-language variants that
+#     caused the two original picker bugs. Dropping it removes both hazards
+#     before the picker ever sees them.
+#   * `default` resolves to android_vr for avc1+mp4a, which works without a PO
+#     token. web / ios / web_safari / tv are SABR-only and yield no usable
+#     formats at all. Left as `default` rather than pinned to `android_vr` so a
+#     yt-dlp update can follow YouTube if android_vr stops working.
+CLIENT_ARGS = "youtube:player_client=default"
+MAX_HEIGHT = 720
+STALL_TIMEOUT = 45.0
+VIDEO_ID_RE = re.compile(r"^[A-Za-z0-9_-]{11}$")
+
+_log_lock = threading.Lock()
+_access_log = None
+
+
+def log(msg):
+    print(f"[ytstream] {msg}", file=sys.stderr, flush=True)
+
+
+def access(msg):
+    """Append to the access log. This is the evidence trail for the scan-probe
+    question, so it is written unbuffered and never dropped."""
+    line = f"{time.strftime('%Y-%m-%dT%H:%M:%S')} {msg}"
+    with _log_lock:
+        print(line, file=sys.stderr, flush=True)
+        if _access_log:
+            with open(_access_log, "a") as f:
+                f.write(line + "\n")
+
+
+# --------------------------------------------------------------------------
+# Extraction and format selection
+# --------------------------------------------------------------------------
+
+def extract_info(url, work):
+    """One extraction. Both downloads reuse it via --load-info-json, which is
+    verified not to re-extract."""
+    info_path = os.path.join(work, "info.json")
+    cmd = [
+        "yt-dlp", "-J", "--no-warnings",
+        "--extractor-args", CLIENT_ARGS,
+        "--extractor-args", POT_ARGS,
+        url,
+    ]
+    t0 = time.monotonic()
+    out = subprocess.run(cmd, capture_output=True, text=True)
+    if out.returncode != 0:
+        raise RuntimeError(f"yt-dlp -J failed: {out.stderr.strip()[:500]}")
+    info = json.loads(out.stdout)
+    with open(info_path, "w") as f:
+        json.dump(info, f)
+    log(f"extracted {info.get('id')} in {time.monotonic() - t0:.1f}s -- "
+        f"{info.get('title')!r}")
+    return info, info_path
+
+
+def pick_formats(info):
+    """Prefer avc1 video and the original-language mp4a audio so the mux is a
+    pure copy and Jellyfin can direct-play."""
+    fmts = info.get("formats", [])
+
+    def usable(f):
+        return f.get("url") and f.get("protocol", "").startswith("http")
+
+    def is_drc(f):
+        # YouTube advertises DRC (dynamic-range-compressed) audio variants that
+        # carry the same abr as their plain counterparts but 403 on download.
+        # Verified against NH2MhBFQm9w: 140-drc -> 403, 140 -> fine.
+        return "-drc" in (f.get("format_id") or "") or "DRC" in (f.get("format_note") or "")
+
+    vids = [
+        f for f in fmts
+        if usable(f)
+        and f.get("vcodec") not in (None, "none")
+        and f.get("acodec") in (None, "none")
+        and (f.get("height") or 0) <= MAX_HEIGHT
+    ]
+    auds = [
+        f for f in fmts
+        if usable(f)
+        and f.get("acodec") not in (None, "none")
+        and f.get("vcodec") in (None, "none")
+    ]
+
+    # Ranking notes, all learned the hard way against real videos:
+    #
+    #  * language_preference outranks everything. Multi-language uploads expose
+    #    140-0..140-N with IDENTICAL abr, so max() would otherwise tie-break on
+    #    list order and silently pick whichever dub YouTube listed first --
+    #    German audio on an English podcast (verified: J9O3sxoMs5U). The
+    #    original track carries language_preference 10, dubs carry -1.
+    #  * non-DRC next, because DRC variants 403. Language correctness ranks
+    #    above this deliberately: a 403 is a loud failure, wrong-language audio
+    #    is a silent one that would ship.
+    #  * only then codec and bitrate.
+    def langpref(f):
+        return f.get("language_preference") or 0
+
+    def vkey(f):
+        return (
+            not is_drc(f),
+            f.get("vcodec", "").startswith("avc1"),
+            f.get("height") or 0,
+            f.get("tbr") or 0,
+        )
+
+    def akey(f):
+        return (
+            langpref(f),
+            not is_drc(f),
+            f.get("acodec", "").startswith("mp4a"),
+            f.get("abr") or 0,
+        )
+
+    if not vids or not auds:
+        raise RuntimeError(
+            "no separate video+audio pair; formats present: " + ", ".join(
+                f"{f.get('format_id')}({f.get('vcodec')}/{f.get('acodec')})"
+                for f in fmts[:15]))
+
+    v = max(vids, key=vkey)
+    a = max(auds, key=akey)
+    log(f"  video {v['format_id']} {v.get('vcodec')} {v.get('height')}p | "
+        f"audio {a['format_id']} {a.get('acodec')} lang={a.get('language')}")
+    if not v.get("vcodec", "").startswith("avc1"):
+        log("  WARNING: no h264 at this height -- Jellyfin may transcode")
+    if len({f.get("language") for f in auds}) > 1 and langpref(a) <= 0:
+        log(f"  WARNING: multi-language upload and the chosen track "
+            f"({a.get('language')}) is not the original -- expect a dub")
+    return v["format_id"], a["format_id"]
+
+
+# --------------------------------------------------------------------------
+# Producer: two yt-dlp -> FIFOs -> ffmpeg -c copy -> fragmented MP4
+# --------------------------------------------------------------------------
+
+def start_producer(info_path, vfmt, afmt, work):
+    vfifo = os.path.join(work, "v.fifo")
+    afifo = os.path.join(work, "a.fifo")
+    for p in (vfifo, afifo):
+        if os.path.exists(p):
+            os.unlink(p)
+        os.mkfifo(p)
+
+    out_path = os.path.join(work, "out.mp4")
+    done_path = out_path + ".done"
+    err_path = out_path + ".err"
+    procs = []
+
+    def feed(fmt, fifo, tag):
+        # Opening a FIFO for write blocks until a reader attaches, so ffmpeg
+        # must already be starting -- it is, just below.
+        fh = open(fifo, "wb")
+        p = subprocess.Popen(
+            [
+                "yt-dlp", "--load-info-json", info_path,
+                "-f", fmt, "-o", "-", "--quiet", "--no-warnings",
+                "--extractor-args", CLIENT_ARGS,
+                "--extractor-args", POT_ARGS,
+            ],
+            stdout=fh, stderr=subprocess.PIPE,
+        )
+        procs.append((tag, p))
+        fh.close()
+
+    ff = subprocess.Popen(
+        [
+            "ffmpeg", "-y", "-loglevel", "error",
+            "-i", vfifo, "-i", afifo,
+            "-c", "copy",
+            "-movflags", "frag_keyframe+empty_moov+default_base_moof",
+            "-f", "mp4", out_path,
+        ],
+        stderr=subprocess.PIPE,
+    )
+
+    threading.Thread(target=feed, args=(vfmt, vfifo, "video"), daemon=True).start()
+    threading.Thread(target=feed, args=(afmt, afifo, "audio"), daemon=True).start()
+
+    def reap():
+        rc = ff.wait()
+        err = ff.stderr.read().decode(errors="replace").strip()
+        problems = []
+        if rc != 0:
+            problems.append(f"ffmpeg exited {rc}: {err[:400]}")
+        for tag, p in procs:
+            if p.poll() not in (0, None):
+                problems.append(
+                    f"yt-dlp[{tag}] exited {p.returncode}: "
+                    f"{p.stderr.read().decode(errors='replace')[:300]}")
+        if problems:
+            # A single 403 on either stream kills the run and there is no retry
+            # yet; record it so /healthz and the caller can see why.
+            with open(err_path, "w") as f:
+                f.write("\n".join(problems))
+            log(f"producer FAILED in {work}: {problems[0]}")
+        else:
+            log(f"mux complete: {out_path} {os.path.getsize(out_path)} bytes")
+        open(done_path, "w").close()
+
+    threading.Thread(target=reap, daemon=True).start()
+    return out_path, done_path, err_path
+
+
+# --------------------------------------------------------------------------
+# Session manager
+# --------------------------------------------------------------------------
+
+class Session:
+    """One video's pipeline and its output file.
+
+    `final` is the terminal signal: set once the session has either produced a
+    complete file or given up after retries. It is deliberately separate from
+    the producer's own done-marker, because a failed attempt writes that marker
+    too and a waiting request must not mistake a retry for a finished file.
+    """
+
+    def __init__(self, video_id, work):
+        self.video_id = video_id
+        self.work = work
+        self.out_path = os.path.join(work, "out.mp4")
+        self.started = time.time()
+        self.last_used = time.time()
+        self.final = threading.Event()
+        self.failed = None
+        self.attempts = 0
+        self.readers = 0
+
+    @property
+    def complete(self):
+        """True only when the file is finished AND good."""
+        return self.final.is_set() and not self.failed
+
+    def size(self):
+        try:
+            return os.path.getsize(self.out_path)
+        except OSError:
+            return 0
+
+
+class Manager:
+    def __init__(self, work_root, max_pipelines, cache_bytes, no_fetch, growing,
+                 max_retries=2, max_starts=20, starts_window=3600.0):
+        self.work_root = work_root
+        self.max_pipelines = max_pipelines
+        self.max_retries = max_retries
+        self.max_starts = max_starts
+        self.starts_window = starts_window
+        # Timestamps of cold starts, pruned to the window. Cache hits are not
+        # recorded: re-watching or resuming must never be rate limited.
+        self.start_log = collections.deque()
+        self.cache_bytes = cache_bytes
+        self.no_fetch = no_fetch
+        self.growing = growing
+        self.lock = threading.Lock()
+        self.sessions = {}
+        self.counters = {"requests": 0, "started": 0, "reused": 0,
+                         "refused_nofetch": 0, "refused_busy": 0,
+                         "refused_ratelimit": 0, "retried": 0, "failed": 0,
+                         "evicted": 0}
+
+    def _active(self):
+        return [s for s in self.sessions.values() if not s.complete]
+
+    def get(self, video_id):
+        """Return (session, error_string). Never raises."""
+        with self.lock:
+            self.counters["requests"] += 1
+            s = self.sessions.get(video_id)
+            if s is not None:
+                s.last_used = time.time()
+                self.counters["reused"] += 1
+                return s, None
+
+            if self.no_fetch:
+                self.counters["refused_nofetch"] += 1
+                return None, "no-fetch mode: refusing to start a pipeline"
+
+            if len(self._active()) >= self.max_pipelines:
+                self.counters["refused_busy"] += 1
+                return None, (f"at pipeline cap ({self.max_pipelines}); "
+                              f"refusing to start another")
+
+            # Cold-start budget. Refusing produces a short retry burst from
+            # libavformat and then aborts whatever refresh triggered it, which
+            # is the intended protective outcome: an aborted refresh costs some
+            # re-derivable metadata, a runaway one costs the whole catalogue in
+            # YouTube traffic.
+            now = time.monotonic()
+            while self.start_log and now - self.start_log[0] > self.starts_window:
+                self.start_log.popleft()
+            if len(self.start_log) >= self.max_starts:
+                self.counters["refused_ratelimit"] += 1
+                oldest = self.starts_window - (now - self.start_log[0])
+                log(f"RATE LIMIT: {len(self.start_log)} cold starts in the last "
+                    f"{self.starts_window / 60:.0f}min, refusing {video_id}. "
+                    f"Budget frees in {oldest:.0f}s. If this was a library "
+                    f"refresh, that refresh is being stopped on purpose.")
+                return None, (f"cold-start budget exhausted "
+                              f"({self.max_starts} per "
+                              f"{self.starts_window / 60:.0f}min)")
+            self.start_log.append(now)
+
+            work = os.path.join(self.work_root, video_id)
+            shutil.rmtree(work, ignore_errors=True)
+            os.makedirs(work, exist_ok=True)
+            s = Session(video_id, work)
+            self.sessions[video_id] = s
+            self.counters["started"] += 1
+
+        threading.Thread(target=self._run, args=(s,), daemon=True).start()
+        return s, None
+
+    def _run(self, s):
+        """Run the pipeline, retrying on failure with a fresh extraction.
+
+        Intermittent 403s do happen -- observed on both the video and the audio
+        stream of videos that succeeded minutes earlier, clustered after heavy
+        use, so most likely transient rate limiting. A single one used to kill
+        playback outright. Each retry re-extracts, because the resolved URLs and
+        their PO token binding are the most likely thing to have gone stale.
+
+        Only retried while nothing has been served yet (size == 0). If bytes
+        already went out we cannot rewind under a reader.
+        """
+        url = f"https://www.youtube.com/watch?v={s.video_id}"
+        last = None
+        for attempt in range(1, self.max_retries + 2):
+            s.attempts = attempt
+            try:
+                shutil.rmtree(s.work, ignore_errors=True)
+                os.makedirs(s.work, exist_ok=True)
+                info, info_path = extract_info(url, s.work)
+                vfmt, afmt = pick_formats(info)
+                _out, done_path, err_path = start_producer(
+                    info_path, vfmt, afmt, s.work)
+
+                while not os.path.exists(done_path):
+                    time.sleep(0.2)
+                if not os.path.exists(err_path):
+                    last = None
+                    break
+                with open(err_path) as f:
+                    last = f.read().strip()
+            except Exception as e:                   # noqa: BLE001 - report anything
+                last = str(e)[:500]
+
+            if s.size() > 0:
+                log(f"{s.video_id}: attempt {attempt} failed after serving "
+                    f"bytes -- not retrying")
+                break
+            if attempt <= self.max_retries:
+                with self.lock:
+                    self.counters["retried"] += 1
+                log(f"{s.video_id}: attempt {attempt} failed "
+                    f"({(last or '')[:90]}) -- retrying")
+                time.sleep(2.0 * attempt)
+
+        s.failed = last
+        if last:
+            with self.lock:
+                self.counters["failed"] += 1
+            log(f"session {s.video_id} gave up after {s.attempts} attempt(s)")
+        s.final.set()
+        self.evict()
+
+    def evict(self):
+        """Drop least-recently-used complete sessions until under budget."""
+        with self.lock:
+            done = [s for s in self.sessions.values()
+                    if s.complete and s.readers == 0]
+            total = sum(s.size() for s in self.sessions.values())
+            for s in sorted(done, key=lambda x: x.last_used):
+                if total <= self.cache_bytes:
+                    break
+                total -= s.size()
+                self.sessions.pop(s.video_id, None)
+                shutil.rmtree(s.work, ignore_errors=True)
+                self.counters["evicted"] += 1
+                log(f"evicted {s.video_id}")
+
+    def status(self):
+        with self.lock:
+            return {
+                "mode": "growing" if self.growing else "wait-for-complete",
+                "no_fetch": self.no_fetch,
+                "max_pipelines": self.max_pipelines,
+                "max_starts": self.max_starts,
+                "starts_in_window": len(self.start_log),
+                "starts_window_min": round(self.starts_window / 60),
+                "cache_gb": round(self.cache_bytes / 2**30, 2),
+                "cache_used_gb": round(
+                    sum(s.size() for s in self.sessions.values()) / 2**30, 3),
+                "counters": dict(self.counters),
+                "sessions": [
+                    {"video_id": s.video_id,
+                     "complete": s.complete,
+                     "bytes": s.size(),
+                     "readers": s.readers,
+                     "age_s": round(time.time() - s.started, 1),
+                     "attempts": s.attempts,
+                     "error": s.failed}
+                    for s in self.sessions.values()
+                ],
+            }
+
+
+# --------------------------------------------------------------------------
+# HTTP
+# --------------------------------------------------------------------------
+
+def make_handler(mgr, wait_timeout):
+
+    class Handler(BaseHTTPRequestHandler):
+        protocol_version = "HTTP/1.1"
+
+        def log_message(self, *a):
+            pass                                     # we do our own logging
+
+        # -- range helpers (verified by test_range.py) ----------------------
+
+        def _size(self, path):
+            return os.path.getsize(path) if os.path.exists(path) else 0
+
+        def _wait_for_bytes(self, path, offset, complete):
+            deadline = time.monotonic() + STALL_TIMEOUT
+            while True:
+                size = self._size(path)
+                if size > offset or complete():
+                    return size
+                if time.monotonic() > deadline:
+                    raise TimeoutError
+                time.sleep(0.05)
+
+        def _parse_range(self, rng, total):
+            """Parse a single byte range against a known total size.
+
+            Returns (start, end) inclusive, "unsatisfiable", or None when there
+            is no usable range and the whole body should be sent. Multi-range
+            requests fall into the None case: answering with the whole body is
+            legal and beats mis-serving one part.
+            """
+            if not rng or not rng.startswith("bytes=") or "," in rng:
+                return None
+            first, sep, last = rng[6:].strip().partition("-")
+            if not sep:
+                return None
+            try:
+                if not first:                        # bytes=-N -> final N bytes
+                    n = int(last)
+                    return (max(0, total - n), total - 1) if n > 0 else "unsatisfiable"
+                start = int(first)
+                end = int(last) if last else total - 1
+            except ValueError:
+                return None
+            if start >= total or start > end:
+                return "unsatisfiable"
+            return start, min(end, total - 1)
+
+        # -- routing --------------------------------------------------------
+
+        def do_HEAD(self):
+            self._route(True)
+
+        def do_GET(self):
+            self._route(False)
+
+        def _route(self, head_only):
+            rng = self.headers.get("Range", "-")
+            ua = (self.headers.get("User-Agent") or "-")[:60]
+            access(f"{self.command} {self.path} range={rng} ua={ua!r}")
+
+            if self.path == "/healthz":
+                body = json.dumps(mgr.status(), indent=2).encode()
+                self.send_response(200)
+                self.send_header("Content-Type", "application/json")
+                self.send_header("Content-Length", str(len(body)))
+                self.end_headers()
+                if not head_only:
+                    self.wfile.write(body)
+                return
+
+            if not self.path.startswith("/watch/"):
+                self._fail(404, "not found")
+                return
+
+            video_id = self.path[len("/watch/"):].split("?")[0]
+            if not VIDEO_ID_RE.match(video_id):
+                self._fail(400, "bad video id")
+                return
+
+            sess, refusal = mgr.get(video_id)
+            if refusal:
+                access(f"  -> 503 {refusal}")
+                self._fail(503, refusal, retry_after=30)
+                return
+
+            if not mgr.growing:
+                # Correct duration and working seeks require a finished file.
+                if not sess.final.is_set():
+                    access(f"  -> waiting for {video_id} to finish muxing")
+                    if not sess.final.wait(wait_timeout):
+                        self._fail(504, "mux did not finish in time")
+                        return
+                if sess.failed:
+                    access(f"  -> 502 producer failed: {sess.failed[:120]}")
+                    self._fail(502, f"producer failed: {sess.failed[:200]}")
+                    return
+            else:
+                # Growing mode still needs the first bytes to exist. Retries
+                # happen underneath while size is still 0, so wait on both.
+                deadline = time.monotonic() + STALL_TIMEOUT
+                while sess.size() == 0 and not sess.final.is_set():
+                    if time.monotonic() > deadline:
+                        self._fail(504, "producer wrote nothing")
+                        return
+                    time.sleep(0.1)
+                if sess.failed and sess.size() == 0:
+                    self._fail(502, f"producer failed: {sess.failed[:200]}")
+                    return
+
+            with mgr.lock:
+                sess.readers += 1
+                sess.last_used = time.time()
+            try:
+                self._serve(sess, head_only)
+            finally:
+                with mgr.lock:
+                    sess.readers -= 1
+                    sess.last_used = time.time()
+
+        def _fail(self, code, msg, retry_after=None):
+            body = (msg + "\n").encode()
+            try:
+                self.send_response(code)
+                if retry_after:
+                    self.send_header("Retry-After", str(retry_after))
+                self.send_header("Content-Type", "text/plain")
+                self.send_header("Content-Length", str(len(body)))
+                self.end_headers()
+                self.wfile.write(body)
+            except (BrokenPipeError, ConnectionResetError):
+                pass
+
+        # -- body -----------------------------------------------------------
+
+        def _serve(self, sess, head_only):
+            path = sess.out_path
+            complete = lambda: sess.complete            # noqa: E731
+
+            # Ranges are only honoured once the file is complete; while it is
+            # still growing there is no reliable time-to-byte mapping into a
+            # fragmented MP4, so we present a non-seekable stream instead.
+            chunked = not complete()
+            start, end, is_range = 0, None, False
+            if not chunked:
+                spec = self._parse_range(self.headers.get("Range"),
+                                         self._size(path))
+                if spec == "unsatisfiable":
+                    total = self._size(path)
+                    self.send_response(416)
+                    self.send_header("Content-Range", f"bytes */{total}")
+                    self.send_header("Content-Length", "0")
+                    self.end_headers()
+                    return
+                if spec:
+                    start, end = spec
+                    is_range = True
+
+            try:
+                self._wait_for_bytes(path, start, complete)
+            except TimeoutError:
+                self._fail(504, "producer stalled")
+                return
+
+            if chunked:
+                self.send_response(200)
+                self.send_header("Transfer-Encoding", "chunked")
+            else:
+                total = self._size(path)
+                if end is None:
+                    end = total - 1
+                if is_range:
+                    self.send_response(206)
+                    self.send_header("Content-Range", f"bytes {start}-{end}/{total}")
+                else:
+                    self.send_response(200)
+                self.send_header("Content-Length", str(end - start + 1))
+                self.send_header("Accept-Ranges", "bytes")
+            self.send_header("Content-Type", "video/mp4")
+            self.end_headers()
+            if head_only:
+                return
+
+            sent = start
+            # None while chunked: the growing case has no known end and stops
+            # when the producer does. A bounded range must send exactly the
+            # bytes it promised in Content-Length, no more.
+            remaining = None if end is None else end - start + 1
+            try:
+                with open(path, "rb") as f:
+                    f.seek(start)
+                    while True:
+                        if remaining is not None and remaining <= 0:
+                            break
+                        buf = f.read(65536 if remaining is None
+                                     else min(65536, remaining))
+                        if buf:
+                            if chunked:
+                                self.wfile.write(b"%X\r\n" % len(buf))
+                                self.wfile.write(buf)
+                                self.wfile.write(b"\r\n")
+                            else:
+                                self.wfile.write(buf)
+                            sent += len(buf)
+                            if remaining is not None:
+                                remaining -= len(buf)
+                            continue
+                        if complete() and sent >= self._size(path):
+                            break
+                        try:
+                            self._wait_for_bytes(path, sent, complete)
+                        except TimeoutError:
+                            break
+                if chunked:
+                    self.wfile.write(b"0\r\n\r\n")
+            except (BrokenPipeError, ConnectionResetError):
+                # Normal: a probe reads the header then hangs up.
+                pass
+
+    return Handler
+
+
+# --------------------------------------------------------------------------
+
+def main():
+    ap = argparse.ArgumentParser(
+        description="just-in-time YouTube streaming proxy for Jellyfin")
+    ap.add_argument("--host", default="127.0.0.1")
+    ap.add_argument("--port", type=int, default=8099)
+    ap.add_argument("--work", default="/dev/shm/ytstream")
+    ap.add_argument("--cache-gb", type=float, default=8.0)
+    ap.add_argument("--max-pipelines", type=int, default=2)
+    ap.add_argument("--wait-timeout", type=float, default=300.0,
+                    help="how long a request may block waiting for the mux")
+    ap.add_argument("--max-starts", type=int, default=20,
+                    help="max cold starts per window; bounds a runaway library "
+                         "refresh (default 20)")
+    ap.add_argument("--starts-window", type=float, default=3600.0,
+                    help="rate-limit window in seconds (default 3600)")
+    ap.add_argument("--max-retries", type=int, default=2,
+                    help="retries after a failed pipeline, each with a fresh "
+                         "extraction (intermittent 403s do happen)")
+    ap.add_argument("--no-fetch", action="store_true",
+                    help="never start a pipeline; log and 503. Use for a first "
+                         "library scan to detect probing with no YouTube traffic")
+    ap.add_argument("--growing", action="store_true",
+                    help="serve while still writing (low TTFB, wrong duration)")
+    ap.add_argument("--access-log", default=None)
+    args = ap.parse_args()
+
+    if not shutil.which("yt-dlp") or not shutil.which("ffmpeg"):
+        sys.exit("need yt-dlp and ffmpeg on PATH")
+
+    ver = subprocess.run(["yt-dlp", "--version"], capture_output=True, text=True)
+    version = ver.stdout.strip()
+    if version < "2025":
+        log(f"WARNING: yt-dlp {version} looks far too old, and a POT plugin is "
+            f"required. Expected the automation venv "
+            f"(/var/lib/youtube-automate/venv/bin) on PATH.")
+    else:
+        log(f"yt-dlp {version}")
+
+    global _access_log
+    _access_log = args.access_log
+
+    os.makedirs(args.work, exist_ok=True)
+    mgr = Manager(args.work, args.max_pipelines,
+                  int(args.cache_gb * 2**30), args.no_fetch, args.growing,
+                  args.max_retries, args.max_starts, args.starts_window)
+
+    if args.no_fetch:
+        log("NO-FETCH MODE: every /watch request will be logged and refused. "
+            "No YouTube traffic will be generated.")
+    if args.growing:
+        log("growing mode: probes will see a partial duration")
+
+    srv = ThreadingHTTPServer((args.host, args.port),
+                              make_handler(mgr, args.wait_timeout))
+    log(f"listening on http://{args.host}:{args.port}  "
+        f"(/watch/<video_id>, /healthz)")
+    try:
+        srv.serve_forever()
+    except KeyboardInterrupt:
+        log("shutting down")
+
+
+if __name__ == "__main__":
+    main()

pyproject.toml

@@ -0,0 +1,33 @@
+[project]
+name = "ytstream"
+version = "1.0.0"
+description = "A just-in-time YouTube library for Jellyfin. Stores no video bytes."
+requires-python = ">=3.11"
+
+# Pins that matter, inherited from youtube-automate's specs.md §3:
+#   - the [default] extra ships yt-dlp-ejs, the JS challenge solver, which is
+#     mandatory. Only the proxy uses yt-dlp; cataloguing runs on the Data API.
+#   - curl-cffi must stay below 0.16 or yt-dlp rejects it as unsupported
+#   - the bgutil plugin must match the POT container tag
+dependencies = [
+    "yt-dlp[default]",
+    "bgutil-ytdlp-pot-provider==1.3.1",
+    "curl-cffi<0.16",
+]
+
+[project.optional-dependencies]
+dev = ["pytest>=8"]
+
+[project.scripts]
+ytstream = "ytstream.cli:main"
+
+[build-system]
+requires = ["setuptools>=68"]
+build-backend = "setuptools.build_meta"
+
+[tool.setuptools]
+packages = ["ytstream", "ytstream.web"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+addopts = "-q"

tests/conftest.py

@@ -0,0 +1,202 @@
+"""Test fixtures.
+
+Every path the application uses is redirected into a tmpdir. No test touches the
+network, a real yt-dlp, a real Jellyfin, or the real media tree — the YouTube API
+client is always a stub (see `FakeApi`), because the point of a test suite here is
+to pin down behaviour that only shows up on the failure paths: a 403, an empty
+response, a video that ages out and must not come back.
+"""
+
+from __future__ import annotations
+
+import os
+import tempfile
+from pathlib import Path
+
+# config resolves its paths at import time, so the environment has to be set
+# before anything from the package is imported.
+_SANDBOX = Path(tempfile.mkdtemp(prefix="yts-tests-"))
+os.environ.setdefault("YTS_STATE_DIR", str(_SANDBOX / "state"))
+os.environ.setdefault("YTS_MEDIA_ROOT", str(_SANDBOX / "media"))
+os.environ.setdefault("YTS_DB_PATH", str(_SANDBOX / "state" / "ytstream.db"))
+os.environ.setdefault("YTS_LOCK_PATH", str(_SANDBOX / "state" / "run.lock"))
+os.environ.setdefault("YTS_VENV_BIN", str(_SANDBOX / "venv" / "bin"))
+
+import pytest  # noqa: E402
+
+from ytstream import api, config, db, util, videos  # noqa: E402
+from ytstream.settings import Settings  # noqa: E402
+
+FIXTURES = Path(__file__).parent / "fixtures"
+
+# A real-looking channel id: UC + 22 chars.
+CHANNEL_ID = "UCW7jUEpYT_t0Gsf632d6_wQ"
+
+
+@pytest.fixture()
+def media_root(tmp_path, monkeypatch):
+    """Point the media root at a per-test tmpdir."""
+    root = tmp_path / "media"
+    root.mkdir(parents=True)
+    monkeypatch.setattr(config, "MEDIA_ROOT", root)
+    return root
+
+
+@pytest.fixture()
+def conn(tmp_path):
+    connection = db.connect(tmp_path / "ytstream.db")
+    yield connection
+    connection.close()
+
+
+@pytest.fixture()
+def settings(conn):
+    settings = Settings(conn)
+    # Tests that reach the API go through FakeApi, but the code refuses to call
+    # out at all without a key, so give it one that is never used for real.
+    settings.set("youtube_api_key", "test-key")
+    return settings
+
+
+@pytest.fixture()
+def channel(conn):
+    """One subscribed channel, returned as a row."""
+    return add_channel(conn, CHANNEL_ID, "clabretro", "clabretro")
+
+
+@pytest.fixture()
+def no_network(monkeypatch):
+    """Fail loudly if anything tries to open a socket.
+
+    Belt and braces: a test that accidentally hits the network would pass locally
+    and fail in a different week for reasons nobody could reproduce.
+    """
+    import urllib.request
+
+    def forbidden(*args, **kwargs):
+        raise AssertionError("test attempted a network call")
+
+    monkeypatch.setattr(urllib.request, "urlopen", forbidden)
+
+
+# --------------------------------------------------------------------- helpers
+
+
+def add_channel(conn, channel_id: str, title: str, dir_name: str, **kwargs):
+    fields = {"source": "youtube", "backfilled": 1, "uploads_playlist": "UULF"}
+    fields.update(kwargs)
+    columns = ", ".join(fields)
+    marks = ", ".join("?" * len(fields))
+    with conn:
+        conn.execute(
+            f"INSERT INTO channel (channel_id, handle, title, description, dir_name, "
+            f"added_at, {columns}) VALUES (?, ?, ?, ?, ?, ?, {marks})",
+            (channel_id, f"@{dir_name}", title, "A channel", dir_name,
+             util.utcnow_iso(), *fields.values()),
+        )
+    return conn.execute(
+        "SELECT * FROM channel WHERE channel_id = ?", (channel_id,)
+    ).fetchone()
+
+
+def add_video(conn, channel_pk, video_id, **kwargs):
+    """Insert a video row with sensible defaults."""
+    defaults = {
+        "title": f"Video {video_id}",
+        "upload_date": "2026-08-01",
+        "state": videos.LISTED,
+        "discovery_source": videos.SOURCE_UULF,
+        "duration": 900,
+    }
+    defaults.update(kwargs)
+    videos.insert(conn, channel_pk=channel_pk, video_id=video_id, **defaults)
+    return videos.get(conn, video_id)
+
+
+def feed_bytes(name: str) -> bytes:
+    return (FIXTURES / name).read_bytes()
+
+
+def make_feed(entries: list[dict], *, playlist_published: str = "2019-01-01T00:00:00+00:00") -> bytes:
+    """Build an Atom feed shaped like YouTube's.
+
+    Includes the feed-level <published> that is NOT an entry, because scraping
+    timestamps instead of walking atom:entry picks it up and yields nonsense
+    upload rates — a mistake made once while measuring.
+    """
+    items = "".join(
+        f"""
+  <entry>
+    <id>yt:video:{e['video_id']}</id>
+    <yt:videoId>{e['video_id']}</yt:videoId>
+    <title>{e.get('title', 'Untitled')}</title>
+    <published>{e['published']}</published>
+  </entry>"""
+        for e in entries
+    )
+    return f"""<?xml version="1.0" encoding="UTF-8"?>
+<feed xmlns:yt="http://www.youtube.com/xml/schemas/2015"
+      xmlns:media="http://search.yahoo.com/mrss/"
+      xmlns="http://www.w3.org/2005/Atom">
+  <title>Videos</title>
+  <published>{playlist_published}</published>{items}
+</feed>""".encode()
+
+
+class FakeApi:
+    """Stand-in for `api.Api` with scriptable responses.
+
+    Counts calls so tests can assert the batching actually batches — the whole
+    quota argument in the plan rests on 50 ids per call, and a regression to one
+    call per video would be silent and expensive.
+    """
+
+    def __init__(self, *, subs=None, uploads=None, durations=None, channel=None,
+                 raises=None):
+        self._subs = subs
+        self._uploads = uploads or []
+        self._durations = durations or {}
+        self._channel = channel
+        self._raises = raises
+        self.calls = 0
+        self.duration_calls = 0
+        self.subscription_calls = 0
+
+    def subscriptions(self, channel_id):
+        self.subscription_calls += 1
+        self.calls += 1
+        if isinstance(self._raises, Exception):
+            raise self._raises
+        return list(self._subs or [])
+
+    def uploads(self, channel_id, *, kind="UULF", since=None, limit=None,
+                page_token=None):
+        self.calls += 1
+        produced = 0
+        for entry, token in self._uploads:
+            if since is not None and entry["published"] < since:
+                return
+            yield entry, token
+            produced += 1
+            if limit is not None and produced >= limit:
+                return
+
+    def durations(self, video_ids):
+        self.duration_calls += 1
+        self.calls += 1
+        return {vid: self._durations[vid] for vid in video_ids
+                if vid in self._durations}
+
+    def channel(self, channel_id):
+        self.calls += 1
+        return self._channel
+
+    def resolve_handle(self, handle):
+        self.calls += 1
+        return self._channel
+
+
+def patch_api(monkeypatch, module, fake: FakeApi):
+    """Make `module.api.Api(...)` return `fake` regardless of arguments."""
+    monkeypatch.setattr(module.api, "Api", lambda *a, **kw: fake)
+    return fake

tests/fixtures/uulf_feed.xml

@@ -0,0 +1,37 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<feed xmlns:yt="http://www.youtube.com/xml/schemas/2015"
+      xmlns:media="http://search.yahoo.com/mrss/"
+      xmlns="http://www.w3.org/2005/Atom">
+  <id>yt:playlist:UULFW7jUEpYT_t0Gsf632d6_wQ</id>
+  <title>Uploads from clabretro</title>
+  <entry>
+    <id>yt:video:08Ajr5fP52I</id>
+    <yt:videoId>08Ajr5fP52I</yt:videoId>
+    <yt:channelId>UCW7jUEpYT_t0Gsf632d6_wQ</yt:channelId>
+    <title>Learning to Design 3D Prints</title>
+    <published>2026-08-07T15:00:11+00:00</published>
+    <media:group>
+      <media:description>Tinkercad &amp; a cheap printer. Part 1/3 &lt;of a series&gt;.</media:description>
+    </media:group>
+  </entry>
+  <entry>
+    <id>yt:video:8k8nAQq0s_s</id>
+    <yt:videoId>8k8nAQq0s_s</yt:videoId>
+    <yt:channelId>UCW7jUEpYT_t0Gsf632d6_wQ</yt:channelId>
+    <title>Trying to use a Nortel PBX: part two</title>
+    <published>2026-08-02T14:30:00+00:00</published>
+    <media:group>
+      <media:description>Telephony experiments.</media:description>
+    </media:group>
+  </entry>
+  <entry>
+    <id>yt:video:vcYYcQyecNQ</id>
+    <yt:videoId>vcYYcQyecNQ</yt:videoId>
+    <yt:channelId>UCW7jUEpYT_t0Gsf632d6_wQ</yt:channelId>
+    <title>IBM Director on an xSeries 346 from 2004</title>
+    <published>2026-06-17T12:00:00+00:00</published>
+    <media:group>
+      <media:description>Old enterprise management software.</media:description>
+    </media:group>
+  </entry>
+</feed>

tests/test_api.py

@@ -0,0 +1,329 @@
+"""The YouTube Data API client, with no YouTube involved.
+
+Two things here earn their keep beyond ordinary coverage:
+
+* **Error classification.** Three distinct operator mistakes all arrive as HTTP
+  403 `forbidden`, and the only reliable signal is `error.details[].reason`. That
+  cost three attempts to diagnose by hand once; it should cost nobody a second one.
+* **`totalResults` is not the list length.** It reported 127 against 119 actually
+  returned, because terminated and private channels still count as subscriptions.
+"""
+
+from __future__ import annotations
+
+import io
+import json
+import urllib.error
+
+import pytest
+
+from ytstream import api
+
+
+# ------------------------------------------------------------------- durations
+
+
+@pytest.mark.parametrize("text,expected", [
+    ("PT1H2M3S", 3723),
+    ("PT21M", 1260),
+    ("PT45S", 45),
+    ("P1DT2H", 93600),
+    ("P1W", 604800),
+    ("PT0S", 0),            # live and upcoming videos report this
+    ("", None),
+    ("garbage", None),
+    ("1H2M", None),
+])
+def test_parse_duration(text, expected):
+    assert api.parse_duration(text) == expected
+
+
+def test_parse_published_keeps_the_exact_string():
+    """The date drives naming; the exact stamp is kept because approximate_date
+    is wrong by up to two days and episode numbers cannot be re-derived later."""
+    day, exact = api.parse_published("2026-08-11T16:32:10Z")
+    assert day.isoformat() == "2026-08-11"
+    assert exact == "2026-08-11T16:32:10Z"
+
+
+def test_parse_published_normalises_to_utc():
+    day, _ = api.parse_published("2026-08-11T23:30:00-08:00")
+    assert day.isoformat() == "2026-08-12"
+
+
+def test_parse_published_tolerates_rubbish():
+    assert api.parse_published("") == (None, None)
+    assert api.parse_published("not a date")[0] is None
+
+
+# ------------------------------------------------------------------- playlists
+
+
+def test_uploads_playlist_id():
+    assert api.uploads_playlist_id("UCjCJ2LaOIsPzOoXUTMDI3wg") == "UULFjCJ2LaOIsPzOoXUTMDI3wg"
+    assert api.uploads_playlist_id("UCjCJ2LaOIsPzOoXUTMDI3wg", "UU") == "UUjCJ2LaOIsPzOoXUTMDI3wg"
+
+
+def test_uploads_playlist_id_rejects_non_channel_ids():
+    with pytest.raises(ValueError):
+        api.uploads_playlist_id("PLnotachannel")
+
+
+# -------------------------------------------------------- error classification
+
+
+def _http_error(code, body):
+    return urllib.error.HTTPError(
+        "https://example/", code, "Forbidden", {},
+        io.BytesIO(json.dumps(body).encode()),
+    )
+
+
+def _classify(code, body):
+    return api.Api("k")._classify(_http_error(code, body))
+
+
+def test_service_disabled_is_not_configured():
+    """The API is not enabled on the project. Carries an activation URL."""
+    error = _classify(403, {"error": {
+        "code": 403,
+        "message": "YouTube Data API v3 has not been used in project 510818173753 "
+                   "before or it is disabled.",
+        "errors": [{"reason": "accessNotConfigured"}],
+        "details": [{"reason": "SERVICE_DISABLED",
+                     "metadata": {"activationUrl": "https://console/enable"}}],
+    }})
+    assert isinstance(error, api.NotConfigured)
+
+
+def test_key_service_blocked_is_not_configured():
+    """A different mistake entirely — the key's own API restrictions — and it
+    reports only `forbidden` in errors[]."""
+    error = _classify(403, {"error": {
+        "code": 403,
+        "message": "Requests to this API youtube method "
+                   "youtube.api.v3.V3DataVideoService.List are blocked.",
+        "errors": [{"reason": "forbidden"}],
+        "details": [{"reason": "API_KEY_SERVICE_BLOCKED"}],
+    }})
+    assert isinstance(error, api.NotConfigured)
+
+
+def test_subscription_forbidden_is_its_own_type():
+    """Must never be mistaken for an empty subscription list."""
+    error = _classify(403, {"error": {
+        "code": 403,
+        "message": "The requester is not allowed to access the requested subscriptions.",
+        "errors": [{"reason": "subscriptionForbidden"}],
+    }})
+    assert isinstance(error, api.SubscriptionsPrivate)
+    assert not isinstance(error, api.NotConfigured)
+
+
+def test_transient_server_error_is_plain_api_error():
+    error = _classify(500, {"error": {
+        "code": 500, "message": "Backend Error",
+        "errors": [{"reason": "backendError"}],
+    }})
+    assert type(error) is api.ApiError
+
+
+def test_quota_exceeded_is_plain_api_error():
+    """Retrying tomorrow helps, so it must not be classed as misconfiguration."""
+    error = _classify(403, {"error": {
+        "code": 403, "message": "The request cannot be completed because you have "
+                                "exceeded your quota.",
+        "errors": [{"reason": "quotaExceeded"}],
+    }})
+    assert type(error) is api.ApiError
+
+
+def test_unparseable_body_still_yields_an_error():
+    exc = urllib.error.HTTPError(
+        "https://example/", 502, "Bad Gateway", {}, io.BytesIO(b"<html>nope</html>")
+    )
+    error = api.Api("k")._classify(exc)
+    assert isinstance(error, api.ApiError)
+    assert error.status == 502
+
+
+def test_missing_key_raises_before_any_request(monkeypatch):
+    import urllib.request
+
+    monkeypatch.setattr(urllib.request, "urlopen", lambda *a, **k:
+                        (_ for _ in ()).throw(AssertionError("should not be called")))
+    with pytest.raises(api.NotConfigured):
+        api.Api("").durations(["dQw4w9WgXcQ"])
+
+
+# -------------------------------------------------------------- request shapes
+
+
+class Recorder:
+    """Captures the URLs requested and replays canned pages."""
+
+    def __init__(self, pages):
+        self.pages = list(pages)
+        self.urls = []
+
+    def __call__(self, request, timeout=None):
+        self.urls.append(request.full_url)
+        payload = json.dumps(self.pages.pop(0)).encode()
+
+        class Response(io.BytesIO):
+            def __enter__(self_inner):
+                return self_inner
+
+            def __exit__(self_inner, *exc):
+                return False
+
+        return Response(payload)
+
+
+@pytest.fixture()
+def recorder(monkeypatch):
+    def install(pages):
+        rec = Recorder(pages)
+        monkeypatch.setattr(api.urllib.request, "urlopen", rec)
+        return rec
+    return install
+
+
+def test_subscriptions_pages_to_exhaustion(recorder):
+    rec = recorder([
+        {"pageInfo": {"totalResults": 127},
+         "items": [{"snippet": {"title": f"C{i}",
+                                "resourceId": {"channelId": f"UC{i:022d}"}}}
+                   for i in range(50)],
+         "nextPageToken": "T2"},
+        {"pageInfo": {"totalResults": 127},
+         "items": [{"snippet": {"title": "Last",
+                                "resourceId": {"channelId": "UC" + "z" * 22}}}]},
+    ])
+
+    found = api.Api("k").subscriptions("UCbrother")
+
+    assert len(found) == 51
+    assert len(rec.urls) == 2
+    assert "pageToken=T2" in rec.urls[1]
+
+
+def test_subscriptions_returns_the_fetched_list_not_total_results(recorder):
+    """Measured: totalResults said 127, pagination returned 119."""
+    recorder([{"pageInfo": {"totalResults": 127},
+               "items": [{"snippet": {"title": "One",
+                                      "resourceId": {"channelId": "UC" + "a" * 22}}}]}])
+
+    found = api.Api("k").subscriptions("UCbrother")
+
+    assert len(found) == 1
+
+
+def test_subscriptions_deduplicates(recorder):
+    recorder([{"items": [
+        {"snippet": {"title": "Dup", "resourceId": {"channelId": "UC" + "a" * 22}}},
+        {"snippet": {"title": "Dup", "resourceId": {"channelId": "UC" + "a" * 22}}},
+    ]}])
+    assert len(api.Api("k").subscriptions("UCbrother")) == 1
+
+
+def test_subscriptions_skips_entries_without_a_channel_id(recorder):
+    recorder([{"items": [
+        {"snippet": {"title": "Broken", "resourceId": {}}},
+        {"snippet": {"title": "Fine", "resourceId": {"channelId": "UC" + "a" * 22}}},
+    ]}])
+    assert [entry["title"] for entry in api.Api("k").subscriptions("UCb")] == ["Fine"]
+
+
+def test_durations_batches_fifty_ids_per_call(recorder):
+    """The whole quota argument rests on this. One call per video would be a
+    silent 50x regression."""
+    ids = [f"vid{i:08d}" for i in range(120)]
+    rec = recorder([
+        {"items": [{"id": vid, "contentDetails": {"duration": "PT10M"}}
+                   for vid in ids[start:start + 50]]}
+        for start in (0, 50, 100)
+    ])
+
+    found = api.Api("k").durations(ids)
+
+    assert len(found) == 120
+    assert len(rec.urls) == 3
+
+
+def test_durations_flags_livestreams(recorder):
+    recorder([{"items": [
+        {"id": "live0000001", "contentDetails": {"duration": "PT0S"},
+         "liveStreamingDetails": {"actualStartTime": "2026-08-01T00:00:00Z"}},
+        {"id": "vod00000001", "contentDetails": {"duration": "PT30M"}},
+    ]}])
+
+    found = api.Api("k").durations(["live0000001", "vod00000001"])
+
+    assert found["live0000001"]["is_live"] is True
+    assert found["vod00000001"]["is_live"] is False
+
+
+def test_uploads_stops_at_the_since_date(recorder):
+    recorder([{"items": [
+        {"contentDetails": {"videoId": "new00000001",
+                            "videoPublishedAt": "2026-08-10T00:00:00Z"}},
+        {"contentDetails": {"videoId": "old00000001",
+                            "videoPublishedAt": "2026-01-01T00:00:00Z"}},
+        {"contentDetails": {"videoId": "old00000002",
+                            "videoPublishedAt": "2025-01-01T00:00:00Z"}},
+    ], "nextPageToken": "T2"}])
+
+    from datetime import date
+
+    got = list(api.Api("k").uploads("UC" + "a" * 22, since=date(2026, 8, 1)))
+
+    assert [entry["video_id"] for entry, _ in got] == ["new00000001"]
+
+
+def test_uploads_honours_the_limit(recorder):
+    recorder([{"items": [
+        {"contentDetails": {"videoId": f"vid{i:08d}",
+                            "videoPublishedAt": "2026-08-10T00:00:00Z"}}
+        for i in range(10)
+    ]}])
+
+    got = list(api.Api("k").uploads("UC" + "a" * 22, limit=3))
+
+    assert len(got) == 3
+
+
+def test_uploads_skips_entries_with_no_publish_date(recorder):
+    """A private or deleted video keeps its playlist slot but loses its date.
+    Skipping is right; stopping would truncate the backfill at that point."""
+    recorder([{"items": [
+        {"contentDetails": {"videoId": "priv0000001"}},
+        {"contentDetails": {"videoId": "good0000001",
+                            "videoPublishedAt": "2026-08-10T00:00:00Z"}},
+    ]}])
+
+    got = list(api.Api("k").uploads("UC" + "a" * 22))
+
+    assert [entry["video_id"] for entry, _ in got] == ["good0000001"]
+
+
+def test_uploads_yields_the_next_page_token_for_resumability(recorder):
+    recorder([{"items": [
+        {"contentDetails": {"videoId": "vid00000001",
+                            "videoPublishedAt": "2026-08-10T00:00:00Z"}}
+    ], "nextPageToken": "CARRY_ON"}])
+
+    # Take only the first entry: the generator would otherwise follow the token
+    # on to a page the recorder has not been given, which is correct paging
+    # behaviour and covered elsewhere.
+    _, token = next(api.Api("k").uploads("UC" + "a" * 22))
+
+    assert token == "CARRY_ON"
+
+
+def test_call_counter_tracks_quota(recorder):
+    recorder([{"items": []}, {"items": []}])
+    client = api.Api("k")
+    client.durations(["a"])
+    client.durations(["b"])
+    assert client.calls == 2

tests/test_auth.py

@@ -0,0 +1,203 @@
+"""Password hashing, session cookies, CSRF tokens and login throttling."""
+
+import time
+
+import pytest
+
+from ytstream.web import auth
+
+
+class TestPasswords:
+    def test_round_trip(self):
+        stored = auth.hash_password("correct horse battery staple")
+        assert auth.verify_password(stored, "correct horse battery staple")
+
+    def test_wrong_password_rejected(self):
+        stored = auth.hash_password("secret")
+        assert not auth.verify_password(stored, "Secret")
+        assert not auth.verify_password(stored, "")
+
+    def test_salt_makes_hashes_unique(self):
+        assert auth.hash_password("same") != auth.hash_password("same")
+
+    def test_hash_is_not_the_plaintext(self):
+        assert "secret" not in auth.hash_password("secret")
+
+    def test_empty_stored_hash_rejects_everything(self):
+        assert not auth.verify_password("", "anything")
+
+    def test_malformed_stored_hash_does_not_raise(self):
+        for junk in ("nonsense", "scrypt$bad", "a$b$c$d$e$f", "scrypt$x$y$z$q$r"):
+            assert auth.verify_password(junk, "anything") is False
+
+    def test_unicode_password(self):
+        stored = auth.hash_password("pässwörd🎬")
+        assert auth.verify_password(stored, "pässwörd🎬")
+
+
+class TestSessions:
+    def test_issue_and_verify(self):
+        secret = auth.new_secret()
+        token = auth.issue_session(secret)
+        assert auth.verify_session(secret, token)
+
+    def test_a_different_secret_rejects(self):
+        token = auth.issue_session(auth.new_secret())
+        assert not auth.verify_session(auth.new_secret(), token)
+
+    def test_tampered_payload_rejected(self):
+        secret = auth.new_secret()
+        token = auth.issue_session(secret)
+        payload, signature = token.split(".", 1)
+        assert not auth.verify_session(secret, f"{payload}x.{signature}")
+
+    def test_tampered_signature_rejected(self):
+        secret = auth.new_secret()
+        payload, _ = auth.issue_session(secret).split(".", 1)
+        assert not auth.verify_session(secret, f"{payload}.deadbeef")
+
+    def test_garbage_rejected(self):
+        secret = auth.new_secret()
+        for junk in ("", "no-dot", "a.b.c", "...."):
+            assert auth.verify_session(secret, junk) is False
+
+    def test_expires_after_a_year(self):
+        secret = auth.new_secret()
+        issued = time.time() - auth.SESSION_MAX_AGE - 10
+        token = auth.issue_session(secret, issued_at=issued)
+        assert not auth.verify_session(secret, token)
+
+    def test_still_valid_just_inside_a_year(self):
+        secret = auth.new_secret()
+        issued = time.time() - auth.SESSION_MAX_AGE + 60
+        token = auth.issue_session(secret, issued_at=issued)
+        assert auth.verify_session(secret, token)
+
+    def test_a_token_from_the_future_is_rejected(self):
+        secret = auth.new_secret()
+        token = auth.issue_session(secret, issued_at=time.time() + 3600)
+        assert not auth.verify_session(secret, token)
+
+
+class TestCookie:
+    def test_carries_the_hardening_flags(self):
+        header = auth.cookie_header("abc")
+        for flag in ("HttpOnly", "Secure", "SameSite=Lax", "Path=/"):
+            assert flag in header
+        assert f"Max-Age={auth.SESSION_MAX_AGE}" in header
+
+    def test_secure_can_be_omitted_for_local_http_testing(self):
+        assert "Secure" not in auth.cookie_header("abc", secure=False)
+
+    def test_clear_cookie_expires_immediately(self):
+        assert "Max-Age=0" in auth.clear_cookie_header()
+
+
+class TestCookieParsing:
+    """Regression tests for a real failure: http.cookies.SimpleCookie silently
+    drops everything after a value it dislikes, which made valid sessions
+    invisible and bounced users back to the login page with no error."""
+
+    def test_finds_our_cookie_alone(self):
+        assert auth.cookie_value("yta_session=abc") == "abc"
+
+    def test_finds_it_after_a_neighbour(self):
+        assert auth.cookie_value("sessionid=xyz; yta_session=abc") == "abc"
+
+    def test_finds_it_before_a_neighbour(self):
+        assert auth.cookie_value("yta_session=abc; sessionid=xyz") == "abc"
+
+    @pytest.mark.parametrize(
+        "neighbour",
+        [
+            'prefs={"a":1}',          # JSON value — what actually broke it
+            "junk=[1,2,3]",
+            "weird=a b c",
+            "empty=",
+            "novalue",
+            "quoted=\"has spaces\"",
+            "path=/a/b/c",
+            "colons=a:b:c",
+            "comma=a,b",
+        ],
+    )
+    def test_survives_hostile_neighbours(self, neighbour):
+        assert auth.cookie_value(f"{neighbour}; yta_session=abc") == "abc"
+        assert auth.cookie_value(f"yta_session=abc; {neighbour}") == "abc"
+
+    def test_strips_surrounding_quotes(self):
+        assert auth.cookie_value('yta_session="abc"') == "abc"
+
+    def test_tolerates_whitespace(self):
+        assert auth.cookie_value("  yta_session = abc  ") == "abc"
+
+    def test_absent_cookie_returns_empty(self):
+        assert auth.cookie_value("sessionid=xyz") == ""
+
+    def test_empty_header_returns_empty(self):
+        assert auth.cookie_value("") == ""
+        assert auth.cookie_value(None) == ""
+
+    def test_does_not_match_a_name_that_merely_contains_ours(self):
+        assert auth.cookie_value("not_yta_session=nope") == ""
+
+    def test_real_token_round_trips_through_the_header(self):
+        secret = auth.new_secret()
+        token = auth.issue_session(secret)
+        header = f'prefs={{"theme":"dark"}}; yta_session={token}; other=1'
+        assert auth.verify_session(secret, auth.cookie_value(header))
+
+
+class TestCsrf:
+    def test_token_verifies(self):
+        secret, session = auth.new_secret(), auth.issue_session(auth.new_secret())
+        token = auth.csrf_token(secret, session)
+        assert auth.verify_csrf(secret, session, token)
+
+    def test_token_is_bound_to_the_session(self):
+        secret = auth.new_secret()
+        one = auth.issue_session(secret, issued_at=1000)
+        two = auth.issue_session(secret, issued_at=2000)
+        assert not auth.verify_csrf(secret, two, auth.csrf_token(secret, one))
+
+    def test_empty_token_rejected(self):
+        secret, session = auth.new_secret(), "sess"
+        assert not auth.verify_csrf(secret, session, "")
+
+    def test_wrong_token_rejected(self):
+        secret, session = auth.new_secret(), "sess"
+        assert not auth.verify_csrf(secret, session, "deadbeef")
+
+
+class TestThrottle:
+    def test_allows_up_to_the_limit(self):
+        throttle = auth.LoginThrottle(max_failures=3, lockout=60)
+        for _ in range(2):
+            throttle.record_failure("1.2.3.4", now=1000)
+        assert not throttle.locked("1.2.3.4", now=1000)
+
+    def test_locks_after_the_limit(self):
+        throttle = auth.LoginThrottle(max_failures=3, lockout=60)
+        for _ in range(3):
+            throttle.record_failure("1.2.3.4", now=1000)
+        assert throttle.locked("1.2.3.4", now=1000)
+
+    def test_lock_expires(self):
+        throttle = auth.LoginThrottle(max_failures=3, lockout=60)
+        for _ in range(3):
+            throttle.record_failure("1.2.3.4", now=1000)
+        assert not throttle.locked("1.2.3.4", now=1061)
+
+    def test_success_clears_the_counter(self):
+        throttle = auth.LoginThrottle(max_failures=3, lockout=60)
+        for _ in range(2):
+            throttle.record_failure("1.2.3.4", now=1000)
+        throttle.record_success("1.2.3.4")
+        assert not throttle.locked("1.2.3.4", now=1000)
+
+    def test_addresses_are_tracked_separately(self):
+        throttle = auth.LoginThrottle(max_failures=2, lockout=60)
+        for _ in range(2):
+            throttle.record_failure("1.1.1.1", now=1000)
+        assert throttle.locked("1.1.1.1", now=1000)
+        assert not throttle.locked("2.2.2.2", now=1000)

tests/test_channels.py

@@ -0,0 +1,151 @@
+"""Channel subscribe/unsubscribe, and the API-only sync path."""
+
+from __future__ import annotations
+
+import pytest
+
+from ytstream import api, channels, config
+
+from conftest import CHANNEL_ID, FakeApi, add_channel, add_video
+
+
+def test_normalise_url_accepts_every_reference_form():
+    assert channels.normalise_url(CHANNEL_ID).endswith("/channel/" + CHANNEL_ID)
+    assert channels.normalise_url("@clabretro") == "https://www.youtube.com/@clabretro"
+    assert channels.normalise_url("clabretro") == "https://www.youtube.com/@clabretro"
+    assert channels.normalise_url("https://youtube.com/x") == "https://youtube.com/x"
+    assert channels.normalise_url("youtube.com/x") == "https://youtube.com/x"
+
+
+def test_normalise_url_rejects_nonsense():
+    with pytest.raises(channels.ResolutionError):
+        channels.normalise_url("")
+    with pytest.raises(channels.ResolutionError):
+        channels.normalise_url("not a handle!!")
+
+
+def test_uulf_playlist_id():
+    assert channels.uulf_playlist_id(CHANNEL_ID) == "UULF" + CHANNEL_ID[2:]
+
+
+# ---------------------------------------------------------- subscribe_from_sync
+
+
+@pytest.fixture()
+def api_channel(monkeypatch):
+    """Patch channels' lazily-imported api module."""
+    fake = FakeApi(channel={"channel_id": "UC" + "a" * 22, "title": "Alpha Channel",
+                            "description": "About alpha", "handle": "@alpha",
+                            "avatar_url": None})
+    monkeypatch.setattr(api, "Api", lambda *a, **kw: fake)
+    return fake
+
+
+def test_subscribe_from_sync_never_calls_yt_dlp(conn, settings, media_root,
+                                                api_channel, monkeypatch):
+    """119 channels' worth of yt-dlp resolution is the request burst this design
+    exists to avoid."""
+    monkeypatch.setattr(channels.ytdlp, "run_json",
+                        lambda *a, **k: pytest.fail("yt-dlp must not be called"))
+
+    row = channels.subscribe_from_sync(conn, settings, "UC" + "a" * 22, "Alpha")
+
+    assert row["title"] == "Alpha Channel"
+    assert row["source"] == "youtube"
+
+
+def test_subscribe_from_sync_uses_the_api_description(conn, settings, media_root,
+                                                      api_channel):
+    row = channels.subscribe_from_sync(conn, settings, "UC" + "a" * 22, "Alpha")
+    assert row["description"] == "About alpha"
+
+
+def test_subscribe_from_sync_is_idempotent(conn, settings, media_root, api_channel):
+    first = channels.subscribe_from_sync(conn, settings, "UC" + "a" * 22, "Alpha")
+    second = channels.subscribe_from_sync(conn, settings, "UC" + "a" * 22, "Alpha")
+    assert first["id"] == second["id"]
+    assert conn.execute("SELECT COUNT(*) FROM channel").fetchone()[0] == 1
+
+
+def test_subscribe_from_sync_survives_a_metadata_failure(conn, settings, media_root,
+                                                         monkeypatch):
+    """A channel we cannot describe is still a channel we can mirror."""
+    class Failing(FakeApi):
+        def channel(self, channel_id):
+            raise api.ApiError(500, "backendError", "boom")
+
+    monkeypatch.setattr(api, "Api", lambda *a, **kw: Failing())
+
+    row = channels.subscribe_from_sync(conn, settings, "UC" + "a" * 22, "Fallback Name")
+
+    assert row["title"] == "Fallback Name"
+
+
+def test_subscribe_from_sync_makes_no_directory(conn, settings, media_root,
+                                                monkeypatch):
+    """The tree appears with the first episode, so a channel with nothing inside
+    the window leaves no empty series behind."""
+    class NoArt(FakeApi):
+        def channel(self, channel_id):
+            return {"channel_id": channel_id, "title": "Quiet", "description": "",
+                    "handle": None, "avatar_url": None}
+
+    monkeypatch.setattr(api, "Api", lambda *a, **kw: NoArt())
+
+    row = channels.subscribe_from_sync(conn, settings, "UC" + "a" * 22, "Quiet")
+
+    assert not (media_root / row["dir_name"]).exists()
+
+
+def test_directory_names_are_deduplicated(conn, settings, media_root, monkeypatch):
+    """Two channels can legitimately share a title; dir_name is UNIQUE."""
+    class Same(FakeApi):
+        def channel(self, channel_id):
+            return {"channel_id": channel_id, "title": "Same Name",
+                    "description": "", "handle": None, "avatar_url": None}
+
+    monkeypatch.setattr(api, "Api", lambda *a, **kw: Same())
+
+    first = channels.subscribe_from_sync(conn, settings, "UC" + "a" * 22, "Same Name")
+    second = channels.subscribe_from_sync(conn, settings, "UC" + "b" * 22, "Same Name")
+
+    assert first["dir_name"] == "Same Name"
+    assert second["dir_name"] == "Same Name (2)"
+
+
+# ------------------------------------------------------------------ unsubscribe
+
+
+def test_unsubscribe_removes_the_tree_and_the_rows(conn, media_root, channel):
+    tree = media_root / "clabretro"
+    (tree / "Season 2026").mkdir(parents=True)
+    (tree / "tvshow.nfo").write_text("<tvshow/>")
+    add_video(conn, channel["id"], "vid00000001")
+
+    title = channels.unsubscribe(conn, channel["id"])
+
+    assert title == "clabretro"
+    assert not tree.exists()
+    assert conn.execute("SELECT COUNT(*) FROM channel").fetchone()[0] == 0
+    assert conn.execute("SELECT COUNT(*) FROM video").fetchone()[0] == 0
+
+
+def test_unsubscribe_rejects_an_unknown_id(conn):
+    with pytest.raises(LookupError):
+        channels.unsubscribe(conn, 999)
+
+
+def test_lookup_helpers(conn, channel):
+    assert channels.get(conn, channel["id"])["title"] == "clabretro"
+    assert channels.get_by_channel_id(conn, CHANNEL_ID)["id"] == channel["id"]
+    assert channels.get(conn, 999) is None
+    assert channels.get_by_channel_id(conn, "UCnope") is None
+
+
+def test_all_channels_is_sorted_case_insensitively(conn):
+    add_channel(conn, "UC" + "a" * 22, "zebra", "zebra")
+    add_channel(conn, "UC" + "b" * 22, "Apple", "Apple")
+
+    titles = [row["title"] for row in channels.all_channels(conn)]
+
+    assert titles == ["Apple", "zebra"]

tests/test_discovery.py

@@ -0,0 +1,391 @@
+"""Discovery: feed parsing, the API backfill, and duration enrichment."""
+
+from __future__ import annotations
+
+from datetime import date, timedelta
+
+import pytest
+
+from ytstream import api, discovery, util, videos
+
+from conftest import FakeApi, add_video, make_feed, patch_api
+
+
+def _entry(video_id, published, title="A video"):
+    return {"video_id": video_id, "published": published, "title": title}
+
+
+@pytest.fixture()
+def offline(monkeypatch):
+    """No feed fetches and no API calls unless a test asks for them."""
+    monkeypatch.setattr(discovery, "fetch_feed",
+                        lambda *a, **k: pytest.fail("unexpected feed fetch"))
+    patch_api(monkeypatch, discovery, FakeApi())
+
+
+# --------------------------------------------------------------- feed parsing
+
+
+def test_parse_entries_reads_only_atom_entries():
+    """The feed-level <published> is the playlist's creation date, sometimes years
+    old. Treating it as a video produced a nonsense 0.01/day upload rate once."""
+    payload = make_feed(
+        [_entry("vid00000001", "2026-08-11T10:00:00+00:00")],
+        playlist_published="2019-03-01T00:00:00+00:00",
+    )
+
+    entries = discovery.parse_entries(payload)
+
+    assert len(entries) == 1
+    assert entries[0]["published"] == date(2026, 8, 11)
+
+
+def test_parse_entries_keeps_the_exact_timestamp():
+    payload = make_feed([_entry("vid00000001", "2026-08-11T16:32:10+00:00")])
+    assert discovery.parse_entries(payload)[0]["published_at"].startswith(
+        "2026-08-11T16:32:10"
+    )
+
+
+def test_parse_entries_skips_undated_entries():
+    payload = make_feed([_entry("vid00000001", "not-a-date"),
+                         _entry("vid00000002", "2026-08-11T10:00:00+00:00")])
+    assert [e["video_id"] for e in discovery.parse_entries(payload)] == ["vid00000002"]
+
+
+def test_parse_entries_rejects_garbage():
+    with pytest.raises(discovery.FeedUnavailable):
+        discovery.parse_entries(b"<not xml")
+
+
+def test_empty_feed_is_not_an_error():
+    assert discovery.parse_entries(make_feed([])) == []
+
+
+def test_feed_urls_use_the_long_form_playlist():
+    url = discovery.uulf_feed_url("UCjCJ2LaOIsPzOoXUTMDI3wg")
+    assert "playlist_id=UULFjCJ2LaOIsPzOoXUTMDI3wg" in url
+    assert "channel_id=UCjCJ2LaOIsPzOoXUTMDI3wg" in discovery.uc_feed_url(
+        "UCjCJ2LaOIsPzOoXUTMDI3wg"
+    )
+
+
+# -------------------------------------------------------------------- polling
+
+
+def _install_feed(monkeypatch, entries, *, uulf=True):
+    payload = make_feed(entries)
+
+    def fetch(url, timeout=30.0):
+        if "playlist_id=UULF" in url:
+            return payload if uulf else None
+        return payload if not uulf else None
+
+    monkeypatch.setattr(discovery, "fetch_feed", fetch)
+
+
+def test_poll_queues_videos_inside_the_window(conn, settings, channel, monkeypatch):
+    today = util.today()
+    _install_feed(monkeypatch, [_entry("vid00000001", f"{today}T10:00:00+00:00")])
+    patch_api(monkeypatch, discovery,
+              FakeApi(durations={"vid00000001": {"duration": 900, "is_live": False}}))
+
+    stats = discovery.poll_channel(conn, settings, channel)
+
+    assert stats["queued"] == 1
+    assert stats["source"] == videos.SOURCE_UULF
+    row = videos.get(conn, "vid00000001")
+    assert row["state"] == videos.LISTED
+    assert row["duration"] == 900
+
+
+def test_poll_marks_older_videos_skipped_old(conn, settings, channel, monkeypatch):
+    old = util.today() - timedelta(days=90)
+    _install_feed(monkeypatch, [_entry("vid00000001", f"{old}T10:00:00+00:00")])
+    patch_api(monkeypatch, discovery, FakeApi())
+
+    stats = discovery.poll_channel(conn, settings, channel)
+
+    assert stats["old"] == 1
+    assert videos.get(conn, "vid00000001")["state"] == videos.SKIPPED_OLD
+
+
+def test_poll_falls_back_to_the_channel_feed(conn, settings, channel, monkeypatch):
+    today = util.today()
+    _install_feed(monkeypatch, [_entry("vid00000001", f"{today}T10:00:00+00:00")],
+                  uulf=False)
+    patch_api(monkeypatch, discovery, FakeApi())
+
+    stats = discovery.poll_channel(conn, settings, channel)
+
+    assert stats["source"] == videos.SOURCE_UC
+    assert videos.get(conn, "vid00000001")["discovery_source"] == videos.SOURCE_UC
+
+
+def test_poll_records_feed_failure_without_raising(conn, settings, channel, monkeypatch):
+    def boom(url, timeout=30.0):
+        raise discovery.FeedUnavailable("HTTP 404")
+
+    monkeypatch.setattr(discovery, "fetch_feed", boom)
+
+    stats = discovery.poll_channel(conn, settings, channel)
+
+    assert "error" in stats
+    row = conn.execute("SELECT * FROM channel WHERE id = ?", (channel["id"],)).fetchone()
+    assert row["last_poll_ok"] == 0
+    assert row["consecutive_poll_failures"] == 1
+
+
+def test_two_of_119_channels_failing_is_survivable(conn, settings, channel, monkeypatch):
+    """Measured: terminated channels stay in the subscription list and 404 here."""
+    monkeypatch.setattr(discovery, "fetch_feed",
+                        lambda *a, **k: (_ for _ in ()).throw(
+                            discovery.FeedUnavailable("HTTP 404")))
+    totals = discovery.poll_all(conn, settings)
+    assert totals["failed"] == 1
+    assert totals["channels"] == 1
+
+
+def test_poll_fills_in_a_missing_title(conn, settings, channel, monkeypatch):
+    """Backfill inserts rows with no title; the feed is where titles come from."""
+    today = util.today()
+    add_video(conn, channel["id"], "vid00000001", title="",
+              upload_date=today.isoformat())
+    _install_feed(monkeypatch,
+                  [_entry("vid00000001", f"{today}T10:00:00+00:00", "Real Title")])
+    patch_api(monkeypatch, discovery, FakeApi())
+
+    stats = discovery.poll_channel(conn, settings, channel)
+
+    assert stats["titled"] == 1
+    assert videos.get(conn, "vid00000001")["title"] == "Real Title"
+
+
+def test_uulf_repairs_a_video_the_fallback_called_short(
+    conn, settings, channel, monkeypatch
+):
+    today = util.today()
+    add_video(conn, channel["id"], "vid00000001", state=videos.SKIPPED_SHORT,
+              discovery_source=videos.SOURCE_UC, upload_date=today.isoformat())
+    _install_feed(monkeypatch, [_entry("vid00000001", f"{today}T10:00:00+00:00")])
+    patch_api(monkeypatch, discovery, FakeApi())
+
+    stats = discovery.poll_channel(conn, settings, channel)
+
+    assert stats["repaired"] == 1
+    row = videos.get(conn, "vid00000001")
+    assert row["state"] == videos.LISTED
+    assert row["discovery_source"] == videos.SOURCE_UULF
+
+
+# ---------------------------------------------------------------- enrichment
+
+
+def test_enrich_filters_shorts(conn, settings, channel, monkeypatch):
+    """Measured: 38 of 50 consecutive uploads on a real channel were <=120s."""
+    add_video(conn, channel["id"], "short000001", duration=None)
+    patch_api(monkeypatch, discovery,
+              FakeApi(durations={"short000001": {"duration": 45, "is_live": False}}))
+
+    stats = discovery.enrich_durations(conn, settings, ["short000001"])
+
+    assert stats["shorts"] == 1
+    assert videos.get(conn, "short000001")["state"] == videos.SKIPPED_SHORT
+
+
+def test_enrich_filters_livestreams_regardless_of_duration(
+    conn, settings, channel, monkeypatch
+):
+    """Live and upcoming both report PT0S, so duration cannot be the signal."""
+    add_video(conn, channel["id"], "live0000001", duration=None)
+    patch_api(monkeypatch, discovery,
+              FakeApi(durations={"live0000001": {"duration": 0, "is_live": True}}))
+
+    stats = discovery.enrich_durations(conn, settings, ["live0000001"])
+
+    assert stats["live"] == 1
+    assert videos.get(conn, "live0000001")["state"] == videos.SKIPPED_LIVE
+
+
+def test_enrich_keeps_long_videos(conn, settings, channel, monkeypatch):
+    add_video(conn, channel["id"], "long0000001", duration=None)
+    patch_api(monkeypatch, discovery,
+              FakeApi(durations={"long0000001": {"duration": 2790, "is_live": False}}))
+
+    discovery.enrich_durations(conn, settings, ["long0000001"])
+
+    row = videos.get(conn, "long0000001")
+    assert row["state"] == videos.LISTED
+    assert row["duration"] == 2790
+
+
+def test_enrich_survives_an_api_failure(conn, settings, channel, monkeypatch):
+    """A NULL duration costs a runtime display, not a working library."""
+    add_video(conn, channel["id"], "vid00000001", duration=None)
+
+    class Failing(FakeApi):
+        def durations(self, ids):
+            raise api.ApiError(500, "backendError", "boom")
+
+    patch_api(monkeypatch, discovery, Failing())
+
+    stats = discovery.enrich_durations(conn, settings, ["vid00000001"])
+
+    assert stats["resolved"] == 0
+    assert videos.get(conn, "vid00000001")["state"] == videos.LISTED
+
+
+def test_enrich_with_no_ids_makes_no_call(conn, settings, monkeypatch):
+    fake = patch_api(monkeypatch, discovery, FakeApi())
+    discovery.enrich_durations(conn, settings, [])
+    assert fake.calls == 0
+
+
+# ------------------------------------------------------------------- backfill
+
+
+def test_backfill_queues_the_window(conn, settings, channel, monkeypatch):
+    today = util.today()
+    uploads = [
+        ({"video_id": f"vid{i:08d}", "published": today - timedelta(days=i),
+          "published_at": f"{today - timedelta(days=i)}T00:00:00Z"}, None)
+        for i in range(3)
+    ]
+    patch_api(monkeypatch, discovery, FakeApi(
+        uploads=uploads,
+        durations={f"vid{i:08d}": {"duration": 900, "is_live": False}
+                   for i in range(3)}))
+
+    stats = discovery.backfill_channel(conn, settings, channel)
+
+    assert stats["queued"] == 3
+    assert videos.get(conn, "vid00000000")["state"] == videos.LISTED
+    assert conn.execute("SELECT backfilled FROM channel WHERE id = ?",
+                        (channel["id"],)).fetchone()[0] == 1
+
+
+def test_backfill_stores_the_exact_publish_time(conn, settings, channel, monkeypatch):
+    today = util.today()
+    patch_api(monkeypatch, discovery, FakeApi(uploads=[
+        ({"video_id": "vid00000001", "published": today,
+          "published_at": "2026-08-11T16:32:10Z"}, None)]))
+
+    discovery.backfill_channel(conn, settings, channel)
+
+    assert videos.get(conn, "vid00000001")["published_at"] == "2026-08-11T16:32:10Z"
+
+
+def test_backfill_respects_the_video_cap(conn, settings, channel, monkeypatch):
+    settings.set("backfill_max_videos", "2")
+    today = util.today()
+    patch_api(monkeypatch, discovery, FakeApi(uploads=[
+        ({"video_id": f"vid{i:08d}", "published": today, "published_at": None}, None)
+        for i in range(10)]))
+
+    stats = discovery.backfill_channel(conn, settings, channel)
+
+    assert stats["queued"] == 2
+
+
+def test_backfill_clears_the_cursor_when_complete(conn, settings, channel, monkeypatch):
+    patch_api(monkeypatch, discovery, FakeApi(uploads=[
+        ({"video_id": "vid00000001", "published": util.today(),
+          "published_at": None}, "TOKEN")]))
+
+    discovery.backfill_channel(conn, settings, channel)
+
+    assert conn.execute("SELECT backfill_cursor FROM channel WHERE id = ?",
+                        (channel["id"],)).fetchone()[0] is None
+
+
+def test_backfill_leaves_the_flag_unset_when_the_api_is_unusable(
+    conn, settings, channel, monkeypatch
+):
+    """So it retries once a key is configured, rather than silently never running."""
+    with conn:
+        conn.execute("UPDATE channel SET backfilled = 0 WHERE id = ?", (channel["id"],))
+
+    class Failing(FakeApi):
+        def uploads(self, *a, **kw):
+            raise api.NotConfigured(403, "forbidden", "blocked")
+            yield  # pragma: no cover
+
+    patch_api(monkeypatch, discovery, Failing())
+
+    stats = discovery.backfill_channel(conn, settings, channel)
+
+    assert "error" in stats
+    assert conn.execute("SELECT backfilled FROM channel WHERE id = ?",
+                        (channel["id"],)).fetchone()[0] == 0
+
+
+def test_backfill_does_not_duplicate_known_videos(conn, settings, channel, monkeypatch):
+    add_video(conn, channel["id"], "vid00000001")
+    patch_api(monkeypatch, discovery, FakeApi(uploads=[
+        ({"video_id": "vid00000001", "published": util.today(),
+          "published_at": None}, None)]))
+
+    stats = discovery.backfill_channel(conn, settings, channel)
+
+    assert stats["queued"] == 0
+    assert conn.execute("SELECT COUNT(*) FROM video").fetchone()[0] == 1
+
+
+# ------------------------------------------------------- titles (regression)
+
+
+def test_backfill_takes_the_title_from_the_api(conn, settings, channel, monkeypatch):
+    """Not an optimisation. RSS returns 15 entries, which for a channel posting
+    under one long-form video a day reaches back only ~23 days against a 30-day
+    window — so the oldest ~5 of every 20-episode backfill was being named after
+    its video id."""
+    patch_api(monkeypatch, discovery, FakeApi(uploads=[
+        ({"video_id": "vid00000001", "published": util.today(),
+          "published_at": None, "title": "A Real Title"}, None)]))
+
+    discovery.backfill_channel(conn, settings, channel)
+
+    assert videos.get(conn, "vid00000001")["title"] == "A Real Title"
+
+
+def test_backfill_tolerates_a_missing_title(conn, settings, channel, monkeypatch):
+    patch_api(monkeypatch, discovery, FakeApi(uploads=[
+        ({"video_id": "vid00000001", "published": util.today(),
+          "published_at": None}, None)]))
+
+    discovery.backfill_channel(conn, settings, channel)
+
+    assert videos.get(conn, "vid00000001")["title"] == ""
+
+
+def test_a_late_title_renames_an_already_materialised_episode(
+    conn, settings, media_root, channel, monkeypatch
+):
+    """The repair has to move the file, not just the row — otherwise the episode
+    keeps its video-id filename forever."""
+    from ytstream import strm
+
+    monkeypatch.setattr(strm, "fetch_thumbnail", lambda *a, **k: False)
+    today = util.today()
+    add_video(conn, channel["id"], "vid00000001", title="",
+              upload_date=today.isoformat())
+    first = strm.materialise(conn, settings, channel,
+                             videos.get(conn, "vid00000001"))
+    assert "vid00000001]" in first["rel_path"]
+    old_path = media_root / first["rel_path"]
+    assert old_path.exists()
+
+    outcome = discovery._record(
+        conn, channel,
+        {"video_id": "vid00000001", "title": "Proper Name", "published": today,
+         "published_at": None},
+        videos.SOURCE_UULF, today - timedelta(days=30),
+    )
+
+    assert outcome == "titled"
+    assert videos.get(conn, "vid00000001")["state"] == videos.LISTED
+    assert not old_path.exists()
+
+    second = strm.materialise(conn, settings, channel,
+                              videos.get(conn, "vid00000001"))
+    assert "Proper Name" in second["rel_path"]

tests/test_naming.py

@@ -0,0 +1,128 @@
+from datetime import date
+
+import pytest
+
+from ytstream import naming
+
+
+class TestSanitise:
+    @pytest.mark.parametrize(
+        "raw, expected",
+        [
+            ("Hermitcraft S11#11: Expanding Business", "Hermitcraft S11#11 Expanding Business"),
+            ("A/B", "A B"),
+            ('Say "hello" <now>', "Say hello now"),
+            ("path\\to\\thing", "path to thing"),
+            ("what?  really *  |  yes", "what really yes"),
+            ("  .leading and trailing.  ", "leading and trailing"),
+            ("line one\nline two", "line one line two"),
+            ("tabs\tand\r\nnewlines", "tabs and newlines"),
+        ],
+    )
+    def test_removes_illegal_characters(self, raw, expected):
+        assert naming.sanitize_component(raw) == expected
+
+    def test_empty_input_is_empty(self):
+        assert naming.sanitize_component("") == ""
+        assert naming.sanitize_component(None) == ""
+
+    def test_title_that_is_only_illegal_characters_collapses_to_empty(self):
+        assert naming.sanitize_component("///???") == ""
+
+    def test_truncates_to_120_characters(self):
+        long = "word " * 60
+        result = naming.sanitize_component(long)
+        assert len(result) <= naming.MAX_TITLE_LEN
+
+    def test_truncation_prefers_a_word_boundary(self):
+        text = "alpha bravo charlie delta echo foxtrot golf hotel india juliet " * 3
+        result = naming.sanitize_component(text)
+        assert not result.endswith(" ")
+        # Should not cut mid-word when a boundary is available late enough.
+        assert result == result.rstrip()
+        assert " " in result
+
+    def test_hard_cuts_when_no_late_word_boundary_exists(self):
+        text = "a" + "b" * 400
+        result = naming.sanitize_component(text)
+        assert len(result) == naming.MAX_TITLE_LEN
+
+
+class TestEpisodeNumbering:
+    @pytest.mark.parametrize(
+        "day, ordinal, expected",
+        [
+            (date(2026, 8, 11), 0, 8110),
+            (date(2026, 8, 11), 1, 8111),
+            (date(2026, 8, 12), 0, 8120),
+            (date(2026, 1, 1), 0, 1010),
+            (date(2026, 12, 31), 0, 12310),
+            (date(2026, 12, 31), 9, 12319),
+        ],
+    )
+    def test_episode_number(self, day, ordinal, expected):
+        assert naming.episode_number(day, ordinal) == expected
+
+    def test_ordinal_is_clamped_at_nine(self):
+        assert naming.episode_number(date(2026, 8, 11), 12) == 8119
+
+    def test_negative_ordinal_clamps_to_zero(self):
+        assert naming.episode_number(date(2026, 8, 11), -3) == 8110
+
+    def test_numbers_sort_chronologically_across_the_year(self):
+        days = [date(2026, 1, 1), date(2026, 6, 15), date(2026, 8, 11), date(2026, 12, 31)]
+        numbers = [naming.episode_number(day, 0) for day in days]
+        assert numbers == sorted(numbers)
+
+    def test_episode_range_covers_ten_slots(self):
+        low, high = naming.episode_range(date(2026, 8, 11))
+        assert (low, high) == (8110, 8119)
+
+    def test_season_is_the_upload_year(self):
+        assert naming.season_for(date(2026, 8, 11)) == 2026
+
+
+class TestParseUploadDate:
+    def test_accepts_ytdlp_compact_form(self):
+        assert naming.parse_upload_date("20260811") == date(2026, 8, 11)
+
+    def test_accepts_iso_form(self):
+        assert naming.parse_upload_date("2026-08-11") == date(2026, 8, 11)
+
+    def test_accepts_iso_timestamp(self):
+        assert naming.parse_upload_date("2026-08-11T12:00:00+00:00") == date(2026, 8, 11)
+
+    def test_passes_through_a_date(self):
+        assert naming.parse_upload_date(date(2026, 8, 11)) == date(2026, 8, 11)
+
+    def test_rejects_nonsense(self):
+        with pytest.raises(ValueError):
+            naming.parse_upload_date("not a date")
+
+
+class TestBasename:
+    def test_includes_video_id_for_uniqueness(self):
+        stem = naming.basename("clabretro", 2026, 8110, "A Title", "dQw4w9WgXcQ")
+        assert stem == "clabretro - S2026E8110 - A Title [dQw4w9WgXcQ]"
+
+    def test_episode_is_unpadded_to_match_the_nfo(self):
+        stem = naming.basename("c", 2026, 8110, "t", "id")
+        assert "S2026E8110" in stem
+        assert "E08110" not in stem
+
+    def test_falls_back_to_video_id_when_the_title_sanitises_away(self):
+        stem = naming.basename("c", 2026, 8110, "///", "dQw4w9WgXcQ")
+        assert stem.endswith("dQw4w9WgXcQ [dQw4w9WgXcQ]")
+
+    def test_two_videos_same_title_differ_by_id(self):
+        one = naming.basename("c", 2026, 8110, "Same", "aaaaaaaaaaa")
+        two = naming.basename("c", 2026, 8111, "Same", "bbbbbbbbbbb")
+        assert one != two
+
+
+class TestChannelDirName:
+    def test_sanitises_the_title(self):
+        assert naming.channel_dir_name("Tom / Jerry", "UCabc") == "Tom Jerry"
+
+    def test_falls_back_to_channel_id_when_title_is_unusable(self):
+        assert naming.channel_dir_name("???", "UCabc") == "UCabc"

tests/test_nfo.py

@@ -0,0 +1,94 @@
+import xml.etree.ElementTree as ET
+
+from ytstream import nfo
+
+HOSTILE = (
+    "Ampersands & angle <brackets> and \"quotes\"\n"
+    "control chars: \x00\x07\x1b\n"
+    "emoji 🎬 and em-dash — and links https://example.com/?a=1&b=2"
+)
+
+
+class TestEpisodeNfo:
+    def build(self, **overrides):
+        kwargs = dict(
+            title="Video Title",
+            show_title="Some Channel",
+            season=2026,
+            episode=8110,
+            plot="A plot.",
+            aired="2026-08-11",
+            duration_seconds=762,
+            video_id="dQw4w9WgXcQ",
+        )
+        kwargs.update(overrides)
+        return nfo.episode_nfo(**kwargs)
+
+    def test_is_well_formed_xml(self):
+        root = ET.fromstring(self.build())
+        assert root.tag == "episodedetails"
+
+    def test_hostile_description_still_parses(self):
+        root = ET.fromstring(self.build(plot=HOSTILE))
+        plot = root.findtext("plot")
+        assert "&" in plot and "<brackets>" in plot
+        assert "🎬" in plot
+
+    def test_control_characters_are_stripped(self):
+        plot = ET.fromstring(self.build(plot=HOSTILE)).findtext("plot")
+        for bad in ("\x00", "\x07", "\x1b"):
+            assert bad not in plot
+
+    def test_newlines_are_preserved(self):
+        plot = ET.fromstring(self.build(plot="one\ntwo")).findtext("plot")
+        assert plot == "one\ntwo"
+
+    def test_runtime_is_rounded_minutes(self):
+        assert ET.fromstring(self.build(duration_seconds=762)).findtext("runtime") == "13"
+
+    def test_short_video_still_gets_at_least_one_minute(self):
+        assert ET.fromstring(self.build(duration_seconds=20)).findtext("runtime") == "1"
+
+    def test_runtime_omitted_when_duration_unknown(self):
+        assert ET.fromstring(self.build(duration_seconds=None)).find("runtime") is None
+
+    def test_unique_id_marks_youtube_as_default(self):
+        unique = ET.fromstring(self.build()).find("uniqueid")
+        assert unique.get("type") == "youtube"
+        assert unique.get("default") == "true"
+        assert unique.text == "dQw4w9WgXcQ"
+
+    def test_season_and_episode_are_present(self):
+        root = ET.fromstring(self.build())
+        assert root.findtext("season") == "2026"
+        assert root.findtext("episode") == "8110"
+
+    def test_title_keeps_characters_that_the_filename_strips(self):
+        root = ET.fromstring(self.build(title="Hermitcraft S11#11: Expanding Business"))
+        assert root.findtext("title") == "Hermitcraft S11#11: Expanding Business"
+
+    def test_empty_plot_does_not_break(self):
+        assert ET.fromstring(self.build(plot=None)).find("plot") is not None
+
+
+class TestTvshowNfo:
+    def test_well_formed_and_carries_channel_id(self):
+        root = ET.fromstring(nfo.tvshow_nfo("clabretro", HOSTILE, "UCabc123"))
+        assert root.tag == "tvshow"
+        assert root.findtext("title") == "clabretro"
+        assert root.findtext("studio") == "YouTube"
+        assert root.find("uniqueid").text == "UCabc123"
+
+
+class TestWrite:
+    def test_write_is_atomic_and_leaves_no_temp_file(self, tmp_path):
+        target = tmp_path / "sub" / "tvshow.nfo"
+        nfo.write(target, b"<tvshow/>")
+        assert target.read_bytes() == b"<tvshow/>"
+        assert list(tmp_path.rglob("*.tmp")) == []
+
+    def test_overwrites_existing(self, tmp_path):
+        target = tmp_path / "tvshow.nfo"
+        nfo.write(target, b"<a/>")
+        nfo.write(target, b"<b/>")
+        assert target.read_bytes() == b"<b/>"

tests/test_proxy.py

@@ -0,0 +1,354 @@
+"""The streaming proxy: HTTP range serving, and the safety rails.
+
+Ported from the standalone `proxy/test_range.py` and `proxy/test_limits.py` so that
+one `pytest` run covers everything. These now drive the real `make_handler(mgr, …)`
+rather than the PoC's `make_handler(path, done)`, which means the routing, the
+video-id validation and the wait-for-complete logic are exercised too.
+
+No network, no yt-dlp, no ffmpeg: the pipeline runner is stubbed and the output
+file is synthetic.
+"""
+
+from __future__ import annotations
+
+import http.client
+import importlib.util
+import json
+import os
+import threading
+import time
+from http.server import ThreadingHTTPServer
+from pathlib import Path
+
+import pytest
+
+PROXY_PATH = Path(__file__).resolve().parent.parent / "proxy" / "ytstream_proxy.py"
+
+
+def _load_proxy():
+    spec = importlib.util.spec_from_file_location("ytstream_proxy", PROXY_PATH)
+    module = importlib.util.module_from_spec(spec)
+    spec.loader.exec_module(module)
+    return module
+
+
+proxy = _load_proxy()
+
+# Position-identifiable body, so a wrong range returns visibly wrong bytes rather
+# than merely the wrong length.
+BODY = bytes(range(256)) * 400
+TOTAL = len(BODY)
+VIDEO_ID = "dQw4w9WgXcQ"
+
+
+class StubSession:
+    """A Session whose output file already exists."""
+
+    def __init__(self, path: Path, *, finished: bool = True):
+        self.video_id = VIDEO_ID
+        self.out_path = str(path)
+        self.final = threading.Event()
+        self.failed = None
+        self.readers = 0
+        self.last_used = time.time()
+        if finished:
+            self.final.set()
+
+    @property
+    def complete(self):
+        return self.final.is_set() and not self.failed
+
+    def size(self):
+        try:
+            return os.path.getsize(self.out_path)
+        except OSError:
+            return 0
+
+
+class StubManager:
+    def __init__(self, session, *, growing=False):
+        self.session = session
+        self.growing = growing
+        self.lock = threading.Lock()
+
+    def get(self, video_id):
+        return self.session, None
+
+    def status(self):
+        return {"mode": "stub", "cache_used_gb": 0.0}
+
+
+@pytest.fixture()
+def complete_server(tmp_path):
+    """A server over a finished file, with ranges honoured."""
+    path = tmp_path / "out.mp4"
+    path.write_bytes(BODY)
+    session = StubSession(path, finished=True)
+    yield from _serve(StubManager(session))
+
+
+@pytest.fixture()
+def growing_server(tmp_path):
+    """A server over a file still being written."""
+    path = tmp_path / "out.mp4"
+    path.write_bytes(BODY[:1000])
+    session = StubSession(path, finished=False)
+    manager = StubManager(session, growing=True)
+
+    def finish():
+        time.sleep(0.2)
+        with open(path, "ab") as handle:
+            handle.write(BODY[1000:])
+        session.final.set()
+
+    threading.Thread(target=finish, daemon=True).start()
+    yield from _serve(manager)
+
+
+def _serve(manager):
+    server = ThreadingHTTPServer(("127.0.0.1", 0), proxy.make_handler(manager, 30))
+    threading.Thread(target=server.serve_forever, daemon=True).start()
+    try:
+        yield server.server_address[1]
+    finally:
+        server.shutdown()
+        server.server_close()
+
+
+def request(port, headers=None, method="GET", path=f"/watch/{VIDEO_ID}"):
+    conn = http.client.HTTPConnection("127.0.0.1", port, timeout=20)
+    conn.request(method, path, headers=headers or {})
+    response = conn.getresponse()
+    body = response.read()
+    head = dict(response.getheaders())
+    conn.close()
+    return response.status, head, body
+
+
+# ------------------------------------------------------------- complete file
+
+
+def test_bounded_range(complete_server):
+    status, head, body = request(complete_server, {"Range": "bytes=1000-1999"})
+    assert status == 206
+    assert len(body) == 1000
+    assert body == BODY[1000:2000]
+    assert head["Content-Range"] == f"bytes 1000-1999/{TOTAL}"
+    assert head["Content-Length"] == "1000"
+
+
+def test_open_ended_range(complete_server):
+    status, head, body = request(complete_server, {"Range": "bytes=102000-"})
+    assert status == 206
+    assert body == BODY[102000:]
+    assert head["Content-Range"] == f"bytes 102000-{TOTAL - 1}/{TOTAL}"
+
+
+def test_suffix_range(complete_server):
+    """bytes=-N means the final N bytes, not the first N."""
+    status, head, body = request(complete_server, {"Range": "bytes=-100"})
+    assert status == 206
+    assert len(body) == 100
+    assert body == BODY[-100:]
+    assert head["Content-Range"] == f"bytes {TOTAL - 100}-{TOTAL - 1}/{TOTAL}"
+
+
+def test_single_byte_range(complete_server):
+    status, _, body = request(complete_server, {"Range": "bytes=5-5"})
+    assert status == 206
+    assert body == BODY[5:6]
+
+
+def test_end_beyond_eof_is_clamped(complete_server):
+    status, head, body = request(complete_server,
+                                 {"Range": f"bytes=102000-{TOTAL + 5000}"})
+    assert status == 206
+    assert body == BODY[102000:]
+    assert head["Content-Range"] == f"bytes 102000-{TOTAL - 1}/{TOTAL}"
+
+
+def test_range_past_eof_is_416(complete_server):
+    """And it must carry Content-Range: bytes */total, or clients retry forever."""
+    status, head, _ = request(complete_server, {"Range": f"bytes={TOTAL + 10}-"})
+    assert status == 416
+    assert head["Content-Range"] == f"bytes */{TOTAL}"
+
+
+def test_no_range_serves_the_whole_body(complete_server):
+    status, head, body = request(complete_server)
+    assert status == 200
+    assert body == BODY
+    assert head["Accept-Ranges"] == "bytes"
+
+
+def test_multi_range_falls_back_to_the_whole_body(complete_server):
+    """Legal, and beats mis-serving one part of a multipart response."""
+    status, _, body = request(complete_server, {"Range": "bytes=0-99,200-299"})
+    assert status == 200
+    assert body == BODY
+
+
+@pytest.mark.parametrize("header", ["bytes=", "bytes=abc-def", "items=0-99", "0-99"])
+def test_unparseable_range_serves_the_whole_body(complete_server, header):
+    status, _, body = request(complete_server, {"Range": header})
+    assert status == 200
+    assert len(body) == TOTAL
+
+
+def test_head_returns_headers_and_no_body(complete_server):
+    status, head, body = request(complete_server, method="HEAD")
+    assert status == 200
+    assert body == b""
+    assert head["Content-Length"] == str(TOTAL)
+
+
+# ------------------------------------------------------------- growing file
+
+
+def test_growing_file_is_chunked_and_tracks_to_eof(growing_server):
+    """Ranges cannot be honoured mid-write: there is no reliable time-to-byte
+    mapping into a fragmented MP4 yet, so it becomes a non-seekable stream."""
+    status, head, body = request(growing_server, {"Range": "bytes=500-999"})
+    assert status == 200
+    assert head.get("Transfer-Encoding") == "chunked"
+    assert body == BODY
+
+
+# ------------------------------------------------------------------- routing
+
+
+def test_unknown_path_is_404(complete_server):
+    status, _, _ = request(complete_server, path="/nope")
+    assert status == 404
+
+
+def test_malformed_video_id_is_400(complete_server):
+    status, _, _ = request(complete_server, path="/watch/short")
+    assert status == 400
+
+
+@pytest.mark.parametrize("bad", ["../../etc/passwd", "abcdefghij", "abcdefghijkl"])
+def test_video_id_must_be_exactly_eleven_safe_chars(complete_server, bad):
+    status, _, _ = request(complete_server, path=f"/watch/{bad}")
+    assert status in (400, 404)
+
+
+def test_healthz_returns_json(complete_server):
+    status, head, body = request(complete_server, path="/healthz")
+    assert status == 200
+    assert head["Content-Type"] == "application/json"
+    assert json.loads(body)["mode"] == "stub"
+
+
+# ------------------------------------------------- safety rails on the manager
+
+
+def vid(n):
+    return f"vid{n:08d}"
+
+
+@pytest.fixture()
+def manager_factory(tmp_path):
+    made = []
+
+    def build(**kwargs):
+        options = dict(max_pipelines=99, cache_bytes=10 ** 12, no_fetch=False,
+                       growing=False, max_retries=0, max_starts=3,
+                       starts_window=3600.0)
+        options.update(kwargs)
+        work = tmp_path / f"work{len(made)}"
+        work.mkdir()
+
+        class Recording(proxy.Manager):
+            """Records starts instead of spawning a pipeline."""
+
+            def __init__(self, *args, **kw):
+                super().__init__(*args, **kw)
+                self.ran = []
+
+            def _run(self, session):
+                self.ran.append(session.video_id)
+                session.final.set()      # completes instantly, never stays active
+
+        manager = Recording(str(work), **options)
+        made.append(manager)
+        return manager
+
+    return build
+
+
+def test_cold_start_budget_is_enforced(manager_factory):
+    manager = manager_factory(max_starts=3)
+    results = [manager.get(vid(i)) for i in range(5)]
+
+    started = [r for r in results if r[0] is not None]
+    refused = [r for r in results if r[0] is None]
+
+    assert len(started) == 3
+    assert len(refused) == 2
+    assert all("cold-start budget" in r[1] for r in refused)
+    assert manager.counters["refused_ratelimit"] == 2
+    assert len(manager.ran) == 3
+
+
+def test_cache_hits_are_never_rate_limited(manager_factory):
+    """Re-watching an already-fetched video must keep working with the budget
+    spent, or a metadata refresh would break normal playback for an hour."""
+    manager = manager_factory(max_starts=3)
+    for i in range(5):
+        manager.get(vid(i))
+
+    session, refusal = manager.get(vid(0))
+
+    assert session is not None and refusal is None
+    assert manager.counters["reused"] >= 1
+    assert len(manager.ran) == 3
+
+
+def test_start_budget_window_prunes(manager_factory):
+    manager = manager_factory(max_starts=2)
+    manager.get(vid(10))
+    manager.get(vid(11))
+    assert manager.get(vid(12))[0] is None
+
+    # Age the recorded starts past the window.
+    manager.start_log = type(manager.start_log)(t - 3601 for t in manager.start_log)
+
+    assert manager.get(vid(13))[0] is not None
+    assert len(manager.start_log) == 1
+
+
+def test_concurrency_cap(tmp_path):
+    class Blocking(proxy.Manager):
+        """Pipelines that never finish, so sessions stay active."""
+
+        def _run(self, session):
+            pass
+
+    work = tmp_path / "blocking"
+    work.mkdir()
+    manager = Blocking(str(work), max_pipelines=2, cache_bytes=10 ** 12,
+                       no_fetch=False, growing=False, max_retries=0,
+                       max_starts=99, starts_window=3600.0)
+
+    results = [manager.get(vid(20 + i)) for i in range(4)]
+    ok = [r for r in results if r[0] is not None]
+    refused = [r for r in results if r[0] is None]
+
+    assert len(ok) == 2
+    assert len(refused) == 2
+    assert all("pipeline cap" in r[1] for r in refused)
+    # A busy refusal must not consume start budget, or a burst of concurrent
+    # requests would exhaust the hourly allowance without fetching anything.
+    assert manager.counters["refused_busy"] == 2
+    assert len(manager.start_log) == 2
+
+
+def test_no_fetch_mode_refuses_everything(manager_factory):
+    manager = manager_factory(no_fetch=True, max_starts=99)
+    session, refusal = manager.get(vid(30))
+
+    assert session is None
+    assert "no-fetch" in refusal
+    assert manager.ran == []
+    assert len(manager.start_log) == 0

tests/test_reap.py

@@ -0,0 +1,201 @@
+"""Retention: the window, the min-keep floor, and the tombstones.
+
+These are the tests that matter most in this suite. Retention is the only part of
+ytstream that deletes things, and two of its rules exist because of measured
+behaviour rather than taste:
+
+* `min_keep_videos` exists because 52 of 117 real channels upload nothing in 30
+  days and would otherwise be empty, flickering Jellyfin series.
+* the `aged_out` tombstone exists because without it the poller re-materialises
+  everything the sweep just deleted, forever.
+"""
+
+from __future__ import annotations
+
+from datetime import timedelta
+
+import pytest
+
+from ytstream import discovery, reap, strm, util, videos
+
+from conftest import add_channel, add_video
+
+
+def _old(days: int) -> str:
+    return (util.today() - timedelta(days=days)).isoformat()
+
+
+@pytest.fixture()
+def materialised(conn, settings, media_root, channel):
+    """Put videos on disk with a spread of ages. Returns their ids, newest first."""
+    counter = {"n": 0}
+
+    def build(ages: list[int], chan=None):
+        chan = chan or channel
+        ids = []
+        for age in ages:
+            video_id = f"vid{counter['n']:08d}"
+            counter["n"] += 1
+            add_video(conn, chan["id"], video_id, upload_date=_old(age))
+            strm.materialise(conn, settings, chan, videos.get(conn, video_id))
+            ids.append(video_id)
+        return ids
+
+    return build
+
+
+def test_video_inside_the_window_is_kept(conn, settings, materialised):
+    materialised([1])
+    assert reap.candidates(conn, settings) == []
+
+
+def test_video_outside_the_window_is_deleted(conn, settings, materialised, media_root):
+    settings.set("min_keep_videos", "0")
+    [video_id] = materialised([99])
+
+    path = media_root / videos.get(conn, video_id)["rel_path"]
+    assert path.exists()
+
+    result = reap.run(conn, settings)
+
+    assert result["aged_out"] == 1
+    assert not path.exists()
+    assert videos.get(conn, video_id)["state"] == videos.AGED_OUT
+
+
+def test_deleting_removes_the_sidecars_too(conn, settings, materialised, media_root):
+    settings.set("min_keep_videos", "0")
+    [video_id] = materialised([99])
+    strm_path = media_root / videos.get(conn, video_id)["rel_path"]
+    stem = strm_path.name[: -len(".strm")]
+    nfo_path = strm_path.with_name(stem + ".nfo")
+    assert nfo_path.exists()
+
+    reap.run(conn, settings)
+
+    assert not nfo_path.exists()
+    assert not strm_path.exists()
+
+
+def test_min_keep_videos_protects_the_newest_regardless_of_age(
+    conn, settings, materialised
+):
+    """The measured common case: a channel whose only videos are all ancient."""
+    settings.set("min_keep_videos", "5")
+    materialised([400, 500, 600, 700, 800])
+
+    assert reap.candidates(conn, settings) == []
+    assert reap.run(conn, settings)["aged_out"] == 0
+
+
+def test_min_keep_videos_releases_once_enough_newer_ones_exist(
+    conn, settings, materialised
+):
+    """Six old videos with a floor of five: exactly the oldest one goes."""
+    settings.set("min_keep_videos", "5")
+    ids = materialised([100, 200, 300, 400, 500, 600])
+    oldest = ids[-1]
+
+    assert reap.run(conn, settings)["aged_out"] == 1
+
+    assert videos.get(conn, oldest)["state"] == videos.AGED_OUT
+    for kept in ids[:-1]:
+        assert videos.get(conn, kept)["state"] == videos.MATERIALISED
+
+
+def test_min_keep_floor_counts_per_channel_not_globally(
+    conn, settings, media_root, channel, materialised
+):
+    other = add_channel(conn, "UCzzzzzzzzzzzzzzzzzzzzzz", "Other", "Other")
+    settings.set("min_keep_videos", "2")
+    materialised([300, 400], chan=channel)
+    materialised([300, 400], chan=other)
+
+    # Two channels, two videos each, floor of two: nothing is eligible. If the
+    # floor were global, two of the four would be deleted.
+    assert reap.candidates(conn, settings) == []
+
+
+def test_channel_retention_override_beats_the_global_setting(
+    conn, settings, media_root, channel, materialised
+):
+    settings.set("min_keep_videos", "0")
+    settings.set("retention_days", "365")
+    materialised([90])
+    assert reap.candidates(conn, settings) == []
+
+    with conn:
+        conn.execute("UPDATE channel SET retention_days = 30 WHERE id = ?",
+                     (channel["id"],))
+    assert len(reap.candidates(conn, settings)) == 1
+
+
+def test_aged_out_row_is_a_tombstone_the_poller_will_not_revive(
+    conn, settings, media_root, channel, materialised
+):
+    """The failure this prevents: sweep deletes, poll re-adds, forever."""
+    settings.set("min_keep_videos", "0")
+    [video_id] = materialised([99])
+    reap.run(conn, settings)
+    assert videos.get(conn, video_id)["state"] == videos.AGED_OUT
+
+    # The video is still in the feed — YouTube has no idea we deleted it.
+    entry = {"video_id": video_id, "title": "Video", "published": util.today(),
+             "published_at": None}
+    outcome = discovery._record(
+        conn, channel, entry, videos.SOURCE_UULF, util.today() - timedelta(days=30)
+    )
+
+    assert outcome == "known"
+    assert videos.get(conn, video_id)["state"] == videos.AGED_OUT
+
+
+def test_rescan_revives_skipped_old_but_never_aged_out(
+    conn, settings, media_root, channel, materialised
+):
+    settings.set("min_keep_videos", "0")
+    [gone] = materialised([99])
+    reap.run(conn, settings)
+    add_video(conn, channel["id"], "skippedold1", upload_date=_old(40),
+              state=videos.SKIPPED_OLD)
+
+    settings.set("retention_days", "365")
+    revived = discovery.rescan_channel(conn, settings, channel)
+
+    assert revived == 1
+    assert videos.get(conn, "skippedold1")["state"] == videos.LISTED
+    assert videos.get(conn, gone)["state"] == videos.AGED_OUT
+
+
+def test_empty_season_directory_is_pruned(conn, settings, materialised, media_root):
+    settings.set("min_keep_videos", "0")
+    [video_id] = materialised([99])
+    season_dir = (media_root / videos.get(conn, video_id)["rel_path"]).parent
+    assert season_dir.is_dir()
+
+    reap.run(conn, settings)
+
+    assert not season_dir.exists()
+    # The channel directory survives: it still holds tvshow.nfo and artwork, and
+    # an active subscription should not vanish from Jellyfin between uploads.
+    assert season_dir.parent.is_dir()
+
+
+def test_prune_never_climbs_past_the_media_root(media_root):
+    nested = media_root / "Chan" / "Season 2026"
+    nested.mkdir(parents=True)
+    removed = util.prune_empty_dirs(nested, media_root)
+    assert removed == 2
+    assert media_root.is_dir()
+
+
+def test_row_without_rel_path_still_gets_a_tombstone(conn, settings, channel):
+    """A video whose files vanished underneath us must not be retried forever."""
+    settings.set("min_keep_videos", "0")
+    add_video(conn, channel["id"], "orphan0001", upload_date=_old(99),
+              state=videos.MATERIALISED)
+    row = videos.get(conn, "orphan0001")
+    assert row["rel_path"] is None
+
+    assert reap.delete_video(conn, row) is False
+    assert videos.get(conn, "orphan0001")["state"] == videos.AGED_OUT

tests/test_runner.py

@@ -0,0 +1,232 @@
+"""Orchestration: ordering, the lock, and what turns the cron check red."""
+
+from __future__ import annotations
+
+import pytest
+
+from ytstream import config, discovery, jellyfin, reap, runner, subsync, videos
+
+from conftest import add_video
+
+
+@pytest.fixture()
+def quiet_jellyfin(monkeypatch):
+    """Jellyfin refresh is best effort; record calls instead of making them."""
+    calls = []
+    monkeypatch.setattr(jellyfin.Jellyfin, "refresh", lambda self: calls.append(1))
+    return calls
+
+
+@pytest.fixture()
+def no_thumbs(monkeypatch):
+    from ytstream import strm
+
+    monkeypatch.setattr(strm, "fetch_thumbnail", lambda *a, **k: False)
+
+
+# ----------------------------------------------------------------------- lock
+
+
+def test_lock_is_exclusive(tmp_path):
+    path = tmp_path / "run.lock"
+    with runner.exclusive_lock(path):
+        with pytest.raises(runner.AlreadyRunning):
+            with runner.exclusive_lock(path):
+                pass  # pragma: no cover
+
+
+def test_lock_is_released_after_use(tmp_path):
+    path = tmp_path / "run.lock"
+    with runner.exclusive_lock(path):
+        pass
+    with runner.exclusive_lock(path):
+        pass
+
+
+def test_lock_is_released_even_when_the_body_raises(tmp_path):
+    path = tmp_path / "run.lock"
+    with pytest.raises(ValueError):
+        with runner.exclusive_lock(path):
+            raise ValueError("boom")
+    with runner.exclusive_lock(path):
+        pass
+
+
+# --------------------------------------------------------------- materialising
+
+
+def test_materialise_all_writes_everything_listed(conn, settings, media_root,
+                                                  channel, no_thumbs):
+    for index in range(3):
+        add_video(conn, channel["id"], f"vid{index:08d}")
+
+    stats = runner.materialise_all(conn, settings)
+
+    assert stats["materialised"] == 3
+    assert stats["errors"] == 0
+    assert videos.queue_depth(conn) == 0
+
+
+def test_tvshow_nfo_is_written_once_per_channel(conn, settings, media_root,
+                                                channel, no_thumbs):
+    for index in range(3):
+        add_video(conn, channel["id"], f"vid{index:08d}")
+
+    stats = runner.materialise_all(conn, settings)
+
+    assert stats["shows"] == 1
+    assert (media_root / "clabretro" / "tvshow.nfo").exists()
+
+
+def test_channel_with_nothing_in_the_window_creates_no_directory(
+    conn, settings, media_root, channel, no_thumbs
+):
+    """52 of 117 measured channels are in this state, and an empty series in
+    Jellyfin looks like a bug rather than a quiet channel."""
+    add_video(conn, channel["id"], "vid00000001", state=videos.SKIPPED_OLD)
+
+    runner.materialise_all(conn, settings)
+
+    assert not (media_root / "clabretro").exists()
+
+
+def test_materialise_all_honours_the_limit(conn, settings, media_root, channel,
+                                           no_thumbs):
+    for index in range(5):
+        add_video(conn, channel["id"], f"vid{index:08d}")
+    stats = runner.materialise_all(conn, settings, limit=2)
+    assert stats["materialised"] == 2
+
+
+def test_materialise_skips_a_video_whose_channel_vanished(conn, settings,
+                                                         media_root, channel,
+                                                         no_thumbs):
+    add_video(conn, channel["id"], "vid00000001")
+    # Simulate the channel being unsubscribed between claiming and writing, with
+    # foreign keys off so the row survives to be found.
+    conn.execute("PRAGMA foreign_keys = OFF")
+    with conn:
+        conn.execute("DELETE FROM channel WHERE id = ?", (channel["id"],))
+
+    stats = runner.materialise_all(conn, settings)
+
+    assert stats["materialised"] == 0
+    assert stats["errors"] == 0
+
+
+# ------------------------------------------------------------------ full cycle
+
+
+def test_run_order_is_sync_poll_materialise_reap(conn, settings, media_root,
+                                                 channel, monkeypatch,
+                                                 quiet_jellyfin):
+    order = []
+    monkeypatch.setattr(subsync, "sync_all",
+                        lambda *a: order.append("sync") or {"added": 0, "queued": 0,
+                                                            "removed": 0, "refused": 0,
+                                                            "sources": 0,
+                                                            "pending_removal": 0})
+    monkeypatch.setattr(discovery, "poll_all",
+                        lambda *a: order.append("poll") or {"queued": 0})
+    monkeypatch.setattr(runner, "materialise_all",
+                        lambda *a, **k: order.append("materialise") or
+                        {"materialised": 0, "shows": 0, "errors": 0})
+    monkeypatch.setattr(reap, "run",
+                        lambda *a: order.append("reap") or {"aged_out": 0})
+
+    runner.run(conn, settings)
+
+    assert order == ["sync", "poll", "materialise", "reap"]
+
+
+def test_single_channel_run_skips_the_sync(conn, settings, channel, monkeypatch,
+                                           quiet_jellyfin):
+    """A targeted run is an operator action, not a mirror pass."""
+    monkeypatch.setattr(subsync, "sync_all",
+                        lambda *a: pytest.fail("sync should not run"))
+    monkeypatch.setattr(discovery, "poll_all", lambda *a: {"queued": 0})
+    monkeypatch.setattr(reap, "run", lambda *a: {"aged_out": 0})
+
+    result = runner.run(conn, settings, channel_pk=channel["id"])
+
+    assert "sync" not in result
+
+
+def test_run_records_last_run_at(conn, settings, channel, monkeypatch,
+                                 quiet_jellyfin):
+    monkeypatch.setattr(subsync, "sync_all", lambda *a: {"added": 0, "queued": 0,
+                                                         "removed": 0, "refused": 0,
+                                                         "sources": 0,
+                                                         "pending_removal": 0})
+    monkeypatch.setattr(discovery, "poll_all", lambda *a: {"queued": 0})
+    monkeypatch.setattr(reap, "run", lambda *a: {"aged_out": 0})
+
+    runner.run(conn, settings)
+
+    assert settings.raw("last_run_at")
+
+
+def test_jellyfin_is_refreshed_only_when_the_tree_changed(
+    conn, settings, media_root, channel, monkeypatch, quiet_jellyfin, no_thumbs
+):
+    monkeypatch.setattr(subsync, "sync_all", lambda *a: {"added": 0, "queued": 0,
+                                                         "removed": 0, "refused": 0,
+                                                         "sources": 0,
+                                                         "pending_removal": 0})
+    monkeypatch.setattr(discovery, "poll_all", lambda *a: {"queued": 0})
+
+    runner.run(conn, settings)
+    assert quiet_jellyfin == []
+
+    add_video(conn, channel["id"], "vid00000001")
+    runner.run(conn, settings)
+    assert len(quiet_jellyfin) == 1
+
+
+# ------------------------------------------------------------------ exit codes
+
+
+def test_refused_sync_turns_the_check_red():
+    """A silently-broken mirror is the worst outcome available: nothing looks
+    wrong until someone asks why a channel never appeared."""
+    assert runner.exit_code({"sync": {"refused": 1}}) == 1
+
+
+def test_materialise_errors_turn_the_check_red():
+    assert runner.exit_code({"materialise": {"errors": 2}}) == 1
+
+
+def test_poll_failures_alone_do_not_turn_the_check_red():
+    """Two of 119 measured channels fail permanently — terminated or private."""
+    assert runner.exit_code({"poll": {"failed": 2}}) == 0
+
+
+def test_clean_run_is_zero():
+    assert runner.exit_code({"sync": {"refused": 0}, "poll": {"failed": 0},
+                             "materialise": {"errors": 0}}) == 0
+
+
+# ------------------------------------------------------------------- summarise
+
+
+def test_summarise_mentions_a_refusal():
+    text = runner.summarise({
+        "sync": {"added": 0, "queued": 0, "removed": 0, "refused": 1},
+        "poll": {"channels": 5, "queued": 0},
+        "materialise": {"materialised": 0},
+        "reap": {"aged_out": 0},
+    })
+    assert "SYNC_REFUSED" in text
+
+
+def test_summarise_is_a_single_line():
+    text = runner.summarise({
+        "sync": {"added": 1, "queued": 2, "removed": 3, "refused": 0},
+        "poll": {"channels": 119, "queued": 4, "shorts": 5, "live": 6, "failed": 2},
+        "materialise": {"materialised": 7, "errors": 0},
+        "reap": {"aged_out": 8},
+    })
+    assert "\n" not in text
+    assert "channels=119" in text
+    assert "materialised=7" in text
+    assert "aged_out=8" in text

tests/test_settings.py

@@ -0,0 +1,125 @@
+"""Typed settings accessors and form validation."""
+
+import pytest
+
+from ytstream import settings as settings_module
+from ytstream.settings import DEFAULTS, Settings, validate, validate_all
+
+
+class TestAccessors:
+    def test_missing_key_returns_the_default(self, settings):
+        assert settings.get_int("retention_days") == 30
+        assert settings.get_int("min_keep_videos") == 5
+        assert settings.get_str("proxy_base_url") == "http://127.0.0.1:8099"
+
+    def test_unknown_key_never_raises(self, settings):
+        assert settings.get_str("no_such_key") == ""
+        assert settings.get_int("no_such_key") == 0
+
+    def test_set_then_get(self, settings):
+        settings.set("retention_days", "30")
+        assert settings.get_int("retention_days") == 30
+
+    def test_set_overwrites(self, settings):
+        settings.set("retention_days", "30")
+        settings.set("retention_days", "45")
+        assert settings.get_int("retention_days") == 45
+
+    def test_corrupt_integer_falls_back_to_the_default(self, settings):
+        settings.set("retention_days", "not a number")
+        assert settings.get_int("retention_days") == 30
+
+    def test_reads_are_not_cached(self, conn, settings):
+        """Rotating the API key must be nothing more than saving a new value —
+        no restart of the hourly job or the admin server."""
+        settings.set("youtube_api_key", "first")
+        assert settings.get_str("youtube_api_key") == "first"
+
+        # A second Settings object standing in for the other process.
+        other = Settings(conn)
+        other.set("youtube_api_key", "second")
+
+        assert settings.get_str("youtube_api_key") == "second"
+
+    def test_all_editable_covers_every_default(self, settings):
+        assert set(settings.all_editable()) == set(DEFAULTS)
+
+    def test_secrets_are_not_editable(self):
+        for key in settings_module.SECRET_KEYS:
+            assert key not in settings_module.EDITABLE
+
+    def test_both_api_keys_are_masked(self):
+        assert "youtube_api_key" in settings_module.MASKED_KEYS
+        assert "jellyfin_api_key" in settings_module.MASKED_KEYS
+
+    def test_download_era_settings_are_gone(self):
+        for gone in ("disk_cap_gb", "write_subs", "sub_langs", "sponsorblock_mark",
+                     "max_attempts", "backfill_days"):
+            assert gone not in DEFAULTS
+
+
+class TestValidation:
+    @pytest.mark.parametrize(
+        "key, value",
+        [
+            ("retention_days", "30"),
+            ("min_keep_videos", "0"),
+            ("min_keep_videos", "5"),
+            ("backfill_max_videos", "300"),
+            ("max_height", "1080"),
+            ("min_duration_seconds", "120"),
+            ("subsync_max_new", "25"),
+            ("subsync_missing_threshold", "3"),
+            ("jellyfin_url", "http://127.0.0.1:8096"),
+            ("pot_provider_url", "https://example.com:4416"),
+            ("proxy_base_url", "http://127.0.0.1:8099"),
+            ("youtube_api_key", ""),
+        ],
+    )
+    def test_accepts_good_values(self, key, value):
+        ok, _ = validate(key, value)
+        assert ok
+
+    @pytest.mark.parametrize(
+        "key, value",
+        [
+            ("retention_days", "abc"),
+            ("retention_days", "0"),
+            ("retention_days", "-5"),
+            ("min_keep_videos", "-1"),
+            ("max_height", "10"),
+            ("min_duration_seconds", "-1"),
+            ("jellyfin_url", "not-a-url"),
+            ("jellyfin_url", "ftp://host/"),
+            ("jellyfin_url", "http://"),
+            ("proxy_base_url", "nonsense"),
+        ],
+    )
+    def test_rejects_bad_values(self, key, value):
+        ok, message = validate(key, value)
+        assert not ok
+        assert message
+
+    def test_zero_missing_threshold_is_rejected(self):
+        """Zero would unsubscribe on the first absent response, which is exactly
+        the failure the threshold exists to prevent."""
+        ok, message = validate("subsync_missing_threshold", "0")
+        assert not ok
+        assert message
+
+    def test_validate_all_reports_each_bad_field(self):
+        errors = validate_all(
+            {"retention_days": "abc", "max_height": "1080", "jellyfin_url": "nope"}
+        )
+        assert set(errors) == {"retention_days", "jellyfin_url"}
+
+    def test_validate_all_ignores_unknown_keys(self):
+        assert validate_all({"not_a_setting": "x"}) == {}
+
+    def test_empty_api_keys_are_acceptable(self):
+        assert validate("jellyfin_api_key", "")[0]
+        assert validate("youtube_api_key", "")[0]
+
+    def test_whitespace_is_tolerated(self):
+        ok, _ = validate("retention_days", "  21  ")
+        assert ok

tests/test_strm.py

@@ -0,0 +1,260 @@
+"""Materialising: .strm contents, NFO sidecars, naming, and idempotency."""
+
+from __future__ import annotations
+
+from datetime import date
+
+import pytest
+
+from ytstream import config, strm, videos
+
+from conftest import add_video
+
+
+@pytest.fixture()
+def video(conn, channel):
+    return add_video(conn, channel["id"], "dQw4w9WgXcQ",
+                     title="A Video: With/Punctuation",
+                     upload_date="2026-08-12", duration=1337)
+
+
+@pytest.fixture()
+def no_thumbs(monkeypatch):
+    """Thumbnails are a network fetch; every test here runs without one."""
+    monkeypatch.setattr(strm, "fetch_thumbnail", lambda *a, **k: False)
+
+
+def test_strm_contains_only_the_proxy_url(conn, settings, media_root, channel,
+                                          video, no_thumbs):
+    result = strm.materialise(conn, settings, channel, video)
+
+    path = media_root / result["rel_path"]
+    assert path.read_text() == "http://127.0.0.1:8099/watch/dQw4w9WgXcQ"
+    # No trailing newline, and nothing else in the file.
+    assert path.read_bytes() == b"http://127.0.0.1:8099/watch/dQw4w9WgXcQ"
+
+
+def test_strm_url_follows_the_configured_base(conn, settings, media_root, channel,
+                                              video, no_thumbs):
+    settings.set("proxy_base_url", "http://127.0.0.1:9999/")
+    result = strm.materialise(conn, settings, channel, video)
+    assert (media_root / result["rel_path"]).read_text().startswith(
+        "http://127.0.0.1:9999/watch/"
+    )
+
+
+def test_layout_matches_the_naming_scheme(conn, settings, media_root, channel,
+                                          video, no_thumbs):
+    result = strm.materialise(conn, settings, channel, video)
+
+    # Season = upload year, episode = MMDD*10 + ordinal.
+    assert result["season"] == 2026
+    assert result["episode"] == 8120
+    assert result["rel_path"] == (
+        "clabretro/Season 2026/"
+        "clabretro - S2026E8120 - A Video With Punctuation [dQw4w9WgXcQ].strm"
+    )
+
+
+def test_nfo_is_written_alongside(conn, settings, media_root, channel, video,
+                                  no_thumbs):
+    result = strm.materialise(conn, settings, channel, video)
+    nfo_path = (media_root / result["rel_path"]).with_suffix(".nfo")
+
+    text = nfo_path.read_text()
+
+    assert "<season>2026</season>" in text
+    assert "<episode>8120</episode>" in text
+    assert "<aired>2026-08-12</aired>" in text
+    # durationinseconds is what stops a .strm episode showing a zero runtime
+    # before it has ever been played.
+    assert "<durationinseconds>1337</durationinseconds>" in text
+    assert 'type="youtube"' in text
+
+
+def test_nfo_has_no_streamdetails(conn, settings, media_root, channel, video,
+                                  no_thumbs):
+    """Pre-seeding them was measured to change nothing about Jellyfin probing."""
+    result = strm.materialise(conn, settings, channel, video)
+    text = (media_root / result["rel_path"]).with_suffix(".nfo").read_text()
+    assert "streamdetails" not in text
+    assert "fileinfo" not in text
+
+
+def test_materialise_is_idempotent(conn, settings, media_root, channel, video,
+                                   no_thumbs):
+    first = strm.materialise(conn, settings, channel, video)
+    before = (media_root / first["rel_path"]).read_bytes()
+
+    second = strm.materialise(
+        conn, settings, channel, videos.get(conn, "dQw4w9WgXcQ")
+    )
+
+    assert second["rel_path"] == first["rel_path"]
+    assert (media_root / second["rel_path"]).read_bytes() == before
+
+
+def test_row_is_marked_materialised(conn, settings, media_root, channel, video,
+                                    no_thumbs):
+    strm.materialise(conn, settings, channel, video)
+    row = videos.get(conn, "dQw4w9WgXcQ")
+    assert row["state"] == videos.MATERIALISED
+    assert row["rel_path"]
+    assert row["materialised_at"]
+
+
+def test_episode_ordinals_increment_within_a_day(conn, settings, media_root,
+                                                 channel, no_thumbs):
+    for index in range(3):
+        row = add_video(conn, channel["id"], f"vid{index:08d}",
+                        upload_date="2026-08-12")
+        result = strm.materialise(conn, settings, channel, row)
+        assert result["episode"] == 8120 + index
+
+
+def test_ordinals_are_stable_when_an_earlier_video_ages_out(
+    conn, settings, media_root, channel, no_thumbs
+):
+    """Aged-out rows keep their episode number, so later ordinals never shift."""
+    first = add_video(conn, channel["id"], "vid00000001", upload_date="2026-08-12")
+    strm.materialise(conn, settings, channel, first)
+    videos.mark_aged_out(conn, "vid00000001")
+
+    second = add_video(conn, channel["id"], "vid00000002", upload_date="2026-08-12")
+    result = strm.materialise(conn, settings, channel, second)
+
+    assert result["episode"] == 8121
+
+
+def test_title_falls_back_to_the_video_id(conn, settings, media_root, channel,
+                                          no_thumbs):
+    """Backfilled rows carry no title until the feed supplies one."""
+    row = add_video(conn, channel["id"], "vid00000001", title="",
+                    upload_date="2026-08-12")
+    result = strm.materialise(conn, settings, channel, row)
+    assert "vid00000001" in result["rel_path"]
+
+
+def test_write_show_creates_tvshow_nfo(media_root, channel):
+    strm.write_show(channel)
+    text = (media_root / "clabretro" / "tvshow.nfo").read_text()
+    assert "<title>clabretro</title>" in text
+    assert channel["channel_id"] in text
+
+
+def test_show_nfo_uses_the_channel_title_not_a_video_title(conn, media_root, channel):
+    """The channel/video join has `title` on both sides, and reading the wrong one
+    renamed every series after whichever episode happened to be first."""
+    add_video(conn, channel["id"], "vid00000001", title="Some Episode Title")
+    strm.write_show(channel)
+    text = (media_root / "clabretro" / "tvshow.nfo").read_text()
+    assert "Some Episode Title" not in text
+
+
+def test_remove_deletes_strm_and_sidecars(conn, settings, media_root, channel,
+                                          video, no_thumbs):
+    result = strm.materialise(conn, settings, channel, video)
+    strm_path = media_root / result["rel_path"]
+    stem = strm_path.name[: -len(".strm")]
+    thumb = strm_path.with_name(stem + "-thumb.jpg")
+    thumb.write_bytes(b"x" * 2000)
+
+    removed = strm.remove(videos.get(conn, "dQw4w9WgXcQ"))
+
+    assert removed == 3          # .strm, .nfo, -thumb.jpg
+    assert not strm_path.exists()
+    assert not thumb.exists()
+
+
+def test_remove_leaves_files_it_does_not_own(conn, settings, media_root, channel,
+                                             video, no_thumbs):
+    result = strm.materialise(conn, settings, channel, video)
+    season_dir = (media_root / result["rel_path"]).parent
+    stranger = season_dir / "someone-elses-file.txt"
+    stranger.write_text("not ours")
+
+    strm.remove(videos.get(conn, "dQw4w9WgXcQ"))
+
+    assert stranger.exists()
+
+
+def test_remove_channel_tree(conn, media_root, channel):
+    tree = media_root / "clabretro"
+    (tree / "Season 2026").mkdir(parents=True)
+    (tree / "tvshow.nfo").write_text("<tvshow/>")
+
+    assert strm.remove_channel_tree(channel) is True
+    assert not tree.exists()
+
+
+def test_remove_channel_tree_refuses_the_media_root(conn, media_root):
+    from conftest import add_channel
+
+    row = add_channel(conn, "UC" + "q" * 22, "Blank", "")
+    keep = media_root / "keep"
+    keep.mkdir()
+
+    assert strm.remove_channel_tree(row) is False
+    assert keep.exists()
+
+
+def test_fetch_thumbnail_rejects_the_grey_placeholder(monkeypatch, tmp_path):
+    """YouTube serves a tiny placeholder rather than a 404 for missing maxres."""
+    import urllib.request
+
+    class Response:
+        status = 200
+
+        def __init__(self, payload):
+            self.payload = payload
+
+        def read(self):
+            return self.payload
+
+        def __enter__(self):
+            return self
+
+        def __exit__(self, *exc):
+            return False
+
+    calls = []
+
+    def fake_open(request, timeout=None):
+        calls.append(request.full_url)
+        # maxres returns the placeholder; hq returns something real.
+        return Response(b"x" * 120 if "maxres" in request.full_url else b"y" * 5000)
+
+    monkeypatch.setattr(urllib.request, "urlopen", fake_open)
+    destination = tmp_path / "out-thumb.jpg"
+
+    assert strm.fetch_thumbnail("dQw4w9WgXcQ", destination) is True
+    assert destination.read_bytes() == b"y" * 5000
+    assert len(calls) == 2
+
+
+def test_fetch_thumbnail_is_skipped_when_one_already_exists(monkeypatch, tmp_path):
+    import urllib.request
+
+    monkeypatch.setattr(urllib.request, "urlopen",
+                        lambda *a, **k: pytest.fail("should not fetch"))
+    existing = tmp_path / "out-thumb.jpg"
+    existing.write_bytes(b"cached")
+
+    assert strm.fetch_thumbnail("dQw4w9WgXcQ", existing) is True
+
+
+def test_untitled_video_does_not_get_its_id_written_back_as_a_title(
+    conn, settings, media_root, channel, no_thumbs
+):
+    """Writing the fallback back to the database makes the row look titled, which
+    permanently disables the title repair in discovery._record. That shipped once
+    and left five of twenty episodes named after their video ids."""
+    row = add_video(conn, channel["id"], "vid00000001", title="",
+                    upload_date="2026-08-12")
+
+    result = strm.materialise(conn, settings, channel, row)
+
+    # The filename falls back to the id...
+    assert "vid00000001]" in result["rel_path"]
+    # ...but the row stays untitled, so a later feed poll can still repair it.
+    assert videos.get(conn, "vid00000001")["title"] == ""

tests/test_subsync.py

@@ -0,0 +1,366 @@
+"""Subscription mirroring, and above all its refusals.
+
+The sync is authoritative in both directions and the removal half deletes a
+channel's whole tree, so most of what needs pinning down here is what it does
+with *bad* data. Every one of these failure modes looks identical to "he
+unsubscribed from everything" on the wire:
+
+    403 subscriptionForbidden    he re-ticked the privacy box
+    network error                susan's link dropped
+    200 with zero items          could be true, could be a broken response
+
+None of them may delete anything. The tests below are the reason that claim can
+be made with a straight face.
+"""
+
+from __future__ import annotations
+
+import pytest
+
+from ytstream import api, subsync, videos
+
+from conftest import CHANNEL_ID, FakeApi, add_channel, add_video, patch_api
+
+BROTHER = "UCPcTWaLV8zwx4WP4QExHj4Q"
+
+
+@pytest.fixture()
+def source(conn):
+    key = subsync.add_source(conn, channel_id=BROTHER, label="C Flux")
+    with conn:
+        conn.execute("UPDATE source SET imported = 1 WHERE key = ?", (key,))
+    return subsync.get_source(conn, key)
+
+
+@pytest.fixture()
+def fresh_source(conn):
+    """A source that has never imported — the first-sync path."""
+    key = subsync.add_source(conn, channel_id=BROTHER, label="C Flux")
+    return subsync.get_source(conn, key)
+
+
+def sub(channel_id, title):
+    return {"channel_id": channel_id, "title": title}
+
+
+def _fake(monkeypatch, **kwargs):
+    return patch_api(monkeypatch, subsync, FakeApi(**kwargs))
+
+
+# ------------------------------------------------------------------- additions
+
+
+def test_first_sync_queues_everything_and_adds_nothing(
+    conn, settings, fresh_source, monkeypatch
+):
+    """119 subscriptions would trip any cap, so day one is approval-only."""
+    _fake(monkeypatch, subs=[sub(f"UC{i:022d}", f"Chan {i}") for i in range(40)])
+
+    stats = subsync.sync_source(conn, settings, fresh_source)
+
+    assert stats["added"] == 0
+    assert stats["queued"] == 40
+    assert conn.execute("SELECT COUNT(*) FROM channel").fetchone()[0] == 0
+    assert len(subsync.pending(conn)) == 40
+    # And it does not queue them again on the next pass.
+    assert subsync.get_source(conn, fresh_source["key"])["imported"] == 1
+
+
+def test_second_sync_adds_within_the_cap(conn, settings, source, monkeypatch):
+    settings.set("subsync_max_new", "25")
+    _fake(monkeypatch, subs=[sub("UCaaaaaaaaaaaaaaaaaaaaaa", "Alpha")],
+          channel={"channel_id": "UCaaaaaaaaaaaaaaaaaaaaaa", "title": "Alpha",
+                   "description": "", "handle": None, "avatar_url": None})
+
+    stats = subsync.sync_source(conn, settings, source)
+
+    assert stats["added"] == 1
+    row = conn.execute("SELECT * FROM channel").fetchone()
+    assert row["title"] == "Alpha"
+    assert row["source"] == "youtube"
+
+
+def test_burst_over_the_cap_adds_nothing_and_queues_all(
+    conn, settings, source, monkeypatch
+):
+    settings.set("subsync_max_new", "3")
+    _fake(monkeypatch, subs=[sub(f"UC{i:022d}", f"Chan {i}") for i in range(10)])
+
+    stats = subsync.sync_source(conn, settings, source)
+
+    assert stats["added"] == 0
+    assert stats["queued"] == 10
+    assert conn.execute("SELECT COUNT(*) FROM channel").fetchone()[0] == 0
+
+
+def test_rejected_channels_are_never_queued_again(
+    conn, settings, fresh_source, monkeypatch
+):
+    entries = [sub("UCaaaaaaaaaaaaaaaaaaaaaa", "Alpha")]
+    _fake(monkeypatch, subs=entries)
+    subsync.sync_source(conn, settings, fresh_source)
+
+    queued = subsync.pending(conn)
+    assert len(queued) == 1
+    subsync.resolve(conn, [queued[0]["id"]], "rejected")
+
+    source = subsync.get_source(conn, fresh_source["key"])
+    stats = subsync.sync_source(conn, settings, source)
+
+    assert stats["queued"] == 0
+    assert stats["added"] == 0
+    assert subsync.pending(conn) == []
+
+
+def test_approving_subscribes(conn, settings, fresh_source, monkeypatch):
+    _fake(monkeypatch, subs=[sub("UCaaaaaaaaaaaaaaaaaaaaaa", "Alpha")],
+          channel={"channel_id": "UCaaaaaaaaaaaaaaaaaaaaaa", "title": "Alpha",
+                   "description": "Desc", "handle": "@alpha", "avatar_url": None})
+    subsync.sync_source(conn, settings, fresh_source)
+    queued = subsync.pending(conn)
+
+    stats = subsync.approve(conn, settings, [row["id"] for row in queued])
+
+    assert stats == {"added": 1, "failed": 0}
+    assert conn.execute("SELECT COUNT(*) FROM channel").fetchone()[0] == 1
+    assert subsync.pending(conn) == []
+
+
+# -------------------------------------------------------------------- refusals
+
+
+def test_private_subscriptions_change_nothing(conn, settings, source, monkeypatch):
+    add_channel(conn, CHANNEL_ID, "Existing", "Existing")
+    _fake(monkeypatch, raises=api.SubscriptionsPrivate(
+        403, "subscriptionForbidden", "not allowed"))
+
+    stats = subsync.sync_source(conn, settings, source)
+
+    assert stats["refused"]
+    assert "private" in stats["refused"]
+    assert stats["removed"] == 0
+    assert conn.execute("SELECT COUNT(*) FROM channel").fetchone()[0] == 1
+    # The channel's miss counter must not move either, or three consecutive
+    # outages would delete the library without a single healthy response.
+    assert conn.execute("SELECT missing_syncs FROM channel").fetchone()[0] == 0
+
+
+def test_empty_response_is_treated_as_suspect(conn, settings, source, monkeypatch):
+    """A genuinely empty list and a broken one are indistinguishable, so assume
+    the harmless reading."""
+    add_channel(conn, CHANNEL_ID, "Existing", "Existing")
+    _fake(monkeypatch, subs=[])
+
+    stats = subsync.sync_source(conn, settings, source)
+
+    assert stats["refused"]
+    assert stats["removed"] == 0
+    assert conn.execute("SELECT COUNT(*) FROM channel").fetchone()[0] == 1
+
+
+def test_api_not_configured_changes_nothing(conn, settings, source, monkeypatch):
+    add_channel(conn, CHANNEL_ID, "Existing", "Existing")
+    _fake(monkeypatch, raises=api.NotConfigured(403, "forbidden", "blocked"))
+
+    stats = subsync.sync_source(conn, settings, source)
+
+    assert stats["refused"]
+    assert conn.execute("SELECT COUNT(*) FROM channel").fetchone()[0] == 1
+
+
+def test_network_error_changes_nothing(conn, settings, source, monkeypatch):
+    add_channel(conn, CHANNEL_ID, "Existing", "Existing")
+    _fake(monkeypatch, raises=api.ApiError(0, "network", "connection reset"))
+
+    stats = subsync.sync_source(conn, settings, source)
+
+    assert stats["refused"]
+    assert conn.execute("SELECT COUNT(*) FROM channel").fetchone()[0] == 1
+
+
+def test_refusal_is_recorded_on_the_source(conn, settings, source, monkeypatch):
+    _fake(monkeypatch, raises=api.ApiError(0, "network", "boom"))
+    subsync.sync_source(conn, settings, source)
+
+    row = subsync.get_source(conn, source["key"])
+    assert row["last_sync_ok"] == 0
+    assert row["consecutive_failures"] == 1
+    assert "boom" in row["last_error"]
+
+
+def test_three_outages_in_a_row_still_delete_nothing(
+    conn, settings, source, monkeypatch
+):
+    """The threshold counts absences from healthy responses, not failures."""
+    add_channel(conn, CHANNEL_ID, "Existing", "Existing")
+    _fake(monkeypatch, raises=api.ApiError(0, "network", "down"))
+
+    for _ in range(5):
+        subsync.sync_source(conn, settings, subsync.get_source(conn, source["key"]))
+
+    assert conn.execute("SELECT COUNT(*) FROM channel").fetchone()[0] == 1
+
+
+# -------------------------------------------------------------------- removals
+
+
+def test_absence_counts_up_but_does_not_delete_below_the_threshold(
+    conn, settings, source, monkeypatch
+):
+    settings.set("subsync_missing_threshold", "3")
+    add_channel(conn, CHANNEL_ID, "Doomed", "Doomed")
+    _fake(monkeypatch, subs=[sub("UCaaaaaaaaaaaaaaaaaaaaaa", "Alpha")])
+    settings.set("subsync_max_new", "0")     # keep the addition path out of this
+
+    for expected in (1, 2):
+        stats = subsync.sync_source(
+            conn, settings, subsync.get_source(conn, source["key"])
+        )
+        assert stats["removed"] == 0
+        assert stats["pending_removal"] == 1
+        assert conn.execute(
+            "SELECT missing_syncs FROM channel WHERE dir_name = 'Doomed'"
+        ).fetchone()[0] == expected
+
+
+def test_deletion_happens_on_the_threshold_sync(
+    conn, settings, source, media_root, monkeypatch
+):
+    settings.set("subsync_missing_threshold", "3")
+    settings.set("subsync_max_new", "0")
+    doomed = add_channel(conn, CHANNEL_ID, "Doomed", "Doomed")
+    tree = media_root / "Doomed"
+    (tree / "Season 2026").mkdir(parents=True)
+    (tree / "tvshow.nfo").write_text("<tvshow/>")
+    add_video(conn, doomed["id"], "vid00000001")
+    _fake(monkeypatch, subs=[sub("UCaaaaaaaaaaaaaaaaaaaaaa", "Alpha")])
+
+    for _ in range(3):
+        stats = subsync.sync_source(
+            conn, settings, subsync.get_source(conn, source["key"])
+        )
+
+    assert stats["removed"] == 1
+    assert conn.execute("SELECT COUNT(*) FROM channel").fetchone()[0] == 0
+    assert not tree.exists()
+    # The videos went with it, via ON DELETE CASCADE.
+    assert conn.execute("SELECT COUNT(*) FROM video").fetchone()[0] == 0
+
+
+def test_reappearing_resets_the_counter(conn, settings, source, monkeypatch):
+    settings.set("subsync_missing_threshold", "3")
+    settings.set("subsync_max_new", "0")
+    add_channel(conn, CHANNEL_ID, "Flaky", "Flaky")
+
+    _fake(monkeypatch, subs=[sub("UCaaaaaaaaaaaaaaaaaaaaaa", "Alpha")])
+    subsync.sync_source(conn, settings, subsync.get_source(conn, source["key"]))
+    assert conn.execute("SELECT missing_syncs FROM channel").fetchone()[0] == 1
+
+    _fake(monkeypatch, subs=[sub(CHANNEL_ID, "Flaky")])
+    subsync.sync_source(conn, settings, subsync.get_source(conn, source["key"]))
+
+    assert conn.execute("SELECT missing_syncs FROM channel").fetchone()[0] == 0
+
+
+def test_manual_channels_are_never_removed(
+    conn, settings, source, media_root, monkeypatch
+):
+    settings.set("subsync_missing_threshold", "1")
+    settings.set("subsync_max_new", "0")
+    add_channel(conn, CHANNEL_ID, "Pinned", "Pinned", source="manual")
+    _fake(monkeypatch, subs=[sub("UCaaaaaaaaaaaaaaaaaaaaaa", "Alpha")])
+
+    for _ in range(3):
+        stats = subsync.sync_source(
+            conn, settings, subsync.get_source(conn, source["key"])
+        )
+
+    assert stats["removed"] == 0
+    assert conn.execute("SELECT COUNT(*) FROM channel").fetchone()[0] == 1
+    assert conn.execute("SELECT missing_syncs FROM channel").fetchone()[0] == 0
+
+
+def test_threshold_of_one_deletes_on_the_first_absence(
+    conn, settings, source, monkeypatch
+):
+    """Configurable so tests need not loop; the default stays at 3."""
+    settings.set("subsync_missing_threshold", "1")
+    settings.set("subsync_max_new", "0")
+    add_channel(conn, CHANNEL_ID, "Doomed", "Doomed")
+    _fake(monkeypatch, subs=[sub("UCaaaaaaaaaaaaaaaaaaaaaa", "Alpha")])
+
+    stats = subsync.sync_source(conn, settings, source)
+
+    assert stats["removed"] == 1
+
+
+def test_zero_threshold_is_clamped_to_one(conn, settings, source, monkeypatch):
+    """A stored 0 must not mean "delete before any absence is confirmed"."""
+    settings.set("subsync_missing_threshold", "0")
+    settings.set("subsync_max_new", "0")
+    add_channel(conn, CHANNEL_ID, "Doomed", "Doomed")
+    _fake(monkeypatch, subs=[sub("UCaaaaaaaaaaaaaaaaaaaaaa", "Alpha")])
+
+    stats = subsync.sync_source(conn, settings, source)
+
+    # Clamped to 1, so the first absence is enough — but it took one absence,
+    # not zero, and the row was actually observed missing.
+    assert stats["removed"] == 1
+    assert stats["seen"] == 1
+
+
+# --------------------------------------------------------------------- general
+
+
+def test_unsafe_channel_dir_is_not_deleted(conn, settings, media_root):
+    """A blank dir_name must never resolve the delete to the media root."""
+    row = add_channel(conn, CHANNEL_ID, "Bad", " ")
+    (media_root / "keepme").mkdir()
+
+    from ytstream import strm
+
+    assert strm.remove_channel_tree(row) is False
+    assert (media_root / "keepme").exists()
+
+
+def test_one_unresolvable_channel_does_not_abort_the_sync(
+    conn, settings, source, monkeypatch
+):
+    class Exploding(FakeApi):
+        def channel(self, channel_id):
+            if channel_id.endswith("bad"):
+                raise api.ApiError(500, "backendError", "boom")
+            return {"channel_id": channel_id, "title": "Fine", "description": "",
+                    "handle": None, "avatar_url": None}
+
+    monkeypatch.setattr(
+        subsync.channels, "subscribe_from_sync",
+        lambda conn, settings, cid, title: (_ for _ in ()).throw(RuntimeError("no"))
+        if cid.endswith("bad") else add_channel(conn, cid, title, title),
+    )
+    patch_api(monkeypatch, subsync,
+              FakeApi(subs=[sub("UC" + "a" * 19 + "bad", "Bad"),
+                            sub("UC" + "b" * 22, "Good")]))
+
+    stats = subsync.sync_source(conn, settings, source)
+
+    assert stats["added"] == 1
+    assert conn.execute("SELECT COUNT(*) FROM channel").fetchone()[0] == 1
+
+
+def test_sync_all_aggregates_and_counts_refusals(conn, settings, source, monkeypatch):
+    _fake(monkeypatch, raises=api.ApiError(0, "network", "boom"))
+    totals = subsync.sync_all(conn, settings)
+    assert totals["sources"] == 1
+    assert totals["refused"] == 1
+
+
+def test_disabled_source_is_skipped(conn, settings, source, monkeypatch):
+    with conn:
+        conn.execute("UPDATE source SET enabled = 0")
+    fake = _fake(monkeypatch, subs=[sub("UCaaaaaaaaaaaaaaaaaaaaaa", "Alpha")])
+
+    totals = subsync.sync_all(conn, settings)
+
+    assert totals["sources"] == 0
+    assert fake.subscription_calls == 0

tests/test_videos.py

@@ -0,0 +1,209 @@
+"""The video state machine and episode numbering."""
+
+from __future__ import annotations
+
+from datetime import date
+
+import pytest
+
+from ytstream import videos
+
+from conftest import add_channel, add_video
+
+
+def test_insert_and_get(conn, channel):
+    row = add_video(conn, channel["id"], "vid00000001")
+    assert row["state"] == videos.LISTED
+    assert videos.exists(conn, "vid00000001")
+    assert not videos.exists(conn, "nosuchvideo")
+
+
+def test_insert_is_idempotent(conn, channel):
+    add_video(conn, channel["id"], "vid00000001", title="First")
+    add_video(conn, channel["id"], "vid00000001", title="Second")
+    assert conn.execute("SELECT COUNT(*) FROM video").fetchone()[0] == 1
+    assert videos.get(conn, "vid00000001")["title"] == "First"
+
+
+def test_aged_out_is_terminal_and_never_revived():
+    assert videos.AGED_OUT in videos.TERMINAL
+    assert videos.AGED_OUT in videos.NEVER_REVIVE
+    # skipped_old is terminal but IS revivable by rescan; conflating the two
+    # would resurrect months of deleted episodes as new.
+    assert videos.SKIPPED_OLD in videos.TERMINAL
+    assert videos.SKIPPED_OLD not in videos.NEVER_REVIVE
+
+
+def test_no_download_era_states_survive():
+    """Materialising a text file cannot fail, so there is no retry ladder."""
+    for gone in ("pending", "downloading", "downloaded", "failed", "deferred"):
+        assert gone not in (
+            videos.LISTED, videos.MATERIALISED, videos.SKIPPED_SHORT,
+            videos.SKIPPED_LIVE, videos.SKIPPED_OLD, videos.AGED_OUT,
+        )
+
+
+def test_mark_aged_out_clears_rel_path_but_keeps_the_row(conn, channel):
+    add_video(conn, channel["id"], "vid00000001", state=videos.MATERIALISED)
+    with conn:
+        conn.execute("UPDATE video SET rel_path = 'a/b.strm' WHERE video_id = ?",
+                     ("vid00000001",))
+
+    videos.mark_aged_out(conn, "vid00000001")
+
+    row = videos.get(conn, "vid00000001")
+    assert row is not None
+    assert row["state"] == videos.AGED_OUT
+    assert row["rel_path"] is None
+    assert row["deleted_at"]
+
+
+def test_mark_materialised_records_everything(conn, channel):
+    add_video(conn, channel["id"], "vid00000001")
+    videos.mark_materialised(conn, "vid00000001", rel_path="a/b.strm", season=2026,
+                             episode=8120, upload_date="2026-08-12", duration=99,
+                             title="T")
+    row = videos.get(conn, "vid00000001")
+    assert (row["state"], row["season"], row["episode"], row["duration"]) == (
+        videos.MATERIALISED, 2026, 8120, 99
+    )
+
+
+def test_set_duration_alone(conn, channel):
+    add_video(conn, channel["id"], "vid00000001", duration=None)
+    videos.set_duration(conn, "vid00000001", 1234)
+    assert videos.get(conn, "vid00000001")["duration"] == 1234
+
+
+# ------------------------------------------------------------ episode numbers
+
+
+def test_first_episode_of_a_day(conn, channel):
+    add_video(conn, channel["id"], "vid00000001")
+    assert videos.next_episode(
+        conn, channel["id"], date(2026, 8, 12), "vid00000001"
+    ) == (2026, 8120)
+
+
+def test_ordinal_counts_existing_rows_for_that_day(conn, channel):
+    add_video(conn, channel["id"], "vid00000001")
+    with conn:
+        conn.execute("UPDATE video SET season = 2026, episode = 8120 "
+                     "WHERE video_id = ?", ("vid00000001",))
+    add_video(conn, channel["id"], "vid00000002")
+
+    assert videos.next_episode(
+        conn, channel["id"], date(2026, 8, 12), "vid00000002"
+    ) == (2026, 8121)
+
+
+def test_ordinal_ignores_other_days(conn, channel):
+    add_video(conn, channel["id"], "vid00000001")
+    with conn:
+        conn.execute("UPDATE video SET season = 2026, episode = 8110 "
+                     "WHERE video_id = ?", ("vid00000001",))
+    add_video(conn, channel["id"], "vid00000002")
+
+    assert videos.next_episode(
+        conn, channel["id"], date(2026, 8, 12), "vid00000002"
+    ) == (2026, 8120)
+
+
+def test_ordinal_ignores_other_channels(conn, channel):
+    other = add_channel(conn, "UC" + "z" * 22, "Other", "Other")
+    add_video(conn, other["id"], "vid00000001")
+    with conn:
+        conn.execute("UPDATE video SET season = 2026, episode = 8120 "
+                     "WHERE video_id = ?", ("vid00000001",))
+    add_video(conn, channel["id"], "vid00000002")
+
+    assert videos.next_episode(
+        conn, channel["id"], date(2026, 8, 12), "vid00000002"
+    ) == (2026, 8120)
+
+
+def test_ordinal_clamps_at_ten_uploads_a_day(conn, channel):
+    for index in range(10):
+        video_id = f"vid{index:08d}"
+        add_video(conn, channel["id"], video_id)
+        with conn:
+            conn.execute("UPDATE video SET season = 2026, episode = ? "
+                         "WHERE video_id = ?", (8120 + index, video_id))
+    add_video(conn, channel["id"], "vid00000099")
+
+    season, episode = videos.next_episode(
+        conn, channel["id"], date(2026, 8, 12), "vid00000099"
+    )
+
+    assert (season, episode) == (2026, 8129)
+
+
+def test_a_videos_own_row_does_not_bump_its_ordinal(conn, channel):
+    """Re-materialising must produce the same number, not the next one."""
+    add_video(conn, channel["id"], "vid00000001")
+    with conn:
+        conn.execute("UPDATE video SET season = 2026, episode = 8120 "
+                     "WHERE video_id = ?", ("vid00000001",))
+
+    assert videos.next_episode(
+        conn, channel["id"], date(2026, 8, 12), "vid00000001"
+    ) == (2026, 8120)
+
+
+# --------------------------------------------------------------------- queues
+
+
+def test_claim_listed_returns_only_listed_rows(conn, channel):
+    add_video(conn, channel["id"], "listed00001", state=videos.LISTED)
+    add_video(conn, channel["id"], "done0000001", state=videos.MATERIALISED)
+    add_video(conn, channel["id"], "short000001", state=videos.SKIPPED_SHORT)
+    add_video(conn, channel["id"], "aged0000001", state=videos.AGED_OUT)
+
+    rows = videos.claim_listed(conn)
+
+    assert [row["video_id"] for row in rows] == ["listed00001"]
+
+
+def test_claim_listed_is_oldest_upload_first(conn, channel):
+    add_video(conn, channel["id"], "newer000001", upload_date="2026-08-10")
+    add_video(conn, channel["id"], "older000001", upload_date="2026-01-01")
+
+    rows = videos.claim_listed(conn)
+
+    assert [row["video_id"] for row in rows] == ["older000001", "newer000001"]
+
+
+def test_claim_listed_honours_the_limit(conn, channel):
+    for index in range(5):
+        add_video(conn, channel["id"], f"vid{index:08d}")
+    assert len(videos.claim_listed(conn, limit=2)) == 2
+
+
+def test_materialised_for_channel_is_newest_first(conn, channel):
+    """The retention sweep counts down from the newest to honour min_keep_videos."""
+    add_video(conn, channel["id"], "older000001", upload_date="2026-01-01",
+              state=videos.MATERIALISED)
+    add_video(conn, channel["id"], "newer000001", upload_date="2026-08-10",
+              state=videos.MATERIALISED)
+
+    rows = videos.materialised_for_channel(conn, channel["id"])
+
+    assert [row["video_id"] for row in rows] == ["newer000001", "older000001"]
+
+
+def test_queue_depth_and_counts(conn, channel):
+    add_video(conn, channel["id"], "listed00001", state=videos.LISTED)
+    add_video(conn, channel["id"], "listed00002", state=videos.LISTED)
+    add_video(conn, channel["id"], "done0000001", state=videos.MATERIALISED)
+
+    assert videos.queue_depth(conn) == 2
+    counts = videos.counts_by_state(conn)
+    assert counts[videos.LISTED] == 2
+    assert counts[videos.MATERIALISED] == 1
+
+
+def test_deleting_a_channel_cascades_to_its_videos(conn, channel):
+    add_video(conn, channel["id"], "vid00000001")
+    with conn:
+        conn.execute("DELETE FROM channel WHERE id = ?", (channel["id"],))
+    assert conn.execute("SELECT COUNT(*) FROM video").fetchone()[0] == 0

tests/test_web.py

@@ -0,0 +1,146 @@
+"""The admin UI: rendering, auth gating and CSRF.
+
+The templates are f-strings over a dict, so a renamed key is a KeyError at render
+time rather than a type error at import time — which is exactly the kind of
+breakage that only shows up when somebody opens the page. These tests render every
+page for real.
+"""
+
+from __future__ import annotations
+
+import pytest
+
+from ytstream import videos
+from ytstream.web import auth, templates
+
+from conftest import add_channel, add_video
+
+
+@pytest.fixture()
+def channel_row(conn, settings, channel):
+    """The dict shape server.py builds for the channel table."""
+    add_video(conn, channel["id"], "vid00000001", state=videos.MATERIALISED)
+    return {
+        "id": channel["id"],
+        "title": channel["title"],
+        "handle": channel["handle"],
+        "channel_id": channel["channel_id"],
+        "retention_days": None,
+        "global_retention": 30,
+        "last_polled_at": "2026-08-12T14:00:00+00:00",
+        "last_poll_ok": 1,
+        "consecutive_poll_failures": 0,
+        "episodes": 1,
+        "source": "youtube",
+        "missing_syncs": 0,
+        "missing_threshold": 3,
+        "latest": "2026-08-12",
+    }
+
+
+def test_login_page_renders(settings):
+    html = templates.login_page().decode()
+    assert "ytstream" in html
+    assert "youtube-automate" not in html
+    assert 'type="password"' in html
+
+
+def test_login_page_shows_an_error():
+    assert "Wrong" in templates.login_page("Wrong password").decode()
+
+
+def test_index_page_renders(channel_row):
+    html = templates.index_page(
+        channels=[channel_row], settings_values={"retention_days": "30"},
+        settings_errors={}, csrf="tok", add_error=None, queue_depth=0,
+    ).decode()
+
+    assert "clabretro" in html
+    assert "ytstream" in html
+    # The provenance column replaced the meaningless size column.
+    assert "youtube" in html
+    assert "Size" not in html
+
+
+def test_index_page_renders_with_no_channels():
+    html = templates.index_page(
+        channels=[], settings_values={}, settings_errors={}, csrf="tok",
+        add_error=None, queue_depth=0,
+    ).decode()
+    assert "0 channel(s)" in html
+
+
+def test_absence_badge_appears_before_the_channel_disappears(channel_row):
+    """A channel counting towards removal must be visible while it still exists."""
+    channel_row["missing_syncs"] = 2
+    html = templates.index_page(
+        channels=[channel_row], settings_values={}, settings_errors={}, csrf="tok",
+        add_error=None, queue_depth=0,
+    ).decode()
+    assert "absent 2/3 syncs" in html
+
+
+def test_no_absence_badge_when_healthy(channel_row):
+    html = templates.index_page(
+        channels=[channel_row], settings_values={}, settings_errors={}, csrf="tok",
+        add_error=None, queue_depth=0,
+    ).decode()
+    assert "absent" not in html
+
+
+def test_hostile_channel_title_is_escaped(channel_row):
+    channel_row["title"] = '<script>alert("x")</script>'
+    html = templates.index_page(
+        channels=[channel_row], settings_values={}, settings_errors={}, csrf="tok",
+        add_error=None, queue_depth=0,
+    ).decode()
+    assert "<script>alert" not in html
+    assert "&lt;script&gt;" in html
+
+
+def test_settings_errors_are_rendered_inline(channel_row):
+    html = templates.index_page(
+        channels=[channel_row], settings_values={"retention_days": "abc"},
+        settings_errors={"retention_days": "must be a whole number"},
+        csrf="tok", add_error=None, queue_depth=0,
+    ).decode()
+    assert "must be a whole number" in html
+
+
+def test_masked_settings_are_not_rendered_in_plaintext(channel_row):
+    """Both API keys are secrets; neither belongs in the HTML."""
+    html = templates.index_page(
+        channels=[channel_row],
+        settings_values={"youtube_api_key": "AIzaSECRETVALUE",
+                         "jellyfin_api_key": "JELLYSECRET"},
+        settings_errors={}, csrf="tok", add_error=None, queue_depth=0,
+    ).decode()
+    assert "AIzaSECRETVALUE" not in html
+    assert "JELLYSECRET" not in html
+
+
+# ------------------------------------------------------------------------ csrf
+
+
+def test_csrf_token_round_trips():
+    secret = auth.new_secret()
+    session = "session-token"
+    token = auth.csrf_token(secret, session)
+    assert auth.verify_csrf(secret, session, token)
+
+
+def test_csrf_token_is_bound_to_the_session():
+    secret = auth.new_secret()
+    token = auth.csrf_token(secret, "session-a")
+    assert not auth.verify_csrf(secret, "session-b", token)
+
+
+def test_csrf_token_rejects_tampering():
+    secret = auth.new_secret()
+    token = auth.csrf_token(secret, "s")
+    assert not auth.verify_csrf(secret, "s", token[:-1] + "x")
+
+
+def test_csrf_rejects_an_empty_token():
+    secret = auth.new_secret()
+    assert not auth.verify_csrf(secret, "s", "")

ytstream/__init__.py

@@ -0,0 +1,7 @@
+"""ytstream — a just-in-time YouTube library for Jellyfin.
+
+Writes .strm files and metadata; a companion proxy materialises video on demand
+when Jellyfin asks for it. No video bytes are ever stored.
+"""
+
+__version__ = "1.0.0"

ytstream/__main__.py

@@ -0,0 +1,4 @@
+from .cli import main
+
+if __name__ == "__main__":
+    raise SystemExit(main())

ytstream/api.py

@@ -0,0 +1,327 @@
+"""YouTube Data API v3 client.
+
+This is the metadata path. yt-dlp is not involved in cataloguing at all — it is
+invoked only by the proxy, for one video, because somebody pressed play. See
+plan.md §3 for why that boundary is worth having.
+
+Everything here is a `list` method costing 1 quota unit per call and returning up
+to 50 items, against a 10,000/day budget. Measured steady state is ~75 units/day,
+so there is no caching layer and no need for one.
+
+`search.list` is never called. It has its own 100-call/day bucket and we have no
+use for it.
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+import re
+import urllib.error
+import urllib.parse
+import urllib.request
+from datetime import date, datetime, timezone
+
+log = logging.getLogger(__name__)
+
+BASE = "https://www.googleapis.com/youtube/v3"
+
+# Pages are capped at 50 by the API itself.
+PAGE_SIZE = 50
+
+# A refusal to page forever if the API ever stops returning a stable
+# nextPageToken. 200 pages is 10,000 items, far past anything we ask for.
+MAX_PAGES = 200
+
+_DURATION = re.compile(
+    r"^P(?:(?P<w>\d+)W)?(?:(?P<d>\d+)D)?"
+    r"(?:T(?:(?P<h>\d+)H)?(?:(?P<m>\d+)M)?(?:(?P<s>\d+)S)?)?$"
+)
+
+
+class ApiError(RuntimeError):
+    """Any non-2xx from the API, with the reason Google actually gave."""
+
+    def __init__(self, status: int, reason: str, message: str):
+        super().__init__(f"HTTP {status} {reason}: {message}")
+        self.status = status
+        self.reason = reason
+        self.message = message
+
+
+class NotConfigured(ApiError):
+    """The key is missing, invalid, restricted, or the API is not enabled.
+
+    Separate from ApiError because it is an operator problem, not a transient
+    one: retrying will not help and the caller must not interpret it as "the
+    remote data changed". See plan.md §16 for the three flavours of this and why
+    they are hard to tell apart.
+    """
+
+
+class SubscriptionsPrivate(ApiError):
+    """403 subscriptionForbidden — the channel's subscriptions are not public.
+
+    Its own type because the sync must never read this as "he unsubscribed from
+    everything" (plan.md §4.4).
+    """
+
+
+def parse_duration(text: str) -> int | None:
+    """ISO-8601 duration to seconds. PT1H2M3S -> 3723. None if unparseable.
+
+    Live and upcoming videos report PT0S, which parses to 0 rather than None —
+    the caller distinguishes those by liveStreamingDetails, not by duration.
+    """
+    match = _DURATION.match((text or "").strip())
+    if not match:
+        return None
+    parts = {key: int(value or 0) for key, value in match.groupdict().items()}
+    return (parts["w"] * 604800 + parts["d"] * 86400
+            + parts["h"] * 3600 + parts["m"] * 60 + parts["s"])
+
+
+def parse_published(text: str) -> tuple[date | None, str | None]:
+    """RFC-3339 to (date, original string).
+
+    The date is what naming uses; the original is kept because it is exact and
+    approximate_date is not (plan.md §3).
+    """
+    raw = (text or "").strip()
+    if not raw:
+        return None, None
+    try:
+        stamp = datetime.fromisoformat(raw.replace("Z", "+00:00"))
+    except ValueError:
+        return None, raw
+    return stamp.astimezone(timezone.utc).date(), raw
+
+
+def uploads_playlist_id(channel_id: str, kind: str = "UULF") -> str:
+    """UULF (long-form only) or UU (everything) for a UC… channel id.
+
+    UULF is undocumented but verified to work through playlistItems.list on
+    2026-08-12, and excludes 54% of a measured catalogue — all Shorts and
+    livestreams. UU is the documented fallback if YouTube ever retires it.
+    """
+    if not channel_id.startswith("UC"):
+        raise ValueError(f"not a channel id: {channel_id!r}")
+    return kind + channel_id[2:]
+
+
+class Api:
+    """Thin client. One instance per run; reads the key at construction."""
+
+    def __init__(self, key: str, *, timeout: float = 30.0):
+        self.key = (key or "").strip()
+        self.timeout = timeout
+        self.calls = 0          # quota units spent, for logging and the doctor
+
+    # ---------------------------------------------------------------- transport
+
+    def _get(self, endpoint: str, **params) -> dict:
+        if not self.key:
+            raise NotConfigured(0, "noKey", "no YouTube API key configured")
+
+        params["key"] = self.key
+        url = f"{BASE}/{endpoint}?" + urllib.parse.urlencode(params)
+        request = urllib.request.Request(url, headers={"Accept": "application/json"})
+        self.calls += 1
+        try:
+            with urllib.request.urlopen(request, timeout=self.timeout) as response:
+                return json.load(response)
+        except urllib.error.HTTPError as exc:
+            raise self._classify(exc) from exc
+        except OSError as exc:
+            # Network-level: transient, and explicitly not NotConfigured.
+            raise ApiError(0, "network", str(exc)) from exc
+
+    def _classify(self, exc: urllib.error.HTTPError) -> ApiError:
+        """Turn Google's 403s into something a caller can branch on.
+
+        The useful signal lives in error.details[].reason, not
+        error.errors[].reason — both SERVICE_DISABLED and
+        API_KEY_SERVICE_BLOCKED surface as a bare `forbidden` in the latter.
+        """
+        raw = exc.read().decode("utf8", "replace")
+        reason = message = ""
+        details: set[str] = set()
+        try:
+            error = json.loads(raw).get("error", {})
+            message = error.get("message", "")
+            errors = error.get("errors", [])
+            reason = errors[0].get("reason", "") if errors else ""
+            for detail in error.get("details", []):
+                if detail.get("reason"):
+                    details.add(detail["reason"])
+        except ValueError:
+            message = raw[:200]
+
+        if reason == "subscriptionForbidden":
+            return SubscriptionsPrivate(exc.code, reason, message)
+        if (details & {"SERVICE_DISABLED", "API_KEY_SERVICE_BLOCKED"}
+                or reason in ("accessNotConfigured", "keyInvalid", "forbidden")
+                or "has not been used in project" in message
+                or "are blocked" in message):
+            return NotConfigured(exc.code, reason or next(iter(details), ""), message)
+        return ApiError(exc.code, reason, message)
+
+    def _paged(self, endpoint: str, **params):
+        """Yield items across all pages, stopping when the token runs out."""
+        token = None
+        for _ in range(MAX_PAGES):
+            if token:
+                params["pageToken"] = token
+            page = self._get(endpoint, **params)
+            for item in page.get("items") or []:
+                yield item
+            token = page.get("nextPageToken")
+            if not token:
+                return
+        log.warning("%s: stopped paging at %d pages", endpoint, MAX_PAGES)
+
+    # ------------------------------------------------------------ subscriptions
+
+    def subscriptions(self, channel_id: str) -> list[dict]:
+        """A channel's public subscriptions as [{channel_id, title}].
+
+        Requires only an API key, provided the account has not ticked "Keep all
+        my subscriptions private" — verified 2026-08-12.
+
+        Deliberately returns the *fetched* list rather than anything derived from
+        pageInfo.totalResults, which overcounts: it reported 127 against 119
+        actually returned, because terminated and private channels still count.
+        """
+        out, seen = [], set()
+        for item in self._paged("subscriptions", part="snippet",
+                                channelId=channel_id, maxResults=PAGE_SIZE):
+            snippet = item.get("snippet") or {}
+            resource = snippet.get("resourceId") or {}
+            sub_id = resource.get("channelId")
+            if not sub_id or sub_id in seen:
+                continue
+            seen.add(sub_id)
+            out.append({"channel_id": sub_id,
+                        "title": (snippet.get("title") or "").strip() or sub_id})
+        return out
+
+    # ----------------------------------------------------------------- channels
+
+    def channel(self, channel_id: str) -> dict | None:
+        """Title, description and artwork for one channel."""
+        items = self._get("channels", part="snippet,contentDetails",
+                          id=channel_id, maxResults=1).get("items") or []
+        if not items:
+            return None
+        item = items[0]
+        snippet = item.get("snippet") or {}
+        thumbs = snippet.get("thumbnails") or {}
+        best = max(thumbs.values(), key=lambda t: t.get("width") or 0, default={})
+        return {
+            "channel_id": item.get("id") or channel_id,
+            "title": (snippet.get("title") or "").strip(),
+            "description": snippet.get("description") or "",
+            "handle": (snippet.get("customUrl") or "") or None,
+            "avatar_url": best.get("url"),
+        }
+
+    def resolve_handle(self, handle: str) -> dict | None:
+        """@handle -> channel. Only needed when subscribing by hand."""
+        handle = handle.strip()
+        if not handle.startswith("@"):
+            handle = "@" + handle
+        items = self._get("channels", part="snippet",
+                          forHandle=handle, maxResults=1).get("items") or []
+        return self.channel(items[0]["id"]) if items else None
+
+    # ------------------------------------------------------------ uploads walk
+
+    def uploads(self, channel_id: str, *, kind: str = "UULF",
+                since: date | None = None, limit: int | None = None,
+                page_token: str | None = None):
+        """Walk a channel's uploads newest-first, yielding (entry, next_token).
+
+        `entry` carries the exact publish timestamp, which is the whole reason
+        this exists rather than `--flat-playlist`: that reports no timestamp at
+        all, and `approximate_date` is wrong by up to two days, which would put
+        episodes in the wrong day.
+
+        Stops at `since` or `limit`, whichever comes first. Yielding the token
+        alongside each entry is what makes a bounded backfill resumable — the
+        caller commits it per page.
+        """
+        playlist_id = uploads_playlist_id(channel_id, kind)
+        token = page_token
+        produced = 0
+
+        for _ in range(MAX_PAGES):
+            # Both parts, because they carry different halves of what we need and
+            # cost the same single unit together as either does alone:
+            #   contentDetails.videoPublishedAt -> when the VIDEO was published
+            #   snippet.title                   -> the title
+            # Do not be tempted by snippet.publishedAt: that is when the video was
+            # added to the playlist, which is not the same thing.
+            #
+            # Asking for the title here rather than letting the RSS poll supply it
+            # is not an optimisation. RSS returns 15 entries, which for a channel
+            # publishing under one long-form video a day reaches back only ~23
+            # days — less than the 30-day window — so the oldest few episodes of
+            # every backfill would otherwise be named after their video id.
+            params = dict(part="snippet,contentDetails", playlistId=playlist_id,
+                          maxResults=PAGE_SIZE)
+            if token:
+                params["pageToken"] = token
+            page = self._get("playlistItems", **params)
+            next_token = page.get("nextPageToken")
+
+            for item in page.get("items") or []:
+                details = item.get("contentDetails") or {}
+                snippet = item.get("snippet") or {}
+                video_id = details.get("videoId")
+                if not video_id:
+                    continue
+                published, exact = parse_published(details.get("videoPublishedAt"))
+                if published is None:
+                    # A private or deleted video still occupies a playlist slot
+                    # and has no publish date. Skip it rather than stopping.
+                    continue
+                if since is not None and published < since:
+                    return
+                title = (snippet.get("title") or "").strip()
+                # YouTube uses these placeholders for videos that have gone away
+                # but still hold a playlist slot. Treated as no title at all.
+                if title in ("Private video", "Deleted video"):
+                    title = ""
+                yield {"video_id": video_id,
+                       "published": published,
+                       "published_at": exact,
+                       "title": title}, next_token
+                produced += 1
+                if limit is not None and produced >= limit:
+                    return
+
+            token = next_token
+            if not token:
+                return
+
+    # ---------------------------------------------------------------- durations
+
+    def durations(self, video_ids: list[str]) -> dict[str, dict]:
+        """{video_id: {duration, is_live}} for up to any number of ids.
+
+        Batched 50 per call, so 441 videos costs 9 units. `is_live` comes from the
+        presence of liveStreamingDetails rather than from the duration, because
+        live and upcoming items both report PT0S.
+        """
+        out: dict[str, dict] = {}
+        for start in range(0, len(video_ids), PAGE_SIZE):
+            batch = video_ids[start:start + PAGE_SIZE]
+            page = self._get("videos", part="contentDetails,liveStreamingDetails",
+                             id=",".join(batch), maxResults=PAGE_SIZE)
+            for item in page.get("items") or []:
+                details = item.get("contentDetails") or {}
+                out[item["id"]] = {
+                    "duration": parse_duration(details.get("duration", "")),
+                    "is_live": bool(item.get("liveStreamingDetails")),
+                }
+        return out

ytstream/channels.py

@@ -0,0 +1,301 @@
+"""Channel resolution, subscribe and unsubscribe."""
+
+from __future__ import annotations
+
+import logging
+import re
+import shutil
+import sqlite3
+import subprocess
+import tempfile
+import urllib.request
+from pathlib import Path
+
+from . import config, naming, nfo, util, ytdlp
+from .settings import Settings
+
+log = logging.getLogger(__name__)
+
+_CHANNEL_ID = re.compile(r"^UC[A-Za-z0-9_-]{22}$")
+_HANDLE = re.compile(r"^@[A-Za-z0-9._-]+$")
+
+# Artwork we try to pull at subscribe time. Best effort — a channel without them
+# still works, it just looks plainer in Jellyfin.
+_ARTWORK = (("avatar_uncropped", "poster.jpg"), ("banner_uncropped", "fanart.jpg"))
+
+
+class ResolutionError(RuntimeError):
+    pass
+
+
+def normalise_url(text: str) -> str:
+    """Turn any accepted channel reference into a URL yt-dlp understands."""
+    text = (text or "").strip()
+    if not text:
+        raise ResolutionError("no channel given")
+
+    if text.startswith(("http://", "https://")):
+        return text
+    if _CHANNEL_ID.match(text):
+        return f"https://www.youtube.com/channel/{text}"
+    if _HANDLE.match(text):
+        return f"https://www.youtube.com/{text}"
+    if text.startswith("www.youtube.com") or text.startswith("youtube.com"):
+        return "https://" + text
+    # Bare word: assume it's a handle without the @.
+    if re.match(r"^[A-Za-z0-9._-]+$", text):
+        return f"https://www.youtube.com/@{text}"
+    raise ResolutionError(f"could not interpret {text!r} as a channel")
+
+
+def resolve(settings: Settings, text: str) -> dict:
+    """Fetch channel metadata without enumerating the uploads."""
+    url = normalise_url(text)
+    args = [
+        "--flat-playlist",
+        "--playlist-items",
+        "0",
+        "-J",
+        "--no-warnings",
+        "--ignore-config",
+        *ytdlp.extractor_args(settings.get_str("pot_provider_url")),
+        url,
+    ]
+    try:
+        data = ytdlp.run_json(args, timeout=180)
+    except ytdlp.YtdlpError as exc:
+        raise ResolutionError(str(exc)) from exc
+
+    channel_id = data.get("channel_id") or data.get("id") or ""
+    if not _CHANNEL_ID.match(channel_id):
+        raise ResolutionError(f"no channel id found for {url}")
+
+    handle = data.get("uploader_id") or ""
+    if handle and not handle.startswith("@"):
+        handle = ""
+
+    return {
+        "channel_id": channel_id,
+        "title": (data.get("channel") or data.get("title") or channel_id).strip(),
+        "description": data.get("description") or "",
+        "handle": handle,
+        "thumbnails": data.get("thumbnails") or [],
+    }
+
+
+def uulf_playlist_id(channel_id: str) -> str:
+    """Long-form-only uploads playlist for a channel (specs.md §4)."""
+    return "UULF" + channel_id[2:]
+
+
+# --------------------------------------------------------------------------
+# artwork
+
+
+def _pick_thumbnail(thumbnails: list[dict], wanted_id: str) -> str | None:
+    for thumb in thumbnails:
+        if str(thumb.get("id", "")) == wanted_id and thumb.get("url"):
+            return thumb["url"]
+    return None
+
+
+def _download_image(url: str, destination: Path) -> bool:
+    """Fetch an image and normalise it to JPEG via ffmpeg.
+
+    YouTube serves avatars as webp as often as jpeg, and naming a webp file
+    .jpg would be a lie some clients notice.
+    """
+    try:
+        request = urllib.request.Request(
+            url, headers={"User-Agent": config.USER_AGENT}
+        )
+        with urllib.request.urlopen(request, timeout=60) as response:
+            payload = response.read()
+    except OSError as exc:
+        log.warning("artwork download failed (%s): %s", url, exc)
+        return False
+
+    with tempfile.NamedTemporaryFile(suffix=".img", delete=True) as raw:
+        raw.write(payload)
+        raw.flush()
+        result = subprocess.run(
+            ["ffmpeg", "-y", "-loglevel", "error", "-i", raw.name, str(destination)],
+            capture_output=True,
+            text=True,
+            timeout=120,
+        )
+    if result.returncode != 0:
+        log.warning("artwork conversion failed: %s", result.stderr.strip()[:200])
+        return False
+    return True
+
+
+def write_channel_metadata(channel_dir: Path, info: dict) -> None:
+    """Write tvshow.nfo and best-effort artwork into the channel directory."""
+    channel_dir.mkdir(parents=True, exist_ok=True)
+    nfo.write(
+        channel_dir / "tvshow.nfo",
+        nfo.tvshow_nfo(info["title"], info.get("description"), info["channel_id"]),
+    )
+    for thumb_id, filename in _ARTWORK:
+        url = _pick_thumbnail(info.get("thumbnails") or [], thumb_id)
+        if url:
+            _download_image(url, channel_dir / filename)
+
+
+# --------------------------------------------------------------------------
+# subscribe / unsubscribe
+
+
+def get(conn: sqlite3.Connection, pk: int) -> sqlite3.Row | None:
+    return conn.execute("SELECT * FROM channel WHERE id = ?", (pk,)).fetchone()
+
+
+def get_by_channel_id(conn: sqlite3.Connection, channel_id: str) -> sqlite3.Row | None:
+    return conn.execute(
+        "SELECT * FROM channel WHERE channel_id = ?", (channel_id,)
+    ).fetchone()
+
+
+def all_channels(conn: sqlite3.Connection) -> list[sqlite3.Row]:
+    return conn.execute("SELECT * FROM channel ORDER BY title COLLATE NOCASE").fetchall()
+
+
+def _unique_dir_name(conn: sqlite3.Connection, base: str) -> str:
+    """dir_name is UNIQUE; two channels can legitimately share a title."""
+    candidate = base
+    suffix = 2
+    while conn.execute(
+        "SELECT 1 FROM channel WHERE dir_name = ?", (candidate,)
+    ).fetchone():
+        candidate = f"{base} ({suffix})"
+        suffix += 1
+    return candidate
+
+
+def subscribe(conn: sqlite3.Connection, settings: Settings, text: str) -> sqlite3.Row:
+    """Resolve, insert and lay down on-disk metadata. Raises ResolutionError."""
+    info = resolve(settings, text)
+
+    existing = get_by_channel_id(conn, info["channel_id"])
+    if existing:
+        raise ResolutionError(f"already subscribed to {existing['title']}")
+
+    dir_name = _unique_dir_name(
+        conn, naming.channel_dir_name(info["title"], info["channel_id"])
+    )
+
+    with conn:
+        cursor = conn.execute(
+            "INSERT INTO channel (channel_id, handle, title, description, dir_name, added_at) "
+            "VALUES (?, ?, ?, ?, ?, ?)",
+            (
+                info["channel_id"],
+                info["handle"],
+                info["title"],
+                info["description"],
+                dir_name,
+                util.utcnow_iso(),
+            ),
+        )
+        pk = cursor.lastrowid
+
+    write_channel_metadata(config.MEDIA_ROOT / dir_name, info)
+    log.info("subscribed to %s (%s)", info["title"], info["channel_id"])
+    return get(conn, pk)
+
+
+def subscribe_from_sync(
+    conn: sqlite3.Connection, settings: Settings, channel_id: str, title: str
+) -> sqlite3.Row:
+    """Subscribe a channel discovered by the subscription mirror.
+
+    Distinct from `subscribe()` because it must not touch yt-dlp: cataloguing runs
+    entirely on the Data API and RSS, and 119 channels' worth of yt-dlp channel
+    resolution is exactly the residential-IP request burst the design avoids
+    (plan.md §3). The API already gave us the id and title; one `channels.list`
+    call fills in description and artwork.
+    """
+    from . import api as ytapi
+
+    existing = get_by_channel_id(conn, channel_id)
+    if existing:
+        return existing
+
+    info = {"channel_id": channel_id, "title": title.strip() or channel_id,
+            "description": "", "handle": None, "avatar_url": None}
+    try:
+        fetched = ytapi.Api(settings.get_str("youtube_api_key")).channel(channel_id)
+        if fetched and fetched.get("title"):
+            info = fetched
+    except ytapi.ApiError as exc:
+        # A channel we cannot describe is still a channel we can mirror.
+        log.warning("could not fetch metadata for %s: %s", channel_id, exc)
+
+    dir_name = _unique_dir_name(
+        conn, naming.channel_dir_name(info["title"], channel_id)
+    )
+    with conn:
+        cursor = conn.execute(
+            "INSERT INTO channel (channel_id, handle, title, description, dir_name, "
+            " added_at, source) VALUES (?, ?, ?, ?, ?, ?, 'youtube')",
+            (channel_id, info.get("handle"), info["title"],
+             info.get("description") or "", dir_name, util.utcnow_iso()),
+        )
+        pk = cursor.lastrowid
+
+    row = get(conn, pk)
+    # tvshow.nfo only. The channel directory is created lazily by the first
+    # episode, so a channel with nothing inside the retention window does not
+    # leave an empty series in Jellyfin (plan.md §5).
+    if info.get("avatar_url"):
+        _write_show_metadata(config.MEDIA_ROOT / dir_name, info)
+    log.info("subscribed to %s (%s) via sync", info["title"], channel_id)
+    return row
+
+
+def _write_show_metadata(channel_dir: Path, info: dict) -> None:
+    """tvshow.nfo plus a poster, for a channel resolved through the API."""
+    channel_dir.mkdir(parents=True, exist_ok=True)
+    nfo.write(
+        channel_dir / "tvshow.nfo",
+        nfo.tvshow_nfo(info["title"], info.get("description"), info["channel_id"]),
+    )
+    if info.get("avatar_url"):
+        _download_image(info["avatar_url"], channel_dir / "poster.jpg")
+
+
+def refresh_metadata(conn: sqlite3.Connection, settings: Settings, pk: int) -> None:
+    """Re-resolve a channel and rewrite tvshow.nfo if the title changed.
+
+    dir_name is deliberately never recomputed — channels rename themselves and
+    we do not want orphaned directories.
+    """
+    row = get(conn, pk)
+    if row is None:
+        return
+    info = resolve(settings, row["channel_id"])
+    if info["title"] != row["title"] or info["description"] != (row["description"] or ""):
+        with conn:
+            conn.execute(
+                "UPDATE channel SET title = ?, description = ? WHERE id = ?",
+                (info["title"], info["description"], pk),
+            )
+        write_channel_metadata(config.MEDIA_ROOT / row["dir_name"], info)
+
+
+def unsubscribe(conn: sqlite3.Connection, pk: int) -> str:
+    """Hard delete: remove the directory tree, then the rows. Irreversible."""
+    row = get(conn, pk)
+    if row is None:
+        raise LookupError(f"no channel with id {pk}")
+
+    title = row["title"]
+    channel_dir = config.MEDIA_ROOT / row["dir_name"]
+    if channel_dir.is_dir():
+        shutil.rmtree(channel_dir, ignore_errors=True)
+
+    with conn:
+        conn.execute("DELETE FROM channel WHERE id = ?", (pk,))
+    log.info("unsubscribed from %s and removed %s", title, channel_dir)
+    return title

ytstream/cli.py

@@ -0,0 +1,556 @@
+"""Command line entry point."""
+
+from __future__ import annotations
+
+import argparse
+import getpass
+import sys
+
+from . import (
+    api,
+    channels,
+    config,
+    db,
+    discovery,
+    doctor,
+    jellyfin,
+    reap,
+    runner,
+    strm,
+    subsync,
+    util,
+    videos,
+)
+from .settings import Settings
+from .web import auth
+
+
+def _open():
+    conn = db.connect()
+    return conn, Settings(conn)
+
+
+# --------------------------------------------------------------------------
+# setup
+
+
+def cmd_doctor(args) -> int:
+    conn, settings = _open()
+    try:
+        text, code = doctor.report(doctor.run_checks(settings))
+        print(text)
+        return code
+    finally:
+        conn.close()
+
+
+def cmd_set_password(args) -> int:
+    conn, settings = _open()
+    try:
+        password = getpass.getpass("New admin password: ")
+        if len(password) < 8:
+            print("Password must be at least 8 characters.", file=sys.stderr)
+            return 1
+        if password != getpass.getpass("Repeat: "):
+            print("Passwords did not match.", file=sys.stderr)
+            return 1
+
+        settings.set("admin_password_hash", auth.hash_password(password))
+        if not settings.raw("session_secret"):
+            settings.set("session_secret", auth.new_secret())
+        print("Admin password set.")
+        return 0
+    finally:
+        conn.close()
+
+
+def cmd_set_jellyfin_key(args) -> int:
+    conn, settings = _open()
+    try:
+        key = (args.key or getpass.getpass("Jellyfin API key: ")).strip()
+        if not key:
+            print("No key given.", file=sys.stderr)
+            return 1
+
+        client = jellyfin.Jellyfin(settings.get_str("jellyfin_url"), key)
+        try:
+            client.virtual_folders()
+        except jellyfin.JellyfinError as exc:
+            print(f"Key rejected by Jellyfin: {exc}", file=sys.stderr)
+            return 1
+
+        settings.set("jellyfin_api_key", key)
+        print("Jellyfin API key stored and verified.")
+        return 0
+    finally:
+        conn.close()
+
+
+def cmd_set_youtube_key(args) -> int:
+    """Store the Data API key, verifying it first.
+
+    Verified rather than trusted because there are three distinct ways for a
+    fresh Google Cloud project to be wrong and they all present as HTTP 403
+    `forbidden` (plan.md §16). Storing a key that cannot work would turn an
+    operator mistake into a mystery an hour later in the cron log.
+    """
+    conn, settings = _open()
+    try:
+        key = (args.key or getpass.getpass("YouTube Data API key: ")).strip()
+        if not key:
+            print("No key given.", file=sys.stderr)
+            return 1
+
+        client = api.Api(key)
+        try:
+            client.durations(["dQw4w9WgXcQ"])
+        except api.NotConfigured as exc:
+            print(f"Key not usable: {exc.message or exc}", file=sys.stderr)
+            print("\nThis is normally one of:", file=sys.stderr)
+            print("  * YouTube Data API v3 not enabled on the project", file=sys.stderr)
+            print("  * the key's API restrictions exclude it", file=sys.stderr)
+            print("  * a console change that has not propagated yet (~2 min)",
+                  file=sys.stderr)
+            return 1
+        except api.ApiError as exc:
+            print(f"Key check failed: {exc}", file=sys.stderr)
+            return 1
+
+        settings.set("youtube_api_key", key)
+        print("YouTube API key stored and verified.")
+        return 0
+    finally:
+        conn.close()
+
+
+def cmd_setup_jellyfin_library(args) -> int:
+    conn, settings = _open()
+    try:
+        client = jellyfin.from_settings(settings)
+        if not client.configured:
+            print("Set jellyfin_url and jellyfin_api_key first.", file=sys.stderr)
+            return 1
+        existing = client.find_library(config.MEDIA_ROOT)
+        if existing:
+            print(f"Library already exists: {existing.get('Name')}")
+            return 0
+        client.create_library(config.MEDIA_ROOT)
+        print(f"Created Jellyfin library for {config.MEDIA_ROOT}.")
+        return 0
+    finally:
+        conn.close()
+
+
+# --------------------------------------------------------------------------
+# subscription sources
+
+
+def cmd_add_source(args) -> int:
+    """Register a YouTube account whose subscriptions we mirror."""
+    conn, settings = _open()
+    try:
+        client = api.Api(settings.get_str("youtube_api_key"))
+        reference = args.channel.strip()
+        try:
+            info = (client.channel(reference) if reference.startswith("UC")
+                    else client.resolve_handle(reference))
+        except api.ApiError as exc:
+            print(f"Could not resolve {reference}: {exc}", file=sys.stderr)
+            return 1
+        if not info:
+            print(f"No such channel: {reference}", file=sys.stderr)
+            return 1
+
+        # Prove the subscriptions are actually readable before storing anything.
+        # A source that 403s is worse than no source: it looks configured.
+        try:
+            found = client.subscriptions(info["channel_id"])
+        except api.SubscriptionsPrivate:
+            print(f"{info['title']}'s subscriptions are private.", file=sys.stderr)
+            print('Fix: YouTube -> Settings -> Privacy -> uncheck "Keep all my '
+                  'subscriptions private".', file=sys.stderr)
+            return 1
+        except api.ApiError as exc:
+            print(f"Could not read subscriptions: {exc}", file=sys.stderr)
+            return 1
+
+        key = subsync.add_source(
+            conn, channel_id=info["channel_id"], label=info["title"]
+        )
+        print(f"Added source {key} ({info['title']}) — {len(found)} subscriptions.")
+        print("Run `ytstream sync` to queue them for approval.")
+        return 0
+    finally:
+        conn.close()
+
+
+def cmd_sources(args) -> int:
+    conn, _ = _open()
+    try:
+        rows = subsync.all_sources(conn)
+        if not rows:
+            print("No subscription sources. Add one with `ytstream add-source`.")
+            return 0
+        for row in rows:
+            state = "ok" if row["last_sync_ok"] else "FAILING"
+            if row["last_sync_ok"] is None:
+                state = "never synced"
+            print(f"{row['key']}  {row['label']}  [{state}]")
+            print(f"    last sync: {row['last_sync_at'] or 'never'}  "
+                  f"failures: {row['consecutive_failures']}  "
+                  f"imported: {'yes' if row['imported'] else 'no'}")
+            if row["last_error"]:
+                print(f"    last error: {row['last_error'][:120]}")
+        return 0
+    finally:
+        conn.close()
+
+
+def cmd_sync(args) -> int:
+    conn, settings = _open()
+    try:
+        totals = subsync.sync_all(conn, settings)
+        print(f"sources={totals['sources']} added={totals['added']} "
+              f"queued={totals['queued']} removed={totals['removed']} "
+              f"pending_removal={totals['pending_removal']} "
+              f"refused={totals['refused']}")
+        return 1 if totals["refused"] else 0
+    finally:
+        conn.close()
+
+
+def cmd_pending(args) -> int:
+    conn, _ = _open()
+    try:
+        rows = subsync.pending(conn)
+        if not rows:
+            print("Nothing awaiting approval.")
+            return 0
+        print(f"{len(rows)} channel(s) awaiting approval:")
+        for row in rows:
+            print(f"{row['id']:>5}  {row['title'][:50]:<50} {row['channel_id']}")
+        print("\nApprove with: ytstream approve <id> [<id>...]  |  --all")
+        return 0
+    finally:
+        conn.close()
+
+
+def cmd_approve(args) -> int:
+    conn, settings = _open()
+    try:
+        if args.all:
+            ids = [row["id"] for row in subsync.pending(conn)]
+        else:
+            ids = args.ids
+        if not ids:
+            print("Nothing to approve.")
+            return 0
+        stats = subsync.approve(conn, settings, ids)
+        print(f"added={stats['added']} failed={stats['failed']}")
+        return 1 if stats["failed"] else 0
+    finally:
+        conn.close()
+
+
+def cmd_reject(args) -> int:
+    conn, _ = _open()
+    try:
+        ids = ([row["id"] for row in subsync.pending(conn)] if args.all else args.ids)
+        count = subsync.resolve(conn, ids, "rejected")
+        print(f"rejected {count} channel(s); they will not be queued again")
+        return 0
+    finally:
+        conn.close()
+
+
+# --------------------------------------------------------------------------
+# channels and catalogue
+
+
+def cmd_subscribe(args) -> int:
+    """Subscribe by hand. Marked `manual`, so the sync never removes it."""
+    conn, settings = _open()
+    try:
+        try:
+            row = channels.subscribe(conn, settings, args.url)
+        except channels.ResolutionError as exc:
+            print(str(exc), file=sys.stderr)
+            return 1
+        with conn:
+            conn.execute("UPDATE channel SET source = 'manual' WHERE id = ?",
+                         (row["id"],))
+        print(f"Subscribed to {row['title']} (id {row['id']}, pinned as manual).")
+        return 0
+    finally:
+        conn.close()
+
+
+def cmd_unsubscribe(args) -> int:
+    conn, settings = _open()
+    try:
+        row = channels.get(conn, args.id)
+        if row is None:
+            print(f"No channel with id {args.id}.", file=sys.stderr)
+            return 1
+        if not args.yes:
+            answer = input(f"Delete {row['title']} and its whole tree? [y/N] ")
+            if answer.strip().lower() not in ("y", "yes"):
+                print("Cancelled.")
+                return 0
+        title = channels.unsubscribe(conn, args.id)
+        jellyfin.from_settings(settings).refresh()
+        print(f"Unsubscribed from {title}.")
+        return 0
+    finally:
+        conn.close()
+
+
+def cmd_channels(args) -> int:
+    conn, _ = _open()
+    try:
+        rows = channels.all_channels(conn)
+        if not rows:
+            print("No channels subscribed.")
+            return 0
+        print(f"{'id':>4}  {'title':<34} {'src':<7} {'eps':>4} {'miss':>4}  last poll")
+        for row in rows:
+            count = conn.execute(
+                "SELECT COUNT(*) FROM video WHERE channel_pk = ? AND state = ?",
+                (row["id"], videos.MATERIALISED),
+            ).fetchone()[0]
+            print(f"{row['id']:>4}  {row['title'][:34]:<34} {row['source']:<7} "
+                  f"{count:>4} {row['missing_syncs']:>4}  "
+                  f"{row['last_polled_at'] or 'never'}")
+        return 0
+    finally:
+        conn.close()
+
+
+def cmd_poll(args) -> int:
+    conn, settings = _open()
+    try:
+        if args.rescan:
+            rows = ([channels.get(conn, args.channel)] if args.channel
+                    else channels.all_channels(conn))
+            total = sum(discovery.rescan_channel(conn, settings, row)
+                        for row in rows if row)
+            print(f"re-queued {total} previously-skipped video(s)")
+
+        totals = discovery.poll_all(conn, settings, args.channel)
+        print(f"channels={totals['channels']} queued={totals['queued']} "
+              f"known={totals['known']} outside-window={totals['old']} "
+              f"repaired={totals['repaired']} titled={totals['titled']} "
+              f"shorts={totals['shorts']} live={totals['live']} "
+              f"failures={totals['failed']}")
+        return 0
+    finally:
+        conn.close()
+
+
+def cmd_materialise(args) -> int:
+    conn, settings = _open()
+    try:
+        if args.all:
+            # Rebuild the tree from the database. This is the supported recovery
+            # from a Jellyfin "replace all metadata" accident — never a refresh
+            # against YouTube.
+            with conn:
+                conn.execute("UPDATE video SET state = ? WHERE state = ?",
+                             (videos.LISTED, videos.MATERIALISED))
+        stats = runner.materialise_all(conn, settings, args.limit)
+        print(f"materialised={stats['materialised']} shows={stats['shows']} "
+              f"errors={stats['errors']}")
+        return 1 if stats["errors"] else 0
+    finally:
+        conn.close()
+
+
+def cmd_reap(args) -> int:
+    conn, settings = _open()
+    try:
+        result = reap.run(conn, settings)
+        print(f"aged_out={result['aged_out']}")
+        return 0
+    finally:
+        conn.close()
+
+
+def cmd_run(args) -> int:
+    try:
+        with runner.exclusive_lock():
+            conn, settings = _open()
+            try:
+                result = runner.run(conn, settings, args.channel)
+                print(runner.summarise(result))
+                return runner.exit_code(result)
+            finally:
+                conn.close()
+    except runner.AlreadyRunning:
+        # Expected when a long first import outlasts the hourly cron tick.
+        return 0
+
+
+def cmd_serve(args) -> int:
+    from .web import server
+
+    server.serve(args.host, args.port, secure_cookies=not args.insecure_cookies)
+    return 0
+
+
+def cmd_set_retention(args) -> int:
+    conn, _ = _open()
+    try:
+        row = channels.get(conn, args.id)
+        if row is None:
+            print(f"No channel with id {args.id}.", file=sys.stderr)
+            return 1
+
+        if args.days.lower() in ("default", "none", "clear"):
+            value = None
+        else:
+            try:
+                value = int(args.days)
+            except ValueError:
+                print("days must be a whole number or 'default'.", file=sys.stderr)
+                return 1
+            if value < 1:
+                print("days must be at least 1.", file=sys.stderr)
+                return 1
+
+        with conn:
+            conn.execute("UPDATE channel SET retention_days = ? WHERE id = ?",
+                         (value, args.id))
+        shown = "the global default" if value is None else f"{value} days"
+        print(f"{row['title']} retention set to {shown}.")
+        return 0
+    finally:
+        conn.close()
+
+
+def cmd_status(args) -> int:
+    conn, settings = _open()
+    try:
+        counts = videos.counts_by_state(conn)
+        total = conn.execute("SELECT COUNT(*) FROM channel").fetchone()[0]
+        print(f"channels: {total}")
+        for state in (videos.MATERIALISED, videos.LISTED, videos.SKIPPED_SHORT,
+                      videos.SKIPPED_LIVE, videos.SKIPPED_OLD, videos.AGED_OUT):
+            print(f"  {state:<14} {counts.get(state, 0)}")
+        print(f"last run: {settings.raw('last_run_at') or 'never'}")
+        awaiting = len(subsync.pending(conn))
+        if awaiting:
+            print(f"awaiting approval: {awaiting} (see `ytstream pending`)")
+        return 0
+    finally:
+        conn.close()
+
+
+# --------------------------------------------------------------------------
+
+
+def build_parser() -> argparse.ArgumentParser:
+    parser = argparse.ArgumentParser(
+        prog="ytstream",
+        description="A just-in-time YouTube library for Jellyfin. Stores no video.",
+    )
+    parser.add_argument("-v", "--verbose", action="store_true", help="debug logging")
+    sub = parser.add_subparsers(dest="command", required=True)
+
+    sub.add_parser("doctor", help="check the installation").set_defaults(
+        func=cmd_doctor)
+    sub.add_parser("status", help="counts by state").set_defaults(func=cmd_status)
+    sub.add_parser("set-password", help="set the admin UI password").set_defaults(
+        func=cmd_set_password)
+
+    jkey = sub.add_parser("set-jellyfin-key", help="store and verify the Jellyfin key")
+    jkey.add_argument("key", nargs="?", help="omit to be prompted (preferred)")
+    jkey.set_defaults(func=cmd_set_jellyfin_key)
+
+    ykey = sub.add_parser("set-youtube-key",
+                          help="store and verify the YouTube Data API key")
+    ykey.add_argument("key", nargs="?", help="omit to be prompted (preferred)")
+    ykey.set_defaults(func=cmd_set_youtube_key)
+
+    sub.add_parser("setup-jellyfin-library",
+                   help="create the Shows library for the media root").set_defaults(
+        func=cmd_setup_jellyfin_library)
+
+    source = sub.add_parser("add-source",
+                            help="mirror a YouTube account's subscriptions")
+    source.add_argument("channel", help="UC... id or @handle")
+    source.set_defaults(func=cmd_add_source)
+
+    sub.add_parser("sources", help="list subscription sources").set_defaults(
+        func=cmd_sources)
+    sub.add_parser("sync", help="pull subscriptions from every source").set_defaults(
+        func=cmd_sync)
+    sub.add_parser("pending", help="channels awaiting approval").set_defaults(
+        func=cmd_pending)
+
+    approve = sub.add_parser("approve", help="subscribe approved channels")
+    approve.add_argument("ids", type=int, nargs="*")
+    approve.add_argument("--all", action="store_true", help="approve everything queued")
+    approve.set_defaults(func=cmd_approve)
+
+    reject = sub.add_parser("reject", help="never queue these channels again")
+    reject.add_argument("ids", type=int, nargs="*")
+    reject.add_argument("--all", action="store_true")
+    reject.set_defaults(func=cmd_reject)
+
+    subscribe = sub.add_parser("subscribe",
+                               help="subscribe by hand (pinned, never auto-removed)")
+    subscribe.add_argument("url", help="channel URL, @handle, or UC... id")
+    subscribe.set_defaults(func=cmd_subscribe)
+
+    unsubscribe = sub.add_parser("unsubscribe",
+                                 help="remove a channel and its whole tree")
+    unsubscribe.add_argument("id", type=int, help="channel id from `channels`")
+    unsubscribe.add_argument("--yes", action="store_true", help="skip confirmation")
+    unsubscribe.set_defaults(func=cmd_unsubscribe)
+
+    sub.add_parser("channels", help="list subscribed channels").set_defaults(
+        func=cmd_channels)
+
+    poll = sub.add_parser("poll", help="discover new videos")
+    poll.add_argument("--channel", type=int, help="restrict to one channel id")
+    poll.add_argument("--rescan", action="store_true",
+                      help="also re-queue videos skipped as too old that the "
+                           "current window now covers (never revives aged-out ones)")
+    poll.set_defaults(func=cmd_poll)
+
+    mat = sub.add_parser("materialise", help="write .strm/.nfo for listed videos")
+    mat.add_argument("--limit", type=int)
+    mat.add_argument("--all", action="store_true",
+                     help="rewrite every episode from the database — the recovery "
+                          "path after a Jellyfin metadata wipe")
+    mat.set_defaults(func=cmd_materialise)
+
+    sub.add_parser("reap", help="delete videos past the retention window").set_defaults(
+        func=cmd_reap)
+
+    run_cmd = sub.add_parser("run", help="sync, poll, materialise, reap (what cron calls)")
+    run_cmd.add_argument("--channel", type=int, help="restrict to one channel")
+    run_cmd.set_defaults(func=cmd_run)
+
+    serve = sub.add_parser("serve", help="run the admin web server")
+    serve.add_argument("--host", default="127.0.0.1")
+    serve.add_argument("--port", type=int, default=8086)
+    serve.add_argument("--insecure-cookies", action="store_true",
+                       help="omit the Secure cookie flag (local http testing only)")
+    serve.set_defaults(func=cmd_serve)
+
+    retention = sub.add_parser("set-retention",
+                               help="set or clear a channel's retention override")
+    retention.add_argument("id", type=int, help="channel id")
+    retention.add_argument("days", help="number of days, or 'default' to clear")
+    retention.set_defaults(func=cmd_set_retention)
+
+    return parser
+
+
+def main(argv: list[str] | None = None) -> int:
+    parser = build_parser()
+    args = parser.parse_args(argv)
+    util.setup_logging(args.verbose)
+    util.apply_umask()
+    return args.func(args)

ytstream/config.py

@@ -0,0 +1,36 @@
+"""Filesystem paths and process-level constants.
+
+Every path is overridable through the environment so the test suite can point the
+whole application at a tmpdir without touching the real media tree.
+"""
+
+from __future__ import annotations
+
+import os
+from pathlib import Path
+
+
+def _path(env: str, default: str) -> Path:
+    return Path(os.environ.get(env, default))
+
+
+STATE_DIR = _path("YTS_STATE_DIR", "/var/lib/ytstream")
+MEDIA_ROOT = _path("YTS_MEDIA_ROOT", "/disks/Plex/_ytstream")
+
+DB_PATH = _path("YTS_DB_PATH", str(STATE_DIR / "ytstream.db"))
+LOCK_PATH = _path("YTS_LOCK_PATH", str(STATE_DIR / "run.lock"))
+VENV_BIN = _path("YTS_VENV_BIN", str(STATE_DIR / "venv" / "bin"))
+
+# Files created by the service must stay group-readable by `mediaserver`, which is
+# how Jellyfin reaches the tree. See plan.md §9.
+UMASK = 0o002
+
+# Sidecars we own and may therefore delete when a video ages out. The `.strm` is
+# the media file itself and is handled separately, so it is deliberately absent.
+SIDECAR_SUFFIXES = (".nfo", "-thumb.jpg")
+
+USER_AGENT = "ytstream/1.0"
+
+# There is deliberately no `.work/` dir under the media root. The proxy's scratch
+# space is tmpfs, outside the library entirely, so ytstream needs neither the
+# dot-prefix nor the `.ignore` file youtube-automate relies on to hide it.

ytstream/db.py

@@ -0,0 +1,149 @@
+"""SQLite access and schema migrations.
+
+WAL is mandatory: the hourly cron job and the long-running admin server both write.
+
+This is schema version 1 of a new database, not a migration of
+youtube-automate's. The two differ enough — no download bookkeeping, a
+subscription source, an approval queue — that carrying the old rows across would
+be more code than the two channels and eighteen rows in it are worth (plan.md §7).
+"""
+
+from __future__ import annotations
+
+import sqlite3
+from pathlib import Path
+
+from . import config
+
+SCHEMA_VERSION = 1
+
+_SCHEMA_V1 = """
+CREATE TABLE IF NOT EXISTS channel (
+  id                INTEGER PRIMARY KEY,
+  channel_id        TEXT NOT NULL UNIQUE,
+  handle            TEXT,
+  title             TEXT NOT NULL,
+  description       TEXT,
+  dir_name          TEXT NOT NULL UNIQUE,
+  added_at          TEXT NOT NULL,
+  backfilled        INTEGER NOT NULL DEFAULT 0,
+
+  -- Which uploads playlist won for this channel: 'UULF' (long-form only) or
+  -- 'UU' (everything, needs duration filtering). Verified 2026-08-12 that the
+  -- API accepts UULF, so this is normally 'UULF'.
+  uploads_playlist  TEXT NOT NULL DEFAULT 'UULF',
+
+  -- Resume point for a bounded backfill: the pageToken of the next page to
+  -- fetch, or NULL when the backfill is complete.
+  backfill_cursor   TEXT,
+
+  -- Per-channel override of the global retention window. NULL = use the setting.
+  retention_days    INTEGER,
+
+  -- Provenance. 'youtube' = came from the mirrored subscription list and may be
+  -- removed by a sync. 'manual' = pinned by hand and never auto-removed.
+  source            TEXT NOT NULL DEFAULT 'youtube',
+
+  -- Consecutive syncs in which this channel was absent from an otherwise-healthy
+  -- subscription response. Only at the threshold is it unsubscribed (plan.md §4.4).
+  missing_syncs     INTEGER NOT NULL DEFAULT 0,
+
+  last_polled_at    TEXT,
+  last_poll_ok      INTEGER,
+  consecutive_poll_failures INTEGER NOT NULL DEFAULT 0
+);
+
+CREATE TABLE IF NOT EXISTS video (
+  id                INTEGER PRIMARY KEY,
+  video_id          TEXT NOT NULL UNIQUE,
+  channel_pk        INTEGER NOT NULL REFERENCES channel(id) ON DELETE CASCADE,
+  title             TEXT,
+
+  -- Date-only, what naming derives season/episode from.
+  upload_date       TEXT,
+  -- Exact RFC-3339 from playlistItems.list. Kept because approximate_date is
+  -- wrong by up to 2 days and episode numbers depend on the date (plan.md §3).
+  published_at      TEXT,
+
+  duration          INTEGER,
+  season            INTEGER,
+  episode           INTEGER,
+  state             TEXT NOT NULL,
+  discovery_source  TEXT NOT NULL,
+
+  -- Path of the .strm relative to the media root, NULL once aged out.
+  rel_path          TEXT,
+
+  discovered_at     TEXT NOT NULL,
+  materialised_at   TEXT,
+  deleted_at        TEXT
+);
+
+CREATE INDEX IF NOT EXISTS idx_video_state ON video(state);
+CREATE INDEX IF NOT EXISTS idx_video_upload_date ON video(upload_date);
+CREATE INDEX IF NOT EXISTS idx_video_channel ON video(channel_pk);
+
+-- One row per mirrored YouTube account. In practice exactly one, but a table
+-- rather than six setting keys because these are fields of one thing that change
+-- together (plan.md §7).
+CREATE TABLE IF NOT EXISTS source (
+  key                  TEXT PRIMARY KEY,
+  label                TEXT NOT NULL,
+  channel_id           TEXT NOT NULL,
+  enabled              INTEGER NOT NULL DEFAULT 1,
+  imported             INTEGER NOT NULL DEFAULT 0,
+  last_sync_at         TEXT,
+  last_sync_ok         INTEGER,
+  consecutive_failures INTEGER NOT NULL DEFAULT 0,
+  last_error           TEXT
+);
+
+-- Channels awaiting a human decision: the whole list on a source's first sync,
+-- and anything over subsync_max_new thereafter.
+CREATE TABLE IF NOT EXISTS pending_approval (
+  id          INTEGER PRIMARY KEY,
+  source      TEXT NOT NULL,
+  channel_id  TEXT NOT NULL,
+  title       TEXT NOT NULL,
+  seen_at     TEXT NOT NULL,
+  resolved_at TEXT,
+  resolution  TEXT,
+  UNIQUE (source, channel_id)
+);
+
+CREATE INDEX IF NOT EXISTS idx_pending_unresolved
+  ON pending_approval(resolved_at);
+
+CREATE TABLE IF NOT EXISTS setting (
+  key   TEXT PRIMARY KEY,
+  value TEXT NOT NULL
+);
+"""
+
+
+def connect(path: Path | None = None) -> sqlite3.Connection:
+    """Open the database, applying migrations if needed."""
+    path = Path(path) if path is not None else config.DB_PATH
+    path.parent.mkdir(parents=True, exist_ok=True)
+
+    conn = sqlite3.connect(path, timeout=30.0, isolation_level=None)
+    conn.row_factory = sqlite3.Row
+    conn.execute("PRAGMA journal_mode = WAL")
+    conn.execute("PRAGMA foreign_keys = ON")
+    conn.execute("PRAGMA busy_timeout = 30000")
+    migrate(conn)
+    return conn
+
+
+def migrate(conn: sqlite3.Connection) -> int:
+    """Bring the schema up to SCHEMA_VERSION. Idempotent."""
+    current = conn.execute("PRAGMA user_version").fetchone()[0]
+    if current >= SCHEMA_VERSION:
+        return current
+
+    with conn:
+        if current < 1:
+            conn.executescript(_SCHEMA_V1)
+        # Future migrations append here, each guarded by `if current < N`.
+        conn.execute(f"PRAGMA user_version = {SCHEMA_VERSION}")
+    return SCHEMA_VERSION

ytstream/discovery.py

@@ -0,0 +1,422 @@
+"""Discovery: RSS polling for what's new, the Data API for the backfill.
+
+Two sources, chosen on cost:
+
+* **RSS** (`feeds/videos.xml?playlist_id=UULF…`) is free, unauthenticated and
+  carries exact publish timestamps. It is the steady-state poller. Measured
+  2026-08-12: all 119 subscribed channels poll in 7.9 s, and for a channel
+  publishing under one long-form video a day the 15-entry feed spans ~23 days —
+  so RSS alone very nearly covers a 30-day retention window.
+* **The Data API** is for the backfill on subscribe, where RSS does not reach far
+  enough, and for durations, which RSS does not carry at all.
+
+The UULF playlist (`UU` with `LF` spliced in) is undocumented but excludes Shorts
+and livestreams at the cheapest possible point — verified both through RSS and,
+on 2026-08-12, through `playlistItems.list`. On a measured channel it excludes 54%
+of the catalogue. The `channel_id` feed is the fallback, and rows discovered that
+way carry `discovery_source='uc_feed'` so the duration pass knows to apply the
+Shorts and livestream filters itself.
+"""
+
+from __future__ import annotations
+
+import logging
+import sqlite3
+import urllib.error
+import urllib.request
+import xml.etree.ElementTree as ET
+from datetime import date, timedelta
+
+from . import api, channels, config, strm, util, videos
+from .settings import Settings
+
+log = logging.getLogger(__name__)
+
+NS = {
+    "atom": "http://www.w3.org/2005/Atom",
+    "yt": "http://www.youtube.com/xml/schemas/2015",
+    "media": "http://search.yahoo.com/mrss/",
+}
+
+FEED_BASE = "https://www.youtube.com/feeds/videos.xml"
+
+
+class FeedUnavailable(Exception):
+    """The feed could not be fetched at all (network/5xx). Not the same as 404."""
+
+
+def uulf_feed_url(channel_id: str) -> str:
+    return f"{FEED_BASE}?playlist_id={channels.uulf_playlist_id(channel_id)}"
+
+
+def uc_feed_url(channel_id: str) -> str:
+    return f"{FEED_BASE}?channel_id={channel_id}"
+
+
+def fetch_feed(url: str, timeout: float = 30.0) -> bytes | None:
+    """Return the feed body, or None if YouTube says it doesn't exist.
+
+    A 404 on UULF means "no such playlist", i.e. the channel has no long-form
+    videos at all — that is not an error. Two of 119 measured channels return
+    hard errors here because they were terminated or made private while still
+    appearing in the subscription list.
+    """
+    request = urllib.request.Request(url, headers={"User-Agent": config.USER_AGENT})
+    try:
+        with urllib.request.urlopen(request, timeout=timeout) as response:
+            return response.read()
+    except urllib.error.HTTPError as exc:
+        if exc.code == 404:
+            return None
+        raise FeedUnavailable(f"HTTP {exc.code}") from exc
+    except OSError as exc:
+        raise FeedUnavailable(str(exc)) from exc
+
+
+def parse_entries(payload: bytes) -> list[dict]:
+    """Parse an Atom feed into video dicts. The feed carries no duration.
+
+    Entries are read from `atom:entry` elements only. The feed also has a
+    top-level `published` (the playlist's own creation date, which can be years
+    old); scraping timestamps with a regex instead picks that up and produces
+    nonsense upload rates.
+    """
+    try:
+        root = ET.fromstring(payload)
+    except ET.ParseError as exc:
+        raise FeedUnavailable(f"unparseable feed: {exc}") from exc
+
+    entries = []
+    for entry in root.findall("atom:entry", NS):
+        video_id = entry.findtext("yt:videoId", "", NS)
+        if not video_id:
+            continue
+        published = entry.findtext("atom:published", "", NS)
+        try:
+            published_date = date.fromisoformat(published[:10])
+        except ValueError:
+            continue
+        entries.append(
+            {
+                "video_id": video_id,
+                "title": (entry.findtext("atom:title", "", NS) or "").strip(),
+                "published": published_date,
+                "published_at": published,
+            }
+        )
+    return entries
+
+
+def effective_retention_days(settings: Settings, channel: sqlite3.Row) -> int:
+    override = channel["retention_days"] if "retention_days" in channel.keys() else None
+    if override:
+        return int(override)
+    return settings.get_int("retention_days")
+
+
+# --------------------------------------------------------------------------
+# recording
+
+
+def _record(
+    conn: sqlite3.Connection,
+    channel: sqlite3.Row,
+    entry: dict,
+    source: str,
+    cutoff: date,
+) -> str:
+    """Insert or repair one discovered video. Returns what happened."""
+    existing = videos.get(conn, entry["video_id"])
+
+    if existing is not None:
+        # An aged-out row is a tombstone. Touching it here is what would make the
+        # retention sweep and the poller fight each other forever.
+        if existing["state"] in videos.NEVER_REVIVE:
+            return "known"
+        # The only repair we perform: a video the fallback path rejected as too
+        # short, later confirmed long-form by the authoritative UULF feed.
+        if (
+            source == videos.SOURCE_UULF
+            and existing["state"] == videos.SKIPPED_SHORT
+            and existing["discovery_source"] == videos.SOURCE_UC
+        ):
+            videos.set_state(conn, entry["video_id"], videos.LISTED)
+            with conn:
+                conn.execute(
+                    "UPDATE video SET discovery_source = ? WHERE video_id = ?",
+                    (videos.SOURCE_UULF, entry["video_id"]),
+                )
+            log.info("re-queued %s: UULF confirms it is long-form", entry["video_id"])
+            return "repaired"
+        # A row can still arrive here untitled — a video that was private when the
+        # backfill saw it, for instance. If a later feed supplies the title, take
+        # it, and if the episode is already on disk under its video id, remove the
+        # files and re-queue so it is rewritten under the real name.
+        if not (existing["title"] or "").strip() and entry["title"]:
+            with conn:
+                conn.execute(
+                    "UPDATE video SET title = ? WHERE video_id = ?",
+                    (entry["title"], entry["video_id"]),
+                )
+            if existing["state"] == videos.MATERIALISED:
+                strm.remove(existing)
+                videos.set_state(conn, entry["video_id"], videos.LISTED)
+                log.info("%s: title arrived late, re-materialising",
+                         entry["video_id"])
+            return "titled"
+        return "known"
+
+    state = videos.LISTED if entry["published"] >= cutoff else videos.SKIPPED_OLD
+    videos.insert(
+        conn,
+        channel_pk=channel["id"],
+        video_id=entry["video_id"],
+        title=entry["title"],
+        upload_date=entry["published"].isoformat(),
+        published_at=entry.get("published_at"),
+        state=state,
+        discovery_source=source,
+    )
+    return "queued" if state == videos.LISTED else "old"
+
+
+def _feed_entries(channel: sqlite3.Row) -> tuple[list[dict], str]:
+    """UULF if it has anything, else the channel feed. Raises FeedUnavailable."""
+    payload = fetch_feed(uulf_feed_url(channel["channel_id"]))
+    entries = parse_entries(payload) if payload else []
+    if entries:
+        return entries, videos.SOURCE_UULF
+
+    log.warning("UULF feed empty for %s, falling back to channel_id feed",
+                channel["title"])
+    payload = fetch_feed(uc_feed_url(channel["channel_id"]))
+    return (parse_entries(payload) if payload else []), videos.SOURCE_UC
+
+
+def poll_channel(
+    conn: sqlite3.Connection, settings: Settings, channel: sqlite3.Row
+) -> dict:
+    """Poll one channel. Never raises for feed problems — records them instead."""
+    stats = {"queued": 0, "old": 0, "known": 0, "repaired": 0, "titled": 0,
+             "source": None}
+
+    try:
+        entries, source = _feed_entries(channel)
+    except FeedUnavailable as exc:
+        _record_poll_failure(conn, channel, str(exc))
+        stats["error"] = str(exc)
+        return stats
+
+    cutoff = util.today() - timedelta(
+        days=effective_retention_days(settings, channel)
+    )
+    fresh: list[str] = []
+    for entry in entries:
+        outcome = _record(conn, channel, entry, source, cutoff)
+        stats[outcome] += 1
+        if outcome == "queued":
+            fresh.append(entry["video_id"])
+
+    # Only the fallback path needs filtering — UULF is pre-filtered by YouTube —
+    # but every new video needs a duration for its NFO either way.
+    if fresh:
+        stats.update(enrich_durations(conn, settings, fresh))
+
+    stats["source"] = source
+    _record_poll_success(conn, channel)
+    return stats
+
+
+def _record_poll_success(conn: sqlite3.Connection, channel: sqlite3.Row) -> None:
+    with conn:
+        conn.execute(
+            "UPDATE channel SET last_polled_at = ?, last_poll_ok = 1, "
+            "consecutive_poll_failures = 0 WHERE id = ?",
+            (util.utcnow_iso(), channel["id"]),
+        )
+
+
+def _record_poll_failure(conn: sqlite3.Connection, channel: sqlite3.Row, error: str) -> None:
+    log.error("poll failed for %s: %s", channel["title"], error)
+    with conn:
+        conn.execute(
+            "UPDATE channel SET last_polled_at = ?, last_poll_ok = 0, "
+            "consecutive_poll_failures = consecutive_poll_failures + 1 WHERE id = ?",
+            (util.utcnow_iso(), channel["id"]),
+        )
+
+
+# --------------------------------------------------------------------------
+# durations
+
+
+def enrich_durations(
+    conn: sqlite3.Connection, settings: Settings, video_ids: list[str]
+) -> dict:
+    """Fill in durations from the API, and filter Shorts and livestreams.
+
+    RSS carries no duration, so without this every episode would show a runtime
+    of zero until it had been played once. Batched 50 per call, so the whole
+    441-video library costs 9 quota units.
+    """
+    stats = {"resolved": 0, "shorts": 0, "live": 0}
+    if not video_ids:
+        return stats
+
+    minimum = settings.get_int("min_duration_seconds")
+    client = api.Api(settings.get_str("youtube_api_key"))
+    try:
+        found = client.durations(video_ids)
+    except api.ApiError as exc:
+        # Durations are an enrichment, not a gate: a NULL duration costs a runtime
+        # display, not a working library.
+        log.warning("duration lookup failed for %d video(s): %s", len(video_ids), exc)
+        return stats
+
+    for video_id, info in found.items():
+        videos.set_duration(conn, video_id, info["duration"])
+        stats["resolved"] += 1
+        if info["is_live"]:
+            videos.set_state(conn, video_id, videos.SKIPPED_LIVE)
+            stats["live"] += 1
+        elif info["duration"] is not None and info["duration"] < minimum:
+            videos.set_state(conn, video_id, videos.SKIPPED_SHORT)
+            stats["shorts"] += 1
+    return stats
+
+
+# --------------------------------------------------------------------------
+# backfill
+
+
+def _set_cursor(conn: sqlite3.Connection, channel_pk: int, token: str | None) -> None:
+    with conn:
+        conn.execute(
+            "UPDATE channel SET backfill_cursor = ? WHERE id = ?", (token, channel_pk)
+        )
+
+
+def backfill_channel(
+    conn: sqlite3.Connection, settings: Settings, channel: sqlite3.Row
+) -> dict:
+    """Walk a channel's uploads back to the retention window, via the Data API.
+
+    This replaces youtube-automate's yt-dlp backfill for one reason above all:
+    `--flat-playlist` reports no timestamp, and `approximate_date` is wrong by up
+    to two days. Season and episode derive from the upload date and Jellyfin
+    caches episode numbers, so a date wrong by a day is a mistake that can only be
+    fixed by wiping metadata. `playlistItems.list` returns the exact publish time.
+
+    Resumable: the page token is committed per page, so a crash mid-backfill
+    resumes rather than restarting the channel.
+    """
+    stats = {"queued": 0, "pages": 0, "resolved": 0, "shorts": 0, "live": 0}
+    cutoff = util.today() - timedelta(
+        days=effective_retention_days(settings, channel)
+    )
+    limit = settings.get_int("backfill_max_videos") or None
+
+    client = api.Api(settings.get_str("youtube_api_key"))
+    kind = channel["uploads_playlist"] or "UULF"
+    fresh: list[str] = []
+
+    try:
+        for entry, next_token in client.uploads(
+            channel["channel_id"], kind=kind, since=cutoff, limit=limit,
+            page_token=channel["backfill_cursor"],
+        ):
+            if not videos.exists(conn, entry["video_id"]):
+                videos.insert(
+                    conn,
+                    channel_pk=channel["id"],
+                    video_id=entry["video_id"],
+                    # The title comes from the same call as the date. Relying on
+                    # the RSS poll instead left the oldest ~5 of every 20-episode
+                    # backfill named after their video id, because RSS reaches
+                    # back only ~23 days against a 30-day window.
+                    title=entry.get("title") or "",
+                    upload_date=entry["published"].isoformat(),
+                    published_at=entry["published_at"],
+                    state=videos.LISTED,
+                    discovery_source=videos.SOURCE_BACKFILL,
+                )
+                fresh.append(entry["video_id"])
+                stats["queued"] += 1
+            _set_cursor(conn, channel["id"], next_token)
+            stats["pages"] += 1
+    except api.NotConfigured as exc:
+        # No key yet. Leave `backfilled` unset so it retries once configured.
+        log.error("backfill skipped for %s: %s", channel["title"], exc)
+        stats["error"] = str(exc)
+        return stats
+    except api.ApiError as exc:
+        log.error("backfill failed for %s: %s", channel["title"], exc)
+        stats["error"] = str(exc)
+        return stats
+
+    stats.update(enrich_durations(conn, settings, fresh))
+    _set_cursor(conn, channel["id"], None)
+    with conn:
+        conn.execute("UPDATE channel SET backfilled = 1 WHERE id = ?",
+                     (channel["id"],))
+    return stats
+
+
+def rescan_channel(
+    conn: sqlite3.Connection, settings: Settings, channel: sqlite3.Row
+) -> int:
+    """Re-queue `skipped_old` rows that a widened retention window now covers.
+
+    `skipped_old` is judged against whatever window was in force at discovery
+    time and is otherwise terminal. Without this, raising retention_days would
+    appear to do nothing for an infrequent uploader.
+
+    `aged_out` rows are deliberately NOT revived. They were on disk once and were
+    deleted; bringing them back would present months of old episodes to Jellyfin
+    as new, and the whole point of the tombstone is that this cannot happen.
+    """
+    cutoff = util.today() - timedelta(
+        days=effective_retention_days(settings, channel)
+    )
+    with conn:
+        cursor = conn.execute(
+            "UPDATE video SET state = ? "
+            "WHERE channel_pk = ? AND state = ? AND upload_date >= ?",
+            (videos.LISTED, channel["id"], videos.SKIPPED_OLD, cutoff.isoformat()),
+        )
+    if cursor.rowcount:
+        log.info("%s: re-queued %d video(s) now inside the %s window",
+                 channel["title"], cursor.rowcount, cutoff.isoformat())
+    return cursor.rowcount
+
+
+def poll_all(
+    conn: sqlite3.Connection, settings: Settings, channel_pk: int | None = None
+) -> dict:
+    """Backfill anything new, then poll everything. Returns aggregate counts."""
+    if channel_pk is not None:
+        rows = [row for row in [channels.get(conn, channel_pk)] if row is not None]
+    else:
+        rows = channels.all_channels(conn)
+
+    totals = {"channels": 0, "queued": 0, "old": 0, "known": 0, "repaired": 0,
+              "titled": 0, "shorts": 0, "live": 0, "failed": 0}
+    for channel in rows:
+        totals["channels"] += 1
+        if not channel["backfilled"]:
+            stats = backfill_channel(conn, settings, channel)
+            # Poll straight after a backfill: it is what supplies the titles the
+            # contentDetails pages do not carry.
+            if "error" not in stats:
+                poll_stats = poll_channel(conn, settings, channel)
+                for key in ("titled", "known"):
+                    stats[key] = stats.get(key, 0) + poll_stats.get(key, 0)
+        else:
+            stats = poll_channel(conn, settings, channel)
+
+        if "error" in stats:
+            totals["failed"] += 1
+        for key in ("queued", "old", "known", "repaired", "titled", "shorts", "live"):
+            totals[key] += stats.get(key, 0)
+        log.info("%s: %s", channel["title"], stats)
+    return totals

ytstream/doctor.py

@@ -0,0 +1,211 @@
+"""Preflight checks.
+
+`doctor` is the first acceptance criterion and the thing to run when something
+breaks. Every check returns a row rather than raising, so one failure doesn't
+hide the others.
+"""
+
+from __future__ import annotations
+
+import grp
+import os
+import stat
+import subprocess
+from dataclasses import dataclass
+from pathlib import Path
+
+from . import config, jellyfin, ytdlp
+from .settings import Settings
+
+MEDIA_GROUP = "mediaserver"
+
+
+@dataclass
+class Check:
+    name: str
+    ok: bool
+    detail: str
+    fatal: bool = True
+
+
+def _deno() -> Check:
+    binary = config.VENV_BIN / "deno"
+    if not binary.exists():
+        return Check(
+            "js runtime",
+            False,
+            f"deno not found at {binary} — yt-dlp cannot solve n challenges (specs.md §3)",
+        )
+    try:
+        result = subprocess.run(
+            [str(binary), "--version"], capture_output=True, text=True, timeout=30
+        )
+    except OSError as exc:
+        return Check("js runtime", False, f"deno unusable: {exc}")
+    if result.returncode != 0:
+        return Check("js runtime", False, "deno --version failed")
+    return Check("js runtime", True, result.stdout.splitlines()[0])
+
+
+def _ytdlp() -> Check:
+    try:
+        return Check("yt-dlp", True, ytdlp.version())
+    except (OSError, ytdlp.YtdlpError) as exc:
+        return Check("yt-dlp", False, str(exc))
+
+
+def _ejs() -> Check:
+    try:
+        from importlib.metadata import version as pkg_version
+
+        return Check("yt-dlp-ejs", True, pkg_version("yt-dlp-ejs"))
+    except Exception:
+        return Check(
+            "yt-dlp-ejs",
+            False,
+            "not installed — reinstall with the yt-dlp[default] extra (specs.md §3)",
+        )
+
+
+def _pot(settings: Settings) -> Check:
+    url = settings.get_str("pot_provider_url")
+    try:
+        info = ytdlp.pot_provider_ping(url)
+    except Exception as exc:
+        return Check("pot provider", False, f"{url}/ping unreachable: {exc}")
+
+    server_version = str(info.get("version", "?"))
+    installed = ytdlp.plugin_version()
+    if installed and installed != server_version:
+        return Check(
+            "pot provider",
+            False,
+            f"version skew: server {server_version} vs plugin {installed}",
+        )
+    return Check("pot provider", True, f"up, version {server_version}")
+
+
+def _database() -> Check:
+    try:
+        from . import db
+
+        conn = db.connect()
+        version = conn.execute("PRAGMA user_version").fetchone()[0]
+        mode = conn.execute("PRAGMA journal_mode").fetchone()[0]
+        conn.close()
+    except Exception as exc:
+        return Check("database", False, str(exc))
+    if str(mode).lower() != "wal":
+        return Check("database", False, f"journal_mode is {mode}, expected wal")
+    return Check("database", True, f"{config.DB_PATH} (schema v{version}, {mode})")
+
+
+def _media_root() -> Check:
+    root = config.MEDIA_ROOT
+    if not root.is_dir():
+        return Check("media root", False, f"{root} does not exist")
+    if not os.access(root, os.W_OK | os.X_OK):
+        return Check("media root", False, f"{root} is not writable")
+
+    info = root.stat()
+    problems = []
+    try:
+        group = grp.getgrgid(info.st_gid).gr_name
+    except KeyError:
+        group = str(info.st_gid)
+    if group != MEDIA_GROUP:
+        problems.append(f"group is {group}, expected {MEDIA_GROUP}")
+    if not info.st_mode & stat.S_ISGID:
+        problems.append("setgid bit not set (new dirs won't inherit the group)")
+
+    if problems:
+        return Check("media root", False, f"{root}: " + "; ".join(problems))
+    return Check("media root", True, f"{root} ({group}, setgid)")
+
+
+def _proxy() -> Check:
+    """The proxy is what turns a .strm into video. Without it nothing plays.
+
+    Replaces youtube-automate's work-dir check: ytstream has no work dir, because
+    the proxy's scratch space is tmpfs outside the media tree entirely.
+    """
+    import json
+    import urllib.error
+    import urllib.request
+
+    url = "http://127.0.0.1:8099/healthz"
+    try:
+        with urllib.request.urlopen(url, timeout=5) as response:
+            payload = json.load(response)
+    except (urllib.error.URLError, OSError, ValueError) as exc:
+        return Check("proxy", False, f"{url} unreachable ({exc}) — nothing will play")
+    mode = payload.get("mode", "?")
+    used = payload.get("cache_used_gb", 0)
+    return Check("proxy", True, f"{url} ok (mode={mode}, cache={used:.2f} GB)")
+
+
+
+
+def _jellyfin(settings: Settings) -> Check:
+    client = jellyfin.from_settings(settings)
+    if not client.base_url:
+        return Check("jellyfin", False, "jellyfin_url not set", fatal=False)
+    try:
+        info = client.public_info()
+    except jellyfin.JellyfinError as exc:
+        return Check("jellyfin", False, str(exc))
+
+    label = f"{info.get('ServerName', '?')} {info.get('Version', '?')}"
+    if not client.api_key:
+        return Check(
+            "jellyfin",
+            False,
+            f"{label} reachable but no API key set (run set-jellyfin-key)",
+            fatal=False,
+        )
+    try:
+        library = client.find_library(config.MEDIA_ROOT)
+    except jellyfin.JellyfinError as exc:
+        return Check("jellyfin", False, f"API key rejected: {exc}")
+    if library is None:
+        return Check(
+            "jellyfin",
+            False,
+            f"{label}, no library for {config.MEDIA_ROOT} (run setup-jellyfin-library)",
+            fatal=False,
+        )
+    return Check("jellyfin", True, f"{label}, library '{library.get('Name')}'")
+
+
+def run_checks(settings: Settings) -> list[Check]:
+    return [
+        _ytdlp(),
+        _ejs(),
+        _deno(),
+        _pot(settings),
+        _database(),
+        _media_root(),
+        _proxy(),
+        _jellyfin(settings),
+    ]
+
+
+def report(checks: list[Check]) -> tuple[str, int]:
+    """Render the checks and return (text, exit_code)."""
+    lines = []
+    failed_fatal = 0
+    for check in checks:
+        if check.ok:
+            mark = "ok  "
+        elif check.fatal:
+            mark = "FAIL"
+            failed_fatal += 1
+        else:
+            mark = "warn"
+        lines.append(f"  [{mark}] {check.name:<14} {check.detail}")
+
+    if failed_fatal:
+        lines.append(f"\n{failed_fatal} fatal problem(s).")
+    else:
+        lines.append("\nAll fatal checks passed.")
+    return "\n".join(lines), (1 if failed_fatal else 0)

ytstream/jellyfin.py

@@ -0,0 +1,145 @@
+"""Minimal Jellyfin API client.
+
+Only three things are needed: check the server is alive, create the Shows library
+with internet metadata providers switched off, and trigger a refresh after we
+change the tree.
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+import urllib.error
+import urllib.parse
+import urllib.request
+from pathlib import Path
+
+from . import config
+
+log = logging.getLogger(__name__)
+
+LIBRARY_NAME = "YouTube (stream)"
+COLLECTION_TYPE = "tvshows"
+
+# Metadata is supplied entirely by our own NFO sidecars, so every fetcher is
+# disabled for all three item types a Shows library resolves.
+_ITEM_TYPES = ("Series", "Season", "Episode")
+
+
+class JellyfinError(RuntimeError):
+    pass
+
+
+class Jellyfin:
+    def __init__(self, base_url: str, api_key: str = "", timeout: float = 30.0):
+        self.base_url = (base_url or "").rstrip("/")
+        self.api_key = api_key or ""
+        self.timeout = timeout
+
+    @property
+    def configured(self) -> bool:
+        return bool(self.base_url and self.api_key)
+
+    def _request(
+        self,
+        method: str,
+        path: str,
+        params: dict | None = None,
+        body: dict | None = None,
+    ):
+        url = self.base_url + path
+        if params:
+            url += "?" + urllib.parse.urlencode(params)
+        data = None
+        headers = {"User-Agent": config.USER_AGENT, "Accept": "application/json"}
+        if self.api_key:
+            headers["X-Emby-Token"] = self.api_key
+        if body is not None:
+            data = json.dumps(body).encode("utf-8")
+            headers["Content-Type"] = "application/json"
+
+        request = urllib.request.Request(url, data=data, headers=headers, method=method)
+        try:
+            with urllib.request.urlopen(request, timeout=self.timeout) as response:
+                payload = response.read()
+        except urllib.error.HTTPError as exc:
+            raise JellyfinError(f"{method} {path} -> HTTP {exc.code}") from exc
+        except OSError as exc:
+            raise JellyfinError(f"{method} {path} -> {exc}") from exc
+
+        if not payload:
+            return None
+        try:
+            return json.loads(payload)
+        except json.JSONDecodeError:
+            return None
+
+    def public_info(self) -> dict:
+        """Unauthenticated liveness check."""
+        return self._request("GET", "/System/Info/Public") or {}
+
+    def virtual_folders(self) -> list[dict]:
+        return self._request("GET", "/Library/VirtualFolders") or []
+
+    def find_library(self, path: Path | str) -> dict | None:
+        target = str(path).rstrip("/")
+        for folder in self.virtual_folders():
+            for location in folder.get("Locations") or []:
+                if str(location).rstrip("/") == target:
+                    return folder
+        return None
+
+    def create_library(self, path: Path | str, name: str = LIBRARY_NAME) -> None:
+        """Create the Shows library with all internet providers disabled."""
+        options = {
+            "EnableInternetProviders": False,
+            "SaveLocalMetadata": True,
+            "EnableRealtimeMonitor": False,
+            "EnableChapterImageExtraction": False,
+            "PathInfos": [{"Path": str(path)}],
+            "TypeOptions": [
+                {
+                    "Type": item_type,
+                    "MetadataFetchers": [],
+                    "MetadataFetcherOrder": [],
+                    "ImageFetchers": [],
+                    "ImageFetcherOrder": [],
+                }
+                for item_type in _ITEM_TYPES
+            ],
+        }
+        self._request(
+            "POST",
+            "/Library/VirtualFolders",
+            params={
+                "name": name,
+                "collectionType": COLLECTION_TYPE,
+                "paths": str(path),
+                "refreshLibrary": "false",
+            },
+            body={"LibraryOptions": options},
+        )
+
+    def refresh(self) -> None:
+        """Trigger a library scan. Best effort — never fatal to the caller.
+
+        Deliberately a plain `/Library/Refresh` with no query parameters. A normal
+        scan was measured to make **zero** media probes, which is the single fact
+        this whole design rests on: a scan of a .strm library must not cause the
+        proxy to fetch every video nobody is watching.
+
+        `metadataRefreshMode=FullRefresh` / `replaceAllMetadata=true` *does* probe.
+        Never add either here, whatever a future caller seems to want — the
+        correct way to rebuild metadata is to re-materialise the tree from the
+        database, which is `ytstream materialise --all`.
+        """
+        try:
+            self._request("POST", "/Library/Refresh")
+        except JellyfinError as exc:
+            log.warning("jellyfin refresh failed: %s", exc)
+
+
+def from_settings(settings) -> Jellyfin:
+    return Jellyfin(
+        settings.get_str("jellyfin_url"), settings.get_str("jellyfin_api_key")
+    )

ytstream/naming.py

@@ -0,0 +1,104 @@
+"""Filename sanitisation and season/episode numbering.
+
+Season is the upload year; episode is ``MMDD * 10 + ordinal_within_day``. That
+scheme sorts correctly across a whole year (1 Jan is 1010, 31 Dec is 12310) and
+leaves room for ten uploads per channel per day.
+"""
+
+from __future__ import annotations
+
+import logging
+import re
+from datetime import date
+
+log = logging.getLogger(__name__)
+
+# Characters that are illegal or awkward in filenames on the platforms Jellyfin
+# clients run on. Replaced with a space rather than deleted so that "A/B" reads
+# as "A B" instead of collapsing into "AB".
+FORBIDDEN = '/\\:*?"<>|'
+
+MAX_TITLE_LEN = 120
+MAX_ORDINAL = 9
+
+_CONTROL = re.compile(r"[\x00-\x1f\x7f]")
+_WHITESPACE = re.compile(r"\s+")
+
+
+def sanitize_component(text: str, max_len: int = MAX_TITLE_LEN) -> str:
+    """Make one path component safe, collapsing whitespace and truncating."""
+    text = _CONTROL.sub(" ", text or "")
+    text = "".join(" " if char in FORBIDDEN else char for char in text)
+    text = _WHITESPACE.sub(" ", text).strip()
+    text = truncate_on_word_boundary(text, max_len)
+    # A component may not begin or end with a dot or space: leading dots hide the
+    # file from Jellyfin, trailing ones confuse some clients.
+    text = text.strip(" .")
+    return text
+
+
+def truncate_on_word_boundary(text: str, max_len: int) -> str:
+    if len(text) <= max_len:
+        return text
+    cut = text[:max_len]
+    space = cut.rfind(" ")
+    # Only honour the word boundary if it doesn't throw away most of the name.
+    if space > max_len * 0.6:
+        cut = cut[:space]
+    return cut.rstrip()
+
+
+def channel_dir_name(title: str, channel_id: str) -> str:
+    """Directory name for a channel. Stored once and never recomputed."""
+    name = sanitize_component(title)
+    return name or channel_id
+
+
+def parse_upload_date(value: str | date) -> date:
+    """Accept yt-dlp's YYYYMMDD, ISO YYYY-MM-DD, or a date."""
+    if isinstance(value, date):
+        return value
+    text = str(value).strip()
+    if len(text) == 8 and text.isdigit():
+        return date(int(text[:4]), int(text[4:6]), int(text[6:8]))
+    return date.fromisoformat(text[:10])
+
+
+def season_for(upload_date: date) -> int:
+    return upload_date.year
+
+
+def episode_base(upload_date: date) -> int:
+    """First episode number available on this date."""
+    return (upload_date.month * 100 + upload_date.day) * 10
+
+
+def episode_number(upload_date: date, ordinal: int) -> int:
+    """Episode number for the nth upload on a given date (n starting at 0)."""
+    if ordinal > MAX_ORDINAL:
+        log.warning(
+            "more than %d uploads on %s; clamping ordinal %d",
+            MAX_ORDINAL + 1,
+            upload_date.isoformat(),
+            ordinal,
+        )
+    return episode_base(upload_date) + min(max(ordinal, 0), MAX_ORDINAL)
+
+
+def episode_range(upload_date: date) -> tuple[int, int]:
+    """Inclusive (low, high) episode numbers belonging to this date."""
+    base = episode_base(upload_date)
+    return base, base + MAX_ORDINAL
+
+
+def season_dir_name(season: int) -> str:
+    return f"Season {season}"
+
+
+def basename(channel_dir: str, season: int, episode: int, title: str, video_id: str) -> str:
+    """Filename stem shared by the media file and every sidecar.
+
+    The [video_id] suffix guarantees uniqueness regardless of title collisions.
+    """
+    safe_title = sanitize_component(title) or video_id
+    return f"{channel_dir} - S{season}E{episode} - {safe_title} [{video_id}]"

ytstream/nfo.py

@@ -0,0 +1,86 @@
+"""Kodi-style NFO sidecars.
+
+Video descriptions are hostile input — they contain ampersands, angle brackets,
+emoji, ASCII art and control characters — so these are always built with
+ElementTree's serialiser and never by string formatting.
+"""
+
+from __future__ import annotations
+
+import re
+import xml.etree.ElementTree as ET
+from pathlib import Path
+
+# XML 1.0 forbids most control characters outright; ElementTree will happily
+# serialise them and produce a document no parser will read back. Written as a
+# raw string so `re` interprets the escapes, not Python.
+_ILLEGAL_XML = re.compile(
+    r"[^\x09\x0a\x0d\x20-퟿-�\U00010000-\U0010ffff]"
+)
+
+
+def clean_text(value: str | None) -> str:
+    return _ILLEGAL_XML.sub("", value or "")
+
+
+def _child(parent: ET.Element, tag: str, text: str | None) -> ET.Element:
+    element = ET.SubElement(parent, tag)
+    element.text = clean_text(text)
+    return element
+
+
+def _serialise(root: ET.Element) -> bytes:
+    ET.indent(root, space="  ")
+    return ET.tostring(root, encoding="utf-8", xml_declaration=True)
+
+
+def tvshow_nfo(title: str, plot: str | None, channel_id: str) -> bytes:
+    root = ET.Element("tvshow")
+    _child(root, "title", title)
+    _child(root, "plot", plot)
+    _child(root, "studio", "YouTube")
+    unique = _child(root, "uniqueid", channel_id)
+    unique.set("type", "youtube")
+    unique.set("default", "true")
+    return _serialise(root)
+
+
+def episode_nfo(
+    *,
+    title: str,
+    show_title: str,
+    season: int,
+    episode: int,
+    plot: str | None,
+    aired: str,
+    duration_seconds: int | None,
+    video_id: str,
+) -> bytes:
+    root = ET.Element("episodedetails")
+    _child(root, "title", title)
+    _child(root, "showtitle", show_title)
+    _child(root, "season", str(season))
+    _child(root, "episode", str(episode))
+    _child(root, "plot", plot)
+    _child(root, "aired", aired)
+    if duration_seconds:
+        # Kodi/Jellyfin expect <runtime> in whole minutes.
+        _child(root, "runtime", str(max(1, round(duration_seconds / 60))))
+        # Jellyfin prefers this when present, and it is what stops a .strm episode
+        # showing a runtime of zero before it has ever been played. Deliberately
+        # NOT wrapped in <fileinfo><streamdetails>: pre-seeding those was measured
+        # to change nothing about whether Jellyfin probes the media (FINDINGS §6).
+        _child(root, "durationinseconds", str(int(duration_seconds)))
+    _child(root, "studio", "YouTube")
+    unique = _child(root, "uniqueid", video_id)
+    unique.set("type", "youtube")
+    unique.set("default", "true")
+    return _serialise(root)
+
+
+def write(path: Path, payload: bytes) -> None:
+    """Write atomically so a crash never leaves Jellyfin a half-written NFO."""
+    path.parent.mkdir(parents=True, exist_ok=True)
+    temporary = path.with_name(path.name + ".tmp")
+    temporary.write_bytes(payload)
+    temporary.replace(path)

ytstream/reap.py

@@ -0,0 +1,89 @@
+"""Retention: delete videos that have aged out of the window.
+
+Two rules, and the second is what makes the first usable:
+
+1. A video older than `retention_days` is deleted from disk and tombstoned.
+2. **Except** the `min_keep_videos` most recent videos of each channel, which are
+   always kept however old they are.
+
+Rule 2 exists because of a measurement, not a preference. Of 117 subscribed
+channels, 52 uploaded nothing long-form in a 30-day window and the median channel
+uploaded once. Under rule 1 alone, most channels would be empty Jellyfin series,
+and a channel uploading every six weeks would flicker in and out of existence as
+its single video crossed the line. Keeping the last few videos regardless of age
+costs ~260 episodes across the whole library and removes the flicker entirely,
+because a video can only leave once `min_keep_videos` newer ones exist.
+
+Deletion leaves the row behind as a tombstone in state `aged_out`. That is
+load-bearing twice over: without it the next poll finds the video in the feed and
+re-materialises it forever, and episode ordinals for a given day would shift as
+videos disappear.
+"""
+
+from __future__ import annotations
+
+import logging
+import sqlite3
+from datetime import timedelta
+
+from . import jellyfin, strm, util, videos
+from .settings import Settings
+
+log = logging.getLogger(__name__)
+
+
+def effective_retention(settings: Settings, channel_override: int | None) -> int:
+    return int(channel_override) if channel_override else settings.get_int("retention_days")
+
+
+def candidates(conn: sqlite3.Connection, settings: Settings) -> list[sqlite3.Row]:
+    """Materialised videos past their window, excluding the protected newest N.
+
+    Evaluated per channel because both bounds are per channel: the window may be
+    overridden on the channel row, and `min_keep_videos` counts within it.
+    """
+    keep = max(0, settings.get_int("min_keep_videos"))
+    today = util.today()
+    due: list[sqlite3.Row] = []
+
+    channels = conn.execute(
+        "SELECT id, title, retention_days FROM channel"
+    ).fetchall()
+
+    for channel in channels:
+        days = effective_retention(settings, channel["retention_days"])
+        cutoff = (today - timedelta(days=days)).isoformat()
+        # Newest first, so everything from index `keep` onwards is unprotected.
+        on_disk = videos.materialised_for_channel(conn, channel["id"])
+        for video in on_disk[keep:]:
+            if video["upload_date"] and video["upload_date"] < cutoff:
+                due.append(video)
+    return due
+
+
+def delete_video(conn: sqlite3.Connection, video: sqlite3.Row) -> bool:
+    """Delete one video's files and leave a tombstone row."""
+    if not video["rel_path"]:
+        # Already gone from disk; still needs the tombstone.
+        videos.mark_aged_out(conn, video["video_id"])
+        return False
+
+    removed = strm.remove(video)
+    videos.mark_aged_out(conn, video["video_id"])
+    log.info("aged out %s (uploaded %s): %d file(s)",
+             video["video_id"], video["upload_date"], removed)
+    return True
+
+
+def run(conn: sqlite3.Connection, settings: Settings) -> dict:
+    """One retention pass."""
+    deleted = 0
+    for video in candidates(conn, settings):
+        if delete_video(conn, video):
+            deleted += 1
+
+    if deleted:
+        # Without this, Jellyfin shows ghost episodes until its own scheduled scan.
+        jellyfin.from_settings(settings).refresh()
+
+    return {"aged_out": deleted}

ytstream/runner.py

@@ -0,0 +1,149 @@
+"""`run` orchestration: sync, then poll, then materialise, then reap — under a lock.
+
+Order matters. The subscription sync runs *first* so that a channel added on
+YouTube at 14:00 has its catalogue built in the same pass rather than an hour
+later. Retention runs *last* so a video discovered and materialised in this run is
+judged against the window once, not twice.
+"""
+
+from __future__ import annotations
+
+import contextlib
+import fcntl
+import logging
+import sqlite3
+from pathlib import Path
+
+from . import config, discovery, jellyfin, reap, strm, subsync, util, videos
+from .settings import Settings
+
+log = logging.getLogger(__name__)
+
+
+class AlreadyRunning(Exception):
+    pass
+
+
+@contextlib.contextmanager
+def exclusive_lock(path: Path | None = None):
+    """Non-blocking flock. Raises AlreadyRunning if another run holds it.
+
+    The cron schedule is hourly and a first-run import of 119 channels can outlast
+    that, so overlapping runs are expected and must be a silent no-op rather than
+    two workers fighting over the same queue.
+    """
+    path = path or config.LOCK_PATH
+    path.parent.mkdir(parents=True, exist_ok=True)
+    handle = path.open("w")
+    try:
+        try:
+            fcntl.flock(handle, fcntl.LOCK_EX | fcntl.LOCK_NB)
+        except OSError as exc:
+            raise AlreadyRunning(f"another run holds {path}") from exc
+        yield
+    finally:
+        with contextlib.suppress(OSError):
+            fcntl.flock(handle, fcntl.LOCK_UN)
+        handle.close()
+
+
+def materialise_all(
+    conn: sqlite3.Connection, settings: Settings, limit: int | None = None
+) -> dict:
+    """Write a `.strm` + `.nfo` + thumbnail for everything in `listed`.
+
+    There is no retry ladder and no failure state. Writing a 50-byte text file
+    either works or the filesystem is broken, and in the latter case the run
+    should stop rather than mark 400 videos as failed.
+    """
+    stats = {"materialised": 0, "shows": 0, "errors": 0}
+    seen_channels: set[int] = set()
+
+    for video in videos.claim_listed(conn, limit):
+        channel = conn.execute(
+            "SELECT * FROM channel WHERE id = ?", (video["channel_pk"],)
+        ).fetchone()
+        if channel is None:                       # channel deleted mid-run
+            continue
+        if channel["id"] not in seen_channels:
+            # Lazily, so a channel with nothing inside the window never creates an
+            # empty series in Jellyfin (plan.md §5).
+            strm.write_show(channel)
+            seen_channels.add(channel["id"])
+            stats["shows"] += 1
+        try:
+            strm.materialise(conn, settings, channel, video)
+            stats["materialised"] += 1
+        except OSError as exc:
+            log.error("could not materialise %s: %s", video["video_id"], exc)
+            stats["errors"] += 1
+    return stats
+
+
+def run(
+    conn: sqlite3.Connection, settings: Settings, channel_pk: int | None = None
+) -> dict:
+    """One full cycle. Assumes the caller holds the lock."""
+    result: dict = {}
+
+    # Sync first: a channel added on YouTube should get its catalogue this run.
+    # Skipped for a single-channel run, which is a targeted operation.
+    if channel_pk is None:
+        result["sync"] = subsync.sync_all(conn, settings)
+
+    result["poll"] = discovery.poll_all(conn, settings, channel_pk)
+    result["materialise"] = materialise_all(conn, settings)
+    result["reap"] = reap.run(conn, settings)
+
+    if result["materialise"]["materialised"] or result["reap"]["aged_out"]:
+        jellyfin.from_settings(settings).refresh()
+
+    settings.set("last_run_at", util.utcnow_iso())
+    return result
+
+
+def summarise(result: dict) -> str:
+    sync = result.get("sync", {})
+    poll = result.get("poll", {})
+    made = result.get("materialise", {})
+    reaped = result.get("reap", {})
+    parts = [
+        f"channels={poll.get('channels', 0)}",
+        f"discovered={poll.get('queued', 0)}",
+        f"materialised={made.get('materialised', 0)}",
+        f"aged_out={reaped.get('aged_out', 0)}",
+        f"shorts={poll.get('shorts', 0)}",
+        f"live={poll.get('live', 0)}",
+        f"poll_failures={poll.get('failed', 0)}",
+    ]
+    if sync:
+        parts[1:1] = [
+            f"subs_added={sync.get('added', 0)}",
+            f"subs_queued={sync.get('queued', 0)}",
+            f"subs_removed={sync.get('removed', 0)}",
+        ]
+        if sync.get("refused"):
+            parts.append(f"SYNC_REFUSED={sync['refused']}")
+    if made.get("errors"):
+        parts.append(f"errors={made['errors']}")
+    return " ".join(parts)
+
+
+def exit_code(result: dict) -> int:
+    """Non-zero when something needs a human.
+
+    The cron entry runs under `runitor`, so this is what turns the healthchecks
+    check red. A silently-broken subscription mirror is the worst outcome
+    available — nothing looks wrong until someone asks why a channel never
+    appeared — so a refused sync must be loud.
+    """
+    sync = result.get("sync", {})
+    if sync.get("refused"):
+        return 1
+    if result.get("materialise", {}).get("errors"):
+        return 1
+    if result.get("poll", {}).get("failed"):
+        # Two of 119 measured channels fail permanently (terminated or private),
+        # so a poll failure is a warning, not a red check.
+        log.warning("%d channel(s) failed to poll", result["poll"]["failed"])
+    return 0

ytstream/settings.py

@@ -0,0 +1,160 @@
+"""Typed settings accessors backed by the `setting` key/value table.
+
+A missing key must never crash anything, so every read falls back to the default
+and every malformed stored value falls back to the default too.
+
+Reads go to the database every time rather than being cached in the process. That
+is deliberate: rotating the YouTube API key has to be nothing more than saving a
+new value in the admin UI, with no restart of the hourly job or the admin server
+(plan.md §11).
+"""
+
+from __future__ import annotations
+
+import sqlite3
+from urllib.parse import urlparse
+
+DEFAULTS: dict[str, str] = {
+    # Retention. One number governs both how far back a new channel is backfilled
+    # and when a video is deleted, so the library cannot grow (plan.md §5).
+    "retention_days": "30",
+    # ...except that 52 of 117 measured channels upload nothing in 30 days, which
+    # would leave them as empty Jellyfin series flickering in and out. So the real
+    # rule is max(retention_days, min_keep_videos most recent).
+    "min_keep_videos": "5",
+    # Runaway guard only: a channel that turns out to upload 50 times a day.
+    "backfill_max_videos": "300",
+
+    "max_height": "1080",
+    "min_duration_seconds": "120",
+
+    "jellyfin_url": "http://127.0.0.1:8096",
+    "jellyfin_api_key": "",
+    "pot_provider_url": "http://127.0.0.1:4416",
+    "youtube_api_key": "",
+    "proxy_base_url": "http://127.0.0.1:8099",
+
+    # Subscription mirroring (plan.md §4.4). max_new is derived from the measured
+    # 119 subscriptions as max(10, ceil(119 * 0.2)).
+    "subsync_max_new": "25",
+    "subsync_missing_threshold": "3",
+}
+
+# Editable through the settings form. Everything else in the table is internal.
+EDITABLE = tuple(DEFAULTS)
+
+# Never rendered, never settable through the web form.
+SECRET_KEYS = ("admin_password_hash", "session_secret")
+
+# Shown as a masked value rather than plaintext.
+MASKED_KEYS = ("jellyfin_api_key", "youtube_api_key")
+
+_INT_KEYS = (
+    "retention_days",
+    "min_keep_videos",
+    "backfill_max_videos",
+    "max_height",
+    "min_duration_seconds",
+    "subsync_max_new",
+    "subsync_missing_threshold",
+)
+_BOOL_KEYS = ()
+_URL_KEYS = ("jellyfin_url", "pot_provider_url", "proxy_base_url")
+
+_TRUE = {"1", "true", "yes", "on"}
+_FALSE = {"0", "false", "no", "off", ""}
+
+
+class Settings:
+    def __init__(self, conn: sqlite3.Connection):
+        self.conn = conn
+
+    def raw(self, key: str) -> str:
+        row = self.conn.execute(
+            "SELECT value FROM setting WHERE key = ?", (key,)
+        ).fetchone()
+        if row is None:
+            return DEFAULTS.get(key, "")
+        return row["value"]
+
+    def get_str(self, key: str) -> str:
+        return self.raw(key)
+
+    def get_int(self, key: str) -> int:
+        try:
+            return int(self.raw(key))
+        except (TypeError, ValueError):
+            return int(DEFAULTS.get(key, "0") or 0)
+
+    def get_bool(self, key: str) -> bool:
+        value = self.raw(key).strip().lower()
+        if value in _TRUE:
+            return True
+        if value in _FALSE:
+            return False
+        return DEFAULTS.get(key, "false").lower() in _TRUE
+
+    def set(self, key: str, value: str) -> None:
+        with self.conn:
+            self.conn.execute(
+                "INSERT INTO setting (key, value) VALUES (?, ?) "
+                "ON CONFLICT(key) DO UPDATE SET value = excluded.value",
+                (key, str(value)),
+            )
+
+    def all_editable(self) -> dict[str, str]:
+        return {key: self.raw(key) for key in EDITABLE}
+
+
+def validate(key: str, value: str) -> tuple[bool, str]:
+    """Validate one submitted setting.
+
+    Returns (ok, message). On failure the caller re-renders the form with the
+    message inline rather than raising.
+    """
+    value = value.strip()
+
+    if key in _INT_KEYS:
+        try:
+            number = int(value)
+        except ValueError:
+            return False, "must be a whole number"
+        if number < 0:
+            return False, "must be zero or greater"
+        if key == "retention_days" and number < 1:
+            return False, "must be at least 1 day"
+        if key == "max_height" and number < 144:
+            return False, "must be at least 144"
+        if key == "subsync_missing_threshold" and number < 1:
+            # Zero would unsubscribe on the first absent response, which is
+            # exactly the failure mode the threshold exists to prevent.
+            return False, "must be at least 1 sync"
+        return True, ""
+
+    if key in _BOOL_KEYS:
+        if value.lower() not in _TRUE | _FALSE:
+            return False, "must be true or false"
+        return True, ""
+
+    if key in _URL_KEYS:
+        parsed = urlparse(value)
+        if parsed.scheme not in ("http", "https") or not parsed.netloc:
+            return False, "must be a http:// or https:// URL"
+        return True, ""
+
+    if key in ("jellyfin_api_key", "youtube_api_key"):
+        return True, ""
+
+    return key in DEFAULTS, "unknown setting"
+
+
+def validate_all(submitted: dict[str, str]) -> dict[str, str]:
+    """Return {key: error} for everything that failed validation."""
+    errors: dict[str, str] = {}
+    for key, value in submitted.items():
+        if key not in EDITABLE:
+            continue
+        ok, message = validate(key, value)
+        if not ok:
+            errors[key] = message
+    return errors

ytstream/strm.py

@@ -0,0 +1,207 @@
+"""Materialising a video: write a `.strm`, an `.nfo` and a thumbnail.
+
+This is what replaced youtube-automate's 330-line `download.py`. There is no
+subprocess, no format selection, no progress parsing and no retry ladder, because
+the whole job is writing a URL into a text file. Everything expensive was moved to
+the moment somebody presses play, which is the point of the design.
+
+Thumbnails come from `i.ytimg.com`, which needs no API key and no quota.
+"""
+
+from __future__ import annotations
+
+import logging
+import shutil
+import sqlite3
+import urllib.error
+import urllib.request
+from pathlib import Path
+
+from . import config, naming, nfo, util, videos
+from .settings import Settings
+
+log = logging.getLogger(__name__)
+
+THUMB_SUFFIX = "-thumb.jpg"
+
+# maxres does not exist for every video; hq always does.
+_THUMB_URLS = (
+    "https://i.ytimg.com/vi/{vid}/maxresdefault.jpg",
+    "https://i.ytimg.com/vi/{vid}/hqdefault.jpg",
+)
+
+
+def watch_url(settings: Settings, video_id: str) -> str:
+    base = settings.get_str("proxy_base_url").rstrip("/")
+    return f"{base}/watch/{video_id}"
+
+
+def channel_dir(channel: sqlite3.Row) -> Path:
+    return config.MEDIA_ROOT / channel["dir_name"]
+
+
+def episode_paths(channel: sqlite3.Row, season: int, episode: int,
+                  title: str, video_id: str) -> tuple[Path, str]:
+    """(.strm path, path relative to the media root)."""
+    stem = naming.basename(channel["dir_name"], season, episode, title, video_id)
+    relative = Path(channel["dir_name"]) / naming.season_dir_name(season) / (stem + ".strm")
+    return config.MEDIA_ROOT / relative, str(relative)
+
+
+def write_show(channel: sqlite3.Row) -> None:
+    """tvshow.nfo for a channel.
+
+    Takes the row's show fields explicitly rather than trusting a joined row: the
+    channel/video join has `title` on both sides, and reading the wrong one
+    renamed every series after whichever episode happened to be first. That
+    shipped once already.
+    """
+    directory = channel_dir(channel)
+    directory.mkdir(parents=True, exist_ok=True)
+    nfo.write(
+        directory / "tvshow.nfo",
+        nfo.tvshow_nfo(
+            title=channel["title"],
+            plot=channel["description"],
+            channel_id=channel["channel_id"],
+        ),
+    )
+
+
+def fetch_thumbnail(video_id: str, destination: Path, *, timeout: float = 20.0) -> bool:
+    """Best effort. A missing thumbnail is cosmetic, never a failure."""
+    if destination.exists():
+        return True
+    for template in _THUMB_URLS:
+        url = template.format(vid=video_id)
+        request = urllib.request.Request(url, headers={"User-Agent": config.USER_AGENT})
+        try:
+            with urllib.request.urlopen(request, timeout=timeout) as response:
+                if response.status != 200:
+                    continue
+                payload = response.read()
+        except (urllib.error.HTTPError, urllib.error.URLError, OSError):
+            continue
+        if len(payload) < 1024:
+            # YouTube serves a 120-byte grey placeholder rather than a 404.
+            continue
+        destination.parent.mkdir(parents=True, exist_ok=True)
+        temporary = destination.with_name(destination.name + ".tmp")
+        temporary.write_bytes(payload)
+        temporary.replace(destination)
+        return True
+    log.debug("no thumbnail available for %s", video_id)
+    return False
+
+
+def materialise(
+    conn: sqlite3.Connection,
+    settings: Settings,
+    channel: sqlite3.Row,
+    video: sqlite3.Row,
+) -> dict:
+    """Write one episode's `.strm`, `.nfo` and thumbnail, and record it.
+
+    Idempotent: re-running over an already-materialised video rewrites the same
+    bytes to the same paths.
+    """
+    upload_date = naming.parse_upload_date(video["upload_date"])
+    season, episode = videos.next_episode(
+        conn, channel["id"], upload_date, video["video_id"]
+    )
+    # `title` is for the filename; `stored_title` is what goes back to the
+    # database. They differ when we have no title yet: writing the video id back
+    # would make the row look titled, permanently disabling the repair path in
+    # discovery._record that fills titles in from a later feed poll. That bug
+    # shipped once and renamed five of twenty episodes after their video ids.
+    stored_title = video["title"] or ""
+    title = stored_title or video["video_id"]
+    strm_path, relative = episode_paths(
+        channel, season, episode, title, video["video_id"]
+    )
+
+    strm_path.parent.mkdir(parents=True, exist_ok=True)
+    # No trailing newline: some Jellyfin versions have historically been fussy
+    # about trailing whitespace in .strm files, and there is nothing to gain.
+    temporary = strm_path.with_name(strm_path.name + ".tmp")
+    temporary.write_text(watch_url(settings, video["video_id"]), encoding="utf-8")
+    temporary.replace(strm_path)
+
+    nfo.write(
+        strm_path.with_suffix(".nfo"),
+        nfo.episode_nfo(
+            title=title,
+            show_title=channel["title"],
+            season=season,
+            episode=episode,
+            plot=None,
+            aired=upload_date.isoformat(),
+            duration_seconds=video["duration"],
+            video_id=video["video_id"],
+        ),
+    )
+
+    stem = strm_path.name[: -len(".strm")]
+    fetch_thumbnail(video["video_id"], strm_path.with_name(stem + THUMB_SUFFIX))
+
+    videos.mark_materialised(
+        conn,
+        video["video_id"],
+        rel_path=relative,
+        season=season,
+        episode=episode,
+        upload_date=upload_date.isoformat(),
+        duration=video["duration"],
+        title=stored_title,
+    )
+    return {"video_id": video["video_id"], "rel_path": relative,
+            "season": season, "episode": episode}
+
+
+def remove(video: sqlite3.Row) -> int:
+    """Delete an episode's files. Returns how many were removed.
+
+    Only touches suffixes we know we wrote. A stray file someone else put in the
+    season directory is not ours to delete.
+    """
+    if not video["rel_path"]:
+        return 0
+    strm_path = config.MEDIA_ROOT / video["rel_path"]
+    stem = strm_path.name[: -len(".strm")]
+    removed = 0
+    for path in [strm_path] + [
+        strm_path.with_name(stem + suffix) for suffix in config.SIDECAR_SUFFIXES
+    ]:
+        try:
+            path.unlink()
+            removed += 1
+        except FileNotFoundError:
+            continue
+        except OSError as exc:
+            log.warning("could not remove %s: %s", path, exc)
+
+    # Prune empty season directories, but stop at the channel directory — it
+    # holds tvshow.nfo and the artwork, and an active subscription must not
+    # vanish from Jellyfin just because it published nothing this year. Passing
+    # MEDIA_ROOT as the boundary instead would delete the channel directory on
+    # any channel whose tvshow.nfo happened to be missing.
+    channel_dir = config.MEDIA_ROOT / Path(video["rel_path"]).parts[0]
+    util.prune_empty_dirs(strm_path.parent, channel_dir)
+    return removed
+
+
+def remove_channel_tree(channel: sqlite3.Row) -> bool:
+    """Delete a whole channel directory, on unsubscribe.
+
+    Guarded against deleting the media root itself, which a channel whose
+    dir_name somehow ended up empty would otherwise do.
+    """
+    directory = channel_dir(channel)
+    if directory == config.MEDIA_ROOT or not str(channel["dir_name"]).strip():
+        log.error("refusing to remove %s: unsafe channel directory", directory)
+        return False
+    if not directory.exists():
+        return False
+    shutil.rmtree(directory)
+    log.info("removed %s", directory)
+    return True

ytstream/subsync.py

@@ -0,0 +1,305 @@
+"""Mirror one YouTube account's public subscriptions.
+
+The mirrored account is the source of truth: what it follows is what exists in
+Jellyfin. That makes the sync authoritative in both directions, and the removal
+half is destructive, so most of this module is about refusing to act on bad data.
+
+The rules, and why each exists (plan.md §4.4):
+
+* **A 403, a network error, or a zero-item response is never a removal.** All
+  three look identical to "he unsubscribed from everything". A genuinely empty
+  list is indistinguishable from a broken one in *consequence*, so we take the
+  harmless reading and change nothing.
+* **A channel must be absent from `subsync_missing_threshold` consecutive healthy
+  responses** before it is unsubscribed. One bad page does not delete a library.
+* **The first sync imports nothing automatically.** With 119 subscriptions the cap
+  would trip on day one whatever it is set to, which trains everyone to ignore it.
+  So the first sync queues the whole list for approval instead.
+* **After that, more than `subsync_max_new` new channels in one run is refused**
+  and queued, on the assumption that an order-of-magnitude jump is a bug.
+
+Everything here treats `source = 'manual'` channels as untouchable.
+"""
+
+from __future__ import annotations
+
+import logging
+import sqlite3
+
+from . import api, channels, strm, util, videos
+from .settings import Settings
+
+log = logging.getLogger(__name__)
+
+MANUAL = "manual"
+YOUTUBE = "youtube"
+
+
+class SyncRefused(RuntimeError):
+    """The response was not trustworthy enough to act on.
+
+    Raised rather than returned so a caller cannot accidentally treat a refusal
+    as an empty subscription list.
+    """
+
+
+# --------------------------------------------------------------------- sources
+
+
+def get_source(conn: sqlite3.Connection, key: str) -> sqlite3.Row | None:
+    return conn.execute("SELECT * FROM source WHERE key = ?", (key,)).fetchone()
+
+
+def all_sources(conn: sqlite3.Connection, *, enabled_only: bool = False) -> list[sqlite3.Row]:
+    sql = "SELECT * FROM source"
+    if enabled_only:
+        sql += " WHERE enabled = 1"
+    return conn.execute(sql + " ORDER BY key").fetchall()
+
+
+def add_source(conn: sqlite3.Connection, *, channel_id: str, label: str) -> str:
+    key = f"youtube:{channel_id}"
+    with conn:
+        conn.execute(
+            "INSERT INTO source (key, label, channel_id) VALUES (?, ?, ?) "
+            "ON CONFLICT(key) DO UPDATE SET label = excluded.label",
+            (key, label, channel_id),
+        )
+    return key
+
+
+def _record_success(conn: sqlite3.Connection, key: str) -> None:
+    with conn:
+        conn.execute(
+            "UPDATE source SET last_sync_at = ?, last_sync_ok = 1, "
+            "consecutive_failures = 0, last_error = NULL WHERE key = ?",
+            (util.utcnow_iso(), key),
+        )
+
+
+def _record_failure(conn: sqlite3.Connection, key: str, error: str) -> None:
+    with conn:
+        conn.execute(
+            "UPDATE source SET last_sync_at = ?, last_sync_ok = 0, "
+            "consecutive_failures = consecutive_failures + 1, last_error = ? "
+            "WHERE key = ?",
+            (util.utcnow_iso(), error[:500], key),
+        )
+
+
+# ------------------------------------------------------------ approval queue
+
+
+def queue_approval(conn: sqlite3.Connection, source: str, entries: list[dict]) -> int:
+    now = util.utcnow_iso()
+    with conn:
+        for entry in entries:
+            conn.execute(
+                "INSERT INTO pending_approval (source, channel_id, title, seen_at) "
+                "VALUES (?, ?, ?, ?) ON CONFLICT(source, channel_id) DO NOTHING",
+                (source, entry["channel_id"], entry["title"], now),
+            )
+    return len(entries)
+
+
+def pending(conn: sqlite3.Connection, source: str | None = None) -> list[sqlite3.Row]:
+    sql = "SELECT * FROM pending_approval WHERE resolved_at IS NULL"
+    params: list = []
+    if source:
+        sql += " AND source = ?"
+        params.append(source)
+    return conn.execute(sql + " ORDER BY title COLLATE NOCASE", params).fetchall()
+
+
+def resolve(conn: sqlite3.Connection, ids: list[int], resolution: str) -> int:
+    if not ids:
+        return 0
+    marks = ",".join("?" * len(ids))
+    with conn:
+        cursor = conn.execute(
+            f"UPDATE pending_approval SET resolved_at = ?, resolution = ? "
+            f"WHERE id IN ({marks}) AND resolved_at IS NULL",
+            [util.utcnow_iso(), resolution, *ids],
+        )
+    return cursor.rowcount
+
+
+# ------------------------------------------------------------------- the sync
+
+
+def _fetch(settings: Settings, source: sqlite3.Row) -> list[dict]:
+    """Fetch the subscription list, or raise SyncRefused.
+
+    Every failure mode collapses to a refusal here so that no caller can mistake
+    one for an empty list.
+    """
+    client = api.Api(settings.get_str("youtube_api_key"))
+    try:
+        remote = client.subscriptions(source["channel_id"])
+    except api.SubscriptionsPrivate as exc:
+        raise SyncRefused(
+            f"subscriptions are private ({exc.reason}) — nothing changed. "
+            'Fix: YouTube → Settings → Privacy → uncheck "Keep all my '
+            'subscriptions private".'
+        ) from exc
+    except api.NotConfigured as exc:
+        raise SyncRefused(f"API not usable: {exc}") from exc
+    except api.ApiError as exc:
+        raise SyncRefused(f"API error: {exc}") from exc
+
+    if not remote:
+        # Could be true, could be a broken response. The consequences differ by
+        # everything, so assume the harmless one.
+        raise SyncRefused(
+            "subscription list came back empty; treating as suspect and "
+            "changing nothing"
+        )
+    return remote
+
+
+def sync_source(
+    conn: sqlite3.Connection, settings: Settings, source: sqlite3.Row
+) -> dict:
+    """One source, one pass. Never raises for remote problems."""
+    stats = {"source": source["key"], "seen": 0, "added": 0, "queued": 0,
+             "removed": 0, "pending_removal": 0, "refused": None}
+
+    try:
+        remote = _fetch(settings, source)
+    except SyncRefused as exc:
+        log.error("%s: %s", source["key"], exc)
+        _record_failure(conn, source["key"], str(exc))
+        stats["refused"] = str(exc)
+        return stats
+
+    stats["seen"] = len(remote)
+    remote_ids = {entry["channel_id"] for entry in remote}
+    known = {
+        row["channel_id"]: row
+        for row in conn.execute("SELECT * FROM channel").fetchall()
+    }
+
+    # ---------------------------------------------------------------- additions
+    new = [entry for entry in remote if entry["channel_id"] not in known]
+    already_queued = {row["channel_id"] for row in pending(conn, source["key"])}
+    resolved = {
+        row["channel_id"]
+        for row in conn.execute(
+            "SELECT channel_id FROM pending_approval "
+            "WHERE source = ? AND resolution = 'rejected'",
+            (source["key"],),
+        ).fetchall()
+    }
+    new = [entry for entry in new
+           if entry["channel_id"] not in already_queued
+           and entry["channel_id"] not in resolved]
+
+    max_new = settings.get_int("subsync_max_new")
+    if not source["imported"]:
+        # First sync: approve-everything-by-hand, never auto-subscribe.
+        if new:
+            stats["queued"] = queue_approval(conn, source["key"], new)
+            log.warning(
+                "%s: first sync — %d channel(s) queued for approval, none added",
+                source["key"], stats["queued"],
+            )
+        with conn:
+            conn.execute("UPDATE source SET imported = 1 WHERE key = ?",
+                         (source["key"],))
+    elif len(new) > max_new:
+        stats["queued"] = queue_approval(conn, source["key"], new)
+        log.warning(
+            "%s: %d new channels exceeds subsync_max_new=%d — none added, all "
+            "queued for approval", source["key"], len(new), max_new,
+        )
+    else:
+        for entry in new:
+            try:
+                channels.subscribe_from_sync(
+                    conn, settings, entry["channel_id"], entry["title"]
+                )
+                stats["added"] += 1
+            except Exception as exc:                      # noqa: BLE001
+                # One unresolvable channel must not abort the whole sync.
+                log.error("%s: could not add %s (%s): %s", source["key"],
+                          entry["title"], entry["channel_id"], exc)
+
+    # ----------------------------------------------------------------- removals
+    threshold = max(1, settings.get_int("subsync_missing_threshold"))
+    for channel_id, row in known.items():
+        if row["source"] == MANUAL:
+            continue
+        if channel_id in remote_ids:
+            if row["missing_syncs"]:
+                with conn:
+                    conn.execute(
+                        "UPDATE channel SET missing_syncs = 0 WHERE id = ?",
+                        (row["id"],),
+                    )
+            continue
+
+        misses = row["missing_syncs"] + 1
+        with conn:
+            conn.execute("UPDATE channel SET missing_syncs = ? WHERE id = ?",
+                         (misses, row["id"]))
+        if misses < threshold:
+            stats["pending_removal"] += 1
+            log.info("%s: %s absent for %d/%d syncs", source["key"],
+                     row["title"], misses, threshold)
+            continue
+
+        log.warning("%s: %s absent for %d syncs — unsubscribing and deleting",
+                    source["key"], row["title"], misses)
+        unsubscribe(conn, row)
+        stats["removed"] += 1
+
+    _record_success(conn, source["key"])
+    return stats
+
+
+def unsubscribe(conn: sqlite3.Connection, channel: sqlite3.Row) -> None:
+    """Delete a channel: its tree, then its rows.
+
+    Tree first. If it fails we still have the rows and can retry; the reverse
+    would orphan a directory Jellyfin keeps showing with nothing to explain it.
+    """
+    strm.remove_channel_tree(channel)
+    with conn:
+        conn.execute("DELETE FROM channel WHERE id = ?", (channel["id"],))
+    log.info("unsubscribed %s (%s)", channel["title"], channel["channel_id"])
+
+
+def approve(conn: sqlite3.Connection, settings: Settings, ids: list[int]) -> dict:
+    """Subscribe the approved rows from the queue."""
+    stats = {"added": 0, "failed": 0}
+    rows = conn.execute(
+        f"SELECT * FROM pending_approval WHERE id IN ({','.join('?' * len(ids))})",
+        ids,
+    ).fetchall() if ids else []
+
+    for row in rows:
+        try:
+            channels.subscribe_from_sync(
+                conn, settings, row["channel_id"], row["title"]
+            )
+            stats["added"] += 1
+        except Exception as exc:                          # noqa: BLE001
+            log.error("could not add %s: %s", row["title"], exc)
+            stats["failed"] += 1
+            continue
+        resolve(conn, [row["id"]], "approved")
+    return stats
+
+
+def sync_all(conn: sqlite3.Connection, settings: Settings) -> dict:
+    totals = {"sources": 0, "added": 0, "queued": 0, "removed": 0,
+              "pending_removal": 0, "refused": 0}
+    for source in all_sources(conn, enabled_only=True):
+        stats = sync_source(conn, settings, source)
+        totals["sources"] += 1
+        if stats["refused"]:
+            totals["refused"] += 1
+        for key in ("added", "queued", "removed", "pending_removal"):
+            totals[key] += stats[key]
+        log.info("%s: %s", source["key"], stats)
+    return totals

ytstream/util.py

@@ -0,0 +1,70 @@
+"""Small shared helpers."""
+
+from __future__ import annotations
+
+import logging
+import os
+from datetime import date, datetime, timezone
+
+from . import config
+
+
+def utcnow() -> datetime:
+    return datetime.now(timezone.utc)
+
+
+def utcnow_iso() -> str:
+    return utcnow().replace(microsecond=0).isoformat()
+
+
+def today() -> date:
+    return utcnow().date()
+
+
+def apply_umask() -> None:
+    """Ensure files land group-writable so Jellyfin's group can read them."""
+    os.umask(config.UMASK)
+
+
+def setup_logging(verbose: bool = False) -> None:
+    logging.basicConfig(
+        level=logging.DEBUG if verbose else logging.INFO,
+        format="%(asctime)s %(levelname)-7s %(name)s: %(message)s",
+        datefmt="%Y-%m-%d %H:%M:%S",
+    )
+    # yt-dlp and urllib are noisy at debug level and we drive them deliberately.
+    logging.getLogger("urllib3").setLevel(logging.WARNING)
+
+
+def prune_empty_dirs(start, stop) -> int:
+    """Remove `start` and its empty parents, never passing `stop`.
+
+    With a rolling retention window, whole season directories empty out as a year
+    rolls over. Leaving them behind gives Jellyfin empty seasons to display, and
+    they accumulate one per channel per year.
+
+    `stop` itself is never removed, and the walk halts at the first non-empty
+    directory.
+    """
+    from pathlib import Path
+
+    start, stop = Path(start), Path(stop)
+    removed = 0
+    current = start
+    while current != stop and stop in current.parents:
+        try:
+            current.rmdir()          # only succeeds when empty
+        except OSError:
+            break
+        removed += 1
+        current = current.parent
+    return removed
+
+
+def human_bytes(value: int | None) -> str:
+    size = float(value or 0)
+    for unit in ("B", "KB", "MB", "GB", "TB"):
+        if size < 1024 or unit == "TB":
+            return f"{size:.0f} {unit}" if unit == "B" else f"{size:.1f} {unit}"
+        size /= 1024
+    return f"{size:.1f} TB"

ytstream/videos.py

@@ -0,0 +1,214 @@
+"""Video row helpers and the state machine.
+
+States (plan.md §7):
+
+    listed         known and inside the window; a .strm should exist for it
+    materialised   .strm + .nfo + thumbnail written, rel_path set
+    skipped_short  below min_duration_seconds; repaired if later seen in UULF
+    skipped_live   livestream, never retried
+    skipped_old    already outside the window when discovered; revivable by rescan
+    aged_out       was materialised, then deleted by the retention sweep
+
+There is no `pending`/`downloading`/`failed` triad. Writing a 50-byte text file
+cannot meaningfully fail, so a video is either listed or it is on disk, and the
+retry machinery youtube-automate needed for 500 MB downloads has no analogue here.
+
+`skipped_old` and `aged_out` are both terminal but are NOT interchangeable:
+raising retention_days should revive the first and must never revive the second,
+or months of previously-deleted episodes would reappear in Jellyfin as new.
+"""
+
+from __future__ import annotations
+
+import sqlite3
+from datetime import date
+
+from . import naming, util
+
+LISTED = "listed"
+MATERIALISED = "materialised"
+SKIPPED_SHORT = "skipped_short"
+SKIPPED_LIVE = "skipped_live"
+SKIPPED_OLD = "skipped_old"
+AGED_OUT = "aged_out"
+
+# We have made a final negative decision about these; discovery must not requeue
+# them. `aged_out` is the tombstone that stops the retention sweep and the poller
+# fighting each other forever.
+TERMINAL = (SKIPPED_LIVE, SKIPPED_OLD, AGED_OUT)
+
+# Never revived by `rescan`, however the retention window changes.
+NEVER_REVIVE = (AGED_OUT,)
+
+SOURCE_UULF = "uulf_feed"
+SOURCE_UC = "uc_feed"
+SOURCE_BACKFILL = "backfill"
+
+
+def get(conn: sqlite3.Connection, video_id: str) -> sqlite3.Row | None:
+    return conn.execute(
+        "SELECT * FROM video WHERE video_id = ?", (video_id,)
+    ).fetchone()
+
+
+def exists(conn: sqlite3.Connection, video_id: str) -> bool:
+    return get(conn, video_id) is not None
+
+
+def insert(
+    conn: sqlite3.Connection,
+    *,
+    channel_pk: int,
+    video_id: str,
+    title: str,
+    upload_date: str | None,
+    state: str,
+    discovery_source: str,
+    duration: int | None = None,
+    published_at: str | None = None,
+) -> None:
+    with conn:
+        conn.execute(
+            "INSERT OR IGNORE INTO video "
+            "(video_id, channel_pk, title, upload_date, published_at, duration, "
+            " state, discovery_source, discovered_at) "
+            "VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)",
+            (
+                video_id,
+                channel_pk,
+                title,
+                upload_date,
+                published_at,
+                duration,
+                state,
+                discovery_source,
+                util.utcnow_iso(),
+            ),
+        )
+
+
+def set_state(conn: sqlite3.Connection, video_id: str, state: str) -> None:
+    with conn:
+        conn.execute(
+            "UPDATE video SET state = ? WHERE video_id = ?", (state, video_id)
+        )
+
+
+def set_duration(conn: sqlite3.Connection, video_id: str, duration: int | None) -> None:
+    with conn:
+        conn.execute(
+            "UPDATE video SET duration = ? WHERE video_id = ?", (duration, video_id)
+        )
+
+
+def mark_materialised(
+    conn: sqlite3.Connection,
+    video_id: str,
+    *,
+    rel_path: str,
+    season: int,
+    episode: int,
+    upload_date: str,
+    duration: int | None,
+    title: str,
+) -> None:
+    with conn:
+        conn.execute(
+            "UPDATE video SET state = ?, rel_path = ?, season = ?, episode = ?, "
+            "upload_date = ?, duration = ?, title = ?, materialised_at = ? "
+            "WHERE video_id = ?",
+            (
+                MATERIALISED,
+                rel_path,
+                season,
+                episode,
+                upload_date,
+                duration,
+                title,
+                util.utcnow_iso(),
+                video_id,
+            ),
+        )
+
+
+def mark_aged_out(conn: sqlite3.Connection, video_id: str) -> None:
+    """Keep the row — it is the tombstone that prevents re-materialising."""
+    with conn:
+        conn.execute(
+            "UPDATE video SET state = ?, rel_path = NULL, deleted_at = ? "
+            "WHERE video_id = ?",
+            (AGED_OUT, util.utcnow_iso(), video_id),
+        )
+
+
+def next_episode(
+    conn: sqlite3.Connection, channel_pk: int, upload_date: date, video_id: str
+) -> tuple[int, int]:
+    """Assign (season, episode) for a video.
+
+    The ordinal is computed against what is already in the database for this
+    channel and date — never against the current batch — so it stays stable
+    across runs and across crashes mid-batch. Aged-out rows keep their season and
+    episode precisely so that this stays stable as videos are deleted too.
+    """
+    season = naming.season_for(upload_date)
+    low, high = naming.episode_range(upload_date)
+    row = conn.execute(
+        "SELECT MAX(episode) AS top FROM video "
+        "WHERE channel_pk = ? AND season = ? AND episode BETWEEN ? AND ? "
+        "AND video_id != ?",
+        (channel_pk, season, low, high, video_id),
+    ).fetchone()
+
+    top = row["top"] if row and row["top"] is not None else None
+    if top is None:
+        return season, low
+    if top >= high:
+        # More than ten uploads in a day; naming.episode_number logs the clamp.
+        return season, high
+    return season, top + 1
+
+
+def claim_listed(
+    conn: sqlite3.Connection, limit: int | None = None
+) -> list[sqlite3.Row]:
+    """Everything waiting to be materialised, oldest upload first."""
+    sql = (
+        "SELECT v.*, c.dir_name, c.title AS channel_title, c.channel_id "
+        "FROM video v JOIN channel c ON c.id = v.channel_pk "
+        "WHERE v.state = ? "
+        "ORDER BY v.upload_date ASC, v.discovered_at ASC"
+    )
+    params: list = [LISTED]
+    if limit:
+        sql += " LIMIT ?"
+        params.append(limit)
+    return conn.execute(sql, params).fetchall()
+
+
+def materialised_for_channel(
+    conn: sqlite3.Connection, channel_pk: int
+) -> list[sqlite3.Row]:
+    """On-disk videos for one channel, newest upload first.
+
+    Newest-first because the retention sweep needs to count down from the most
+    recent to honour min_keep_videos.
+    """
+    return conn.execute(
+        "SELECT * FROM video WHERE channel_pk = ? AND state = ? "
+        "ORDER BY upload_date DESC, episode DESC",
+        (channel_pk, MATERIALISED),
+    ).fetchall()
+
+
+def queue_depth(conn: sqlite3.Connection) -> int:
+    return conn.execute(
+        "SELECT COUNT(*) FROM video WHERE state = ?", (LISTED,)
+    ).fetchone()[0]
+
+
+def counts_by_state(conn: sqlite3.Connection) -> dict[str, int]:
+    rows = conn.execute(
+        "SELECT state, COUNT(*) AS n FROM video GROUP BY state"
+    ).fetchall()
+    return {row["state"]: row["n"] for row in rows}

ytstream/web/__init__.py

@@ -0,0 +1 @@
+"""Admin web UI."""

ytstream/web/auth.py

@@ -0,0 +1,191 @@
+"""Password hashing, session cookies and CSRF tokens.
+
+The admin UI is publicly reachable over HTTPS, so this has to be real. The design
+goal from specs.md §11 is "log in once per device, effectively never again",
+which means a long-lived signed cookie rather than HTTP basic auth.
+"""
+
+from __future__ import annotations
+
+import base64
+import hashlib
+import hmac
+import json
+import secrets
+import time
+
+SCRYPT_N = 2**14
+SCRYPT_R = 8
+SCRYPT_P = 1
+DKLEN = 32
+
+SESSION_MAX_AGE = 365 * 24 * 3600  # one year
+COOKIE_NAME = "yta_session"
+
+# Login throttling: after this many consecutive failures from one address, refuse
+# for LOCKOUT_SECONDS regardless of whether the password is right.
+MAX_FAILURES = 5
+LOCKOUT_SECONDS = 60
+
+
+def _b64(raw: bytes) -> str:
+    return base64.urlsafe_b64encode(raw).decode("ascii").rstrip("=")
+
+
+def _unb64(text: str) -> bytes:
+    padding = "=" * (-len(text) % 4)
+    return base64.urlsafe_b64decode(text + padding)
+
+
+# --------------------------------------------------------------------------
+# passwords
+
+
+def hash_password(password: str, *, salt: bytes | None = None) -> str:
+    salt = salt if salt is not None else secrets.token_bytes(16)
+    derived = hashlib.scrypt(
+        password.encode("utf-8"),
+        salt=salt,
+        n=SCRYPT_N,
+        r=SCRYPT_R,
+        p=SCRYPT_P,
+        dklen=DKLEN,
+    )
+    return f"scrypt${SCRYPT_N}${SCRYPT_R}${SCRYPT_P}${_b64(salt)}${_b64(derived)}"
+
+
+def verify_password(stored: str, password: str) -> bool:
+    if not stored:
+        return False
+    try:
+        scheme, n, r, p, salt_b64, hash_b64 = stored.split("$")
+        if scheme != "scrypt":
+            return False
+        derived = hashlib.scrypt(
+            password.encode("utf-8"),
+            salt=_unb64(salt_b64),
+            n=int(n),
+            r=int(r),
+            p=int(p),
+            dklen=len(_unb64(hash_b64)),
+        )
+    except (ValueError, TypeError):
+        return False
+    return hmac.compare_digest(derived, _unb64(hash_b64))
+
+
+# --------------------------------------------------------------------------
+# sessions
+
+
+def new_secret() -> str:
+    return _b64(secrets.token_bytes(32))
+
+
+def _sign(secret: str, payload: bytes) -> str:
+    return _b64(hmac.new(secret.encode("utf-8"), payload, hashlib.sha256).digest())
+
+
+def issue_session(secret: str, *, issued_at: float | None = None) -> str:
+    payload = json.dumps(
+        {"iat": int(issued_at if issued_at is not None else time.time())},
+        separators=(",", ":"),
+    ).encode("utf-8")
+    return f"{_b64(payload)}.{_sign(secret, payload)}"
+
+
+def verify_session(secret: str, token: str, *, now: float | None = None) -> bool:
+    if not token or not secret:
+        return False
+    try:
+        payload_b64, signature = token.split(".", 1)
+        payload = _unb64(payload_b64)
+    except (ValueError, TypeError):
+        return False
+    if not hmac.compare_digest(_sign(secret, payload), signature):
+        return False
+    try:
+        issued_at = int(json.loads(payload)["iat"])
+    except (ValueError, KeyError, TypeError):
+        return False
+    age = (now if now is not None else time.time()) - issued_at
+    return 0 <= age <= SESSION_MAX_AGE
+
+
+def cookie_header(token: str, *, secure: bool = True) -> str:
+    parts = [
+        f"{COOKIE_NAME}={token}",
+        "Path=/",
+        "HttpOnly",
+        "SameSite=Lax",
+        f"Max-Age={SESSION_MAX_AGE}",
+    ]
+    if secure:
+        parts.insert(2, "Secure")
+    return "; ".join(parts)
+
+
+def clear_cookie_header() -> str:
+    return f"{COOKIE_NAME}=; Path=/; HttpOnly; SameSite=Lax; Max-Age=0"
+
+
+def cookie_value(header: str, name: str = COOKIE_NAME) -> str:
+    """Pull one cookie out of a request's Cookie header.
+
+    Deliberately hand-rolled rather than using http.cookies.SimpleCookie. That
+    parser silently discards the remainder of the header the moment it meets a
+    value it considers illegal — a JSON-ish value such as `prefs={"a":1}` is
+    enough — so any cookie appearing after it becomes invisible. A browser sends
+    us every cookie on the domain, including ones set by unrelated services, so
+    one stray value would otherwise make a perfectly valid session vanish and
+    bounce the user back to the login page with no error shown.
+    """
+    for part in (header or "").split(";"):
+        candidate, separator, value = part.strip().partition("=")
+        if separator and candidate.strip() == name:
+            return value.strip().strip('"')
+    return ""
+
+
+# --------------------------------------------------------------------------
+# CSRF
+
+
+def csrf_token(secret: str, session_token: str) -> str:
+    return _sign(secret, b"csrf:" + session_token.encode("utf-8"))
+
+
+def verify_csrf(secret: str, session_token: str, submitted: str) -> bool:
+    if not submitted:
+        return False
+    return hmac.compare_digest(csrf_token(secret, session_token), submitted)
+
+
+# --------------------------------------------------------------------------
+# throttling
+
+
+class LoginThrottle:
+    """In-memory consecutive-failure tracker keyed by remote address."""
+
+    def __init__(self, max_failures: int = MAX_FAILURES, lockout: int = LOCKOUT_SECONDS):
+        self.max_failures = max_failures
+        self.lockout = lockout
+        self._state: dict[str, tuple[int, float]] = {}
+
+    def locked(self, key: str, *, now: float | None = None) -> bool:
+        failures, last = self._state.get(key, (0, 0.0))
+        if failures < self.max_failures:
+            return False
+        elapsed = (now if now is not None else time.time()) - last
+        if elapsed >= self.lockout:
+            self._state.pop(key, None)
+            return False
+        return True
+
+    def record_failure(self, key: str, *, now: float | None = None) -> None:
+        failures, _ = self._state.get(key, (0, 0.0))
+        self._state[key] = (failures + 1, now if now is not None else time.time())
+
+    def record_success(self, key: str) -> None:
+        self._state.pop(key, None)

ytstream/web/server.py

@@ -0,0 +1,399 @@
+"""The admin HTTP server.
+
+Stdlib only. Binds to localhost; nginx terminates TLS in front of it at
+tube.jihakuz.xyz. Because that hostname is public, this carries real
+authentication, CSRF tokens on every state-changing request, and login
+throttling.
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+import subprocess
+import sys
+import threading
+import urllib.parse
+from http.server import BaseHTTPRequestHandler, ThreadingHTTPServer
+
+from .. import channels, config, db, discovery, jellyfin, util, videos, ytdlp
+from ..settings import EDITABLE, MASKED_KEYS, Settings, validate_all
+from . import auth, templates
+
+log = logging.getLogger(__name__)
+
+MAX_BODY = 64 * 1024
+
+
+class AdminServer(ThreadingHTTPServer):
+    daemon_threads = True
+    allow_reuse_address = True
+
+    def __init__(self, address, handler, *, secure_cookies: bool = True):
+        super().__init__(address, handler)
+        self.throttle = auth.LoginThrottle()
+        self.secure_cookies = secure_cookies
+        self.db_lock = threading.Lock()
+
+
+class Handler(BaseHTTPRequestHandler):
+    server_version = "ytstream"
+    protocol_version = "HTTP/1.1"
+
+    # ---------------------------------------------------------------- utils
+
+    def log_message(self, fmt, *args):  # noqa: A003 - stdlib signature
+        log.debug("%s - %s", self.client_address[0], fmt % args)
+
+    def _client_key(self) -> str:
+        """Real client address, since we always sit behind nginx."""
+        forwarded = self.headers.get("X-Forwarded-For", "")
+        if forwarded:
+            return forwarded.split(",")[0].strip()
+        return self.client_address[0]
+
+    def _send(self, status: int, body: bytes, headers: dict | None = None) -> None:
+        self.send_response(status)
+        self.send_header("Content-Type", "text/html; charset=utf-8")
+        self.send_header("Content-Length", str(len(body)))
+        self.send_header("X-Content-Type-Options", "nosniff")
+        self.send_header("Referrer-Policy", "same-origin")
+        self.send_header("X-Frame-Options", "DENY")
+        for key, value in (headers or {}).items():
+            self.send_header(key, value)
+        self.end_headers()
+        self.wfile.write(body)
+
+    def _redirect(self, location: str, headers: dict | None = None) -> None:
+        combined = {"Location": location}
+        combined.update(headers or {})
+        self._send(303, b"", combined)
+
+    def _json(self, status: int, payload: dict) -> None:
+        body = json.dumps(payload, indent=2).encode("utf-8")
+        self.send_response(status)
+        self.send_header("Content-Type", "application/json")
+        self.send_header("Content-Length", str(len(body)))
+        self.end_headers()
+        self.wfile.write(body)
+
+    def _form(self) -> dict[str, str]:
+        length = int(self.headers.get("Content-Length") or 0)
+        if length <= 0 or length > MAX_BODY:
+            return {}
+        raw = self.rfile.read(length).decode("utf-8", "replace")
+        return {
+            key: values[-1]
+            for key, values in urllib.parse.parse_qs(raw, keep_blank_values=True).items()
+        }
+
+    def _cookie_token(self) -> str:
+        return auth.cookie_value(self.headers.get("Cookie") or "")
+
+    # ------------------------------------------------------------- session
+
+    def _open(self):
+        conn = db.connect()
+        return conn, Settings(conn)
+
+    def _secret(self, settings: Settings) -> str:
+        secret = settings.raw("session_secret")
+        if not secret:
+            secret = auth.new_secret()
+            settings.set("session_secret", secret)
+        return secret
+
+    def _authenticated(self, settings: Settings) -> str | None:
+        """Return the session token if the request is signed in, else None."""
+        token = self._cookie_token()
+        if token and auth.verify_session(self._secret(settings), token):
+            return token
+        return None
+
+    def _check_csrf(self, settings: Settings, token: str, form: dict) -> bool:
+        return auth.verify_csrf(self._secret(settings), token, form.get("csrf", ""))
+
+    # ---------------------------------------------------------------- GET
+
+    def do_GET(self) -> None:  # noqa: N802 - stdlib signature
+        path = urllib.parse.urlparse(self.path).path.rstrip("/") or "/"
+        conn, settings = self._open()
+        try:
+            token = self._authenticated(settings)
+
+            # /health was specced as unauthenticated back when the UI was going
+            # to be tailnet-only. On a public hostname it is gratuitous
+            # fingerprinting surface (yt-dlp version, queue depth), so it needs
+            # a session like everything else.
+            if path == "/health":
+                if not token:
+                    return self._json(401, {"error": "authentication required"})
+                return self._health(conn, settings)
+
+            if path == "/login":
+                if token:
+                    return self._redirect("/")
+                return self._send(200, templates.login_page(self._login_hint(settings)))
+
+            if not token:
+                return self._redirect("/login")
+
+            if path == "/":
+                return self._send(200, self._render_index(conn, settings, token))
+
+            return self._send(404, templates.page("Not found", "<h1>Not found</h1>"))
+        finally:
+            conn.close()
+
+    def _login_hint(self, settings: Settings) -> str | None:
+        if not settings.raw("admin_password_hash"):
+            return "No password is set yet. Run `ytstream set-password` on susan."
+        return None
+
+    def _health(self, conn, settings: Settings) -> None:
+        try:
+            ytdlp_version = ytdlp.version()
+        except Exception as exc:  # noqa: BLE001
+            ytdlp_version = f"error: {exc}"
+        try:
+            ytdlp.pot_provider_ping(settings.get_str("pot_provider_url"))
+            pot_up = True
+        except Exception:  # noqa: BLE001
+            pot_up = False
+
+        self._json(
+            200,
+            {
+                "yt_dlp_version": ytdlp_version,
+                "pot_provider_up": pot_up,
+                "last_run_at": settings.raw("last_run_at") or None,
+                "queue_depth": videos.queue_depth(conn),
+                "channels": len(channels.all_channels(conn)),
+            },
+        )
+
+    # --------------------------------------------------------------- POST
+
+    def do_POST(self) -> None:  # noqa: N802 - stdlib signature
+        path = urllib.parse.urlparse(self.path).path.rstrip("/") or "/"
+        conn, settings = self._open()
+        try:
+            form = self._form()
+
+            if path == "/login":
+                return self._login(settings, form)
+
+            token = self._authenticated(settings)
+            if not token:
+                return self._redirect("/login")
+
+            if not self._check_csrf(settings, token, form):
+                log.warning("CSRF check failed for %s from %s", path, self._client_key())
+                return self._send(
+                    400,
+                    templates.page(
+                        "Bad request",
+                        "<h1>Bad request</h1><p>Invalid form token. "
+                        '<a href="/">Go back</a> and try again.</p>',
+                    ),
+                )
+
+            if path == "/logout":
+                return self._redirect("/login", {"Set-Cookie": auth.clear_cookie_header()})
+            if path == "/channels":
+                return self._add_channel(conn, settings, token, form)
+            if path == "/settings":
+                return self._save_settings(conn, settings, token, form)
+
+            parts = path.strip("/").split("/")
+            if len(parts) == 3 and parts[0] == "channels" and parts[1].isdigit():
+                pk = int(parts[1])
+                if parts[2] == "delete":
+                    return self._delete_channel(conn, settings, pk)
+                if parts[2] == "retention":
+                    return self._set_retention(conn, pk, form)
+                if parts[2] == "rescan":
+                    return self._rescan(conn, settings, pk)
+
+            return self._send(404, templates.page("Not found", "<h1>Not found</h1>"))
+        finally:
+            conn.close()
+
+    def _login(self, settings: Settings, form: dict) -> None:
+        key = self._client_key()
+        if self.server.throttle.locked(key):
+            return self._send(
+                429, templates.login_page("Too many attempts. Wait a minute.")
+            )
+
+        stored = settings.raw("admin_password_hash")
+        if not stored:
+            log.warning("login attempted from %s but no password is set", key)
+
+        if stored and auth.verify_password(stored, form.get("password", "")):
+            self.server.throttle.record_success(key)
+            # Logged at INFO so "did my login work?" is answerable from the
+            # journal. A rejected login only ever re-renders the form, which
+            # looks identical to a session that failed to stick.
+            log.info("successful login from %s", key)
+            token = auth.issue_session(self._secret(settings))
+            return self._redirect(
+                "/",
+                {
+                    "Set-Cookie": auth.cookie_header(
+                        token, secure=self.server.secure_cookies
+                    )
+                },
+            )
+
+        self.server.throttle.record_failure(key)
+        log.warning("failed login from %s", key)
+        return self._send(
+            401, templates.login_page(self._login_hint(settings) or "Wrong password.")
+        )
+
+    # ------------------------------------------------------------ actions
+
+    def _add_channel(self, conn, settings: Settings, token: str, form: dict) -> None:
+        url = (form.get("url") or "").strip()
+        try:
+            row = channels.subscribe(conn, settings, url)
+        except channels.ResolutionError as exc:
+            body = self._render_index(conn, settings, token, add_error=str(exc))
+            return self._send(400, body)
+
+        self._spawn_backfill(row["id"])
+        return self._redirect("/")
+
+    def _spawn_backfill(self, channel_pk: int) -> None:
+        """Kick off discovery immediately rather than waiting for the hourly cron."""
+        try:
+            subprocess.Popen(  # noqa: S603
+                [sys.executable, "-m", "ytstream", "run", "--channel",
+                 str(channel_pk)],
+                stdout=subprocess.DEVNULL,
+                stderr=subprocess.DEVNULL,
+                start_new_session=True,
+            )
+        except OSError as exc:  # pragma: no cover
+            log.warning("could not spawn backfill for channel %d: %s", channel_pk, exc)
+
+    def _delete_channel(self, conn, settings: Settings, pk: int) -> None:
+        try:
+            channels.unsubscribe(conn, pk)
+        except LookupError:
+            return self._send(404, templates.page("Not found", "<h1>Not found</h1>"))
+        jellyfin.from_settings(settings).refresh()
+        return self._redirect("/")
+
+    def _set_retention(self, conn, pk: int, form: dict) -> None:
+        raw = (form.get("days") or "").strip()
+        value: int | None
+        if not raw:
+            value = None
+        else:
+            try:
+                value = max(1, int(raw))
+            except ValueError:
+                return self._redirect("/")
+        with conn:
+            conn.execute(
+                "UPDATE channel SET retention_days = ? WHERE id = ?", (value, pk)
+            )
+        return self._redirect("/")
+
+    def _rescan(self, conn, settings: Settings, pk: int) -> None:
+        channel = channels.get(conn, pk)
+        if channel is None:
+            return self._send(404, templates.page("Not found", "<h1>Not found</h1>"))
+        discovery.rescan_channel(conn, settings, channel)
+        self._spawn_backfill(pk)
+        return self._redirect("/")
+
+    def _save_settings(self, conn, settings: Settings, token: str, form: dict) -> None:
+        submitted = {key: form.get(key, "") for key in EDITABLE if key in form}
+
+        # A blank masked field means "keep what is stored", not "clear it".
+        for key in MASKED_KEYS:
+            if key in submitted and not submitted[key].strip():
+                submitted.pop(key)
+
+        errors = validate_all(submitted)
+        if errors:
+            body = self._render_index(
+                conn, settings, token, settings_errors=errors, submitted=submitted
+            )
+            return self._send(400, body)
+
+        for key, value in submitted.items():
+            settings.set(key, value.strip())
+        return self._redirect("/")
+
+    # ------------------------------------------------------------- render
+
+    def _render_index(
+        self,
+        conn,
+        settings: Settings,
+        token: str,
+        *,
+        add_error: str | None = None,
+        settings_errors: dict | None = None,
+        submitted: dict | None = None,
+    ) -> bytes:
+        global_retention = settings.get_int("retention_days")
+        rows = []
+        for channel in channels.all_channels(conn):
+            stats = conn.execute(
+                "SELECT COUNT(*) AS n, MAX(upload_date) AS latest FROM video "
+                "WHERE channel_pk = ? AND state = ?",
+                (channel["id"], videos.MATERIALISED),
+            ).fetchone()
+            latest_any = conn.execute(
+                "SELECT MAX(upload_date) AS latest FROM video WHERE channel_pk = ?",
+                (channel["id"],),
+            ).fetchone()
+            rows.append(
+                {
+                    "id": channel["id"],
+                    "title": channel["title"],
+                    "handle": channel["handle"],
+                    "channel_id": channel["channel_id"],
+                    "retention_days": channel["retention_days"],
+                    "global_retention": global_retention,
+                    "last_polled_at": channel["last_polled_at"],
+                    "last_poll_ok": channel["last_poll_ok"],
+                    "consecutive_poll_failures": channel["consecutive_poll_failures"],
+                    "episodes": stats["n"],
+                    "source": channel["source"],
+                    "missing_syncs": channel["missing_syncs"],
+                    "missing_threshold": settings.get_int(
+                        "subsync_missing_threshold"),
+                    "latest": latest_any["latest"],
+                }
+            )
+
+        values = settings.all_editable()
+        if submitted:
+            values.update(submitted)
+
+        return templates.index_page(
+            channels=rows,
+            settings_values=values,
+            settings_errors=settings_errors or {},
+            csrf=auth.csrf_token(self._secret(settings), token),
+            add_error=add_error,
+            queue_depth=videos.queue_depth(conn),
+        )
+
+
+def serve(host: str = "127.0.0.1", port: int = 8085, *, secure_cookies: bool = True) -> None:
+    util.apply_umask()
+    server = AdminServer((host, port), Handler, secure_cookies=secure_cookies)
+    log.info("admin server listening on http://%s:%d", host, port)
+    try:
+        server.serve_forever()
+    except KeyboardInterrupt:  # pragma: no cover
+        pass
+    finally:
+        server.server_close()

ytstream/web/templates.py

@@ -0,0 +1,253 @@
+"""Server-rendered HTML. One embedded stylesheet, no JavaScript beyond a
+confirm() on the destructive buttons."""
+
+from __future__ import annotations
+
+import html
+from datetime import date
+
+from .. import util
+from ..settings import DEFAULTS, EDITABLE, MASKED_KEYS
+
+STYLE = """
+:root {
+  --bg: #14161a; --panel: #1c1f26; --line: #2c313b; --text: #e6e8ec;
+  --muted: #99a0ae; --accent: #6aa9ff; --warn: #ffb454; --bad: #ff6b6b;
+  --good: #6ade9b;
+}
+* { box-sizing: border-box; }
+body { margin: 0; background: var(--bg); color: var(--text);
+  font: 15px/1.5 ui-sans-serif, system-ui, -apple-system, "Segoe UI", sans-serif; }
+main { max-width: 62rem; margin: 0 auto; padding: 1.5rem 1rem 4rem; }
+h1 { font-size: 1.4rem; margin: 0; }
+h2 { font-size: 1.05rem; margin: 2rem 0 .75rem; color: var(--muted);
+  text-transform: uppercase; letter-spacing: .06em; }
+header { display: flex; align-items: baseline; justify-content: space-between;
+  gap: 1rem; border-bottom: 1px solid var(--line); padding-bottom: .75rem; }
+header .sub { color: var(--muted); font-size: .85rem; }
+a { color: var(--accent); }
+.panel { background: var(--panel); border: 1px solid var(--line);
+  border-radius: 10px; padding: 1rem; }
+table { width: 100%; border-collapse: collapse; }
+th, td { text-align: left; padding: .55rem .5rem; border-bottom: 1px solid var(--line);
+  vertical-align: middle; }
+th { color: var(--muted); font-weight: 600; font-size: .78rem;
+  text-transform: uppercase; letter-spacing: .05em; }
+td.num { text-align: right; font-variant-numeric: tabular-nums; }
+.muted { color: var(--muted); }
+.badge { display: inline-block; padding: .1rem .45rem; border-radius: 999px;
+  font-size: .75rem; border: 1px solid currentColor; }
+.badge.warn { color: var(--warn); }
+.badge.good { color: var(--good); }
+.badge.bad { color: var(--bad); }
+input[type=text], input[type=password], input[type=number], select {
+  background: #12141a; color: var(--text); border: 1px solid var(--line);
+  border-radius: 7px; padding: .45rem .55rem; font: inherit; width: 100%; }
+button { background: var(--accent); color: #0b1017; border: 0; border-radius: 7px;
+  padding: .5rem .9rem; font: inherit; font-weight: 600; cursor: pointer; }
+button.secondary { background: #2b3140; color: var(--text); }
+button.danger { background: transparent; color: var(--bad);
+  border: 1px solid var(--bad); font-weight: 500; padding: .3rem .6rem; }
+button.link { background: transparent; color: var(--accent); border: 0;
+  padding: .3rem .4rem; font-weight: 500; }
+form.inline { display: inline; }
+.row { display: flex; gap: .6rem; align-items: center; }
+.grid { display: grid; grid-template-columns: repeat(auto-fill, minmax(15rem, 1fr));
+  gap: .85rem; }
+label { display: block; font-size: .82rem; color: var(--muted);
+  margin-bottom: .25rem; }
+.field { margin-bottom: .3rem; }
+.error { color: var(--bad); font-size: .8rem; margin-top: .2rem; }
+.flash { border-radius: 8px; padding: .6rem .8rem; margin-bottom: 1rem;
+  border: 1px solid; }
+.flash.ok { color: var(--good); border-color: var(--good); }
+.flash.bad { color: var(--bad); border-color: var(--bad); }
+.login { max-width: 21rem; margin: 6rem auto; }
+footer { margin-top: 2.5rem; color: var(--muted); font-size: .8rem; }
+@media (max-width: 40rem) {
+  th.hide, td.hide { display: none; }
+}
+"""
+
+
+def _e(value) -> str:
+    return html.escape("" if value is None else str(value), quote=True)
+
+
+def page(title: str, body: str) -> bytes:
+    return f"""<!doctype html>
+<html lang="en"><head>
+<meta charset="utf-8">
+<meta name="viewport" content="width=device-width, initial-scale=1">
+<title>{_e(title)}</title>
+<style>{STYLE}</style>
+</head><body><main>{body}</main></body></html>""".encode("utf-8")
+
+
+def login_page(error: str | None = None) -> bytes:
+    alert = f'<div class="flash bad">{_e(error)}</div>' if error else ""
+    body = f"""
+<div class="login">
+  <h1>ytstream</h1>
+  <p class="muted">Sign in to manage subscriptions.</p>
+  {alert}
+  <form method="post" action="/login" class="panel">
+    <div class="field">
+      <label for="password">Password</label>
+      <input type="password" id="password" name="password" autofocus
+             autocomplete="current-password">
+    </div>
+    <div style="margin-top:.8rem"><button type="submit">Sign in</button></div>
+  </form>
+  <footer>You will stay signed in on this device for a year.</footer>
+</div>"""
+    return page("Sign in — ytstream", body)
+
+
+def _channel_row(channel: dict, csrf: str) -> str:
+    failures = channel["consecutive_poll_failures"]
+    if failures > 2:
+        badge = f'<span class="badge bad">{failures} failed polls</span>'
+    elif channel["last_poll_ok"] == 0:
+        badge = '<span class="badge warn">last poll failed</span>'
+    else:
+        badge = ""
+
+    # A channel counting up towards being unsubscribed should be visible before
+    # it disappears, not afterwards.
+    missing = ""
+    if channel.get("missing_syncs"):
+        missing = (f' <span class="badge warn">absent {channel["missing_syncs"]}'
+                   f'/{channel.get("missing_threshold", 3)} syncs</span>')
+
+    retention = channel["retention_days"]
+    retention_value = "" if retention is None else str(retention)
+    placeholder = f"default ({channel['global_retention']})"
+
+    return f"""
+<tr>
+  <td>
+    <strong>{_e(channel['title'])}</strong> {badge}<br>
+    <span class="muted">{_e(channel['handle'] or channel['channel_id'])}</span>
+  </td>
+  <td class="num">{channel['episodes']}</td>
+  <td class="hide muted">{_e(channel['source'])}{missing}</td>
+  <td class="hide muted">{_e(channel['latest'] or '—')}</td>
+  <td class="hide muted">{_e(channel['last_polled_at'] or 'never')}</td>
+  <td>
+    <form method="post" action="/channels/{channel['id']}/retention" class="row">
+      <input type="hidden" name="csrf" value="{_e(csrf)}">
+      <input type="number" name="days" min="1" style="width:6.5rem"
+             value="{_e(retention_value)}" placeholder="{_e(placeholder)}">
+      <button class="link" type="submit">save</button>
+    </form>
+  </td>
+  <td>
+    <form method="post" action="/channels/{channel['id']}/rescan" class="inline">
+      <input type="hidden" name="csrf" value="{_e(csrf)}">
+      <button class="link" type="submit"
+        title="Re-queue videos previously skipped as too old that the current
+retention window now covers">rescan</button>
+    </form>
+    <form method="post" action="/channels/{channel['id']}/delete" class="inline"
+          onsubmit="return confirm('Permanently delete {_e(channel['title'])} and its whole tree? This cannot be undone.');">
+      <input type="hidden" name="csrf" value="{_e(csrf)}">
+      <button class="danger" type="submit">remove</button>
+    </form>
+  </td>
+</tr>"""
+
+
+def _settings_form(values: dict, errors: dict, csrf: str) -> str:
+    fields = []
+    for key in EDITABLE:
+        value = values.get(key, DEFAULTS[key])
+        error = errors.get(key)
+        if key in MASKED_KEYS and value:
+            shown, placeholder = "", "stored — leave blank to keep"
+        else:
+            shown, placeholder = value, ""
+        input_type = "password" if key in MASKED_KEYS else "text"
+        fields.append(
+            f"""<div class="field">
+  <label for="{_e(key)}">{_e(key.replace('_', ' '))}</label>
+  <input type="{input_type}" id="{_e(key)}" name="{_e(key)}"
+         value="{_e(shown)}" placeholder="{_e(placeholder)}" autocomplete="off">
+  {f'<div class="error">{_e(error)}</div>' if error else ''}
+</div>"""
+        )
+
+    return f"""
+<form method="post" action="/settings" class="panel">
+  <input type="hidden" name="csrf" value="{_e(csrf)}">
+  <div class="grid">{''.join(fields)}</div>
+  <div style="margin-top:1rem"><button type="submit">Save settings</button></div>
+</form>"""
+
+
+def index_page(
+    *,
+    channels: list[dict],
+    settings_values: dict,
+    settings_errors: dict,
+    csrf: str,
+    flash: tuple[str, str] | None = None,
+    add_error: str | None = None,
+    queue_depth: int = 0,
+) -> bytes:
+    flash_html = ""
+    if flash:
+        kind, message = flash
+        flash_html = f'<div class="flash {kind}">{_e(message)}</div>'
+
+    if channels:
+        rows = "".join(_channel_row(channel, csrf) for channel in channels)
+        table = f"""
+<div class="panel">
+<table>
+<thead><tr>
+  <th>Channel</th><th class="num">Episodes</th><th class="hide">Source</th>
+  <th class="hide">Latest upload</th><th class="hide">Last poll</th>
+  <th>Retention (days)</th><th></th>
+</tr></thead>
+<tbody>{rows}</tbody>
+</table>
+</div>"""
+    else:
+        table = '<div class="panel muted">No channels yet. Add one below.</div>'
+
+    add_error_html = f'<div class="error">{_e(add_error)}</div>' if add_error else ""
+
+    body = f"""
+<header>
+  <h1>ytstream</h1>
+  <div class="sub">
+    {len(channels)} channel(s) · {queue_depth} queued
+    · <form method="post" action="/logout" class="inline">
+        <input type="hidden" name="csrf" value="{_e(csrf)}">
+        <button class="link" type="submit">sign out</button>
+      </form>
+  </div>
+</header>
+
+{flash_html}
+
+<h2>Channels</h2>
+{table}
+
+<h2>Add a channel</h2>
+<form method="post" action="/channels" class="panel">
+  <input type="hidden" name="csrf" value="{_e(csrf)}">
+  <div class="row">
+    <input type="text" name="url" placeholder="https://www.youtube.com/@handle, @handle, or UC..." autocomplete="off">
+    <button type="submit">Add</button>
+  </div>
+  {add_error_html}
+</form>
+
+<h2>Settings</h2>
+{_settings_form(settings_values, settings_errors, csrf)}
+
+<footer>Downloads run hourly. Videos are deleted once they pass the retention
+window for their channel — this is a DVR, not an archive.</footer>"""
+    return page("ytstream", body)

ytstream/ytdlp.py

@@ -0,0 +1,109 @@
+"""Thin wrapper around the venv's yt-dlp binary.
+
+Everything that shells out to yt-dlp goes through here so the PATH handling (Deno
+must be discoverable — see specs.md §3) and the shared extractor args live in one
+place.
+"""
+
+from __future__ import annotations
+
+import json
+import logging
+import os
+import subprocess
+import urllib.request
+from pathlib import Path
+
+from . import config
+
+log = logging.getLogger(__name__)
+
+
+class YtdlpError(RuntimeError):
+    pass
+
+
+def binary() -> Path:
+    return config.VENV_BIN / "yt-dlp"
+
+
+def environment() -> dict[str, str]:
+    """Env for a yt-dlp subprocess.
+
+    yt-dlp locates the JS runtime by searching PATH, so the venv's bin directory
+    must come first — that is where Deno lives.
+    """
+    env = dict(os.environ)
+    env["PATH"] = f"{config.VENV_BIN}:{env.get('PATH', '')}"
+    return env
+
+
+def version() -> str:
+    result = subprocess.run(
+        [str(binary()), "--version"],
+        capture_output=True,
+        text=True,
+        env=environment(),
+        timeout=60,
+    )
+    if result.returncode != 0:
+        raise YtdlpError(result.stderr.strip() or "yt-dlp --version failed")
+    return result.stdout.strip()
+
+
+def extractor_args(pot_provider_url: str) -> list[str]:
+    return [
+        "--extractor-args",
+        "youtube:player_client=default,mweb",
+        "--extractor-args",
+        f"youtubepot-bgutilhttp:base_url={pot_provider_url}",
+    ]
+
+
+def run_json(args: list[str], timeout: int = 300) -> dict:
+    """Run yt-dlp with -J and parse the single JSON document it prints."""
+    cmd = [str(binary()), *args]
+    log.debug("yt-dlp %s", " ".join(args))
+    result = subprocess.run(
+        cmd, capture_output=True, text=True, env=environment(), timeout=timeout
+    )
+    if result.returncode != 0:
+        raise YtdlpError(first_error(result.stderr) or "yt-dlp failed")
+    try:
+        return json.loads(result.stdout)
+    except json.JSONDecodeError as exc:
+        raise YtdlpError(f"could not parse yt-dlp output: {exc}") from exc
+
+
+def run(args: list[str], timeout: int = 7200) -> subprocess.CompletedProcess:
+    """Run yt-dlp for its side effects, returning the completed process."""
+    cmd = [str(binary()), *args]
+    log.debug("yt-dlp %s", " ".join(args))
+    return subprocess.run(
+        cmd, capture_output=True, text=True, env=environment(), timeout=timeout
+    )
+
+
+def first_error(stderr: str) -> str:
+    for line in (stderr or "").splitlines():
+        if line.startswith("ERROR:"):
+            return line[len("ERROR:") :].strip()
+    return (stderr or "").strip().splitlines()[-1] if stderr.strip() else ""
+
+
+def pot_provider_ping(base_url: str, timeout: float = 5.0) -> dict:
+    """GET /ping on the bgutil provider. Raises on any failure."""
+    url = base_url.rstrip("/") + "/ping"
+    request = urllib.request.Request(url, headers={"User-Agent": config.USER_AGENT})
+    with urllib.request.urlopen(request, timeout=timeout) as response:
+        return json.loads(response.read().decode("utf-8"))
+
+
+def plugin_version() -> str | None:
+    """Installed bgutil plugin version, for comparison against the server's."""
+    try:
+        from importlib.metadata import version as pkg_version
+
+        return pkg_version("bgutil-ytdlp-pot-provider")
+    except Exception:  # pragma: no cover - only when the plugin is absent
+        return None