Files
ytstream/tests/conftest.py
T
Tom FluxandClaude Opus 5 155f05773d Build ytstream: catalogue, retention, subscription mirror, proxy
Phases 1-4 of plan.md §13. Forked from youtube-automate as planned rather than
written from scratch: naming, NFO, auth, the admin UI, settings and the DB layer
came across largely unchanged, download.py is gone, and the pieces that only make
sense for a streaming library are new.

  api.py       YouTube Data API v3 client. The whole metadata path.
  strm.py      Materialising: a .strm, an .nfo and a thumbnail. Replaces the
               330-line download.py, because the job is writing a URL to a file.
  subsync.py   The subscription mirror, most of which is refusals.
  reap.py      Retention, rewritten around the 30-day window and min_keep_videos.
  discovery.py RSS polling plus an API-backed, resumable, bounded backfill.
  proxy/       The verified PoC, moved in with a systemd unit.

330 tests, all passing, no network and no yt-dlp in any of them. The suite leans
towards the failure paths, because that is where this design can actually hurt
someone: a 403 that looks like an unsubscribe, a video that ages out and comes
back, a title that never arrives. tests/test_proxy.py replaces the two standalone
scripts under proxy/ and now drives the real make_handler(mgr, ...) rather than the
PoC's make_handler(path, done), so routing and video-id validation are covered too.

Ran it end to end against the live API and it found three real bugs.

The first was mine and the tests caught it: strm.remove pruned empty directories
up to the media root, so a channel directory whose tvshow.nfo happened to be
missing would be deleted along with the season. It only looked safe because
tvshow.nfo normally stops the walk. The prune boundary is now the channel
directory explicitly.

The other two only showed up against real data, and they compounded. The backfill
inserted rows with no title and left the RSS poll to fill them in — but RSS returns
15 entries, which for Pitch Side spans 23 days against a 30-day window, so five of
twenty episodes were named after their video ids. Worse, strm.materialise wrote
that fallback back to the database as the title, which made the row look titled and
permanently disabled the repair path. Both fixed: playlistItems.list now requests
snippet as well as contentDetails, which costs the same single quota unit and
carries the title alongside the exact publish date, and the fallback is used for the
filename without being persisted. A title that does arrive late now also removes the
badly-named files and re-queues, so the episode is rewritten rather than keeping its
video-id name forever. Verified against the live API: all twenty Pitch Side episodes
now carry real titles.

Measured on the real account: 119 subscriptions queued for approval and none added
on the first sync, then a two-channel run backfilled and materialised 26 episodes in
under seven seconds.

Two deliberate departures from plan.md, both recorded there:

min_keep_videos defaults to 5 rather than being left as an open question. Without
it 52 of 117 measured channels are empty Jellyfin series that flicker in and out as
their single video crosses the retention line, and the plan already recommended it.

The Jellyfin refresh is a bare /Library/Refresh with a comment explaining why it
must stay that way. A normal scan makes zero media probes; FullRefresh does probe,
and at 400 episodes that is 400 cold starts.

Not yet done: no systemd units are installed (needs root — deploy/deploy.sh), the
admin UI has no routes for sources or the approval queue yet, and nothing has been
pointed at the real media root.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-08-12 16:35:23 +01:00

203 lines
6.6 KiB
Python

"""Test fixtures.
Every path the application uses is redirected into a tmpdir. No test touches the
network, a real yt-dlp, a real Jellyfin, or the real media tree — the YouTube API
client is always a stub (see `FakeApi`), because the point of a test suite here is
to pin down behaviour that only shows up on the failure paths: a 403, an empty
response, a video that ages out and must not come back.
"""
from __future__ import annotations
import os
import tempfile
from pathlib import Path
# config resolves its paths at import time, so the environment has to be set
# before anything from the package is imported.
_SANDBOX = Path(tempfile.mkdtemp(prefix="yts-tests-"))
os.environ.setdefault("YTS_STATE_DIR", str(_SANDBOX / "state"))
os.environ.setdefault("YTS_MEDIA_ROOT", str(_SANDBOX / "media"))
os.environ.setdefault("YTS_DB_PATH", str(_SANDBOX / "state" / "ytstream.db"))
os.environ.setdefault("YTS_LOCK_PATH", str(_SANDBOX / "state" / "run.lock"))
os.environ.setdefault("YTS_VENV_BIN", str(_SANDBOX / "venv" / "bin"))
import pytest # noqa: E402
from ytstream import api, config, db, util, videos # noqa: E402
from ytstream.settings import Settings # noqa: E402
FIXTURES = Path(__file__).parent / "fixtures"
# A real-looking channel id: UC + 22 chars.
CHANNEL_ID = "UCW7jUEpYT_t0Gsf632d6_wQ"
@pytest.fixture()
def media_root(tmp_path, monkeypatch):
"""Point the media root at a per-test tmpdir."""
root = tmp_path / "media"
root.mkdir(parents=True)
monkeypatch.setattr(config, "MEDIA_ROOT", root)
return root
@pytest.fixture()
def conn(tmp_path):
connection = db.connect(tmp_path / "ytstream.db")
yield connection
connection.close()
@pytest.fixture()
def settings(conn):
settings = Settings(conn)
# Tests that reach the API go through FakeApi, but the code refuses to call
# out at all without a key, so give it one that is never used for real.
settings.set("youtube_api_key", "test-key")
return settings
@pytest.fixture()
def channel(conn):
"""One subscribed channel, returned as a row."""
return add_channel(conn, CHANNEL_ID, "clabretro", "clabretro")
@pytest.fixture()
def no_network(monkeypatch):
"""Fail loudly if anything tries to open a socket.
Belt and braces: a test that accidentally hits the network would pass locally
and fail in a different week for reasons nobody could reproduce.
"""
import urllib.request
def forbidden(*args, **kwargs):
raise AssertionError("test attempted a network call")
monkeypatch.setattr(urllib.request, "urlopen", forbidden)
# --------------------------------------------------------------------- helpers
def add_channel(conn, channel_id: str, title: str, dir_name: str, **kwargs):
fields = {"source": "youtube", "backfilled": 1, "uploads_playlist": "UULF"}
fields.update(kwargs)
columns = ", ".join(fields)
marks = ", ".join("?" * len(fields))
with conn:
conn.execute(
f"INSERT INTO channel (channel_id, handle, title, description, dir_name, "
f"added_at, {columns}) VALUES (?, ?, ?, ?, ?, ?, {marks})",
(channel_id, f"@{dir_name}", title, "A channel", dir_name,
util.utcnow_iso(), *fields.values()),
)
return conn.execute(
"SELECT * FROM channel WHERE channel_id = ?", (channel_id,)
).fetchone()
def add_video(conn, channel_pk, video_id, **kwargs):
"""Insert a video row with sensible defaults."""
defaults = {
"title": f"Video {video_id}",
"upload_date": "2026-08-01",
"state": videos.LISTED,
"discovery_source": videos.SOURCE_UULF,
"duration": 900,
}
defaults.update(kwargs)
videos.insert(conn, channel_pk=channel_pk, video_id=video_id, **defaults)
return videos.get(conn, video_id)
def feed_bytes(name: str) -> bytes:
return (FIXTURES / name).read_bytes()
def make_feed(entries: list[dict], *, playlist_published: str = "2019-01-01T00:00:00+00:00") -> bytes:
"""Build an Atom feed shaped like YouTube's.
Includes the feed-level <published> that is NOT an entry, because scraping
timestamps instead of walking atom:entry picks it up and yields nonsense
upload rates — a mistake made once while measuring.
"""
items = "".join(
f"""
<entry>
<id>yt:video:{e['video_id']}</id>
<yt:videoId>{e['video_id']}</yt:videoId>
<title>{e.get('title', 'Untitled')}</title>
<published>{e['published']}</published>
</entry>"""
for e in entries
)
return f"""<?xml version="1.0" encoding="UTF-8"?>
<feed xmlns:yt="http://www.youtube.com/xml/schemas/2015"
xmlns:media="http://search.yahoo.com/mrss/"
xmlns="http://www.w3.org/2005/Atom">
<title>Videos</title>
<published>{playlist_published}</published>{items}
</feed>""".encode()
class FakeApi:
"""Stand-in for `api.Api` with scriptable responses.
Counts calls so tests can assert the batching actually batches — the whole
quota argument in the plan rests on 50 ids per call, and a regression to one
call per video would be silent and expensive.
"""
def __init__(self, *, subs=None, uploads=None, durations=None, channel=None,
raises=None):
self._subs = subs
self._uploads = uploads or []
self._durations = durations or {}
self._channel = channel
self._raises = raises
self.calls = 0
self.duration_calls = 0
self.subscription_calls = 0
def subscriptions(self, channel_id):
self.subscription_calls += 1
self.calls += 1
if isinstance(self._raises, Exception):
raise self._raises
return list(self._subs or [])
def uploads(self, channel_id, *, kind="UULF", since=None, limit=None,
page_token=None):
self.calls += 1
produced = 0
for entry, token in self._uploads:
if since is not None and entry["published"] < since:
return
yield entry, token
produced += 1
if limit is not None and produced >= limit:
return
def durations(self, video_ids):
self.duration_calls += 1
self.calls += 1
return {vid: self._durations[vid] for vid in video_ids
if vid in self._durations}
def channel(self, channel_id):
self.calls += 1
return self._channel
def resolve_handle(self, handle):
self.calls += 1
return self._channel
def patch_api(monkeypatch, module, fake: FakeApi):
"""Make `module.api.Api(...)` return `fake` regardless of arguments."""
monkeypatch.setattr(module.api, "Api", lambda *a, **kw: fake)
return fake