diff --git a/plan.md b/plan.md index b6e0f15..74599ac 100644 --- a/plan.md +++ b/plan.md @@ -211,6 +211,9 @@ short of §4.2. 4. Restrict the key: Application restrictions → *None* (it is called from a server, so referrer and Android/iOS restrictions do not apply; an IP restriction is optional and breaks if susan's residential IP rotates). API restrictions → **YouTube Data API v3 only**. + **Do step 2 before this step.** YouTube Data API v3 does not appear in the API-restriction picker + until it is enabled on the project, so restricting first produces a key that blocks the one API it + exists for — see §16, which is exactly what happened. 5. Paste it into the ytstream admin UI. It is stored in the `setting` table like `jellyfin_api_key` already is — never in the repo, never in a systemd unit. @@ -778,6 +781,11 @@ Things already paid for once. All of these are verified. spend an afternoon trying to scrape it; the API is the only route (§4). - **`pkill -f 'ytstream.py'` kills the shell that runs it**, because the command string contains its own pattern. Bracket it: `pkill -f 'ytstrea[m].py'`. +- **Google returns 403 `forbidden` for two unrelated setup mistakes**, and the useful signal is in + `error.details[].reason`, not `error.errors[].reason`. `SERVICE_DISABLED` means the API is not + enabled on the project (and carries an `activationUrl` naming the project number); + `API_KEY_SERVICE_BLOCKED` means the API is enabled but *this key's* restrictions exclude it. They + are fixed on different console screens. `tools/verify_api.py` distinguishes them and prints the fix. --- @@ -800,12 +808,24 @@ Things already paid for once. All of these are verified. - **`tools/verify_api.py` written and self-tested** (ISO-8601 duration parser unit-checked against six cases; argparse and import verified). It runs all three API checks in one command. -### Blocked on one thing only +### Key created, project 510818173753 — two setup steps deep, one to go -**There is no Google API key on this machine** — I searched for `AIza`-shaped strings across -`/opt/*`, `/home/susan`, the settings table and the config trees, and there is none. Creating one -needs a browser and Tom's Google account (§4.1, ~5 minutes, free, no billing). Until then these three -remain unverified: +Progress on the key itself, all of it diagnosed from the error bodies: + +1. **Key created** and reaching Google — it authenticates, so the key string is good. +2. **`SERVICE_DISABLED`** — YouTube Data API v3 was not enabled on project `510818173753`. Fixed by + enabling it. +3. **`API_KEY_SERVICE_BLOCKED`** ← *current state.* The API is now enabled, but the key's own API + restrictions exclude it, so every method returns "Requests to this API youtube method … are + blocked". Fix at + `https://console.cloud.google.com/apis/credentials?project=510818173753` → the key → API + restrictions → *Don't restrict key*, or tick YouTube Data API v3. + +The ordering trap is worth remembering rather than rediscovering: the API must be enabled **before** +the key can be restricted to it, because it is absent from the picker until then. §4.1 step 4 now +says so. + +Until the key answers, these three assumptions remain unverified: | # | Assumption | Consequence if it fails | |---|---|---| @@ -814,8 +834,9 @@ remain unverified: | 3 | `videos.list` returns parseable `contentDetails.duration` | No `` in NFOs without a yt-dlp extraction per video. Degrades, does not block. | Only #1 is a genuine blocker, and it also carries the number that sets `subsync_max_new`. Two -prerequisites, both external: **Tom creates the API key**, and **his brother unchecks "Keep all my -subscriptions private"**. Then one command closes Phase 0: +external prerequisites remain: **the key's API restriction** (above), and **his brother unchecking +"Keep all my subscriptions private"** — note that nothing so far has tested the second, because every +call has failed at the key before reaching YouTube's privacy check. Then one command closes Phase 0: ```sh python3 /opt/ytstream/tools/verify_api.py --key AIza... diff --git a/tools/verify_api.py b/tools/verify_api.py index 3b285fd..2ab2e2f 100755 --- a/tools/verify_api.py +++ b/tools/verify_api.py @@ -38,15 +38,40 @@ ISO8601 = re.compile( class ApiError(Exception): - def __init__(self, status, reason, body): + def __init__(self, status, reason, body, message="", activation_url=""): super().__init__(f"HTTP {status} {reason}") self.status = status self.reason = reason self.body = body + self.message = message + self.activation_url = activation_url + + +class ServiceDisabled(ApiError): + """YouTube Data API v3 is not enabled on the key's project. + + Signalled by `accessNotConfigured` in errors[] or `SERVICE_DISABLED` in + details[], and carries an activationUrl naming the project number. + """ + + +class KeyRestricted(ApiError): + """The API is enabled, but this key's API restrictions exclude it. + + Signalled by `API_KEY_SERVICE_BLOCKED` in details[] with the message + "Requests to this API youtube method ... are blocked". Distinct from + ServiceDisabled and fixed in a completely different console screen, which is + why the two are separate types. + + Both were observed from one key on 2026-08-12, in this order, and the + ordering is the trap: YouTube Data API v3 does not appear in the key's API + restriction picker until the API is enabled on the project. Enable first, + restrict second, or the key is created blocking the very API it is for. + """ def call(endpoint: str, key: str, **params) -> dict: - """One API call. Raises ApiError with the parsed reason on 4xx/5xx.""" + """One API call. Raises ApiError (or ServiceDisabled) on 4xx/5xx.""" params["key"] = key url = f"{API}/{endpoint}?" + urllib.parse.urlencode(params) try: @@ -54,13 +79,34 @@ def call(endpoint: str, key: str, **params) -> dict: return json.load(response) except urllib.error.HTTPError as exc: raw = exc.read().decode("utf8", "replace") - reason = "" + reason = message = activation = "" + detail_reasons = set() try: - errors = json.loads(raw).get("error", {}).get("errors", []) + error = json.loads(raw).get("error", {}) + message = error.get("message", "") + errors = error.get("errors", []) reason = errors[0].get("reason", "") if errors else "" + for detail in error.get("details", []): + meta = detail.get("metadata") or {} + if meta.get("activationUrl"): + activation = meta["activationUrl"] + if detail.get("reason"): + detail_reasons.add(detail["reason"]) except ValueError: pass - raise ApiError(exc.code, reason, raw[:400]) from exc + + # `forbidden` is used for both of these, so the details[] reason is what + # actually distinguishes them. Check the specific ones before it. + if "API_KEY_SERVICE_BLOCKED" in detail_reasons: + cls = KeyRestricted + elif ("SERVICE_DISABLED" in detail_reasons + or reason == "accessNotConfigured" + or "has not been used in project" in message): + cls = ServiceDisabled + else: + cls = ApiError + raise cls(exc.code, reason or next(iter(detail_reasons), ""), + raw[:400], message, activation) from exc def iso8601_seconds(text: str) -> int | None: @@ -198,6 +244,49 @@ def check_durations(key: str, channel_id: str) -> None: f"sample: " + ", ".join(f"{v}={s}s" for v, s in parsed[:5])) +def preflight(key: str) -> None: + """One cheap call to separate "project not set up" from "assumption wrong". + + Without this, a disabled API fails all three checks with three different + messages and none of them says what to do about it. + """ + try: + call("videos", key, part="id", id="dQw4w9WgXcQ") + except ServiceDisabled as exc: + print("\nSTOP: the key is valid, but YouTube Data API v3 is not enabled " + "on its project.") + print(f"\n {exc.message}\n") + if exc.activation_url: + print(" Enable it here, then wait ~2 minutes and re-run:") + print(f" {exc.activation_url}\n") + print(" Creating an API key and enabling the API are separate steps in " + "the console;\n only the first one has been done.") + raise SystemExit(2) + except KeyRestricted as exc: + project = "" + match = re.search(r"projects?/(\d+)", exc.body) or \ + re.search(r"project=(\d+)", exc.body) + if match: + project = f"?project={match.group(1)}" + print("\nSTOP: the API is enabled, but this key's API restrictions block " + "it (API_KEY_SERVICE_BLOCKED).") + print(f"\n {exc.message}\n") + print(" Fix: Credentials -> click the key -> API restrictions -> either") + print(' "Don\'t restrict key", or tick "YouTube Data API v3" in the list.') + print(f" https://console.cloud.google.com/apis/credentials{project}\n") + print(" Note the ordering trap: YouTube Data API v3 is absent from that") + print(" picker until the API is enabled on the project, so a key created") + print(" and restricted first ends up blocking the API it was made for.") + raise SystemExit(2) + except ApiError as exc: + if exc.status in (400, 403) and exc.reason in ("badRequest", "keyInvalid", + "API_KEY_INVALID"): + print(f"\nSTOP: the key was rejected -- {exc.reason}: {exc.message}") + raise SystemExit(2) + # Anything else is worth letting the real checks characterise. + print(f"\n (preflight warning: {exc} reason={exc.reason!r})") + + def main() -> int: parser = argparse.ArgumentParser() parser.add_argument("--key", required=True, help="YouTube Data API v3 key") @@ -208,6 +297,7 @@ def main() -> int: args = parser.parse_args() print(f"Phase 0 verification against the live API (~5 quota units of 10,000/day)") + preflight(args.key) check_subscriptions(args.key, args.brother) check_uploads_playlist(args.key, args.sample) check_durations(args.key, args.sample)