242 lines
14 KiB
Markdown
242 lines
14 KiB
Markdown
# zapret2-autotune
|
||
|
||
Go CLI for scanning and checking YouTube TLS bypass strategies with `blockcheckw`.
|
||
|
||
## Manual managed-test activation (not deployed)
|
||
|
||
`managed init|status|active-ids|apply` requires exact role `managed-test`, target ID
|
||
`zapret-autotune-managed-v1`, a provisioned runtime marker and zapret executables,
|
||
and a private dedicated mount at `/var/lib/zapret-autotune/target`.
|
||
`ZAPRET_CONFIG` must equal `/var/lib/zapret-autotune/target/config`: the standalone
|
||
full config is allowed only in that fresh target volume, never over an existing
|
||
service config. Existing-service include fragments are not implemented.
|
||
The managed proxy must equal `socks5://127.0.0.1:$SOCKS_PORT` and that listener
|
||
must be owned by target-local `ss-local`; arbitrary loopback/host ports fail.
|
||
Init creates a private disabled baseline and identity/journal without overwriting
|
||
existing config. Status shows generation/hash/groups, not secrets or live health.
|
||
Apply accepts either or both of `--frontend-candidate ID` and
|
||
`--googlevideo-candidate ID` and requires one qualified candidate per selected
|
||
group, fixed target init executable and explicit runtime-config
|
||
binding. It uses full target-local firewall/daemon restart and Player API post-check, with
|
||
atomic rollback on failure. See [MANAGED_TARGET.md](MANAGED_TARGET.md) for guards
|
||
and provisioning prerequisites. An isolated pinned deployment definition is in
|
||
[deploy/managed-target](deploy/managed-target/README.md); build/run are deferred.
|
||
An injected bootstrap/saved-first recovery adapter now connects the persisted
|
||
monitor foundation to blockcheck, immutable candidates and transactional apply.
|
||
`managed monitor` and the dedicated target entrypoint now wire these adapters;
|
||
this increment is not tested by the producer or live/runtime verified.
|
||
No remote activation or Xray changes are included. Image
|
||
compatibility and route/rollback gates are not yet remotely proven; adapter
|
||
tests/build are deferred to the orchestrator.
|
||
|
||
The apply flags are the entire desired group set, not a merge: a single-group
|
||
apply removes the other group. Frontend-only supports bootstrap discovery;
|
||
exact-host GGC-only is also valid. Status lists missing or unverifiable groups
|
||
as unresolved, never as healthy or a complete playback configuration. Player
|
||
API post-checks do not establish either group's health. `active-ids` fails closed
|
||
on pending, legacy or unverifiable state; journal IDs and config bytes commit
|
||
and roll back together, including after a crash.
|
||
|
||
## One-shot Player API health
|
||
|
||
Internal recovery building blocks enumerate qualified immutable candidates
|
||
newest-first with a 1–10 limit and recheck saved arguments on an exact endpoint
|
||
through an injected combined TLS checker. Only complete, valid two-TLS 3-pass
|
||
nonpassing evidence permits the nonpass result; malformed reports and runner,
|
||
filesystem or context failures remain infrastructure errors. These helpers do
|
||
not update stored evidence, apply configuration or wire managed recovery/monitor
|
||
commands. Their local fixture tests/build are deferred to the orchestrator.
|
||
|
||
```sh
|
||
zapret-autotune health youtube
|
||
```
|
||
|
||
This makes one bounded Android Player API request through the explicitly
|
||
configured **loopback SOCKS5** endpoint. Healthy means only
|
||
`playabilityStatus.status == "OK"`; no streaming URL is required and no media is
|
||
fetched. **API health is not playback health.** Non-OK/missing status, malformed
|
||
responses and request failures return exit 1; invalid config returns exit 2.
|
||
Curl configuration and proxy bypasses are disabled; ambient proxy settings are
|
||
not used and there is no direct fallback. The request times out after 30 seconds
|
||
(35-second process deadline), with a 1 MiB response limit and sanitized errors.
|
||
|
||
| Environment variable | Default |
|
||
| --- | --- |
|
||
| `ZAPRET_AUTOTUNE_SOCKS_PROXY` | `socks5://127.0.0.1:11080` |
|
||
| `ZAPRET_AUTOTUNE_VIDEO_ID` | `dQw4w9WgXcQ` |
|
||
| `ZAPRET_AUTOTUNE_HEALTH_INTERVAL` | `5m` |
|
||
| `ZAPRET_AUTOTUNE_FAILURE_THRESHOLD` | `3` |
|
||
| `ZAPRET_AUTOTUNE_RECOVERY_COOLDOWN` | `30m` |
|
||
|
||
Configuration is parsed once per health invocation. The proxy must use `socks5`
|
||
with a loopback IP or `localhost`, port 1–65535, and no credentials/path/query.
|
||
Video IDs must be 11 characters; durations and the failure threshold must be
|
||
positive. Explicit empty values are invalid. Interval, consecutive-failure
|
||
threshold and cooldown configure `managed monitor` (the one-shot command still
|
||
does not schedule anything). **CLI/entrypoint integration exists; tests/build
|
||
and live target validation remain deferred.**
|
||
|
||
The future managed target is a **brand-new prod-like zapret2-ss container in the
|
||
test environment only**, not the existing container. Its stable configurable
|
||
SOCKS endpoint must be loopback-accessible to this probe and same-host Xray.
|
||
The one-shot command does not persist health. This command does not create or change any container, apply
|
||
configuration, restart services or roll back anything. Xray configuration is
|
||
out of scope and will be configured separately by the user. Run this command in
|
||
the host network context where that loopback endpoint is reachable; Docker
|
||
`localhost` otherwise refers to the CLI container itself.
|
||
|
||
## Exact cache-host proposal
|
||
|
||
```sh
|
||
zapret-autotune tune googlevideo --video-id dQw4w9WgXcQ --proxy socks5://localhost:1080
|
||
```
|
||
|
||
Both flags are required. The validated Android 21.02.35 player request discovers
|
||
exactly one current cache hostname through the explicit proxy. `blockcheckw`
|
||
then scans only that hostname for TLS1.2/TLS1.3, checks the shortlist with
|
||
`--take 10 --passes 3`, ranks fully passing strategies by measured data-transfer
|
||
speed/latency, and checks the exact combined arguments 3/3 on both protocols.
|
||
The proxy is for discovery; blockcheckw uses its own network path, not that proxy.
|
||
No Selectel speedtest is used. This is not a YouTube playback/SABR test.
|
||
|
||
Only after successful transactional persistence does this command print one
|
||
inert `NFQWS2_OPT` profile filtered by the **exact discovered hostname**, never
|
||
the `googlevideo.com` wildcard. Cache hosts rotate; one host is not wildcard
|
||
evidence. GGC proposals use `--hostlist-domains=^<discovered-host>`: the `^`
|
||
prefix disables Zapret2's default subdomain matching, so untested subdomains
|
||
are not included. Only validated lowercase `*.googlevideo.com` hostnames are
|
||
accepted; commas and a leading `^` in the discovered hostname are rejected.
|
||
Schema-v3 candidates use group `googlevideo`, one exact endpoint, and
|
||
both endpoint×TLS measurements with raw JSON and timestamps. An additive v3
|
||
cache-run journal records failed attempts without candidates and successful
|
||
attempts atomically with their candidate/evidence; legacy frontend runs keep
|
||
their existing target constraint. Database failures emit no options.
|
||
`history` and `strategies youtube` include the cache group and exact endpoint;
|
||
`status` still describes the legacy frontend journal only.
|
||
|
||
Signed URLs are not printed. Scanner diagnostics are retained in restricted
|
||
artifact files rather than relayed to the terminal. No configuration is applied,
|
||
no profile is marked active, and no service is restarted. The staged pipeline is
|
||
scan/propose → guarded managed-test activation → dedicated target monitoring;
|
||
the deployment definition exists but is not deployed. `managed monitor` wires
|
||
the scheduler and recovery; live compatibility remains unverified. Manual
|
||
proposal-only recovery is available below.
|
||
No-argument `tune youtube` remains frontend-only and unchanged.
|
||
|
||
## Alternatives-first recovery (proposal only)
|
||
|
||
```sh
|
||
zapret-autotune recover googlevideo --video-id dQw4w9WgXcQ --proxy socks5://localhost:1080
|
||
```
|
||
|
||
Recovery discovers the current exact cache host once, then rechecks qualified
|
||
schema-v3 `googlevideo` candidates newest-first using their exact saved arguments.
|
||
Qualification covers every saved endpoint × TLS protocol; revalidation checks
|
||
the fresh host for combined TLS1.2+TLS1.3 3/3. The first passing scenario is saved
|
||
as a new immutable candidate/run scoped to that host and emitted as an exact-host
|
||
proposal, **not active**. No scan runs after a passing saved scenario.
|
||
Only no eligible candidates or complete, valid non-passing evidence permit the
|
||
normal tune scan on the already-discovered host. Database, discovery, cancellation,
|
||
runner and malformed/incomplete-report errors fail closed without a fallback scan.
|
||
This command does not apply configuration, restart services or start monitoring.
|
||
|
||
## Cache-host discovery
|
||
|
||
```sh
|
||
zapret-autotune discover youtube --video-id dQw4w9WgXcQ --proxy socks5://localhost:1080
|
||
```
|
||
|
||
Both flags are required. Use a proxy address reachable from the CLI; in Docker,
|
||
`localhost` is the container itself, not the Docker host. Proxy credentials,
|
||
cookies, and API credentials are not supported. This command requires `curl`
|
||
with `--json` support and POSTs to the YouTube player API through SOCKS5 using
|
||
Android client 21.02.35, SDK 30, Android 11. Requests time out after 30 seconds;
|
||
responses are limited to 1 MiB and must have playability status `OK` and a valid
|
||
HTTPS `serverAbrStreamingUrl` under `.googlevideo.com`.
|
||
|
||
Output is JSON containing only `videoId` and `hostnames`; signed streaming URLs
|
||
are never printed. Cache hosts may rotate. This is **cache-host discovery only**,
|
||
not verification of real video transfer: a discovered SABR URL can return
|
||
`sabr.malformed_config` on GET. Discovery does not fetch media, persist results,
|
||
apply production configuration, or restart services. Invalid input and failed
|
||
requests return nonzero with sanitized errors.
|
||
|
||
Build the Docker image with:
|
||
|
||
```sh
|
||
docker build -t zapret-autotune .
|
||
```
|
||
|
||
The `tune youtube` command scans and checks candidate strategies; it prints a
|
||
proposed `NFQWS2_OPT` and does not apply configuration or restart services.
|
||
|
||
Tune runs are recorded in pure-Go SQLite (no CGO). A verified combined candidate
|
||
and its TLS1.2/TLS1.3 3/3 pass results are committed together before options are
|
||
printed; failed scans/checks are recorded as failed runs without a candidate.
|
||
Database initialization or persistence errors return nonzero and emit no options.
|
||
`status` shows the latest recorded run, or explicitly says there is no recorded
|
||
run. Candidates are **PROPOSED, not active**; this journal does not track managed activation and
|
||
production health is not checked. A later failed run is shown instead of an older
|
||
proposal. An output-write failure can leave a verified proposal recorded, but
|
||
still returns nonzero.
|
||
|
||
Set `ZAPRET_AUTOTUNE_DB` to override the default database path
|
||
`/var/lib/zapret-autotune/autotune.db`. The parent directory is created if needed;
|
||
it must be writable. In Docker, mount that directory to retain history:
|
||
|
||
```sh
|
||
docker run --rm -v autotune-data:/var/lib/zapret-autotune zapret-autotune status
|
||
```
|
||
|
||
Schema version 5 is initialized transactionally on `tune` and `status`; subsequent
|
||
opens are idempotent and newer, unsupported schema versions are rejected.
|
||
|
||
Schema v4 adds a policy-state foundation keyed by stable service ID (not a cache
|
||
hostname or SOCKS address): last probe time, consecutive failures, recovery lock
|
||
and cooldown expiry. Internal operations accept injected time: success resets the
|
||
streak, failures increment it, and an atomic recovery acquisition requires the
|
||
configured threshold, an elapsed cooldown and no running recovery. Completion,
|
||
successful or failed, requires the exact per-acquisition ownership token returned
|
||
by acquisition, clears the lock and starts cooldown from completion time. Stale
|
||
or duplicate completion is rejected without changing the lock, token or cooldown,
|
||
even after a newer acquisition starts. State survives reopening the database.
|
||
The internal `runManagedMonitor` supervisor probes immediately and periodically,
|
||
persists observations, and invokes injected recovery only at threshold outside
|
||
cooldown. Unresolved GGC is reported without stopping monitoring; DB, safety,
|
||
runner, lock and finalization errors are fatal. Cooldown starts when recovery returns, including
|
||
failure or cancellation. Callbacks must honor cancellation and join their children.
|
||
Existing target guards run before a lifetime nonblocking `flock` in the private
|
||
target state directory. With that exclusive singleton held, one-shot startup
|
||
reconciliation uses schema-v5 PID/starttime identity: only a proven dead or
|
||
PID-reused owner permits clearing its exact lease and starting crash cooldown.
|
||
A live owner remains busy; legacy or unverifiable owners fail closed. Probe
|
||
history is preserved. The lock file is never unlinked.
|
||
The scheduler core has fake-time tests for scheduling, reset, cooldown,
|
||
cancellation, parallel starts, fatal DB errors, reopen and safe startup
|
||
reconciliation. Internal injected managed recovery and monitor callbacks now
|
||
support saved-first frontend checks, scan fallback only for valid non-pass
|
||
evidence, and fresh exact-host GGC recovery with a frontend-only rollback boundary.
|
||
Checks remove prior combined reports; missing/malformed reports fail closed.
|
||
Scan shortlists must be regular files no older than the scan start's whole second
|
||
(production scans use unique temporary paths). Fake-only orchestration tests are
|
||
added; verification is deferred. **Monitor CLI/entrypoint wiring exists but no
|
||
live target has been deployed**; existing container/Xray are untouched.
|
||
Manual managed activation remains separate. Verification of this increment is
|
||
deferred to the orchestrator.
|
||
|
||
Schema v3 adds immutable strategy candidate definitions (group, declared exact
|
||
endpoints, args and normalized-args fingerprint) and endpoint/protocol evidence.
|
||
Each newly successful tune saves its exact verified combined frontend profile for
|
||
`youtube.com`, both TLS measurements, raw measurement JSON and checked timestamps
|
||
in the same transaction as the existing run. Qualification requires every declared
|
||
endpoint to pass exactly 3/3 for both TLS1.2 and TLS1.3, with full success/coverage.
|
||
Existing v2 history is unchanged and is not backfilled into independent candidates.
|
||
Candidates have no DB active/installed state; managed activation uses its own
|
||
durable config journal. Scheduling and recovery are wired only to the dedicated
|
||
managed target; live monitoring has not been verified or deployed.
|
||
The explicit `tune googlevideo` path adds exact-cache proposals only.
|
||
Proposed runs store a SHA-256 fingerprint of validated arguments with separator
|
||
whitespace normalized and argument order preserved; repeated proposals retain
|
||
separate history rows. Failed runs have no fingerprint. Version 1 history is
|
||
upgraded atomically, rejecting invalid proposed arguments without changing it.
|