Files
zapret-autotune/deploy/managed-target/README.md
T
ZeusinaandGPT-6 Bot 3fc8563995 feat: discover and qualify Googlevideo CDNs autonomously
Retain strict three-host native qualification, discover fresh representative inventory automatically, and recheck deployed media before promotion. Document offline-only verification and outstanding field gates.

Co-authored-by: GPT-6 <gpt-6@openai.com>
2026-10-13 10:12:10 +03:00

307 lines
19 KiB
Markdown

# Isolated managed-test target
This Compose project provisions a new, isolated target. It does not attach to,
reconfigure, or stop the existing `zapret2-ss-autotune` container. The target
starts from a disabled managed baseline; `managed monitor` bootstraps the
frontend/API strategy, then discovers and applies an exact-host GGC strategy.
## New autonomous three-CDN source candidate (2026-10-13): offline only
Service-mode source now discovers representative inventory automatically and can
promote an existing experiment only through the unchanged version-1 owned native
proof factory: **three actual media hosts**, at least two roots and two prefixes,
post-warmup 3/3 on both TLS protocols. Discovery is bounded to three rounds,
15 requests maximum and three minutes; signed MP4 URLs need twenty minutes left.
VideoIDs are not CDN evidence. No manual domain list or single-CDN relaxation is
introduced. Missing/empty inventory defaults and explicit overrides are below.
Apply requires qualified frontend and fresh deployed checks of all three media
resources, not avatar readiness. Whole-service health remains DEGRADED until avatar
checks pass. Insufficient inventory preserves the committed experiment and uses
configured cooldown; true strategy exhaustion retains its minimum two-hour pause.
Suffix authorization remains extrapolative, not proof of every Googlevideo host.
This source is offline unit/fixture-verified only; no new image deployment or
autonomous three-CDN field acceptance is claimed. The historical results below
do not close that gate. Maximum strategy collection and PID1 global-failure
stay-alive are unfinished future increments, as is arbitrary declarative service
configuration. See [MANAGED_TARGET.md](../../MANAGED_TARGET.md) and
[GOOGLEVIDEO_DISCOVERY.md](../../GOOGLEVIDEO_DISCOVERY.md).
## Recorded Googlevideo suffix experiment (2026-10-12)
### Final field follow-up: DEGRADED / NOT READY
Supplied results for source `fd25cb89398300bb9849db4b5511af75bf86eccf`, correct
managed image `sha256:73e955e7fdc6efaef88c62ebe4f7afeab71c29d29ba3c88b2a2b7547bb647da5`,
container `a4e845a6d7…`, official managed PID 1 and SOCKS11082 supersede the
earlier pending media/fallback labels below. The `37823…` prototype is not active.
Watch `20:15:07.686Z` → `20:30:54.826Z` on 2026-10-12 completed over 15 minutes:
restart count0, monitor alive; a fresh post-report check returned SOCKS `0500`,
WWW HTTP200/SSLVerifyResult0. Native `socks5h` probes respect SS `-d 8.8.8.8`.
Post-warmup TLS1.2 and TLS1.3 each passed 3/3 on
`rr4---sn-axq7sn7z.googlevideo.com`, `rr5---sn-n8v7kn76.googlevideo.com`, and
`rr18---sn-axq7sn7z.googlevideo.com`: **18 actual HTTP206, 65536-byte MP4 `ftyp`
transfers**, valid Range `0..65535`. Initial warmup is separate: TLS1.2 three
timeouts, TLS1.3 three passes. No cold/all-host claim; the old rr5 `sn-ixh`
counterexample was not retested. Thumbnail JPEG decoding passed 3/3 per TLS
(2888 bytes, SSLVerifyResult0). Watch metadata found an allowed `yt3.ggpht.com`
avatar URL, but actual both-TLS avatar transfers returned curl28, zero samples.
There is no saved avatar evidence or qualified avatar profile.
Recovery now continued through validated peer-TLS/payload nonpasses with current
job-bound transaction/store/ownership guards: genuine no-qualified-strategy
cooldown advanced naturally while the monitor stayed alive, without a reset.
Snapshot generation9/IDs[1,3,9]/hash `0e34…`/experimental scope were unchanged;
candidate3 has no qualification/evidence. This is degraded recovery acceptance,
not successful avatar repair, durable family qualification or promotion.
Actual avatar transfer/strategy qualification and durable representative
multi-host family/mode qualification remain unfinished; automatic family
qualification is not implemented. Other targets on 1080/11080/11081 and Xray
were unchanged. No remote operations are performed by this docs update.
Private sanitized proof: `/home/agent/zapret2-autotune-rc-fd25cb8/proof/final.safe.json`.
Exact results and limits: [PROGRESS.md](../../PROGRESS.md).
### Historical initial experiment
Source RC `eade051c2bc335b71b1616d697326ca4c8019d70`, immutable image
`sha256:fdb60593e29d7c337a413a47aaddfbcaaeabdeb877042c58f506d4abdc98a13b`,
isolated SOCKS `127.0.0.1:11082`: recorded journal generation3, active [1,2,3],
scope `experimental_googlevideo_suffix`. Frontend `www.youtube.com` and
thumbnail `i.ytimg.com` profiles were preserved; older targets were unchanged.
These recorded observations predate this docs commit; no deployment or fresh
network check is implied here.
Fresh signed media on `rr16---sn-n8v7znsz.googlevideo.com` returned HTTP206,
65536 bytes with MP4 `ftyp` over TLS1.2. The user confirmed browser videos work
on 2026-10-12 and accepts initial learning. The shared suffix profile is native
circular A/E: **two strategies with per-host learning**, not a single uniform
static strategy or proof of all-host cold 3/3. Its explicit operator experimental
policy persists across restart under hash/rollback guards, but is **not
family-qualified or READY**. Browser-visible images do not qualify the avatar
profile; only the thumbnail image profile is currently active.
Service-mode health already probes fresh actual images and media using both TLS
protocols. Experimental recovery deliberately does not automatically qualify or
replace this approved profile. Remaining acceptance is representative signed
media coverage (at least three hosts across two roots/prefixes, post-warmup 3/3
on both TLS protocols plus media), avatar qualification, actual restart/recreation
recovery of this suffix RC, and autonomous experimental recovery/autoqualification
policy with accurate covered-versus-qualified status. These were that initial
RC's pending gates; see the final follow-up above for later field results and
[PROGRESS.md](../../PROGRESS.md) for current remaining work. The older clean-run
PASS below does not close family/avatar qualification.
## Provisioning
From this directory, create a private `.env` with a fresh independent password
and the configured host SOCKS port:
```sh
(umask 077; set -C; printf 'SS_PASSWORD=%s\nZAPRET_AUTOTUNE_HOST_SOCKS_PORT=11080\n' "$(openssl rand -hex 32)" > .env)
```
This refuses to overwrite an existing `.env`; do not print it or commit it.
Compose passes `SS_PASSWORD` as a container environment variable. It is removed
from the entrypoint environment before Shadowsocks children start and is never
placed in argv or logs. Docker administrators/root can still read container
environment and runtime config, so use this only for the isolated test target.
Create a **new empty** `./target-state`, owned by host UID 0 (runtime user),
mode `0700`; rootless/user-remapped Docker needs the mapped owner. Never reuse
state. Compose refuses a missing bind source. Retain state for recreation;
deletion loses transaction identity, journal, backups and candidate DB.
Validate without printing interpolated values:
`docker compose --env-file .env -f compose.yaml config --quiet`.
Then build and start only this Compose project. The rendered configuration
contains the environment password, so never use plain `docker compose config`
in logs or shared terminals.
By default Compose uses the locally built `zapret-autotune-managed-target:local`.
Tags are only local build/test candidates, not production release references.
For production promotion, set `ZAPRET_AUTOTUNE_IMAGE=<registry>/<managed-target>@sha256:<digest>`
in the private `.env` and pull that immutable image before starting the target.
Use a managed target image, not the generic image. Do not deploy production
with a tag or local default.
Build from the repository root with an explicit Dockerfile and assert the
image contract **before promotion**, even if its tag says managed-target:
```sh
IMAGE=zapret-autotune-managed-target:local
docker build --pull -f Dockerfile.target -t "$IMAGE" .
test "$(docker image inspect --format '{{json .Config.Entrypoint}}' "$IMAGE")" = '["/bin/sh","/opt/managed-target-entrypoint.sh"]'
docker run --rm --network none --entrypoint /bin/sh "$IMAGE" -ec 'test "$(cat /etc/zapret-autotune/target-id)" = zapret-autotune-managed-v1; test -f /opt/managed-target-entrypoint.sh'
```
Root `Dockerfile` is the proposal/prototype image, not the managed PID 1 service;
omitting `-f Dockerfile.target` previously produced a mistagged prototype.
These positive entrypoint/marker assertions do not replace live acceptance.
Promote the same immutable image as described in
[MANAGED_TARGET.md](../../MANAGED_TARGET.md#build-once-promote-the-same-image).
Offline static guard (from repository root):
`python3 deploy/managed-target/check_static.py`.
Offline real pre-launch validation and private Compose forwarding fixtures:
`python3 deploy/managed-target/check_config.py` (requires local Go and Compose).
Isolated, accelerated shell supervision fixtures (no network):
`python3 deploy/managed-target/check_supervision.py`.
## In-container supervision
### Opt-in service/adaptive and target-native DNS
Compose forwards `ZAPRET_AUTOTUNE_YOUTUBE_SERVICE` (`0` or `1`, default `0`),
`ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE` (`cold` or `adaptive`, default `cold`),
`ZAPRET_AUTOTUNE_VIDEO_ID` (default `dQw4w9WgXcQ`) and
`ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS` (optional JSON array of 1..5 unique VideoIDs).
Missing/empty inventory selects automatic service-mode samples: primary plus
`aqz-KE-bpKQ` and `jNQXAC9IVRw`, deduplicated. A nonempty explicit inventory
overrides those defaults, including an intentional single video. Invalid nonempty
settings are rejected. Adaptive requires service `1`. See `.env.example`.
Service `0` preserves legacy Player API/exact-host behavior.
Adaptive qualification is **post-warmup**, distinct from cold qualification;
initial connections may fail and native learning must repeat after restart.
It requires at least **three real unique signed ordinary media host samples**.
Three sample videos may all resolve to the same CDN host: that means
`DEGRADED`, no family-qualified googlevideo suffix, not fabricated coverage. The
explicit operator experiment above is separate from this qualification. Service
asset checks cover sampled thumbnail/avatar routes, not the whole thumbnail pool.
The current service/adaptive hardware acceptance gate is **pending**; the old
clean-run PASS below does not qualify these new modes.
Optional `SS_DNS_SERVER` is an unbracketed literal IPv4/IPv6 address only (for
example `8.8.8.8` or `2001:4860:4860::8888`); empty preserves existing resolver
behavior. No domain, port, zone, control characters or option strings are
accepted. Validation runs before service launch and reports opaque `configerror`
without echoing input. DNS applies only to this target's native `ss-server`.
The image build checks installed `ss-server -h` for native `-d <addr>` support;
the validated nonsecret address is passed as a separate `-d` argv value, not
eval or an assumed JSON key. Passwords still go only to private JSON configs.
Managed probes use `socks5h://127.0.0.1:1080`: Player, metadata, image,
media, recovery and warmup hostnames go through the owned SS listener rather than
curl's local DNS. Older private `socks5` overrides are validated and canonicalized
internally; credentials and unowned endpoints remain forbidden. Public legacy
discovery and root checker local `--resolve` pinning are unchanged.
This corrects a DNS path bypass, not a proven cause of earlier peer certificate
failures, and does not fix/prove all thumbnail routes. It is **not full asset
availability proof**. No DoH bridge, permanent Google IP pins or global host DNS
changes are supplied. PID 1 remains the sole startup/recovery owner.
Compose has no automatic restart. PID 1 restarts the entire owned runtime only
for technical failures, with three starts per rolling six hours (including initial
startup) and cancellable 15/30-second cooldowns. Clean typed recovery timeouts,
unexpected monitor exits/crashes, and Shadowsocks failures qualify only if
shutdown cleanup succeeds. Configuration (64), safety/database/rollback/cleanup
failures (78), and graceful SIGTERM (0) never request restart. Exhausting the
restart budget exits 78. Investigate and
repair nonretry failures before manually starting again; do not clear state.
Monitor cancellation/join is bounded to 45 seconds, reconciliation gets 45 seconds
plus five to force kill, and final target-local zapret
stop gets 30 seconds then forced kill after five more, and each Shadowsocks join
gets five seconds. Exceeding a join bound fails closed. Compose grants 240
seconds. Reconcile after joining the monitor and before even stopping zapret:
stop also sources config, so unknown state is never sourced. The sequential
cleanup budget is 146 seconds; 240 seconds also covers an in-flight startup
command and scheduling grace. Watchdogs run in fresh shell interpreters and
cancel/join their own sleepers, including during PID 1's EXIT trap.
The monitor has a verified private session. After a crash, remaining check/NFQ
workers in that session are terminated and verified gone before reconciliation
or reactivation; Shadowsocks, watchdogs and unrelated exec peers are excluded.
## Isolation and runtime contract
Only SOCKS TCP/UDP is published on host
`127.0.0.1:${ZAPRET_AUTOTUNE_HOST_SOCKS_PORT:-11080}` for optional same-host
Xray use; no Xray edits are supplied. SS port `8388` is internal and never
published. The server binds `0.0.0.0:8388` only inside the dedicated bridge
namespace; ss-local binds `0.0.0.0:1080`. Startup validates a hex-only password
of at least 32 characters, writes JSON configs to target-owned tmpfs
`/run/zapret-autotune-ss` (directory `0700`, files `0600`), then unsets the
password before starting child processes.
The dedicated bridge is not shared with existing services. No host networking,
Docker socket, existing mounts, host Lua/fakes or privileged mode; all
capabilities are dropped except `NET_ADMIN`, `NET_RAW`, `SETUID`, `SETGID`, `KILL`,
`SYS_PTRACE`.
`NET_ADMIN` covers nft/NFQUEUE; `NET_RAW` covers nfqws2 raw sends. Zapret2's
`nfqws2 --user=tpws` needs `SETUID`/`SETGID` for UID/GID switching.
`KILL` allows bounded cleanup of workers that have dropped their UID.
`SYS_PTRACE` permits inspecting socket FD symlinks of privilege-dropped,
nondumpable nfqws workers to prove NFQUEUE ownership by inode. The native owned
worker path fails fast without effective `KILL`, `NET_ADMIN` and `SYS_PTRACE`;
legacy modes do not gain this requirement. Permission denial is an inspection
failure, not evidence that another process owns the queue.
This capability is confined to the target's **private PID namespace**: never
use host or shared/container PID namespaces. It does not grant inspection of
broader host processes. The full service/adaptive hardware gate remains
**pending**, not PASS; this ownership fix alone does not qualify it.
Startup order: ss-server/ss-local → listener 1080 → guarded managed init on
fresh state → `managed reconcile` on every startup → target-local `zapret2 start`
→ supervised managed monitor. Reconciliation uses the fixed private target DB
and restores an interrupted prepared transaction's qualified last-good bytes
before any daemon reload/start. Drift, unknown state or failed rollback exits
78 (no automatic retry); nothing is adopted. Restore a consistent whole-state
backup if needed; do not wipe/re-init persisted state.
Startup init and reconcile each have a 45-second deadline; initial daemon start
has 30 seconds, with forced kill after five more seconds for each command.
Any startup command failure/timeout exits 78, never the retry code 75. These
bounds do not shorten the monitor's 90-minute bootstrap/recovery scan budget.
SIGTERM/SIGINT cancels and waits for the monitor before stopping target-local
zapret and Shadowsocks children. Nonretry monitor fatal exits stop the
entrypoint; technical exits and Shadowsocks failures restart from private configs
only after safe cleanup. Legacy monitor health is YouTube Player API status
through the owned SOCKS, not video playback/SABR; legacy GGC candidates are scoped
to the exact discovered hostname. Opt-in service health also checks actual fresh
images and media; the experimental suffix scope above is not family qualification.
## Recorded clean-run acceptance (2026-10-11): PASS
Tested source RC: `d8316e695bc36e1243c111e19bb1e527355907b6`; local tag
`zapret-autotune-managed-target:rc-d8316e6`; immutable image
`sha256:5636aa9031fd0f6a0513f2e22cc0c39b1c806e9714b7f9ac040c8713d6a26c6c`.
The image was built from that source git archive with pinned inputs, not from
the later documentation commit. Private source root:
`/home/agent/zapret2-autotune-rc-d8316e6`.
The additional clean Compose project `zapret-autotune-clean-d8316e6-45bb14ba`
uses root `/home/agent/zapret2-managed-clean-d8316e6-45bb14ba` and host SOCKS
TCP/UDP `127.0.0.1:11081`. It is left running for inspection; the existing
managed target on `11080` and older SS target remain untouched.
- Before launch, state was empty, root-owned and mode `0700`; the independent
secret was 64 hex characters and `.env` mode `0600`. Real `blockcheckw` was
used, without mocks or copied config/candidates. Fresh init was observed;
fast bootstrap meant no exact generation-0 status snapshot was captured.
- Bootstrap committed generation2, frontend ID1 (verified layout-safe fake +
multidisorder seed) and exact-host GGC ID2 for
`rr17---sn-n8v7kn7d.googlevideo.com`. Both have valid raw TLS1.2/TLS1.3
evidence, 3/3 each. Periodic `lastProbeAt` advanced, counter0, recovery idle.
- Monitor SIGKILL changed host PID522765→525271 in the same container/PID1,
DockerRestartCount0, preserving IDs[1,2] and config hash
`57e639c98420be73a8ec4754b20ba975d81aced44215ed547b1ac029c89c14de`.
Graceful stop exited0 in 0.61s with no worker leaks. Forced recreation
(`5af686e2ad94…`, host PID525625) retained generation2, IDs and hash.
- Post-recreation counters1 then2 were observed; their cause is unproven.
A fresh-discovery GGC mismatch sample is not proof of the actual error cause.
The next default probe at 07:54 reset the counter to0, no active recovery.
Player API was OK and host SOCKS curl to `www.youtube.com` returned HTTP200.
- Both older targets' images, IDs, PIDs, start times and ports were unchanged
(managed PID518298, old SS PID266922); no host-manager or Xray changes.
Private PASS report: `final-cleanrun-report.json` under the clean root above;
raw artifacts: `final-evidence/`. This is isolated clean-run acceptance, not
blanket production approval or media playback/SABR verification. No real
90-minute watchdog wait was performed; shortened/injected tests cover expiry.
Saved-first recovery, rollback and other fault-injection results are recorded
separately in [PROGRESS.md](../../PROGRESS.md), not implied by this clean run.
For any new rollout, validate its own image binding, routing and lifecycle gates
on the new target only; never widen capabilities or reuse existing config/network
as a workaround.