Compare commits
18
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
2242643000 | ||
|
|
5c85f3f0b9 | ||
|
|
4efc066625 | ||
|
|
cfce8170d7 | ||
|
|
4d7c4b33c3 | ||
|
|
0623fa3323 | ||
|
|
544dda7c85 | ||
|
|
721cad42ce | ||
|
|
367d956ae3 | ||
|
|
ea886e5bc8 | ||
|
|
3fc8563995 | ||
|
|
9984f21466 | ||
|
|
fd25cb8939 | ||
|
|
1afee7d3bd | ||
|
|
e9a28c8cd3 | ||
|
|
2804d97063 | ||
|
|
50a78608ce | ||
|
|
a60f62de22 |
No files matched your search
@@ -17,6 +17,9 @@ RUN if [ ! -e /opt/zapret2/lua ] && [ ! -L /opt/zapret2/lua ]; then \
|
|||||||
fi \
|
fi \
|
||||||
&& test -d /opt/zapret2/lua \
|
&& test -d /opt/zapret2/lua \
|
||||||
&& command -v setsid \
|
&& command -v setsid \
|
||||||
|
&& command -v stat \
|
||||||
|
&& test "$(/usr/local/bin/zapret-autotune dns-forwarder mode)" = doh \
|
||||||
|
&& test "$(/usr/local/bin/zapret-autotune dns-forwarder address)" = 127.0.0.1:5353 \
|
||||||
&& (ss-server -h 2>&1 | grep -q -- '-d <addr>') \
|
&& (ss-server -h 2>&1 | grep -q -- '-d <addr>') \
|
||||||
&& mkdir -p /etc/zapret-autotune /var/lib/zapret-autotune/target \
|
&& mkdir -p /etc/zapret-autotune /var/lib/zapret-autotune/target \
|
||||||
&& printf 'zapret-autotune-managed-v1\n' > /etc/zapret-autotune/target-id \
|
&& printf 'zapret-autotune-managed-v1\n' > /etc/zapret-autotune/target-id \
|
||||||
|
|||||||
@@ -0,0 +1,34 @@
|
|||||||
|
# Bounded representative Googlevideo discovery
|
||||||
|
|
||||||
|
The YouTube default inventory is the configured primary plus `aqz-KE-bpKQ` and
|
||||||
|
`jNQXAC9IVRw`, deduplicated. Explicit configured inventories override it. The
|
||||||
|
inventory (one to five IDs) is queried freshly
|
||||||
|
in round-robin order for up to three rounds, bounded by three minutes. No new
|
||||||
|
manual CDN addresses, host rewriting or IP pinning is added.
|
||||||
|
With only one configured video, diversity depends on that video's real Player
|
||||||
|
backend rotation; if it does not rotate, qualification remains unresolved.
|
||||||
|
|
||||||
|
Discovery deduplicates actual ordinary signed MP4 media hosts, refetches expired
|
||||||
|
inventory and selects three hosts passing the existing root/prefix diversity
|
||||||
|
gate. URLs need twenty minutes remaining for the three sequential owned native
|
||||||
|
warmups and verified TLS transfers. URLs stay transient; unresolved diagnostics
|
||||||
|
contain only attempt/host and rejection counts. Discovery is not media proof.
|
||||||
|
|
||||||
|
Production adaptive recovery uses this collector. Existing experimental suffixes
|
||||||
|
retain their one-host operational recheck; promotion requires fresh three-host
|
||||||
|
inventory and the unchanged version-1 owned native proof factory. Insufficient
|
||||||
|
inventory leaves the committed experimental profile intact and retries after
|
||||||
|
the configured cooldown, not the exhausted-strategy two-hour pause. Native GGC
|
||||||
|
scope requires qualified frontend and any supplied asset candidates remain
|
||||||
|
qualified and nonoverlapping; missing avatar inventory is not its prerequisite.
|
||||||
|
Apply checks frontend and all three actual media resources in the newly deployed
|
||||||
|
daemon, including bounded learning and three verified transfers per TLS version.
|
||||||
|
Whole-service monitoring still reports avatar degradation independently.
|
||||||
|
|
||||||
|
Single-CDN version-2 qualification has been withdrawn. Maximum strategy catalogs,
|
||||||
|
new video discovery sources and controller stay-alive changes are separate work.
|
||||||
|
The three-CDN requirement was reaffirmed on 2026-10-13. The common suffix remains
|
||||||
|
authorized extrapolative coverage; only exact measured endpoints have transfer
|
||||||
|
evidence. This candidate is offline unit/fixture-verified only: owned factory,
|
||||||
|
autonomous discovery/promotion and deployed learning still need a scoped actual
|
||||||
|
three-CDN field test. No remote network checks or deployment accompany this update.
|
||||||
+257
-26
@@ -1,4 +1,154 @@
|
|||||||
# Fresh managed-test target contract (not deployed)
|
# Fresh managed-test target contract (isolated acceptance, not production approval)
|
||||||
|
|
||||||
|
## Declarative HTTPS resource checks v1 (check-only)
|
||||||
|
|
||||||
|
Inside the provisioned owned managed target, with the tracked example available:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
zapret-autotune check --config resource-check.example.json
|
||||||
|
```
|
||||||
|
|
||||||
|
This is not a standalone host checker for arbitrary external proxies: owned-target,
|
||||||
|
SS and configured wire-DNS readiness/ownership guards must pass before probes.
|
||||||
|
Version1 supports `kind: http`, GET/HEAD on HTTPS **DNS-hostname** URLs,
|
||||||
|
TLS1.2/1.3/both, timeout and expected HTTP statuses, MIME content type and
|
||||||
|
minimum/maximum bytes. Outcomes identify stable configured service/resource IDs
|
||||||
|
and report HTTP status, bytes and latency. URL/header secrets come from env via
|
||||||
|
private stdin, not argv; keep the config and env private when sensitive.
|
||||||
|
|
||||||
|
All resolved IPs are policy-checked before numeric CONNECT pinning; TLS verifies
|
||||||
|
the original hostname/SNI. Redirects and system-DNS fallback are forbidden.
|
||||||
|
Private destinations require explicit opt-in **and** CIDR restriction. IPv6
|
||||||
|
literal URLs and address failover are not supported in v1. Image/media decoding,
|
||||||
|
Range and suffix-scope qualification are not generic HTTP-check features.
|
||||||
|
|
||||||
|
Schema/loader and executable checks have unit and isolated remote acceptance for
|
||||||
|
source `4d7c4b33c36186d43ea89fd24354b1834b21692c`, image
|
||||||
|
`sha256:be5a1c45c6d40b390a29bfc9a5ab3975c3a8e0c015b98a524696312807dad9a4`.
|
||||||
|
The orchestrator supplied verified private evidence, not reread by this docs-only
|
||||||
|
update; see [PROGRESS.md](PROGRESS.md) for gates and cleanup. The positive private
|
||||||
|
TLS fixture trusted a **staging-only test CA**, never bypassed TLS or changed
|
||||||
|
production CAs. The existing DNS backend and earlier DoH outage evidence are reused,
|
||||||
|
not a new DoH implementation or new outage test here.
|
||||||
|
|
||||||
|
**Acceptance scope:** staging YouTube monitoring was SIGSTOP-isolated from checks,
|
||||||
|
then SIGCONT-resumed before graceful stop (exit0, owned cleanup, foreign baseline
|
||||||
|
unchanged). A prior managed monitor exit78 remains a runtime caveat, not proof of
|
||||||
|
whole-controller resilience. LIVE11082 was already stopped, not upgraded/restarted;
|
||||||
|
old SS1080/Xray were untouched. Historical LIVE observations below are not current
|
||||||
|
production availability. No network operations or deployment accompany this update.
|
||||||
|
|
||||||
|
Check-only neither selects/applies strategies nor writes strategy DB entries or
|
||||||
|
declares full-service READY. Periodic checks/state/status/reload/independent retry
|
||||||
|
and generic PID1 BLOCKED foundations now have narrow remote fault acceptance;
|
||||||
|
unlike the earlier check-only run, this monitor ran without SIGSTOP.
|
||||||
|
Use `managed monitor --resource-config FILE`, or PID1 environment
|
||||||
|
`ZAPRET_AUTOTUNE_RESOURCE_CONFIG`; `managed resource-status` reads that same env
|
||||||
|
source. See [RESOURCE_MONITOR.md](RESOURCE_MONITOR.md) for ownership and repair.
|
||||||
|
HEALTHY is CHECKS_ONLY, not automatic strategy assignment. Generic strategy DB,
|
||||||
|
domain qualification matrix, runtime coverage/profile-group rendering and apply,
|
||||||
|
plugins/autotuning, maximum strategy collection and Prometheus remain TODO.
|
||||||
|
Three-CDN discovery/qualification is a YouTube
|
||||||
|
adapter contract, not a universal arbitrary-service requirement.
|
||||||
|
|
||||||
|
## Managed DoH live acceptance (2026-10-13): DNS verified, NOT READY
|
||||||
|
|
||||||
|
Supplied verified reports bind source `544dda7c85965fc3200de3af17fca9ef9999b2ce`
|
||||||
|
(DoH `367d956`, TCP deadline `721cad4`, public-health `socks5h` fix `544dda7`)
|
||||||
|
to image `sha256:b3085b7ceca4a629c28bd58f23bf97f02a34c2a38df08b57756e40594ba2d427`,
|
||||||
|
container `4ad54175a54c`, owned SOCKS `127.0.0.1:11082`.
|
||||||
|
The 303-second LIVE snapshot retained root/DNS22/SS59,60/monitor253 PIDs,
|
||||||
|
children PPID1, owned readiness ACK and parent FD8/9 anchors. UDP/TCP DNS
|
||||||
|
IDs/questions matched, rcode0, answers2/8; SOCKS `0500`, WWW HTTP200/SSL0
|
||||||
|
and configured `health youtube` exit0 passed. Wire evidence: 41 marked direct
|
||||||
|
DoH connections, zero external port53. Scope `suffix`, generation9, IDs `[1,9]`
|
||||||
|
and hash preservation versus staging are measured snapshots, not universal
|
||||||
|
state; autonomous updates have occurred since the earlier generation8 snapshot.
|
||||||
|
|
||||||
|
In isolated staging, blocking both bootstrap IPs for 61 seconds returned
|
||||||
|
UDP/TCP SERVFAIL/rcode2 at about 5000ms without changing root, DNS, SS or monitor
|
||||||
|
PIDs or sending external port53 traffic. DNS recovered immediately after removal;
|
||||||
|
WWW HTTP200 passed after 46 seconds without restart. Graceful stop: 1 second,
|
||||||
|
exit0, no survivors. Staging nft fault rules were removed; this is a tested DNS
|
||||||
|
dependency failure, not all-outage or nonconfiguration-guard acceptance.
|
||||||
|
No new runtime firewall policy beyond normal generated native rules is supplied.
|
||||||
|
|
||||||
|
Existing shared GGC qualification is separate evidence, not an 18-transfer rerun
|
||||||
|
under DoH. Fresh TLS1.3 media: HTTP206/65536 bytes/MP4 `ftyp`/SSL0; TLS1.2
|
||||||
|
TIMEOUT is not qualified. Avatar/thumbnail qualification remains independently
|
||||||
|
outstanding; no whole-service READY claim. Historical media results below do
|
||||||
|
not close these gates on this image. Generic service integration and maximum
|
||||||
|
strategy collection remain future work, without a new policy framework.
|
||||||
|
|
||||||
|
Private supplied reports (not reread here):
|
||||||
|
`/home/agent/zapret2-autotune-rc-doh-721cad4/live-upgrade-private/`, then latest
|
||||||
|
`/home/agent/zapret2-autotune-rc-doh-544dda7/live-upgrade-private/`.
|
||||||
|
No deployment/probes accompany this docs update; later docs are not tested source.
|
||||||
|
State/credentials/private env were preserved, staging containers removed, old
|
||||||
|
user SS1080 and user-stopped proxy unchanged. See the deployment guide for the
|
||||||
|
four DNS settings, LIVE env location, privacy boundary and recreation lifecycle.
|
||||||
|
|
||||||
|
## Autonomous three-CDN acceptance (2026-10-13): media qualified, NOT READY
|
||||||
|
|
||||||
|
Supplied field results bind source `3fc8563995e890310580ba8f78850b92fea42136`
|
||||||
|
to managed image `sha256:a902e4efd949a7ffce4414fd99ab4a815a643b40cf9b9e30e2bd2db7b452817a`,
|
||||||
|
container `4c6b34…`, Compose project `rc0398e6f…`, SOCKS `127.0.0.1:11082`.
|
||||||
|
Native post-cooldown recovery at 08:07–08:09 automatically discovered three actual
|
||||||
|
ordinary signed-media hosts without manual domains and promoted Adaptive candidate10
|
||||||
|
through fresh version-1 `owned-nfq-media-range-v1` proof and deployed checks.
|
||||||
|
Generation9→10, IDs `[1,3,9]`→`[1,9,10]`, scope
|
||||||
|
`experimental_googlevideo_suffix`→`suffix`; frontend1, thumbnail9 and common
|
||||||
|
`googlevideo.com` hostlist were preserved (config hash prefix `dc997ac91…`).
|
||||||
|
|
||||||
|
Measured hosts were `rr16---sn-n8v7znsz.googlevideo.com`,
|
||||||
|
`rr18---sn-n8v7zns6.googlevideo.com`, and `rr5---sn-n8v7kn76.googlevideo.com`:
|
||||||
|
each passed post-warmup 3/3 on TLS1.2 and TLS1.3, all18 HTTP206/65536-byte
|
||||||
|
MP4 `ftyp` transfers with Range `0..65535` and valid SSL verification.
|
||||||
|
Separate warm attempts/failures were 2/0, 3/1, 3/1 respectively, not cold evidence.
|
||||||
|
Fresh owned proof checked at `2026-10-13T08:08:28.752Z` retains actual endpoints,
|
||||||
|
source IDs and native-owner Lua binding; three roots/prefixes were measured.
|
||||||
|
Native request statistics were not persisted; manual request count is unknown.
|
||||||
|
|
||||||
|
At 08:21:48 the reported container and monitor were alive, but whole-service
|
||||||
|
health remained false / DEGRADED. Probe failures405 at 08:20:37 do not independently
|
||||||
|
establish an avatar-specific cause; yesterday's avatar timeouts are separate.
|
||||||
|
The orchestrator's fresh owned-SOCKS greeting was `0500`; native `socks5h` uses
|
||||||
|
SS DNS `8.8.8.8`. The passive SSH watch vanished after about ten minutes;
|
||||||
|
no normally completed 18-minute watch is claimed. No cooldown was reset.
|
||||||
|
Private supplied proof: `/home/agent/zapret2-autotune-rc-3fc8563/field-private/proof/final.safe.json`
|
||||||
|
(not independently reread by this docs update). No new deployment/probes here.
|
||||||
|
See [PROGRESS.md](PROGRESS.md) for the measured snapshot and remaining gates.
|
||||||
|
|
||||||
|
The user's three-CDN requirement is retained; the proposed single-CDN relaxation
|
||||||
|
was withdrawn. Service-mode discovery needs no manual domains: missing/empty
|
||||||
|
`ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS` uses primary `ZAPRET_AUTOTUNE_VIDEO_ID`
|
||||||
|
(default `dQw4w9WgXcQ`) plus `aqz-KE-bpKQ` and `jNQXAC9IVRw`, deduplicated.
|
||||||
|
A nonempty JSON array of 1..5 unique VideoIDs overrides this inventory. VideoIDs
|
||||||
|
are discovery inputs, not proof of distinct CDNs.
|
||||||
|
|
||||||
|
Fresh round-robin Player requests are bounded to three rounds (at most 15 requests)
|
||||||
|
and three minutes. Qualification selects three actual signed MP4 media hosts,
|
||||||
|
with at least two roots and two prefixes, and twenty-minute remaining URL TTL.
|
||||||
|
Only the existing version-1 owned native proof factory authorizes qualification:
|
||||||
|
post-warmup actual media 3/3 on both TLS protocols for each measured endpoint.
|
||||||
|
The common `googlevideo.com` suffix is an authorized extrapolative scope, not
|
||||||
|
measured evidence for every Googlevideo host or cold/uniform behavior.
|
||||||
|
|
||||||
|
Experimental promotion uses the same proof and fresh deployed three-host media
|
||||||
|
checks before committing apply. Insufficient inventory preserves the current
|
||||||
|
experiment and uses configured retry cooldown; genuine exhausted strategy search
|
||||||
|
still waits at least two hours. Qualified frontend is required; avatar success
|
||||||
|
does not block this media stage, but whole-service health remains DEGRADED until
|
||||||
|
actual avatar checks pass. Legacy service-off/exact-host behavior is unchanged.
|
||||||
|
|
||||||
|
The representative three-CDN Adaptive promotion is field-qualified above;
|
||||||
|
whole-service/avatar acceptance remains open. The 2026-10-12 measured results
|
||||||
|
below are historical, not the new factory evidence. New images still require
|
||||||
|
their own source/image binding and guarded apply/rollback acceptance. Maximum
|
||||||
|
strategy collection/search diversity and PID1 global-failure stay-alive remain unfinished future
|
||||||
|
increments. Declarative check-only configuration is accepted above; generic
|
||||||
|
controller integration remains future work. No new generic policy framework is
|
||||||
|
supplied. See [GOOGLEVIDEO_DISCOVERY.md](GOOGLEVIDEO_DISCOVERY.md).
|
||||||
|
|
||||||
`managed init|reconcile|status|active-ids|apply` is exclusively for a **new dedicated container**
|
`managed init|reconcile|status|active-ids|apply` is exclusively for a **new dedicated container**
|
||||||
which owns its entire config. The existing/dev/production container must refuse
|
which owns its entire config. The existing/dev/production container must refuse
|
||||||
@@ -12,7 +162,7 @@ must supply all of the following before any managed command can run:
|
|||||||
| `ZAPRET_AUTOTUNE_TARGET_DIR` | `/var/lib/zapret-autotune/target` |
|
| `ZAPRET_AUTOTUNE_TARGET_DIR` | `/var/lib/zapret-autotune/target` |
|
||||||
| `ZAPRET_CONFIG` | `/var/lib/zapret-autotune/target/config` |
|
| `ZAPRET_CONFIG` | `/var/lib/zapret-autotune/target/config` |
|
||||||
| `SOCKS_PORT` | Base image's actual ss-local service port (canonical decimal, 1–65535) |
|
| `SOCKS_PORT` | Base image's actual ss-local service port (canonical decimal, 1–65535) |
|
||||||
| `ZAPRET_AUTOTUNE_SOCKS_PROXY` | `socks5://127.0.0.1:$SOCKS_PORT` (expanded value) |
|
| `ZAPRET_AUTOTUNE_SOCKS_PROXY` | `socks5h://127.0.0.1:$SOCKS_PORT` (expanded value; legacy `socks5` validated and canonicalized internally) |
|
||||||
|
|
||||||
The fixed directory must already be a private runtime-user-owned **dedicated
|
The fixed directory must already be a private runtime-user-owned **dedicated
|
||||||
persistent volume mountpoint**, not just a directory under another mount.
|
persistent volume mountpoint**, not just a directory under another mount.
|
||||||
@@ -27,8 +177,9 @@ deliberately forging a provisioned runtime.
|
|||||||
|
|
||||||
The managed `config` is a standalone **full config only in this fresh volume**.
|
The managed `config` is a standalone **full config only in this fresh volume**.
|
||||||
Never replace, adopt, or symlink over `/opt/zapret2/config` or any existing
|
Never replace, adopt, or symlink over `/opt/zapret2/config` or any existing
|
||||||
service config. The new deployment entrypoint must ensure the base daemon
|
service config. The deployment entrypoint must ensure the base daemon actually
|
||||||
actually honors the exact `ZAPRET_CONFIG` path; compatibility remains unproven.
|
honors the exact `ZAPRET_CONFIG` path; isolated RC acceptance does not waive
|
||||||
|
that check for a new image.
|
||||||
Existing services would require a separately designed managed **include
|
Existing services would require a separately designed managed **include
|
||||||
fragment**, not this full-config adapter. No such integration is implemented.
|
fragment**, not this full-config adapter. No such integration is implemented.
|
||||||
|
|
||||||
@@ -104,15 +255,15 @@ Directories must not be writable by untrusted processes; hostile concurrent
|
|||||||
filesystem mutation is outside the transaction boundary. Shell config safety
|
filesystem mutation is outside the transaction boundary. Shell config safety
|
||||||
and exact GGC filtering are unchanged.
|
and exact GGC filtering are unchanged.
|
||||||
|
|
||||||
The isolated [Compose definition](deploy/managed-target/README.md) now supplies
|
The isolated [Compose definition](deploy/managed-target/README.md) supplies
|
||||||
the marker, mount and ordered startup; it has not been built or deployed.
|
the marker, mount and ordered startup; isolated deployments are recorded below.
|
||||||
No Docker socket, remote/container operations, old target paths, Xray edits,
|
No Docker socket, old target paths, Xray edits or host firewall provisioning
|
||||||
or host firewall provisioning are included. Before deployment,
|
are included. Before each new deployment,
|
||||||
provisioning must verify the pinned image honors `ZAPRET_CONFIG`, supports all
|
provisioning must verify the pinned image honors `ZAPRET_CONFIG`, supports all
|
||||||
listed variables, and starts ss-local while the baseline leaves nfqws2 disabled.
|
listed variables, and starts ss-local while the baseline leaves nfqws2 disabled.
|
||||||
Start the fresh target with that baseline, then verify active apply installs
|
Start the fresh target with that baseline, then verify active apply installs
|
||||||
target-local nft NFQUEUE rules and starts nfqws2, and rollback restores both.
|
target-local nft NFQUEUE rules and starts nfqws2, and rollback restores both.
|
||||||
These image/startup/routing gates remain unverified: if any is unsupported or
|
If any image/startup/routing requirement is unsupported or
|
||||||
uncertain, deployment must fail closed, not substitute `restart-daemons` or
|
uncertain, deployment must fail closed, not substitute `restart-daemons` or
|
||||||
alter an old config. Persistent volume durability is a provisioning
|
alter an old config. Persistent volume durability is a provisioning
|
||||||
requirement; mountinfo alone cannot establish the storage lifecycle.
|
requirement; mountinfo alone cannot establish the storage lifecycle.
|
||||||
@@ -131,9 +282,9 @@ Cancelled/failed downtime restores the committed configuration and runs fixed
|
|||||||
target `start` with an independent 30-second cleanup deadline. Failed cleanup or
|
target `start` with an independent 30-second cleanup deadline. Failed cleanup or
|
||||||
rollback is fatal to the supervisor; unresolved journal state is never started.
|
rollback is fatal to the supervisor; unresolved journal state is never started.
|
||||||
The callback caller must hold the monitor singleton; the wiring repeats runtime
|
The callback caller must hold the monitor singleton; the wiring repeats runtime
|
||||||
identity/SOCKS guards. Fake-only checks are added, but tests/build are deferred
|
identity/SOCKS guards. `managed monitor` binds those callbacks and only the
|
||||||
to the orchestrator. `managed monitor` now binds those callbacks and only the
|
dedicated target entrypoint starts it; local and isolated runtime validation
|
||||||
dedicated target entrypoint starts it; no live/runtime validation has occurred.
|
results, with their limits, are recorded in [PROGRESS.md](PROGRESS.md).
|
||||||
|
|
||||||
## Dedicated monitor integration (isolated clean-run acceptance passed)
|
## Dedicated monitor integration (isolated clean-run acceptance passed)
|
||||||
|
|
||||||
@@ -160,9 +311,9 @@ DB/safety/journal/runner failures stop nonzero; joined cancellation plus a clean
|
|||||||
failure is also fatal. A qualified frontend at startup is probed immediately,
|
failure is also fatal. A qualified frontend at startup is probed immediately,
|
||||||
not rescanned merely because the process restarted.
|
not rescanned merely because the process restarted.
|
||||||
|
|
||||||
Entrypoint ordering is SS children → listener → fresh managed init if needed → managed reconcile → zapret start
|
Entrypoint ordering is owned DNS readiness (default DoH) → SS children → listener → fresh managed init if needed → managed reconcile → zapret start
|
||||||
→ one installed `/usr/local/bin/zapret-autotune managed monitor` child. It supervises
|
→ one installed `/usr/local/bin/zapret-autotune managed monitor` child. It supervises
|
||||||
all three children; unexpected monitor exit is not a successful shutdown.
|
the DNS, SS and monitor children; unexpected monitor exit is not a successful shutdown.
|
||||||
SIGTERM/SIGINT cancels and waits the monitor before target-local zapret stop and
|
SIGTERM/SIGINT cancels and waits the monitor before target-local zapret stop and
|
||||||
SS child termination/wait. Existing targets and Xray remain unchanged. Deployment
|
SS child termination/wait. Existing targets and Xray remain unchanged. Deployment
|
||||||
allows 240s Compose shutdown grace for bounded
|
allows 240s Compose shutdown grace for bounded
|
||||||
@@ -211,6 +362,8 @@ Example (replace registry and release; never reuse a release tag):
|
|||||||
```sh
|
```sh
|
||||||
IMAGE=registry.example/team/zapret-autotune-managed-target:2026-10-10-r1
|
IMAGE=registry.example/team/zapret-autotune-managed-target:2026-10-10-r1
|
||||||
docker build --pull -f Dockerfile.target -t "$IMAGE" .
|
docker build --pull -f Dockerfile.target -t "$IMAGE" .
|
||||||
|
test "$(docker image inspect --format '{{json .Config.Entrypoint}}' "$IMAGE")" = '["/bin/sh","/opt/managed-target-entrypoint.sh"]'
|
||||||
|
docker run --rm --network none --entrypoint /bin/sh "$IMAGE" -ec 'test "$(cat /etc/zapret-autotune/target-id)" = zapret-autotune-managed-v1; test -f /opt/managed-target-entrypoint.sh'
|
||||||
docker push "$IMAGE"
|
docker push "$IMAGE"
|
||||||
docker image inspect --format '{{index .RepoDigests 0}}' "$IMAGE"
|
docker image inspect --format '{{index .RepoDigests 0}}' "$IMAGE"
|
||||||
```
|
```
|
||||||
@@ -230,6 +383,11 @@ docker compose --env-file deploy/managed-target/.env -f deploy/managed-target/co
|
|||||||
docker compose --env-file deploy/managed-target/.env -f deploy/managed-target/compose.yaml pull target
|
docker compose --env-file deploy/managed-target/.env -f deploy/managed-target/compose.yaml pull target
|
||||||
```
|
```
|
||||||
|
|
||||||
|
Do not omit `-f Dockerfile.target`: root `Dockerfile` builds the proposal/prototype,
|
||||||
|
not the PID-1-managed target. Assert both entrypoint and provisioned marker on the
|
||||||
|
exact selected image before promotion; the assertions above are image-contract
|
||||||
|
checks, not live resource qualification. A managed-looking tag is insufficient.
|
||||||
|
|
||||||
Start the selected container image with Compose `up --no-build`;
|
Start the selected container image with Compose `up --no-build`;
|
||||||
no host service installation is required. Keep the selected digest and
|
no host service installation is required. Keep the selected digest and
|
||||||
private configuration together for upgrades and rollback. Never run a build
|
private configuration together for upgrades and rollback. Never run a build
|
||||||
@@ -251,19 +409,31 @@ signed URLs; use `docker compose ... config --quiet` for validation.
|
|||||||
| --- | --- |
|
| --- | --- |
|
||||||
| `ZAPRET_AUTOTUNE_IMAGE` | Local build/test default `zapret-autotune-managed-target:local`; production promotion requires a managed-target `repository@sha256:<digest>` |
|
| `ZAPRET_AUTOTUNE_IMAGE` | Local build/test default `zapret-autotune-managed-target:local`; production promotion requires a managed-target `repository@sha256:<digest>` |
|
||||||
| `SS_PASSWORD` | Required independent hex-only secret, at least 32 characters; deployment README generates 64 |
|
| `SS_PASSWORD` | Required independent hex-only secret, at least 32 characters; deployment README generates 64 |
|
||||||
| `SS_DNS_SERVER` | Empty (unchanged resolver); optional literal IPv4/IPv6 for target-only native ss-server DNS, validated before launch |
|
| `ZAPRET_AUTOTUNE_DNS_MODE` | `doh` default; explicit `native` opts into the old resolver, never automatic fallback |
|
||||||
|
| `ZAPRET_AUTOTUNE_DOH_URL` | `https://dns.google/dns-query`; HTTPS/443 path, no userinfo/query/fragment; path may be secret |
|
||||||
|
| `ZAPRET_AUTOTUNE_DOH_BOOTSTRAP_IPS` | Compose default empty: parser selects `8.8.8.8,8.8.4.4` only for exact default Google URL; custom provider requires explicit 1–4 comma-separated literal IPs without spaces |
|
||||||
|
| `ZAPRET_AUTOTUNE_DOH_TIMEOUT_SEC` | `5`; integer 1–30 |
|
||||||
|
| `SS_DNS_SERVER` | Explicit native mode only: empty retains base resolver or literal IPv4/IPv6; ignored in DoH mode |
|
||||||
| `ZAPRET_AUTOTUNE_YOUTUBE_SERVICE` | `0` (legacy) or opt-in `1`; explicitly forwarded by Compose |
|
| `ZAPRET_AUTOTUNE_YOUTUBE_SERVICE` | `0` (legacy) or opt-in `1`; explicitly forwarded by Compose |
|
||||||
| `ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE` | `cold` default or `adaptive` (requires service `1`); explicitly forwarded |
|
| `ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE` | `cold` default or `adaptive` (requires service `1`); explicitly forwarded |
|
||||||
| `ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS` | JSON array of 1..5 unique VideoIDs; defaults to only the primary ID; explicitly forwarded |
|
| `ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS` | JSON array of 1..5 unique VideoIDs; defaults to only the primary ID; explicitly forwarded |
|
||||||
| `ZAPRET_AUTOTUNE_HOST_SOCKS_PORT` | `11080`; choose an available TCP/UDP port 1–65535, published only on `127.0.0.1` |
|
| `ZAPRET_AUTOTUNE_HOST_SOCKS_PORT` | `11080`; choose an available TCP/UDP port 1–65535, published only on `127.0.0.1` |
|
||||||
| `SOCKS_PORT` | Supplied deployment fixes internal listener to `1080`; not a host-port override |
|
| `SOCKS_PORT` | Supplied deployment fixes internal listener to `1080`; not a host-port override |
|
||||||
| `ZAPRET_AUTOTUNE_SOCKS_PROXY` | Fixed `socks5://127.0.0.1:1080` inside this target, no credentials/direct fallback |
|
| `ZAPRET_AUTOTUNE_SOCKS_PROXY` | Fixed `socks5h://127.0.0.1:1080` inside this target, no credentials/direct fallback |
|
||||||
| `ZAPRET_AUTOTUNE_VIDEO_ID` | `dQw4w9WgXcQ`; exactly 11 ASCII letters, digits, `_` or `-` |
|
| `ZAPRET_AUTOTUNE_VIDEO_ID` | `dQw4w9WgXcQ`; exactly 11 ASCII letters, digits, `_` or `-` |
|
||||||
| `ZAPRET_AUTOTUNE_HEALTH_INTERVAL` | `5m`; duration in `(0,24h]` for monitor |
|
| `ZAPRET_AUTOTUNE_HEALTH_INTERVAL` | `5m`; duration in `(0,24h]` for monitor |
|
||||||
| `ZAPRET_AUTOTUNE_FAILURE_THRESHOLD` | `3`; integer 1–1000 for monitor |
|
| `ZAPRET_AUTOTUNE_FAILURE_THRESHOLD` | `3`; integer 1–1000 for monitor |
|
||||||
| `ZAPRET_AUTOTUNE_RECOVERY_COOLDOWN` | `30m`; duration in `(0,168h]` for monitor |
|
| `ZAPRET_AUTOTUNE_RECOVERY_COOLDOWN` | `30m`; duration in `(0,168h]` for monitor |
|
||||||
| `ZAPRET_AUTOTUNE_DB` | Fixed `/var/lib/zapret-autotune/target/candidates.db` |
|
| `ZAPRET_AUTOTUNE_DB` | Fixed `/var/lib/zapret-autotune/target/candidates.db` |
|
||||||
|
|
||||||
|
Managed Player, metadata, image, media and warmup requests send hostnames through
|
||||||
|
the owned SS listener (`socks5h`), so they respect target-native DNS instead of
|
||||||
|
bypassing it with curl's local resolution. Older private environment overrides
|
||||||
|
using `socks5` are canonicalized internally after validation; arbitrary proxies
|
||||||
|
remain rejected by the listener/port/process ownership guard. Legacy public CLI
|
||||||
|
discovery and root checker `--resolve` pinning are unchanged. This corrects a DNS
|
||||||
|
path bypass, not a proven explanation of earlier peer certificate failures.
|
||||||
|
|
||||||
The monitor's three policy flags override corresponding environment defaults;
|
The monitor's three policy flags override corresponding environment defaults;
|
||||||
explicit empty/invalid environment settings are rejected. The entrypoint accepts
|
explicit empty/invalid environment settings are rejected. The entrypoint accepts
|
||||||
no arguments. Compose forwards the primary/sample VideoIDs and service/mode settings.
|
no arguments. Compose forwards the primary/sample VideoIDs and service/mode settings.
|
||||||
@@ -281,10 +451,11 @@ signed ordinary media host samples are required; multiple VideoIDs may yield one
|
|||||||
CDN host and leave `DEGRADED` with no authorized suffix. Sampled thumbnail/avatar
|
CDN host and leave `DEGRADED` with no authorized suffix. Sampled thumbnail/avatar
|
||||||
routes do not prove availability of the whole thumbnail pool. The reported
|
routes do not prove availability of the whole thumbnail pool. The reported
|
||||||
`8.8.8.8` DNS diagnostic corrected a tampered avatar, not all thumbnail routes;
|
`8.8.8.8` DNS diagnostic corrected a tampered avatar, not all thumbnail routes;
|
||||||
native DNS correction is not full asset availability proof. Target DNS uses
|
native DNS correction is not full asset availability proof. Default DoH uses
|
||||||
validated native `-d` argv (nonsecret), with installed-option support checked
|
SS `-d 127.0.0.1:5353`; explicit native mode uses validated literal `-d` argv,
|
||||||
at image build; passwords remain private JSON only. No DoH bridge, permanent
|
with installed-option support checked at image build. Passwords remain private
|
||||||
Google IP pins, global host DNS edits or external startup manager are introduced.
|
JSON only. No permanent service IP pins, global host DNS edits or external
|
||||||
|
startup manager are introduced.
|
||||||
|
|
||||||
### Supervision and failure handling
|
### Supervision and failure handling
|
||||||
|
|
||||||
@@ -298,10 +469,30 @@ errors never authorize retry. No new dependency, host service installation or
|
|||||||
external process manager is required. SIGTERM/SIGINT cancels backoff or the running
|
external process manager is required. SIGTERM/SIGINT cancels backoff or the running
|
||||||
monitor, waits for bounded restoration and joins children before exit.
|
monitor, waits for bounded restoration and joins children before exit.
|
||||||
|
|
||||||
|
In default DoH mode PID 1 first starts the stdlib UDP/TCP loopback DNS forwarder
|
||||||
|
inside the existing Go binary (no dependency or new container),
|
||||||
|
waits at most five seconds for its owned private PID marker, and only then starts
|
||||||
|
ss-server with `-d 127.0.0.1:5353`. No host DNS port or DNS/iptables changes are
|
||||||
|
made. Hostname SOCKS traffic and managed `socks5h` checks use the same encrypted
|
||||||
|
HTTPS resolver; clients must enable remote SOCKS DNS, not resolve locally or
|
||||||
|
send IP literals. Bootstrap uses configured literal IPs with verified TLS and
|
||||||
|
mark `0x50000000` direct sockets excluding probe/live NFQUEUE, with no system DNS, proxy
|
||||||
|
or plaintext fallback. The DoH resolver sees queries directly, not SS-tunneled;
|
||||||
|
the plaintext helper exchange stays on loopback5353. Endpoint failures produce
|
||||||
|
SERVFAIL without child exit or plaintext fallback. A forwarder crash, unlike an
|
||||||
|
upstream outage, uses the same bounded technical retry budget. The DNS child
|
||||||
|
stays alive through monitor restoration, zapret stop and SS joins; it is then
|
||||||
|
terminated/joined within five seconds and only its validated readiness inode is
|
||||||
|
removed using PID1-held directory/file anchors. Foreign/stale/replaced readiness
|
||||||
|
is preserved and vetoes startup/retry; SIGKILL does not lose owned cleanup proof. Explicit
|
||||||
|
native mode has no DNS child. See the deployment README for provider settings
|
||||||
|
and scope limits. Supplied staged-outage and LIVE DNS acceptance are recorded
|
||||||
|
above; they do not establish all-outage resilience or whole-service readiness.
|
||||||
|
|
||||||
Compose restart remains disabled: a blanket container restart could loop over a
|
Compose restart remains disabled: a blanket container restart could loop over a
|
||||||
safety refusal or reset the in-container retry budget. Compose stop grace is 240s:
|
safety refusal or reset the in-container retry budget. Compose stop grace is 240s:
|
||||||
the sequential cleanup budget is 140s (45s monitor join, 45s reconciliation plus
|
the sequential cleanup budget is 145s (45s monitor join, 45s reconciliation plus
|
||||||
5s kill grace, 30s zapret stop plus 5s kill grace, and two 5s SS joins), with room
|
5s kill grace, 30s zapret stop plus 5s kill grace, two 5s SS joins and a 5s DNS join), with room
|
||||||
for a deferred 45s startup command plus kill grace and scheduling overhead.
|
for a deferred 45s startup command plus kill grace and scheduling overhead.
|
||||||
The join watchdog runs in a fresh shell and is cancelled when its child finishes;
|
The join watchdog runs in a fresh shell and is cancelled when its child finishes;
|
||||||
graceful shutdown does not wait out the watchdog's full deadline.
|
graceful shutdown does not wait out the watchdog's full deadline.
|
||||||
@@ -419,7 +610,47 @@ do not blindly retry it. Use the matched whole-state/image rollback above.
|
|||||||
Candidate-ID CLI apply cannot select/inherit the experimental policy; explicit
|
Candidate-ID CLI apply cannot select/inherit the experimental policy; explicit
|
||||||
native-circular authorization is required. Internal journal recovery is unchanged.
|
native-circular authorization is required. Internal journal recovery is unchanged.
|
||||||
|
|
||||||
Resource probes still run. This policy remains `DEGRADED`/GGC-unresolved even
|
Resource probes still run. In service mode (legacy or adaptive), healthy actual
|
||||||
when a bounded media check succeeds: neither playback READY nor family
|
image/media probes short-circuit recovery, but operational health does not mean
|
||||||
qualification is inferred. Automatic frontend/image repairs preserve the
|
family qualification or playback READY. Experimental media recovery preserves
|
||||||
operator suffix; automatic GGC recovery does not replace or qualify it.
|
the operator suffix and all committed profile IDs: a validated transfer failure
|
||||||
|
permits one guarded restart of that same deployed daemon, then at most 10 native
|
||||||
|
relearning attempts within 120s and a final fresh signed media 3/3 for each of
|
||||||
|
TLS1.2 and TLS1.3. Learning stays in the persistent daemon's RAM; no temporary
|
||||||
|
NFQ worker or replacement/qualified candidate is created. Expiring media is
|
||||||
|
rediscovered, never rewritten or persisted. Missing inventory remains degraded,
|
||||||
|
not strategy absence; mixed, guard, infrastructure and cancellation errors do
|
||||||
|
not authorize relearning. Unrelated missing image inventory may coexist with
|
||||||
|
usable media, but successful media recovery does not repair or qualify it.
|
||||||
|
|
||||||
|
Avatar inventory first parses fresh homepage `ytInitialData`. Only absent metadata
|
||||||
|
or no allowed avatar permits one fallback to the configured video's watch page,
|
||||||
|
using a validated video ID, the same trusted HTTPS WWW origin and SOCKS proxy,
|
||||||
|
bounded time/body size, no redirects, and discarded diagnostics. Malformed JSON,
|
||||||
|
transfer errors and safety/cancellation failures do not trigger fallback. Both
|
||||||
|
pages and discovered resource URLs remain ephemeral; discovering an allowed URL
|
||||||
|
is not image-transfer qualification or a guarantee of full-family coverage.
|
||||||
|
|
||||||
|
`managed status` reports active-profile `covered=` separately from qualified
|
||||||
|
`groups` and `unresolved_groups`; `managed active-ids` adds `covered_groups` and
|
||||||
|
`googlevideo_scope` to its JSON. An experimental Googlevideo profile can be
|
||||||
|
covered yet qualification-unresolved; one image profile does not establish full
|
||||||
|
asset-inventory success. Status is read-only and reports live health NOT CHECKED.
|
||||||
|
Historical `a60f62d` restart/recreation and status checks passed, but its upgrade
|
||||||
|
hit `BrokenPipe` and rolled back; that run did not accept media recovery. Later
|
||||||
|
supplied final field results for source `fd25cb8`, managed image
|
||||||
|
`sha256:73e955e7fdc6efaef88c62ebe4f7afeab71c29d29ba3c88b2a2b7547bb647da5`
|
||||||
|
on SOCKS11082 completed a 15-minute watch with restart count0 and a live monitor.
|
||||||
|
Validated rejection/no-qualified-strategy recovery continued with naturally
|
||||||
|
advanced cooldown and unchanged experimental profiles; it did not repair avatars.
|
||||||
|
Three fresh media hosts across two roots passed post-warmup 3/3 on each TLS
|
||||||
|
protocol (18 actual HTTP206/65536-byte MP4 transfers). This is field evidence,
|
||||||
|
not persisted family qualification, a cold/all-host claim or candidate promotion.
|
||||||
|
Watch fallback discovered allowed avatar metadata, but both-TLS curl28 transfers
|
||||||
|
had zero samples: avatar evidence/profile qualification is still absent and
|
||||||
|
overall health is **DEGRADED / NOT READY**. Automatic durable multi-host family
|
||||||
|
qualification remains unimplemented. See [PROGRESS.md](PROGRESS.md) for exact
|
||||||
|
hosts, separate warmup results, timestamps, private proof and remaining gates.
|
||||||
|
No new network checks accompany this documentation. Generic production
|
||||||
|
media behavior and signed-URL/SS secret privacy remain unchanged, as do the
|
||||||
|
persisted minimum two-hour strategy-absence pause and 90-minute whole-job budget.
|
||||||
+117
-7
@@ -1,23 +1,133 @@
|
|||||||
# Прогресс реализации
|
# Прогресс реализации
|
||||||
|
|
||||||
Срез на **2026-10-12**. Этот файл описывает фактическое состояние кода, а не весь объём требований из `Requirements.md` и `Architecture.md`.
|
Срез на **2026-10-14**. Этот файл описывает фактическое состояние кода, а не весь объём требований из `Requirements.md` и `Architecture.md`.
|
||||||
|
|
||||||
## Текущий статус
|
## Текущий статус
|
||||||
|
|
||||||
|
### Generic periodic monitor fault acceptance: narrow PASS, not autotuning/READY
|
||||||
|
|
||||||
|
- Supplied orchestrator-verified privileged read of `/home/agent/zapret2-autotune-monitor-5c85f3f/field-private/report.safe.json` covers `4efc066` plus **`5c85f3f0b9ef04e600f67593d02cae526fac901a`**. Not independently reread here; native full/race checks and independent security review were reported GO. This docs-only update performs no probes/deployment; later docs commits are not tested source.
|
||||||
|
- Actual periodic monitoring ran **without SIGSTOP**: latest window 21:50:22–21:54:30 retained PID132 and three rows across two services (alpha stable/flaky, beta slow). Healthy rows progressed while flaky was DEGRADED, then repair made all healthy; earlier `4efc066` ran stably for 305s with independent failure/retry and valid reload/removal.
|
||||||
|
- Snapshot/key inode replacement, wrong UID/mode, symlink, corrupt/missing files and readiness tampering yielded **BLOCKED, stale=true**; private DNS/SNI/server counters were unchanged during faults. Invalid source/wrong ownership made zero DNS requests. Foreign bytes/unknown files were preserved; original-inode/operator repair resumed the same PID. Root wrong-owner field test passed (local unit case skipped).
|
||||||
|
- Public state was authenticated with a private 32-byte root-owned mode0600 HMAC key; secret-marker/address scans were clean. Idle 30s left content/inode/mtime unchanged. Cancelling an actual active curl left the unfinished row's LastCompleted/Failures/NextDue untouched; completed rows remained durable.
|
||||||
|
- Graceful stop: **1.45s**, exit0, all owned PIDs joined and network namespace gone; restart PID121 accepted HMAC/prior rows, force-recreate PID126 retained the state-directory inode and accepted HMAC. Additional stops: 0.59s/0.57s. Offline native worst-case cleanup verifies bounded final reaps within 90s, not a measured remote 80s worst-case wait.
|
||||||
|
- Strategy config/journal hashes were unchanged. Old SS1080, stopped proxy and unrelated stopped RC11082/exit78 were unchanged; legacy YouTube exit78 is **not fixed**. HEALTHY means **CHECKS_ONLY**, not generic strategy recovery, whole-production READY or full P0 acceptance.
|
||||||
|
- Periodic controller/state/status/reload/independent retry foundations are DONE. Next P0: actual per-domain qualification matrix, then runtime coverage/profile groups, renderer transactions and saving multiple qualified candidates; generic strategy DB/migration/apply remain TODO. Prometheus P1 docs were researched, but no metrics code exists; future labels must stay bounded to configured IDs/enums.
|
||||||
|
|
||||||
|
### Declarative resource-check acceptance (2026-10-13): check-only PASS, not controller acceptance
|
||||||
|
|
||||||
|
- Supplied orchestrator-verified remote evidence binds source **`4d7c4b33c36186d43ea89fd24354b1834b21692c`** to image **`sha256:be5a1c45c6d40b390a29bfc9a5ab3975c3a8e0c015b98a524696312807dad9a4`**. Private report: `/home/agent/zapret2-autotune-resources-4d7c4b3/field-private/final.safe.json`; privileged-read by the orchestrator, **not independently reread here**. Reported `checker_gates_passed=true`; later documentation commits are not the tested source. No new network checks or deployment accompany this update.
|
||||||
|
- JSON v1 schema/loader and executable HTTPS GET/HEAD checks are unit-verified and now remotely accepted: public GET/HEAD on TLS1.2/1.3; private opt-in plus CIDR with a staging-only trusted test CA, original TLS/SNI hostname and numeric CONNECT pin. No `--insecure`, production trust-store pollution or DoH implementation change. Checks reuse the owned SS/configured wire-DNS backend; the earlier DoH outage acceptance recorded below remains separate evidence.
|
||||||
|
- Default/private-policy, mixed DNS answers, metadata/link-local/ULA/loopback denials made **zero destination connections**; rebinding used the first pinned address and denied the later answer. Ownership-negative cases made zero DNS/resource probes. Cancellation exited1 with joined workers in **0.38s**; the secret sentinel was absent from argv, output and persisted artifacts.
|
||||||
|
- **Check-only isolation:** staging YouTube monitor was SIGSTOP-isolated during checks, then SIGCONT-resumed **before** graceful stop; reported `monitor_resumed=true`, Docker exit0. Cleanup left zero owned host PIDs, no owned namespace or temporary containers/networks, and an unchanged foreign baseline. An earlier existing managed monitor **exit78** contributed before isolation: this does **not** prove whole-controller resilience, generic recovery or production availability. LIVE11082 was already **stopped** before the task and was **not upgraded or restarted**; old SS1080/Xray were untouched. Earlier LIVE snapshots below are historical, not current uptime claims.
|
||||||
|
- At this earlier check-only acceptance, periodic integration was pending; the narrow controller/BLOCKED foundations are now accepted above. Remaining P0: config/strategy DB integration, matrix and plugins/autotuning; maximum strategy collection and Prometheus P1 remain TODO. Three-CDN qualification belongs to the YouTube adapter, not every arbitrary service. See [MANAGED_TARGET.md](MANAGED_TARGET.md#declarative-https-resource-checks-v1-check-only) and the tracked [resource-check.example.json](resource-check.example.json).
|
||||||
|
|
||||||
|
### Managed DoH acceptance (2026-10-13): live DNS path and staged outage verified, NOT READY
|
||||||
|
|
||||||
|
- Managed target defaults to the stdlib Go UDP/TCP loopback DoH forwarder; PID 1 validates owned readiness before ss-server `-d 127.0.0.1:5353`. SOCKS remote-hostname traffic and managed `socks5h` checks share this path. Provider URL/bootstrap IPs/timeout are private-env configurable; explicit native mode alone preserves legacy DNS, never automatic fallback.
|
||||||
|
- DNS stays available during monitor restoration, zapret stop and SS joins, then is terminated/joined with a five-second watchdog and readiness cleanup. DNS child death uses existing bounded retries; upstream failures return SERVFAIL without stopping the forwarder. No host DNS port, global DNS changes, new dependency or capability.
|
||||||
|
- Security follow-up: PID 1 retains directory/readiness inode descriptors for validated owned cleanup after graceful exit or SIGKILL; foreign/stale/replaced markers are preserved and fail closed. Compose passes empty bootstrap by default, allowing Google defaults only for the exact Google URL; custom URLs require explicit IPs.
|
||||||
|
- Earlier offline shell lifecycle/static/config fixtures (real readiness CLI with inherited anchors), sh/bash syntax checks, full Go and targeted readiness/entrypoint race checks passed as previously reported. Coverage includes foreign regular/symlink preservation, SIGKILL owned cleanup/retry, custom-provider Compose validation, signal/cooldown cancellation and DNS-last cleanup. This docs-only update adds no source/runtime changes or probes and does not rerun the Go suite.
|
||||||
|
- Supplied verified reports bind DoH implementation `367d956`, TCP reply deadline fix `721cad4`, and explicit public-health `socks5h` fix **`544dda7c85965fc3200de3af17fca9ef9999b2ce`** to LIVE image **`sha256:b3085b7ceca4a629c28bd58f23bf97f02a34c2a38df08b57756e40594ba2d427`**, container **`4ad54175a54c`**, owned SOCKS **`127.0.0.1:11082`**. Private reports: `/home/agent/zapret2-autotune-rc-doh-721cad4/live-upgrade-private/`, then latest `/home/agent/zapret2-autotune-rc-doh-544dda7/live-upgrade-private/`; not independently reread here. Later documentation commits are not the tested source.
|
||||||
|
- Real isolated staging blocked **both** bootstrap IPs for **61 seconds**: UDP and TCP replies returned SERVFAIL (rcode2) at about **5000ms**; DNS, monitor, SS and root PIDs stayed unchanged, with zero external port53 traffic. After removal, DNS recovered immediately and WWW HTTP200 was verified after **46 seconds**, without process restart. Graceful stop took **1 second**, exit0, no survivors. This proves the tested DNS dependency outage, not every conceivable outage or nonconfiguration guard behavior. Staging-only nft fault rules were removed; no new runtime firewall policy was added beyond normal generated native rules.
|
||||||
|
- LIVE **303-second** measured snapshot: same root and children DNS22, SS59/60, monitor253, all children PPID1; owned readiness ACK and PID1-held FD8/9 anchors were positive. UDP/TCP DNS IDs and questions matched, rcode0, answer counts2/8. SOCKS greeting `0500`, WWW HTTP200/SSLVerifyResult0, and configured `health youtube` exit0 passed. Wire evidence counted **41 marked DoH connections**, zero external port53; direct HTTPS sockets use mark **`0x50000000`** to bypass live/probe NFQUEUE. Scope `suffix`, generation9, IDs `[1,9]`, config hash preserved versus staging are measured snapshot values, not universal constants: autonomous state has advanced since an earlier generation8 observation.
|
||||||
|
- Existing normal shared GGC qualification remains separate proof; no new 18-transfer qualification under DoH is claimed. Fresh media TLS1.3 returned HTTP206/65536 bytes/MP4 `ftyp`/SSLVerifyResult0; TLS1.2 timed out and is **not qualified**. Avatar and thumbnail qualification remain independent outstanding gates; no full-service READY claim. Maximum strategy collection and generic service integration remain future work, not a new framework.
|
||||||
|
- DNS configuration is the private mode0600 LIVE `/home/agent/zapret-autotune-cleanops-3fc8563-1791882112/.env`; the four public DNS keys and defaults are documented in the deployment guide. URL/bootstrap come from env, never argv; changing DNS requires official Compose recreation under the controlled PID1 lifecycle. Credentials/private env and state were preserved. Staging cleanup left no new running test containers; the old user SS1080 and user-stopped proxy were unchanged.
|
||||||
|
- Default `https://dns.google/dns-query` selects literal `8.8.8.8,8.8.4.4` bootstrap only when missing/empty; custom endpoints require explicit IPs. TLS is verified, with no system-DNS, proxy or plaintext fallback; native mode is manual and explicit only. The direct DoH resolver sees queries: they are **not SS-tunneled**. Plaintext UDP/TCP exists only at internal loopback5353. IP-transmitting SOCKS clients using local DNS are outside this guarantee; use remote DNS/`socks5h`. The existing Go binary supplies the stdlib forwarder, no dependency or extra container, bounded to 4096-byte queries, 65535-byte replies, 8192-byte HTTP headers and 16 workers, with no forwarder cache.
|
||||||
|
|
||||||
|
### Autonomous three-CDN field acceptance (2026-10-13): media qualified, NOT READY
|
||||||
|
|
||||||
|
- Supplied field results for source `3fc8563995e890310580ba8f78850b92fea42136`, managed `Dockerfile.target` image `sha256:a902e4efd949a7ffce4414fd99ab4a815a643b40cf9b9e30e2bd2db7b452817a`, container `4c6b34…`, Compose project `rc0398e6f…`, isolated SOCKS `127.0.0.1:11082`. This documentation update performs no deployment or network probes. Private sanitized artifact: `/home/agent/zapret2-autotune-rc-3fc8563/field-private/proof/final.safe.json`; its contents are reported by the orchestrator, not independently reread here. Later docs commits are not the tested image source.
|
||||||
|
- From the existing 08:01:50 cooldown state, native recovery ran naturally at 08:07–08:09 without resetting cooldown. Missing/blank `ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS` selected primary `dQw4w9WgXcQ`, `aqz-KE-bpKQ`, and `jNQXAC9IVRw`. The program discovered three actual ordinary signed-media hosts without manual domains; VideoIDs are inputs, not CDN evidence.
|
||||||
|
|
||||||
|
| Actual discovered host | TLS1.2 steady media | TLS1.3 steady media | Separate warm attempts / failures |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| `rr16---sn-n8v7znsz.googlevideo.com` | 3/3 | 3/3 | 2 / 0 |
|
||||||
|
| `rr18---sn-n8v7zns6.googlevideo.com` | 3/3 | 3/3 | 3 / 1 |
|
||||||
|
| `rr5---sn-n8v7kn76.googlevideo.com` | 3/3 | 3/3 | 3 / 1 |
|
||||||
|
|
||||||
|
All **18 post-warmup transfers** returned HTTP206, valid Range `0..65535`, 65536 bytes, MP4 `ftyp`, and valid SSL verification. Warm failures are separate, not cold qualification. Fresh version-1 `owned-nfq-media-range-v1` attestation in Adaptive mode retained actual host/endpoints, source IDs and native-owner Lua binding; proof checked at `2026-10-13T08:08:28.752Z`, with three roots and three prefixes. Native request statistics were not persisted: manual request count is unknown, not zero or an invented total.
|
||||||
|
- The operator automatically **PROMOTED** one newly qualified Adaptive candidate10: generation9→10, active IDs `[1,3,9]`→`[1,9,10]`, scope `experimental_googlevideo_suffix`→`suffix`, current config hash prefix `dc997ac91…`. Common hostlist `googlevideo.com`, frontend1 and thumbnail9 were retained. Qualification and fresh deployed checks used the transaction-bound apply safety path (asset/frontend hooks have source test coverage). This qualifies the measured representative three-CDN composite profile, not every Googlevideo host, cold behavior, or a maximum strategy catalogue. The shared profile remains two native strategies with per-host learning.
|
||||||
|
- Final reported snapshot at **08:21:48**: container running and monitor alive; whole-service health still false / **DEGRADED, NOT READY**. Probe failures reached405 at 08:20:37; their avatar-specific cause was not independently measured in this phase. The prior day's avatar timeouts are historical, not today's diagnosis. The orchestrator's fresh owned-SOCKS check returned `0500`; role-owned `socks5h` uses native SS DNS `8.8.8.8`. The passive SSH watch session vanished after about ten minutes: no normally completed 18-minute watch or interruption of the controller is claimed.
|
||||||
|
- Remaining: actual avatar/service health qualification, maximum strategy collection and broader search diversity, and controller security/global-failure stay-alive policy (current safety guards remain fatal). Generic periodic per-service failure isolation/controller integration remain future requirements; declarative check-only configuration is accepted above, not a new plugin framework. No further build, probes or cooldown changes accompany these docs.
|
||||||
|
|
||||||
|
### Autonomous three-CDN source behavior (2026-10-13)
|
||||||
|
|
||||||
|
- User reaffirmed **three actual CDN checks**, discovered by the program without
|
||||||
|
manual domains; the earlier single-CDN/version-2 proposal is withdrawn. Missing
|
||||||
|
or empty service-mode VideoID inventory selects primary (default `dQw4w9WgXcQ`)
|
||||||
|
plus `aqz-KE-bpKQ` and `jNQXAC9IVRw`, deduplicated. A nonempty 1..5 unique-ID
|
||||||
|
override is respected; three videos do not prove three CDN hosts.
|
||||||
|
- Fresh discovery runs up to three round-robin rounds, at most 15 requests in
|
||||||
|
three minutes; it deduplicates actual signed ordinary MP4 hosts, requires
|
||||||
|
twenty-minute remaining URL TTL, and selects three hosts across at least two
|
||||||
|
roots and two prefixes. The unchanged version-1 owned native proof factory
|
||||||
|
requires actual post-warmup media 3/3 on both TLS protocols at each endpoint.
|
||||||
|
Common Googlevideo suffix authorization is extrapolative, not all-host evidence.
|
||||||
|
- Existing experiments can now seek promotion through that shared qualification
|
||||||
|
path, then frontend and fresh deployed three-host media checks under the current
|
||||||
|
transaction/ownership guards. Insufficient inventory leaves the experiment
|
||||||
|
committed and retries at configured cooldown; genuine strategy exhaustion keeps
|
||||||
|
its minimum two-hour pause. Missing avatar does not block the media stage;
|
||||||
|
whole-service monitoring remains DEGRADED until actual avatar health passes.
|
||||||
|
Legacy service-off/exact-host behavior and cancellation safety remain unchanged.
|
||||||
|
- The source was unit/fixture-tested and subsequently field-accepted for the
|
||||||
|
representative media promotion recorded above; whole-service/avatar acceptance
|
||||||
|
remains open. Earlier statements below that automatic qualification is
|
||||||
|
unimplemented describe historical RCs, not this source; historical measurements
|
||||||
|
are not the new owned factory evidence.
|
||||||
|
- Maximum strategy collection and PID1 global-failure stay-alive are still future
|
||||||
|
increments; broad maximum-profile live staging is deferred. Declarative checks
|
||||||
|
are accepted above; generic service controller integration remains future work. Details:
|
||||||
|
[GOOGLEVIDEO_DISCOVERY.md](GOOGLEVIDEO_DISCOVERY.md).
|
||||||
|
|
||||||
|
### Final managed RC field acceptance (2026-10-12): DEGRADED, not READY
|
||||||
|
|
||||||
|
- Supplied final field results, not network checks rerun by this docs update: source `fd25cb89398300bb9849db4b5511af75bf86eccf`, correct managed image `sha256:73e955e7fdc6efaef88c62ebe4f7afeab71c29d29ba3c88b2a2b7547bb647da5`, active container `a4e845a6d7…`, official managed PID 1, isolated SOCKS `127.0.0.1:11082`. The earlier `37823…` prototype image is **not active** and is not this acceptance image. Private sanitized proof: `/home/agent/zapret2-autotune-rc-fd25cb8/proof/final.safe.json`. The later docs commit is not the tested image source.
|
||||||
|
- Watch started `2026-10-12T20:15:07.686Z`; check at `2026-10-12T20:30:54.826Z` completed over 15 minutes with DockerRestartCount0 and monitor alive. A fresh orchestrator check after the report returned SOCKS greeting `0500` and WWW HTTP200 with SSLVerifyResult0. Managed `socks5h` probes respect native SS DNS `-d 8.8.8.8`; no local-resolution bypass or certificate bypass is claimed.
|
||||||
|
- At this observation, experimental journal generation9, IDs [1,3,9], hash prefix `0e34…` and covered scope were retained unchanged. Saved candidate3 remains experimental, without qualification/evidence. These are snapshot values, not universal IDs/generation. Runtime recovery classification accepted validated peer-TLS/payload rejections as nonpass, with writable-store/ownership guards and the current job-bound transaction hooks intact. Genuine no-qualified-strategy recovery advanced its cooldown **naturally** while the monitor stayed alive; no cooldown reset or fabricated ERROR/source result was used. This validates degraded recovery continuation, not successful avatar repair or family promotion.
|
||||||
|
- Fresh actual Range media results, **post-warmup**, not cold qualification:
|
||||||
|
|
||||||
|
| Sampled hostname | TLS1.2 | TLS1.3 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `rr4---sn-axq7sn7z.googlevideo.com` | 3/3 | 3/3 |
|
||||||
|
| `rr5---sn-n8v7kn76.googlevideo.com` | 3/3 | 3/3 |
|
||||||
|
| `rr18---sn-axq7sn7z.googlevideo.com` | 3/3 | 3/3 |
|
||||||
|
|
||||||
|
All **18** transfers verified HTTP206, 65536 bytes, MP4 `ftyp` and valid Range `0..65535`. Separately, initial TLS1.2 warmup had three timeouts; initial TLS1.3 warmup had three passes. Three hosts across two roots provide representative field transfer evidence only; no cold/all-host or durable family qualification was created. The older rr5 `sn-ixh` counterexample was **not tested** here and is not resolved by the new rr5 `sn-n8v7kn76` sample.
|
||||||
|
- Thumbnail `i.ytimg.com`: JPEG, 2888 bytes, SSLVerifyResult0, decoded 3/3 on each TLS protocol. Fresh watch metadata discovered an allowed exact `yt3.ggpht.com` avatar URL, but actual avatar transfers returned curl28 on both TLS protocols, zero samples. No saved avatar evidence or qualified avatar profile exists. Overall live health remains **DEGRADED / NOT READY**; URL discovery and successful media do not close the avatar gate.
|
||||||
|
- Other production/test targets on 1080/11080/11081 retained their IDs/start times; Xray was untouched. No further remote operations accompany this docs update. Build explicitly with `-f Dockerfile.target` and assert the managed entrypoint plus target marker **before promotion**: a tag alone previously mistook root `Dockerfile`'s prototype for the managed image.
|
||||||
|
|
||||||
|
Remaining work: actual avatar transfer/strategy qualification, then durable representative multi-host family qualification and mode acceptance. Automatic family qualification is not implemented; no experimental candidate was promoted. Restart/recreation observations from older RCs remain separate from this final 15-minute watch. The historical open-media/fallback/deployment labels below are superseded only by the specific final results above, not by blanket production approval.
|
||||||
|
|
||||||
|
### Earlier peer TLS rejection classification fix (2026-10-12)
|
||||||
|
|
||||||
|
- Follow-up independent production repro exposed a separate decoder rejection bug: a bounded/origin-verified truncated image degraded live health but failed owned verification as infrastructure before remaining seeds. Image decoding now emits typed payload rejections (invalid magic/dimensions, truncated PNG/JPEG/GIF, body budget and content-type mismatch); both live health and candidate verification share that class, without accepting any payload or fabricating evidence. Missing/corrupt normalized evidence, malformed transfer metrics/Player JSON, arbitrary same-text errors, explicit guards and mixed cancellation remain fatal; image rejection cannot become media absence. Permanent production monitor tests cover joined workers, seed exhaustion/two-hour pause and a later seed's fresh both-TLS 3/3 evidence with preserved frontend/GGC profiles. The independent overlay repro, affected Go tests and race tests, vet, CLI build and diff checks passed locally; no remote deployment was performed.
|
||||||
|
|
||||||
|
- Read-only field inspection: RC source `50a78608ce40dfc26289b5a63d01eeb3fc569ace`, image `sha256:1a97f34cfbe3bd606d5fb13599767f8a51273b374fa441f5e67802c74d814701`, container `d361b5d581bf710af28f14d3998d6a9a699661b827248c12c9352182df94f98d` is now exited78 (finished `2026-10-12T18:44:23.941802233Z`); host port11082 has no TCP listener. The earlier `proof-v2/final.safe.json` observation at 18:44:10 reported it running, so it is not current uptime evidence. Private artifacts: `/home/agent/zapret2-autotune-rc-50a7860/proof-v2/`.
|
||||||
|
- That safe collector evidence contains a fresh allowed avatar URL, avatar TLS1.2 curl60 and TLS1.3 curl35, both zero samples/FAIL with generic fatal wrappers; thumbnail and sampled media passed. Terminal safe logs subsequently show avatar degradation, **thumbnail** degradation, recovery failure, and the generic safety/database/runtime terminal branch. They do not identify the exact terminal underlying error: do not claim avatar TLS alone is proven to have caused exit78 or that a real local guard failed.
|
||||||
|
- Source fix distinguishes validated resource-curl peer TLS rejection (35/51/60) from process-wide operational safety. It remains rejected/unqualified, with certificate verification, origin checks and SSLVerifyResult=0 qualification unchanged. Probe/recovery can remain DEGRADED; genuine exhausted asset verification enters the existing minimum two-hour pause with workers joined and committed profiles preserved. Local proxy connection7, client-certificate configuration58, unknown/mixed cancellation and explicit guard/cleanup failures remain infrastructure/safety failures. No certificate bypass, host pinning, DNS change, family promotion or READY claim.
|
||||||
|
- The corrected source has not been deployed or image-qualified. Existing remote containers, profiles and other working ports were not restarted or modified by this fix; a tested new image and renewed actual-resource checks are the next dispatch.
|
||||||
|
- Local affected-area verification: managed/resource/experimental/asset/Googlevideo/monitor tests passed normally and under `-race`; local CLI build and `git diff --check` passed. Fixture checks cover production avatar35/60 degradation, owned verification exhaustion and post-apply rejection/rollback, joined workers, durable two-hour cooldown, experimental same-profile media35/60 recovery, and retained local/mixed safety failures. These are not live image acceptance.
|
||||||
|
|
||||||
### Googlevideo suffix RC — browser-verified experiment (2026-10-12)
|
### Googlevideo suffix RC — browser-verified experiment (2026-10-12)
|
||||||
|
|
||||||
- Source RC `eade051c2bc335b71b1616d697326ca4c8019d70`, immutable image `sha256:fdb60593e29d7c337a413a47aaddfbcaaeabdeb877042c58f506d4abdc98a13b`, isolated host SOCKS `127.0.0.1:11082`. Recorded journal: generation3, active candidates [1,2,3], scope `experimental_googlevideo_suffix`; `www.youtube.com` and `i.ytimg.com` profiles preserved. Older targets were unchanged. These are recorded field observations, not a new probe/deployment by this documentation commit; its SHA is not the tested image source.
|
- Initial source RC `eade051c2bc335b71b1616d697326ca4c8019d70`, immutable image `sha256:fdb60593e29d7c337a413a47aaddfbcaaeabdeb877042c58f506d4abdc98a13b`, isolated host SOCKS `127.0.0.1:11082`. Historical initial journal: generation3, active candidates [1,2,3], scope `experimental_googlevideo_suffix`; `www.youtube.com` and `i.ytimg.com` profiles preserved. Older targets were unchanged. These are recorded field observations, not current candidate IDs or a new probe/deployment by this source update; its SHA is not the tested image source.
|
||||||
- Fresh signed media on `rr16---sn-n8v7znsz.googlevideo.com` returned HTTP206, 65536 bytes, MP4 `ftyp`, TLS1.2. On **2026-10-12 the user confirmed browser videos work** with this live RC and accepted one shared profile for Googlevideo, including initial learning. Browser images also appeared, but that does not qualify every image route.
|
- Fresh signed media on `rr16---sn-n8v7znsz.googlevideo.com` returned HTTP206, 65536 bytes, MP4 `ftyp`, TLS1.2. On **2026-10-12 the user confirmed browser videos work** with this live RC and accepted one shared profile for Googlevideo, including initial learning. Browser images also appeared, but that does not qualify every image route.
|
||||||
- One suffix profile is a **composite native circular A/E profile with two strategies and per-host learning**, not one uniform static strategy. Explicit operator experimental policy is durable across restart with hash/rollback guards; it is neither family-qualified nor `READY`, and no all-host cold 3/3 result is claimed.
|
- One suffix profile is a **composite native circular A/E profile with two strategies and per-host learning**, not one uniform static strategy. Explicit operator experimental policy is durable across restart with hash/rollback guards; it is neither family-qualified nor `READY`, and no all-host cold 3/3 result is claimed.
|
||||||
- Service health already checks actual fresh image resources and media with both TLS protocols. Experimental recovery intentionally returns without automatically qualifying or replacing the operator-approved experiment: the remaining gap is autonomous experimental recovery/autoqualification, not missing probe implementation. Operator scope coverage and qualification are separate states.
|
- Source now wires autonomous experimental media recovery in either configured service mode. Healthy actual-resource probes short-circuit recovery; a validated media transfer failure permits one guarded restart of the same committed profiles and bounded native relearning in the persistent deployed daemon (10 attempts / 120s), followed by fresh signed media 3/3 on both TLS protocols. Learning stays in daemon RAM, not a temporary verification worker; expired inventory is refreshed, never rewritten or persisted. Missing inventory is degraded, not strategy absence; guard/infrastructure/cancellation and mixed failures remain fail-closed. No experimental candidate is replaced or promoted to qualified.
|
||||||
|
- Status distinguishes active-profile coverage (`covered=` / JSON `covered_groups`, `googlevideo_scope`) from qualified `groups` and `unresolved_groups`. Experimental Googlevideo remains unresolved for qualification even when operational media is healthy; a thumbnail profile does not mean the full asset inventory is qualified. Status itself performs no live health check. Generic production media handling, signed-URL/SS privacy, persisted minimum two-hour strategy-absence pause and the separate 90-minute whole-job budget are unchanged.
|
||||||
|
- **Reported field follow-up for `a60f62d`:** privileged checks passed restart/recreation and covered-versus-qualified status checks. The live upgrade encountered a runner `BrokenPipe` and automatically rolled back to the same last-good image; this is not a successful autonomous media-recovery acceptance. Fresh Range media passed once on TLS1.2, then failed on repeat; no TLS1.3 media PASS is claimed. The underlying field failure remains unknown. These are supplied field results, not network checks rerun by this source update.
|
||||||
|
- Avatar discovery now conditionally falls back from a homepage with absent metadata/no allowed avatar to one fresh watch page for the validated configured video ID, on the same trusted HTTPS WWW origin and proxy. Malformed JSON, request failures, cancellation and safety failures do not authorize fallback. Reported watch metadata contained an allowed avatar URL where the homepage did not; the updated fallback has not been field-deployed or image-qualified. URL discovery alone does not qualify the avatar route/profile or close a family gate.
|
||||||
|
- Local source verification for the watch fallback: fresh `go test ./... -count=1 -timeout=180s`, focused avatar inventory/parser/classifier, experimental production-probe and private-inventory regression tests under `-race`, and `git diff --check` passed. These are unit/fixture checks only, not new field evidence.
|
||||||
|
|
||||||
Remaining acceptance gates (not asserted defects):
|
Historical acceptance gates at that RC (updated by final field results above):
|
||||||
|
|
||||||
1. Representative fresh signed media evidence: at least three unique hosts across at least two roots/prefixes, post-warmup 3/3 for both TLS protocols plus actual media transfer. Do not impose uniform static/cold behavior on the accepted learning experiment or label it normally qualified without evidence.
|
1. Representative fresh signed media evidence: at least three unique hosts across at least two roots/prefixes, post-warmup 3/3 for both TLS protocols plus actual media transfer. Do not impose uniform static/cold behavior on the accepted learning experiment or label it normally qualified without evidence.
|
||||||
2. Qualify the avatar route/profile: only the thumbnail image profile is currently active; browser-visible images do not close this gate.
|
2. Qualify the avatar route/profile: only the thumbnail image profile is currently active; browser-visible images do not close this gate.
|
||||||
3. Field-accept actual restart/recreation recovery after the all-Googlevideo suffix change, including renewed native learning; older exact-host lifecycle PASS does not cover this RC.
|
3. Restart/recreation checks are reported PASS for `a60f62d`; renewed learning with repeatable actual media success remains an open gate.
|
||||||
4. Resolve autonomous recovery/autoqualification policy for the approved experiment without silently replacing or promoting its operator-approved scope.
|
4. Field-accept the new autonomous same-profile experimental recovery and resource-error classifier without replacing or promoting operator-approved scope. Unit coverage exists; live upgrade/rollback observations do not prove successful recovery. Automatic family qualification is not implemented.
|
||||||
5. Reconcile service status with those results: distinguish operator-covered suffix from unresolved qualification/assets; no family-qualified/`READY` or blanket production claim yet.
|
5. Covered-versus-qualified field checks are reported PASS for `a60f62d`; validate the new conditional watch fallback and qualify actual avatar image transfers separately. No family-qualified/`READY` or blanket production claim yet.
|
||||||
|
|
||||||
### Earlier isolated-target acceptance (2026-10-11)
|
### Earlier isolated-target acceptance (2026-10-11)
|
||||||
|
|
||||||
|
|||||||
@@ -4,6 +4,71 @@
|
|||||||
|
|
||||||
Go CLI for scanning and checking YouTube TLS bypass strategies with `blockcheckw`.
|
Go CLI for scanning and checking YouTube TLS bypass strategies with `blockcheckw`.
|
||||||
|
|
||||||
|
The managed target now defaults to encrypted DoH for both SOCKS hostname traffic
|
||||||
|
and managed checks, with no plaintext fallback. Configure provider URL/bootstrap
|
||||||
|
IPs in the private deployment `.env`; native DNS requires an explicit opt-out.
|
||||||
|
Clients must enable SOCKS5 remote DNS (e.g. browser remote DNS on the chosen
|
||||||
|
loopback SOCKS port). See [DNS configuration and limits](deploy/managed-target/README.md).
|
||||||
|
This new integration is not covered by the earlier clean-run acceptance.
|
||||||
|
|
||||||
|
## Configured HTTPS checks (checks only)
|
||||||
|
|
||||||
|
Inside the owned managed target, with its configured DNS/SOCKS environment, run:
|
||||||
|
```sh
|
||||||
|
zapret-autotune check --config /path/to/resource-check.example.json
|
||||||
|
```
|
||||||
|
See [the public example](resource-check.example.json). This does **not** select
|
||||||
|
strategies, migrate candidates/transactions, implement metrics, or declare READY;
|
||||||
|
`check youtube` and the YouTube qualification contract remain unchanged.
|
||||||
|
|
||||||
|
Version 1 supports `kind: "http"`, GET/HEAD, exact HTTPS DNS-hostname origins
|
||||||
|
(port 443 only), `tls12`, `tls13`, or `both`. All fields shown in the example are
|
||||||
|
required except `content_type` and `min_bytes` (default zero); IDs/group are lowercase bounded identifiers.
|
||||||
|
There are at most 16 services and 64 resources, 100–30000 ms per TLS probe,
|
||||||
|
and 1–5242880 maximum decoded bytes. HEAD requires `min_bytes: 0`; its advertised
|
||||||
|
Content-Length is not downloaded or used as a body limit. No redirects are accepted.
|
||||||
|
Expected statuses are explicit, non-redirect 200–599 codes; MIME is an optional
|
||||||
|
exact media type (parameters are ignored in the response).
|
||||||
|
This slice checks HTTP status, MIME and byte bounds only: no image/media decode,
|
||||||
|
Range probes, suffix scopes or per-resource DNS settings.
|
||||||
|
|
||||||
|
Replace `url` with `url_env` for a private URL; optional `headers_env` maps header
|
||||||
|
names to environment variable names (e.g. `{"Authorization":"CHECK_TOKEN"}`).
|
||||||
|
Secrets travel only over private child stdin, never argv/results/database.
|
||||||
|
Transport/framing headers (including Range and Accept-Encoding) cannot be
|
||||||
|
overridden. Range probes need a future explicit probe field, not a secret header.
|
||||||
|
Config files must be regular, non-symlink files no larger than 1 MiB; permissions
|
||||||
|
are left untouched. Unknown fields, duplicates, missing/invalid settings and
|
||||||
|
environment references reject the entire configuration before any probes.
|
||||||
|
|
||||||
|
Checks issue explicit framed A/AAAA DNS queries to the configured local DoH
|
||||||
|
forwarder: no `/etc/hosts`, search domains or system DNS fallback. CNAME chains
|
||||||
|
are bounded and validated; unrelated Additional glue is never a destination.
|
||||||
|
The root-owned forwarder must match PID 1's private readiness anchors, executable,
|
||||||
|
PID/network namespaces and both UDP/TCP listeners; the owned Shadowsocks chain
|
||||||
|
must actually route its resolver to that same listener before any probes.
|
||||||
|
Explicit native mode requires an IP-valued `SS_DNS_SERVER` and is **plaintext DNS,
|
||||||
|
not encrypted DoH**. Every answer must pass
|
||||||
|
the destination policy, then the chosen IP is pinned for SOCKS CONNECT while the
|
||||||
|
original hostname remains the TLS/SNI/certificate authority. Addresses are resolved
|
||||||
|
fresh per probe, not stored. Private targets require `allow_private: true` **and**
|
||||||
|
explicit `allowed_networks` CIDRs. CIDRs always restrict; metadata, link-local,
|
||||||
|
CGNAT, loopback, benchmark and other reserved ranges remain denied even with that opt-in.
|
||||||
|
|
||||||
|
Output is one JSON line per resource/TLS probe: IDs, bounded outcome, TLS,
|
||||||
|
decoded bytes, milliseconds and HTTP status only. Exit 0 means all checks passed;
|
||||||
|
1 means a resource failed/cancellation/output failure; 2 means invalid config or
|
||||||
|
unavailable ownership. Resource failures continue to other resources; ownership
|
||||||
|
failure stops all probes. These checks have unit/loopback-fixture coverage only,
|
||||||
|
not new remote acceptance or generic automated recovery/controller support.
|
||||||
|
|
||||||
|
Generic `managed resource-status` validates the authenticated snapshot against
|
||||||
|
the live controller's PID/starttime, executable, command, UID, namespaces and
|
||||||
|
held directory/key/state/singleton inodes. Cached check results alone cannot
|
||||||
|
report current health: missing ownership or an identical signed replacement
|
||||||
|
returns `BLOCKED` with `stale: true` without modifying the foreign file.
|
||||||
|
`CHECKS_ONLY`/`HEALTHY` still do not mean candidate or production readiness.
|
||||||
|
|
||||||
## Production preparation (isolated clean-run acceptance passed)
|
## Production preparation (isolated clean-run acceptance passed)
|
||||||
|
|
||||||
The standalone deployment candidate is a **new isolated target**, never an
|
The standalone deployment candidate is a **new isolated target**, never an
|
||||||
@@ -102,8 +167,9 @@ not used and there is no direct fallback. The request times out after 30 seconds
|
|||||||
| `ZAPRET_AUTOTUNE_FAILURE_THRESHOLD` | `3` |
|
| `ZAPRET_AUTOTUNE_FAILURE_THRESHOLD` | `3` |
|
||||||
| `ZAPRET_AUTOTUNE_RECOVERY_COOLDOWN` | `30m` |
|
| `ZAPRET_AUTOTUNE_RECOVERY_COOLDOWN` | `30m` |
|
||||||
|
|
||||||
Configuration is parsed once per health invocation. The proxy must use `socks5`
|
Configuration is parsed once per health invocation. The proxy must use `socks5` or `socks5h`
|
||||||
with a loopback IP or `localhost`, port 1–65535, and no credentials/path/query.
|
with a loopback IP or `localhost`, port 1–65535, and no credentials/path/query/fragment.
|
||||||
|
Explicit `socks5h` uses proxy-side hostname resolution (encrypted DNS only with managed DoH configured); explicit `socks5` retains local DNS semantics.
|
||||||
Video IDs must be 11 characters; durations and the failure threshold must be
|
Video IDs must be 11 characters; durations and the failure threshold must be
|
||||||
positive. Explicit empty values are invalid. Interval, consecutive-failure
|
positive. Explicit empty values are invalid. Interval, consecutive-failure
|
||||||
threshold and cooldown configure `managed monitor` (the one-shot command still
|
threshold and cooldown configure `managed monitor` (the one-shot command still
|
||||||
|
|||||||
@@ -0,0 +1,72 @@
|
|||||||
|
# Periodic configured resource checks
|
||||||
|
|
||||||
|
`managed monitor --resource-config FILE` runs resource-config v1 periodically.
|
||||||
|
PID 1 selects the same mode when `ZAPRET_AUTOTUNE_RESOURCE_CONFIG` is nonempty;
|
||||||
|
unset preserves the existing YouTube monitor. The modes share one singleton
|
||||||
|
lock. Supply an absolute container-visible config path (for example mount a
|
||||||
|
read-only operator config directory using a Compose override). Configuration
|
||||||
|
must be a regular root/runtime-owned, non-writable-by-others file without
|
||||||
|
symlink/hardlink aliases. Environment references are resolved on every reload.
|
||||||
|
|
||||||
|
Each resource accepts `interval_ms` (default 300000) and `cooldown_ms` (default
|
||||||
|
30000), both integers in [100, 86400000]. Checks are sequential round robin
|
||||||
|
with independent due times; `tls: both` requires both passes. Ordinary DNS,
|
||||||
|
TLS, timeout or expectation failures mark only that resource DEGRADED and
|
||||||
|
retry after its cooldown. There is **no strategy selection, recovery or apply**,
|
||||||
|
and no no-strategy two-hour pause. HEALTHY means configured checks passed,
|
||||||
|
not playback health, generic recovery or whole-production READY: HEALTHY is
|
||||||
|
**CHECKS_ONLY**, never READY. The scheduling, status, persistence and lifecycle
|
||||||
|
base passed native full/race checks, independent security review and isolated
|
||||||
|
remote fault acceptance for `4efc066` plus ownership-status fix `5c85f3f`.
|
||||||
|
See [PROGRESS.md](PROGRESS.md) for supplied evidence and its narrow scope.
|
||||||
|
Version1 is bounded to 16 services / 64 resources with stable configured IDs;
|
||||||
|
valid configuration/state does not imply an automatic strategy assignment.
|
||||||
|
|
||||||
|
Invalid resource JSON initially or after reload is BLOCKED, with no DNS or
|
||||||
|
transfers until repaired. Bad static target role, proxy or DNS settings still
|
||||||
|
fail startup validation; resource JSON is deliberately not part of
|
||||||
|
`target-config-check`. Ownership and fresh source snapshots are checked before
|
||||||
|
DNS and again before transfer; existing DNS/IP policy, pinned SOCKS transfers,
|
||||||
|
TLS verification and private stdin are unchanged. No cached addresses bypass
|
||||||
|
DNS policy. Existing zapret reconciliation remains mandatory and generic
|
||||||
|
startup never consumes experimental apply requests.
|
||||||
|
|
||||||
|
State is independent of recovery SQL leases, fixed in the private target
|
||||||
|
directory as `.resource-monitor-state.json` and `.resource-monitor-key`.
|
||||||
|
Both are private 0600 files. Fingerprints HMAC the resolved configuration
|
||||||
|
(including secrets); state is also authenticated and never contains URLs,
|
||||||
|
headers, addresses or raw errors. Missing state initializes UNKNOWN. A missing
|
||||||
|
key with existing state, corrupt or unsafe file is preserved and BLOCKED,
|
||||||
|
never silently replaced. Restore the original key and a valid version-1 signed
|
||||||
|
state from a trusted backup to repair it. The controller holds directory, key
|
||||||
|
and state inode anchors: an identical signed replacement is still BLOCKED and
|
||||||
|
preserved. Repair on the original held inode is reloaded only after version/MAC
|
||||||
|
validation; replacing an inode requires deliberate operator repair/restart.
|
||||||
|
A persistence failure pauses probes;
|
||||||
|
deliberate repair requires a different valid authenticated snapshot, not just
|
||||||
|
retrying a failed write. Never chmod/adopt unknown files to bypass validation.
|
||||||
|
This is not isolation from a hostile same-UID actor: that actor can read the
|
||||||
|
HMAC key and race filesystem publication; root/container administrators can
|
||||||
|
also read private data. HMAC fingerprints protect stored secret values from
|
||||||
|
unkeyed guessing, not from an actor with access to the key.
|
||||||
|
|
||||||
|
`managed resource-status` is read-only and reports BLOCKED or STALE explicitly
|
||||||
|
(a check overdue by >65s, unknown state age >65s, or changed configuration).
|
||||||
|
Idle reloads do not write a disk heartbeat: only configured state changes,
|
||||||
|
BLOCKED transitions and completed checks are flushed. Cancelled checks join
|
||||||
|
their children but do not change prior health, counts or completion/due times.
|
||||||
|
It reads the config path from
|
||||||
|
`ZAPRET_AUTOTUNE_RESOURCE_CONFIG`; use that environment setting also for manual
|
||||||
|
monitor/status invocations. Status rows report only configured-check health.
|
||||||
|
|
||||||
|
TERM cancels/joins transfers before finalizing a completed snapshot and exits
|
||||||
|
0 on safe cleanup. PID 1 retains DNS through monitor/SS cleanup. In generic
|
||||||
|
mode transient child restart-budget exhaustion pauses until the rolling window
|
||||||
|
opens, rather than terminating the container. Unknown/unsafe cleanup stays
|
||||||
|
BLOCKED pending operator repair/restart; no unsafe daemon sourcing is allowed.
|
||||||
|
Generic cleanup shares an 80-second budget, including watchdog setup and
|
||||||
|
command kill escalation, with bounded final reaping; it is designed to fit
|
||||||
|
within a 90-second Docker stop limit (the supplied Compose grants 240 seconds).
|
||||||
|
Bounded final reaps/worst-case cleanup were verified offline; measured remote
|
||||||
|
normal stops were 1.45s, 0.59s and 0.57s, not an 80-second runtime worst-case test.
|
||||||
|
These changes do not claim to fix legacy YouTube exit 78.
|
||||||
@@ -177,27 +177,36 @@ func validateAssetTransfers(m assetMeasurement) (float64, float64, error) {
|
|||||||
return latency[1], speed[1], nil
|
return latency[1], speed[1], nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Only supplied body bytes can produce this typed rejection. Missing/corrupt
|
||||||
|
// normalized evidence remains a local format error, never candidate absence.
|
||||||
|
type imageBodyRejection string
|
||||||
|
|
||||||
|
func (e imageBodyRejection) Error() string { return string(e) }
|
||||||
|
|
||||||
// Bodies are transient. Stored evidence carries only verified normalized facts.
|
// Bodies are transient. Stored evidence carries only verified normalized facts.
|
||||||
func normalizeAssetBody(t *assetTransfer) error {
|
func normalizeAssetBody(t *assetTransfer) error {
|
||||||
if len(t.Body) != 0 {
|
if len(t.Body) != 0 {
|
||||||
if len(t.Body) > 5<<20 {
|
if len(t.Body) > 5<<20 {
|
||||||
return fmt.Errorf("image body exceeds limit")
|
return imageBodyRejection("image body exceeds limit")
|
||||||
}
|
}
|
||||||
cfg, format, err := image.DecodeConfig(bytes.NewReader(t.Body))
|
cfg, format, err := image.DecodeConfig(bytes.NewReader(t.Body))
|
||||||
if err != nil || cfg.Width <= 0 || cfg.Height <= 0 || cfg.Width > (16<<20)/cfg.Height {
|
if err != nil || cfg.Width <= 0 || cfg.Height <= 0 || cfg.Width > (16<<20)/cfg.Height {
|
||||||
return fmt.Errorf("invalid image dimensions")
|
return imageBodyRejection("invalid image dimensions")
|
||||||
}
|
}
|
||||||
if format == "gif" {
|
if format == "gif" {
|
||||||
if _, err := gif.DecodeAll(bytes.NewReader(t.Body)); err != nil {
|
if _, err := gif.DecodeAll(bytes.NewReader(t.Body)); err != nil {
|
||||||
return fmt.Errorf("invalid GIF body")
|
return imageBodyRejection("invalid GIF body")
|
||||||
}
|
}
|
||||||
} else if _, _, err := image.Decode(bytes.NewReader(t.Body)); err != nil {
|
} else if _, _, err := image.Decode(bytes.NewReader(t.Body)); err != nil {
|
||||||
return fmt.Errorf("invalid image body")
|
return imageBodyRejection("invalid image body")
|
||||||
}
|
}
|
||||||
t.Format, t.Size, t.BodyHash = format, len(t.Body), fmt.Sprintf("%x", sha256.Sum256(t.Body))
|
t.Format, t.Size, t.BodyHash = format, len(t.Body), fmt.Sprintf("%x", sha256.Sum256(t.Body))
|
||||||
}
|
}
|
||||||
want := map[string]string{"png": "image/png", "jpeg": "image/jpeg", "gif": "image/gif"}[t.Format]
|
want := map[string]string{"png": "image/png", "jpeg": "image/jpeg", "gif": "image/gif"}[t.Format]
|
||||||
if want == "" || t.ContentType != want || t.Size <= 0 || t.Size > 5<<20 || len(t.BodyHash) != 64 || strings.Trim(t.BodyHash, "0123456789abcdef") != "" {
|
if want == "" || t.ContentType != want || t.Size <= 0 || t.Size > 5<<20 || len(t.BodyHash) != 64 || strings.Trim(t.BodyHash, "0123456789abcdef") != "" {
|
||||||
|
if len(t.Body) != 0 {
|
||||||
|
return imageBodyRejection("invalid image content type")
|
||||||
|
}
|
||||||
return fmt.Errorf("invalid normalized image evidence")
|
return fmt.Errorf("invalid normalized image evidence")
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
|
|||||||
@@ -33,7 +33,7 @@ func parseHealthConfig(lookup func(string) (string, bool)) (healthConfig, error)
|
|||||||
return healthConfig{}, err
|
return healthConfig{}, err
|
||||||
}
|
}
|
||||||
if !validLoopbackSOCKS(cfg.SOCKSProxy) {
|
if !validLoopbackSOCKS(cfg.SOCKSProxy) {
|
||||||
return healthConfig{}, fmt.Errorf("ZAPRET_AUTOTUNE_SOCKS_PROXY requires socks5://LOOPBACK:PORT without credentials")
|
return healthConfig{}, fmt.Errorf("ZAPRET_AUTOTUNE_SOCKS_PROXY requires socks5:// or socks5h://LOOPBACK:PORT without credentials")
|
||||||
}
|
}
|
||||||
if !videoIDPattern.MatchString(cfg.VideoID) {
|
if !videoIDPattern.MatchString(cfg.VideoID) {
|
||||||
return healthConfig{}, fmt.Errorf("ZAPRET_AUTOTUNE_VIDEO_ID requires an 11-character video ID")
|
return healthConfig{}, fmt.Errorf("ZAPRET_AUTOTUNE_VIDEO_ID requires an 11-character video ID")
|
||||||
@@ -75,9 +75,17 @@ func managedGooglevideoConfig(lookup func(string) (string, bool), primary string
|
|||||||
if mode == QualificationAdaptive && !service {
|
if mode == QualificationAdaptive && !service {
|
||||||
return "", nil, fmt.Errorf("adaptive googlevideo requires ZAPRET_AUTOTUNE_YOUTUBE_SERVICE=1")
|
return "", nil, fmt.Errorf("adaptive googlevideo requires ZAPRET_AUTOTUNE_YOUTUBE_SERVICE=1")
|
||||||
}
|
}
|
||||||
// Only the configured primary is sampled by default; no implicit public playlist.
|
|
||||||
ids := []string{primary}
|
ids := []string{primary}
|
||||||
if raw, ok := lookup("ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS"); ok {
|
// YouTube-specific public discovery samples, not CDN addresses. Explicit
|
||||||
|
// inventories below override these defaults; health/discovery primary stays put.
|
||||||
|
if service {
|
||||||
|
for _, id := range []string{"aqz-KE-bpKQ", "jNQXAC9IVRw"} {
|
||||||
|
if id != primary {
|
||||||
|
ids = append(ids, id)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if raw, ok := lookup("ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS"); ok && raw != "" {
|
||||||
if json.Unmarshal([]byte(raw), &ids) != nil || len(ids) == 0 || len(ids) > 5 {
|
if json.Unmarshal([]byte(raw), &ids) != nil || len(ids) == 0 || len(ids) > 5 {
|
||||||
return "", nil, fmt.Errorf("GOOGLEVIDEO_VIDEO_IDS requires a JSON array of 1..5 video IDs")
|
return "", nil, fmt.Errorf("GOOGLEVIDEO_VIDEO_IDS requires a JSON array of 1..5 video IDs")
|
||||||
}
|
}
|
||||||
@@ -104,13 +112,45 @@ func managedServiceEnabled(lookup func(string) (string, bool)) (bool, error) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func validLoopbackSOCKS(raw string) bool {
|
func validLoopbackSOCKS(raw string) bool {
|
||||||
if !validDiscoveryProxy(raw) {
|
u, err := url.Parse(raw)
|
||||||
|
if err != nil || (u.Scheme != "socks5" && u.Scheme != "socks5h") || strings.IndexFunc(raw, func(r rune) bool { return r < 32 || r == 127 }) >= 0 {
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
u, err := url.Parse(raw)
|
if !validDiscoveryProxy(strings.Replace(raw, "socks5h://", "socks5://", 1)) {
|
||||||
if err != nil {
|
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
host := u.Hostname()
|
host := u.Hostname()
|
||||||
return strings.EqualFold(host, "localhost") || net.ParseIP(host).IsLoopback()
|
return strings.EqualFold(host, "localhost") || net.ParseIP(host).IsLoopback()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Managed probes send hostnames through the owned SS listener, not local DNS.
|
||||||
|
// Public health preserves the explicit scheme; public discovery keeps socks5.
|
||||||
|
func managedProbeProxy(raw string) (string, error) {
|
||||||
|
u, err := url.Parse(raw)
|
||||||
|
if err != nil || (u.Scheme != "socks5" && u.Scheme != "socks5h") || strings.IndexFunc(raw, func(r rune) bool { return r < 32 || r == 127 }) >= 0 {
|
||||||
|
return "", fmt.Errorf("invalid managed SOCKS proxy")
|
||||||
|
}
|
||||||
|
if !validLoopbackSOCKS(strings.Replace(raw, "socks5h://", "socks5://", 1)) {
|
||||||
|
return "", fmt.Errorf("invalid managed SOCKS proxy")
|
||||||
|
}
|
||||||
|
u.Scheme = "socks5h"
|
||||||
|
return u.String(), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseManagedHealthConfig(lookup func(string) (string, bool)) (healthConfig, error) {
|
||||||
|
// Reuse all legacy setting validation without changing its public contract.
|
||||||
|
cfg, err := parseHealthConfig(func(key string) (string, bool) {
|
||||||
|
raw, ok := lookup(key)
|
||||||
|
if key == "ZAPRET_AUTOTUNE_SOCKS_PROXY" && ok {
|
||||||
|
if proxy, e := managedProbeProxy(raw); e == nil {
|
||||||
|
return strings.Replace(proxy, "socks5h://", "socks5://", 1), true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return raw, ok
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
return cfg, err
|
||||||
|
}
|
||||||
|
cfg.SOCKSProxy, err = managedProbeProxy(cfg.SOCKSProxy)
|
||||||
|
return cfg, err
|
||||||
|
}
|
||||||
@@ -0,0 +1,676 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"crypto/tls"
|
||||||
|
"encoding/binary"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"mime"
|
||||||
|
"net"
|
||||||
|
"net/http"
|
||||||
|
"os"
|
||||||
|
"os/signal"
|
||||||
|
"path/filepath"
|
||||||
|
"strconv"
|
||||||
|
"sync"
|
||||||
|
"sync/atomic"
|
||||||
|
"syscall"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
const dnsSocketMark = 0x50000000 // Exclude both deployed (0x40000000) and probe (0x10000000) queues.
|
||||||
|
const dnsMaxQuery = 4096
|
||||||
|
const dnsConcurrency = 16
|
||||||
|
const dnsMaxUDPPayload = 65507 // Fixed IPv4 listener: maximum UDP payload, excluding IP/UDP headers.
|
||||||
|
|
||||||
|
var errDNSForwarder = errors.New("DNS forwarder unavailable")
|
||||||
|
|
||||||
|
func markDNSSocket(_, _ string, raw syscall.RawConn) error {
|
||||||
|
return setDNSSocketMark(raw, syscall.SetsockoptInt)
|
||||||
|
}
|
||||||
|
|
||||||
|
func setDNSSocketMark(raw syscall.RawConn, set func(int, int, int, int) error) error {
|
||||||
|
var markErr error
|
||||||
|
err := raw.Control(func(fd uintptr) { markErr = set(int(fd), syscall.SOL_SOCKET, syscall.SO_MARK, dnsSocketMark) })
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return markErr
|
||||||
|
}
|
||||||
|
|
||||||
|
type dnsDialFunc func(context.Context, string, string) (net.Conn, error)
|
||||||
|
|
||||||
|
func dnsBootstrapDial(cfg managedDNSConfig, dial dnsDialFunc) dnsDialFunc {
|
||||||
|
var next atomic.Uint64
|
||||||
|
return func(ctx context.Context, network, address string) (net.Conn, error) {
|
||||||
|
if network != "tcp" || address != net.JoinHostPort(cfg.Host, "443") || len(cfg.BootstrapIPs) == 0 {
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
}
|
||||||
|
ctx, cancelAll := context.WithTimeout(ctx, cfg.Timeout)
|
||||||
|
defer cancelAll()
|
||||||
|
start := int((next.Add(1) - 1) % uint64(len(cfg.BootstrapIPs)))
|
||||||
|
for i := range cfg.BootstrapIPs {
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
// Give every bootstrap a bounded share of the remaining deadline.
|
||||||
|
remaining := cfg.Timeout
|
||||||
|
if deadline, ok := ctx.Deadline(); ok {
|
||||||
|
remaining = time.Until(deadline)
|
||||||
|
}
|
||||||
|
attempt, cancel := context.WithTimeout(ctx, remaining/time.Duration(len(cfg.BootstrapIPs)-i))
|
||||||
|
conn, err := dial(attempt, "tcp", net.JoinHostPort(cfg.BootstrapIPs[(start+i)%len(cfg.BootstrapIPs)].String(), "443"))
|
||||||
|
cancel()
|
||||||
|
if err == nil {
|
||||||
|
return conn, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func newDNSHTTPClient(cfg managedDNSConfig, control func(string, string, syscall.RawConn) error) (*http.Client, *http.Transport) {
|
||||||
|
dialer := &net.Dialer{Timeout: cfg.Timeout, Control: control}
|
||||||
|
transport := &http.Transport{
|
||||||
|
Proxy: nil, DialContext: dnsBootstrapDial(cfg, dialer.DialContext),
|
||||||
|
TLSClientConfig: &tls.Config{MinVersion: tls.VersionTLS12, ServerName: cfg.Host},
|
||||||
|
TLSHandshakeTimeout: cfg.Timeout, ResponseHeaderTimeout: cfg.Timeout,
|
||||||
|
DisableCompression: true, MaxConnsPerHost: dnsConcurrency,
|
||||||
|
MaxIdleConns: dnsConcurrency, MaxIdleConnsPerHost: dnsConcurrency,
|
||||||
|
IdleConnTimeout: 30 * time.Second, MaxResponseHeaderBytes: 8192,
|
||||||
|
}
|
||||||
|
return &http.Client{Transport: transport, Timeout: cfg.Timeout, CheckRedirect: func(*http.Request, []*http.Request) error { return errDNSForwarder }}, transport
|
||||||
|
}
|
||||||
|
|
||||||
|
// DNS names may use compression. Bound both pointer traversal and expanded name size.
|
||||||
|
func dnsNameEnd(wire []byte, start int) (int, bool) {
|
||||||
|
end, _, ok := dnsName(wire, start)
|
||||||
|
return end, ok
|
||||||
|
}
|
||||||
|
|
||||||
|
func dnsName(wire []byte, start int) (int, []byte, bool) {
|
||||||
|
pos, end, expanded := start, -1, 0
|
||||||
|
var name []byte
|
||||||
|
for steps := 0; steps < 256; steps++ {
|
||||||
|
if pos < 12 || pos >= len(wire) {
|
||||||
|
return 0, nil, false
|
||||||
|
}
|
||||||
|
n := int(wire[pos])
|
||||||
|
pos++
|
||||||
|
if n == 0 {
|
||||||
|
if end < 0 {
|
||||||
|
end = pos
|
||||||
|
}
|
||||||
|
return end, append(name, 0), true
|
||||||
|
}
|
||||||
|
if n&0xc0 == 0xc0 {
|
||||||
|
if pos >= len(wire) {
|
||||||
|
return 0, nil, false
|
||||||
|
}
|
||||||
|
target := (n&0x3f)<<8 | int(wire[pos])
|
||||||
|
if target < 12 || target >= len(wire) {
|
||||||
|
return 0, nil, false
|
||||||
|
}
|
||||||
|
if end < 0 {
|
||||||
|
end = pos + 1
|
||||||
|
}
|
||||||
|
pos = target
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if n > 63 || pos+n >= len(wire) {
|
||||||
|
return 0, nil, false
|
||||||
|
}
|
||||||
|
expanded += n + 1
|
||||||
|
if expanded > 254 {
|
||||||
|
return 0, nil, false
|
||||||
|
}
|
||||||
|
name = append(name, byte(n))
|
||||||
|
name = append(name, wire[pos:pos+n]...)
|
||||||
|
pos += n
|
||||||
|
}
|
||||||
|
return 0, nil, false
|
||||||
|
}
|
||||||
|
|
||||||
|
func dnsQuestionEnd(wire []byte) (int, bool) {
|
||||||
|
end, _, ok := dnsQuestion(wire)
|
||||||
|
return end, ok
|
||||||
|
}
|
||||||
|
|
||||||
|
func dnsQuestion(wire []byte) (int, []byte, bool) {
|
||||||
|
if len(wire) < 12 || binary.BigEndian.Uint16(wire[4:6]) != 1 {
|
||||||
|
return 0, nil, false
|
||||||
|
}
|
||||||
|
end, name, ok := dnsName(wire, 12)
|
||||||
|
if !ok || end+4 > len(wire) {
|
||||||
|
return 0, nil, false
|
||||||
|
}
|
||||||
|
if binary.BigEndian.Uint16(wire[end:end+2]) == 0 || binary.BigEndian.Uint16(wire[end+2:end+4]) == 0 {
|
||||||
|
return 0, nil, false
|
||||||
|
}
|
||||||
|
return end + 4, append(name, wire[end:end+4]...), true
|
||||||
|
}
|
||||||
|
|
||||||
|
func validDNSQuery(wire []byte) bool {
|
||||||
|
if len(wire) < 12 || len(wire) > dnsMaxQuery || wire[2]&0xfe != 0 || wire[3]&0xcf != 0 || binary.BigEndian.Uint16(wire[6:8]) != 0 || binary.BigEndian.Uint16(wire[8:10]) != 0 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
end, ok := dnsQuestionEnd(wire)
|
||||||
|
if !ok {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
_, ok = dnsQueryUDPLimit(wire, end)
|
||||||
|
return ok
|
||||||
|
}
|
||||||
|
|
||||||
|
// Query additional records have the same framing as answers. Only OPT affects
|
||||||
|
// downstream UDP sizing; its root owner and option framing must be valid.
|
||||||
|
func dnsQueryUDPLimit(wire []byte, end int) (int, bool) {
|
||||||
|
limit, seenOPT := 512, false
|
||||||
|
for count := int(binary.BigEndian.Uint16(wire[10:12])); count > 0; count-- {
|
||||||
|
header, name, ok := dnsName(wire, end)
|
||||||
|
if !ok || header+10 > len(wire) {
|
||||||
|
return 0, false
|
||||||
|
}
|
||||||
|
end = header + 10 + int(binary.BigEndian.Uint16(wire[header+8:header+10]))
|
||||||
|
if end > len(wire) {
|
||||||
|
return 0, false
|
||||||
|
}
|
||||||
|
if binary.BigEndian.Uint16(wire[header:header+2]) != 41 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if seenOPT || len(name) != 1 || wire[header+4] != 0 || wire[header+5] != 0 || binary.BigEndian.Uint16(wire[header+6:header+8])&0x7fff != 0 {
|
||||||
|
return 0, false
|
||||||
|
}
|
||||||
|
seenOPT = true
|
||||||
|
limit = int(binary.BigEndian.Uint16(wire[header+2 : header+4]))
|
||||||
|
if limit < 512 {
|
||||||
|
limit = 512
|
||||||
|
}
|
||||||
|
if limit > dnsMaxUDPPayload {
|
||||||
|
limit = dnsMaxUDPPayload
|
||||||
|
}
|
||||||
|
for option := header + 10; option < end; {
|
||||||
|
if option+4 > end {
|
||||||
|
return 0, false
|
||||||
|
}
|
||||||
|
option += 4 + int(binary.BigEndian.Uint16(wire[option+2:option+4]))
|
||||||
|
if option > end {
|
||||||
|
return 0, false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return limit, end == len(wire)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Validate only record boundaries (including EDNS/DNSSEC), never decode or rewrite RDATA.
|
||||||
|
func validDNSRecordFraming(wire []byte, end, count int) bool {
|
||||||
|
for ; count > 0; count-- {
|
||||||
|
var ok bool
|
||||||
|
end, ok = dnsNameEnd(wire, end)
|
||||||
|
if !ok || end+10 > len(wire) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
end += 10 + int(binary.BigEndian.Uint16(wire[end+8:end+10]))
|
||||||
|
if end > len(wire) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return end == len(wire)
|
||||||
|
}
|
||||||
|
|
||||||
|
func validDNSResponse(query, response []byte) bool {
|
||||||
|
if len(query) < 12 || len(response) < 12 || len(response) > 65535 || response[2]&0x80 == 0 || response[2]&0x78 != query[2]&0x78 || !bytes.Equal(query[:2], response[:2]) || response[3]&0x40 != 0 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
_, question, qok := dnsQuestion(query)
|
||||||
|
re, responseQuestion, rok := dnsQuestion(response)
|
||||||
|
if !qok || !rok || !bytes.Equal(question, responseQuestion) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
count := int(binary.BigEndian.Uint16(response[6:8])) + int(binary.BigEndian.Uint16(response[8:10])) + int(binary.BigEndian.Uint16(response[10:12]))
|
||||||
|
return validDNSRecordFraming(response, re, count)
|
||||||
|
}
|
||||||
|
|
||||||
|
func dnsServfail(query []byte) []byte {
|
||||||
|
if !validDNSQuery(query) {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
reply := dnsQuestionReply(query, query)
|
||||||
|
reply[2] = 0x80 | query[2]&0x79 // QR, opcode and RD, not AA/TC.
|
||||||
|
reply[3] = 0x02 // SERVFAIL, no fabricated recursion/DNSSEC assertions.
|
||||||
|
return reply
|
||||||
|
}
|
||||||
|
|
||||||
|
// Re-encode the question without pointers into records omitted from this reply.
|
||||||
|
func dnsQuestionReply(query, header []byte) []byte {
|
||||||
|
_, question, ok := dnsQuestion(query)
|
||||||
|
if !ok || len(header) < 12 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
reply := append([]byte(nil), header[:12]...)
|
||||||
|
clear(reply[6:12])
|
||||||
|
return append(reply, question...)
|
||||||
|
}
|
||||||
|
|
||||||
|
func dnsUDPReply(query, response []byte) []byte {
|
||||||
|
if response == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
end, ok := dnsQuestionEnd(query)
|
||||||
|
if !ok {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
limit, ok := dnsQueryUDPLimit(query, end)
|
||||||
|
if !ok {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if len(response) <= limit {
|
||||||
|
return response
|
||||||
|
}
|
||||||
|
reply := dnsQuestionReply(query, response)
|
||||||
|
if reply != nil {
|
||||||
|
reply[2] |= 0x82 // QR and TC; preserve upstream opcode, RD, AA and other flags.
|
||||||
|
reply[3] &^= 0x40 // Reserved Z bit; no records/assertions are invented.
|
||||||
|
}
|
||||||
|
return reply
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeDNSDatagram(udp net.PacketConn, addr net.Addr, query, response []byte) {
|
||||||
|
if reply := dnsUDPReply(query, response); reply != nil {
|
||||||
|
if _, err := udp.WriteTo(reply, addr); err != nil {
|
||||||
|
return
|
||||||
|
} // Drop only this datagram; never retry plaintext or stop the server.
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type dnsForwarder struct {
|
||||||
|
cfg managedDNSConfig
|
||||||
|
client *http.Client
|
||||||
|
slots chan struct{}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *dnsForwarder) exchange(ctx context.Context, query []byte) []byte {
|
||||||
|
if !validDNSQuery(query) {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithTimeout(ctx, f.cfg.Timeout)
|
||||||
|
defer cancel()
|
||||||
|
req, err := http.NewRequestWithContext(ctx, http.MethodPost, f.cfg.URL, bytes.NewReader(query))
|
||||||
|
if err != nil {
|
||||||
|
return dnsServfail(query)
|
||||||
|
}
|
||||||
|
req.Header.Set("Content-Type", "application/dns-message")
|
||||||
|
req.Header.Set("Accept", "application/dns-message")
|
||||||
|
resp, err := f.client.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
return dnsServfail(query)
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
typ, _, err := mime.ParseMediaType(resp.Header.Get("Content-Type"))
|
||||||
|
if err != nil || typ != "application/dns-message" || resp.StatusCode != http.StatusOK || resp.Header.Get("Content-Encoding") != "" {
|
||||||
|
return dnsServfail(query)
|
||||||
|
}
|
||||||
|
body, err := io.ReadAll(io.LimitReader(resp.Body, 65536))
|
||||||
|
if err != nil || resp.ContentLength >= 0 && resp.ContentLength != int64(len(body)) || !validDNSResponse(query, body) {
|
||||||
|
return dnsServfail(query)
|
||||||
|
}
|
||||||
|
return body
|
||||||
|
}
|
||||||
|
|
||||||
|
// One connection owns one request slot; frames are sequential, capped in number
|
||||||
|
// and total lifetime, so slow/idle clients cannot create unbounded workers.
|
||||||
|
func (f *dnsForwarder) tcpWorker(ctx context.Context, conn net.Conn) {
|
||||||
|
defer conn.Close()
|
||||||
|
stop := context.AfterFunc(ctx, func() { conn.Close() })
|
||||||
|
defer stop()
|
||||||
|
lifetime := time.Now().Add(16 * f.cfg.Timeout)
|
||||||
|
for frame := 0; frame < 16; frame++ {
|
||||||
|
deadline := time.Now().Add(f.cfg.Timeout)
|
||||||
|
if lifetime.Before(deadline) {
|
||||||
|
deadline = lifetime
|
||||||
|
}
|
||||||
|
if err := conn.SetReadDeadline(deadline); err != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
var size [2]byte
|
||||||
|
if _, err := io.ReadFull(conn, size[:]); err != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
n := int(binary.BigEndian.Uint16(size[:]))
|
||||||
|
if n < 12 || n > dnsMaxQuery {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
query := make([]byte, n)
|
||||||
|
if _, err := io.ReadFull(conn, query); err != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if !time.Now().Before(lifetime) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
requestCtx, cancel := context.WithDeadline(ctx, lifetime)
|
||||||
|
reply := f.exchange(requestCtx, query)
|
||||||
|
cancel()
|
||||||
|
if reply == nil || !time.Now().Before(lifetime) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
// An upstream timeout must still leave a bounded window to send SERVFAIL.
|
||||||
|
deadline = time.Now().Add(f.cfg.Timeout)
|
||||||
|
if lifetime.Before(deadline) {
|
||||||
|
deadline = lifetime
|
||||||
|
}
|
||||||
|
if err := conn.SetWriteDeadline(deadline); err != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
binary.BigEndian.PutUint16(size[:], uint16(len(reply)))
|
||||||
|
if err := writeDNSFrame(conn, size[:], reply); err != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func writeDNSFrame(w io.Writer, size, reply []byte) error {
|
||||||
|
for _, part := range [][]byte{size, reply} {
|
||||||
|
for len(part) > 0 {
|
||||||
|
n, err := w.Write(part)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if n == 0 {
|
||||||
|
return io.ErrShortWrite
|
||||||
|
}
|
||||||
|
part = part[n:]
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (f *dnsForwarder) run(ctx context.Context, udp net.PacketConn, tcp net.Listener, ready func() (func(), error)) error {
|
||||||
|
ctx, cancel := context.WithCancel(ctx)
|
||||||
|
defer cancel()
|
||||||
|
stop := context.AfterFunc(ctx, func() { udp.Close(); tcp.Close() })
|
||||||
|
defer stop()
|
||||||
|
defer udp.Close()
|
||||||
|
defer tcp.Close()
|
||||||
|
cleanup, err := ready()
|
||||||
|
if err != nil {
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
defer cleanup()
|
||||||
|
var workers sync.WaitGroup
|
||||||
|
var loops sync.WaitGroup
|
||||||
|
var failed atomic.Bool
|
||||||
|
loops.Add(2)
|
||||||
|
go func() {
|
||||||
|
defer loops.Done()
|
||||||
|
buffer := make([]byte, dnsMaxQuery+1)
|
||||||
|
for {
|
||||||
|
n, addr, err := udp.ReadFrom(buffer)
|
||||||
|
if err != nil {
|
||||||
|
if ctx.Err() == nil {
|
||||||
|
failed.Store(true)
|
||||||
|
cancel()
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if !validDNSQuery(buffer[:n]) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
select {
|
||||||
|
case f.slots <- struct{}{}:
|
||||||
|
query := append([]byte(nil), buffer[:n]...)
|
||||||
|
workers.Add(1)
|
||||||
|
go func() {
|
||||||
|
defer workers.Done()
|
||||||
|
defer func() { <-f.slots }()
|
||||||
|
if reply := f.exchange(ctx, query); reply != nil {
|
||||||
|
writeDNSDatagram(udp, addr, query, reply)
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
default:
|
||||||
|
writeDNSDatagram(udp, addr, buffer[:n], dnsServfail(buffer[:n]))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
go func() {
|
||||||
|
defer loops.Done()
|
||||||
|
for {
|
||||||
|
conn, err := tcp.Accept()
|
||||||
|
if err != nil {
|
||||||
|
if ctx.Err() == nil {
|
||||||
|
failed.Store(true)
|
||||||
|
cancel()
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
select {
|
||||||
|
case f.slots <- struct{}{}:
|
||||||
|
workers.Add(1)
|
||||||
|
go func() { defer workers.Done(); defer func() { <-f.slots }(); f.tcpWorker(ctx, conn) }()
|
||||||
|
default:
|
||||||
|
conn.Close()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
loops.Wait()
|
||||||
|
workers.Wait()
|
||||||
|
if failed.Load() {
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func createDNSReady(path string) (func(), error) {
|
||||||
|
dir := filepath.Dir(path)
|
||||||
|
if err := managedDirectory(dir); err != nil {
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
}
|
||||||
|
info, err := os.Lstat(dir)
|
||||||
|
if err != nil || info.Mode().Perm() != 0700 || info.Mode()&(os.ModeSetuid|os.ModeSetgid|os.ModeSticky) != 0 {
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
}
|
||||||
|
fd, err := syscall.Open(dir, syscall.O_RDONLY|syscall.O_DIRECTORY|syscall.O_NOFOLLOW|syscall.O_CLOEXEC, 0)
|
||||||
|
if err != nil {
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
}
|
||||||
|
parent := os.NewFile(uintptr(fd), dir)
|
||||||
|
opened, err := parent.Stat()
|
||||||
|
if err != nil || !os.SameFile(info, opened) {
|
||||||
|
parent.Close()
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
}
|
||||||
|
heldDir := fmt.Sprintf("/proc/self/fd/%d", fd)
|
||||||
|
file, err := os.CreateTemp(heldDir, ".dns-ready-")
|
||||||
|
if err != nil {
|
||||||
|
parent.Close()
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
}
|
||||||
|
owned, statErr := file.Stat()
|
||||||
|
if statErr == nil && owned.Mode().Perm() != 0600 {
|
||||||
|
statErr = errDNSForwarder
|
||||||
|
}
|
||||||
|
var once sync.Once
|
||||||
|
cleanup := func() {
|
||||||
|
once.Do(func() {
|
||||||
|
// Inspect and unlink relative to the held private parent, not a replacement directory.
|
||||||
|
for _, name := range []string{filepath.Base(path), filepath.Base(file.Name())} {
|
||||||
|
current, e := os.Lstat(filepath.Join(heldDir, name))
|
||||||
|
if e == nil && owned != nil && os.SameFile(owned, current) {
|
||||||
|
syscall.Unlinkat(fd, name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
parent.Close()
|
||||||
|
// Hold the inode until all identity checks/unlinks finish, preventing reuse.
|
||||||
|
file.Close()
|
||||||
|
})
|
||||||
|
}
|
||||||
|
if statErr == nil {
|
||||||
|
err = publishDNSReady(file, filepath.Join(heldDir, filepath.Base(path)))
|
||||||
|
}
|
||||||
|
if statErr != nil || err != nil {
|
||||||
|
cleanup()
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
}
|
||||||
|
if err := syscall.Unlinkat(fd, filepath.Base(file.Name())); err != nil {
|
||||||
|
cleanup()
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
}
|
||||||
|
return cleanup, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func publishDNSReady(file *os.File, destination string) error {
|
||||||
|
if _, err := fmt.Fprintf(file, "%d\n", os.Getpid()); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := file.Sync(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return os.Link(file.Name(), destination) // Atomic complete-PID publication, EXCL even for a symlink destination.
|
||||||
|
}
|
||||||
|
|
||||||
|
func serveDNSForwarder(ctx context.Context, cfg managedDNSConfig) error {
|
||||||
|
udp, err := net.ListenPacket("udp4", dnsListenAddress)
|
||||||
|
if err != nil {
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
tcp, err := net.Listen("tcp4", dnsListenAddress)
|
||||||
|
if err != nil {
|
||||||
|
udp.Close()
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
client, transport := newDNSHTTPClient(cfg, markDNSSocket)
|
||||||
|
defer transport.CloseIdleConnections()
|
||||||
|
f := &dnsForwarder{cfg: cfg, client: client, slots: make(chan struct{}, dnsConcurrency)}
|
||||||
|
return f.run(ctx, udp, tcp, func() (func(), error) { return createDNSReady(dnsReadyPath) })
|
||||||
|
}
|
||||||
|
|
||||||
|
func dnsForwarderCLI(args []string, stdout, stderr io.Writer) int {
|
||||||
|
if len(args) > 0 && (args[0] == "readiness-check" || args[0] == "readiness-cleanup") {
|
||||||
|
return dnsReadinessCLI(args, stderr)
|
||||||
|
}
|
||||||
|
if len(args) != 1 || (args[0] != "mode" && args[0] != "address" && args[0] != "serve") {
|
||||||
|
fmt.Fprintln(stderr, "configerror: invalid DNS forwarder command")
|
||||||
|
return 64
|
||||||
|
}
|
||||||
|
cfg, err := parseManagedDNSConfig(os.LookupEnv)
|
||||||
|
if err != nil || args[0] == "serve" && cfg.Mode != "doh" {
|
||||||
|
fmt.Fprintln(stderr, "configerror: invalid managed DNS configuration")
|
||||||
|
return 64
|
||||||
|
}
|
||||||
|
switch args[0] {
|
||||||
|
case "mode":
|
||||||
|
_, err = fmt.Fprintln(stdout, cfg.Mode)
|
||||||
|
case "address":
|
||||||
|
_, err = fmt.Fprintln(stdout, cfg.serverAddress())
|
||||||
|
case "serve":
|
||||||
|
ctx, stop := signal.NotifyContext(context.Background(), syscall.SIGTERM, syscall.SIGINT)
|
||||||
|
defer stop()
|
||||||
|
err = serveDNSForwarder(ctx, cfg)
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintln(stderr, "error: DNS forwarder unavailable")
|
||||||
|
return 75
|
||||||
|
}
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
// PID 1 holds FD 8 (private directory) and FD 9 (readiness inode) across the
|
||||||
|
// child's lifetime, including SIGKILL. Helpers borrow duplicates, never offsets.
|
||||||
|
func duplicateDNSAnchor(fd int) (*os.File, error) {
|
||||||
|
// Keep duplicates above both reserved descriptors; a missing anchor must
|
||||||
|
// never be filled by duplicating the other one.
|
||||||
|
duplicate, _, err := syscall.Syscall(syscall.SYS_FCNTL, uintptr(fd), syscall.F_DUPFD_CLOEXEC, 10)
|
||||||
|
if err != 0 {
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
}
|
||||||
|
return os.NewFile(duplicate, "DNS readiness anchor"), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func dnsReadinessCLI(args []string, stderr io.Writer) int {
|
||||||
|
if len(args) != 2 {
|
||||||
|
fmt.Fprintln(stderr, "configerror: invalid DNS readiness command")
|
||||||
|
return 64
|
||||||
|
}
|
||||||
|
pid, err := strconv.ParseInt(args[1], 10, 32)
|
||||||
|
if err != nil || pid <= 1 || strconv.FormatInt(pid, 10) != args[1] {
|
||||||
|
fmt.Fprintln(stderr, "configerror: invalid DNS readiness command")
|
||||||
|
return 64
|
||||||
|
}
|
||||||
|
cfg, err := parseManagedDNSConfig(os.LookupEnv)
|
||||||
|
if err != nil || cfg.Mode != "doh" {
|
||||||
|
fmt.Fprintln(stderr, "configerror: invalid managed DNS configuration")
|
||||||
|
return 64
|
||||||
|
}
|
||||||
|
anchor, _ := duplicateDNSAnchor(9)
|
||||||
|
if anchor != nil {
|
||||||
|
defer anchor.Close()
|
||||||
|
}
|
||||||
|
parent, err := duplicateDNSAnchor(8)
|
||||||
|
if err == nil {
|
||||||
|
defer parent.Close()
|
||||||
|
err = dnsReadinessAction(parent, anchor, int(pid), args[0] == "readiness-cleanup")
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintln(stderr, "error: DNS readiness ownership unavailable")
|
||||||
|
return 78
|
||||||
|
}
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
func dnsOwnedAnchor(info os.FileInfo, directory bool) bool {
|
||||||
|
stat, ok := info.Sys().(*syscall.Stat_t)
|
||||||
|
if !ok || stat.Uid != uint32(os.Geteuid()) || info.Mode()&(os.ModeSetuid|os.ModeSetgid|os.ModeSticky) != 0 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if directory {
|
||||||
|
return info.IsDir() && info.Mode().Perm() == 0700
|
||||||
|
}
|
||||||
|
return info.Mode().IsRegular() && info.Mode().Perm() == 0600
|
||||||
|
}
|
||||||
|
|
||||||
|
func dnsReadinessAction(parent, anchor *os.File, expectedPID int, cleanup bool) error {
|
||||||
|
if parent == nil || expectedPID <= 1 || int64(expectedPID) > 2147483647 {
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
info, err := parent.Stat()
|
||||||
|
if err != nil || !dnsOwnedAnchor(info, true) {
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
name := filepath.Base(dnsReadyPath)
|
||||||
|
fd, err := syscall.Openat(int(parent.Fd()), name, syscall.O_RDONLY|syscall.O_NONBLOCK|syscall.O_NOFOLLOW|syscall.O_CLOEXEC, 0)
|
||||||
|
if err != nil {
|
||||||
|
if cleanup && errors.Is(err, syscall.ENOENT) {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
current := os.NewFile(uintptr(fd), "DNS readiness entry")
|
||||||
|
defer current.Close()
|
||||||
|
if anchor == nil {
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
owned, err := anchor.Stat()
|
||||||
|
if err != nil || !dnsOwnedAnchor(owned, false) {
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
entry, err := current.Stat()
|
||||||
|
if err != nil || !dnsOwnedAnchor(entry, false) || !os.SameFile(owned, entry) {
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
expected := strconv.Itoa(expectedPID) + "\n"
|
||||||
|
if owned.Size() != int64(len(expected)) {
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
buffer := make([]byte, len(expected))
|
||||||
|
if _, err := anchor.ReadAt(buffer, 0); err != nil || string(buffer) != expected {
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
if cleanup {
|
||||||
|
if err := syscall.Unlinkat(int(parent.Fd()), name); err != nil && !errors.Is(err, syscall.ENOENT) {
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,105 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"net"
|
||||||
|
"net/url"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
const defaultDoHURL = "https://dns.google/dns-query"
|
||||||
|
const dnsListenAddress = "127.0.0.1:5353"
|
||||||
|
const dnsReadyPath = "/run/zapret-autotune-ss/dns-ready"
|
||||||
|
|
||||||
|
var errDNSConfig = errors.New("invalid managed DNS configuration")
|
||||||
|
|
||||||
|
type managedDNSConfig struct {
|
||||||
|
Mode string
|
||||||
|
URL string
|
||||||
|
Host string
|
||||||
|
BootstrapIPs []net.IP
|
||||||
|
Timeout time.Duration
|
||||||
|
NativeServer string
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseManagedDNSConfig(lookup func(string) (string, bool)) (managedDNSConfig, error) {
|
||||||
|
cfg := managedDNSConfig{Mode: "doh", URL: defaultDoHURL, Timeout: 5 * time.Second}
|
||||||
|
if v, ok := lookup("ZAPRET_AUTOTUNE_DNS_MODE"); ok {
|
||||||
|
cfg.Mode = v
|
||||||
|
}
|
||||||
|
if cfg.Mode == "native" {
|
||||||
|
cfg.NativeServer, _ = lookup("SS_DNS_SERVER")
|
||||||
|
if cfg.NativeServer != "" && net.ParseIP(cfg.NativeServer) == nil {
|
||||||
|
return cfg, errDNSConfig
|
||||||
|
}
|
||||||
|
return cfg, nil
|
||||||
|
}
|
||||||
|
if cfg.Mode != "doh" {
|
||||||
|
return cfg, errDNSConfig
|
||||||
|
}
|
||||||
|
if v, ok := lookup("ZAPRET_AUTOTUNE_DOH_URL"); ok {
|
||||||
|
cfg.URL = v
|
||||||
|
}
|
||||||
|
u, err := url.Parse(cfg.URL)
|
||||||
|
if err != nil || u.Scheme != "https" || u.Opaque != "" || u.User != nil || u.RawQuery != "" || u.ForceQuery || u.Fragment != "" || strings.Contains(cfg.URL, "#") || u.Host == "" || u.Path == "" || !strings.HasPrefix(u.Path, "/") || (u.Port() != "" && u.Port() != "443") {
|
||||||
|
return cfg, errDNSConfig
|
||||||
|
}
|
||||||
|
cfg.Host = u.Hostname()
|
||||||
|
// Do not accept opaque/control-bearing authorities, zone IDs or ambiguous ports.
|
||||||
|
if cfg.Host == "" || strings.ContainsAny(cfg.URL, "\x00\r\n\t ") || strings.Contains(cfg.Host, "%") {
|
||||||
|
return cfg, errDNSConfig
|
||||||
|
}
|
||||||
|
if net.ParseIP(cfg.Host) == nil {
|
||||||
|
for _, label := range strings.Split(cfg.Host, ".") {
|
||||||
|
if len(label) == 0 || len(label) > 63 || label[0] == '-' || label[len(label)-1] == '-' {
|
||||||
|
return cfg, errDNSConfig
|
||||||
|
}
|
||||||
|
for _, c := range label {
|
||||||
|
if !(c >= 'a' && c <= 'z' || c >= 'A' && c <= 'Z' || c >= '0' && c <= '9' || c == '-') {
|
||||||
|
return cfg, errDNSConfig
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(cfg.Host) > 253 {
|
||||||
|
return cfg, errDNSConfig
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if u.Host != net.JoinHostPort(cfg.Host, "443") && u.Host != cfg.Host && u.Host != "["+cfg.Host+"]" {
|
||||||
|
return cfg, errDNSConfig
|
||||||
|
}
|
||||||
|
bootstrap, explicit := lookup("ZAPRET_AUTOTUNE_DOH_BOOTSTRAP_IPS")
|
||||||
|
if !explicit || bootstrap == "" {
|
||||||
|
if cfg.URL != defaultDoHURL {
|
||||||
|
return cfg, errDNSConfig
|
||||||
|
}
|
||||||
|
bootstrap = "8.8.8.8,8.8.4.4"
|
||||||
|
}
|
||||||
|
parts := strings.Split(bootstrap, ",")
|
||||||
|
if len(parts) > 4 {
|
||||||
|
return cfg, errDNSConfig
|
||||||
|
}
|
||||||
|
for _, part := range parts {
|
||||||
|
ip := net.ParseIP(part)
|
||||||
|
if ip == nil {
|
||||||
|
return cfg, errDNSConfig
|
||||||
|
}
|
||||||
|
cfg.BootstrapIPs = append(cfg.BootstrapIPs, ip)
|
||||||
|
}
|
||||||
|
if v, ok := lookup("ZAPRET_AUTOTUNE_DOH_TIMEOUT_SEC"); ok {
|
||||||
|
seconds, err := strconv.Atoi(v)
|
||||||
|
if err != nil || seconds < 1 || seconds > 30 || strconv.Itoa(seconds) != v {
|
||||||
|
return cfg, errDNSConfig
|
||||||
|
}
|
||||||
|
cfg.Timeout = time.Duration(seconds) * time.Second
|
||||||
|
}
|
||||||
|
return cfg, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (c managedDNSConfig) serverAddress() string {
|
||||||
|
if c.Mode == "doh" {
|
||||||
|
return dnsListenAddress
|
||||||
|
}
|
||||||
|
return c.NativeServer
|
||||||
|
}
|
||||||
@@ -0,0 +1,91 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func dnsConfigForTest(t *testing.T, env map[string]string) managedDNSConfig {
|
||||||
|
t.Helper()
|
||||||
|
cfg, err := parseManagedDNSConfig(func(k string) (string, bool) { v, ok := env[k]; return v, ok })
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return cfg
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDNSConfigDefaultsAndNative(t *testing.T) {
|
||||||
|
cfg := dnsConfigForTest(t, map[string]string{"SS_DNS_SERVER": "ignored\nsecret"})
|
||||||
|
if cfg.Mode != "doh" || cfg.URL != defaultDoHURL || cfg.Host != "dns.google" || cfg.Timeout != 5*time.Second || cfg.serverAddress() != dnsListenAddress || len(cfg.BootstrapIPs) != 2 {
|
||||||
|
t.Fatal(cfg)
|
||||||
|
}
|
||||||
|
emptyBootstrap := dnsConfigForTest(t, map[string]string{"ZAPRET_AUTOTUNE_DOH_BOOTSTRAP_IPS": ""})
|
||||||
|
if len(emptyBootstrap.BootstrapIPs) != 2 || emptyBootstrap.BootstrapIPs[0].String() != "8.8.8.8" || emptyBootstrap.BootstrapIPs[1].String() != "8.8.4.4" {
|
||||||
|
t.Fatal("empty default bootstrap did not use default IPs")
|
||||||
|
}
|
||||||
|
for _, ip := range []string{"", "8.8.8.8", "::1"} {
|
||||||
|
cfg = dnsConfigForTest(t, map[string]string{"ZAPRET_AUTOTUNE_DNS_MODE": "native", "SS_DNS_SERVER": ip, "ZAPRET_AUTOTUNE_DOH_URL": "ignored"})
|
||||||
|
if cfg.serverAddress() != ip {
|
||||||
|
t.Fatal(cfg)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
cfg = dnsConfigForTest(t, map[string]string{"ZAPRET_AUTOTUNE_DOH_URL": "https://resolver.example:443/dns-query", "ZAPRET_AUTOTUNE_DOH_BOOTSTRAP_IPS": "1.1.1.1,2001:db8::1", "ZAPRET_AUTOTUNE_DOH_TIMEOUT_SEC": "30"})
|
||||||
|
if cfg.Host != "resolver.example" || cfg.Timeout != 30*time.Second {
|
||||||
|
t.Fatal(cfg)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDNSConfigRejectsOpaqueInputs(t *testing.T) {
|
||||||
|
cases := []map[string]string{
|
||||||
|
{"ZAPRET_AUTOTUNE_DNS_MODE": ""}, {"ZAPRET_AUTOTUNE_DNS_MODE": "auto"},
|
||||||
|
{"ZAPRET_AUTOTUNE_DNS_MODE": "native", "SS_DNS_SERVER": "dns.google"},
|
||||||
|
{"ZAPRET_AUTOTUNE_DNS_MODE": "native", "SS_DNS_SERVER": "8.8.8.8:53"},
|
||||||
|
{"ZAPRET_AUTOTUNE_DOH_URL": "https://other.example/dns-query"},
|
||||||
|
{"ZAPRET_AUTOTUNE_DOH_URL": "https://other.example/dns-query", "ZAPRET_AUTOTUNE_DOH_BOOTSTRAP_IPS": ""},
|
||||||
|
{"ZAPRET_AUTOTUNE_DOH_URL": "https://dns.google/custom-path", "ZAPRET_AUTOTUNE_DOH_BOOTSTRAP_IPS": ""},
|
||||||
|
}
|
||||||
|
for _, raw := range []string{"", "http://dns.google/dns-query", "https://token@dns.google/dns-query", "https://dns.google/dns-query?token=secret", "https://dns.google/dns-query#", "https://dns.google:5353/dns-query", "https://dns.google:/dns-query", "https://dns.google", "https://dns.google/\nsecret", "https://dns_google/dns-query"} {
|
||||||
|
cases = append(cases, map[string]string{"ZAPRET_AUTOTUNE_DOH_URL": raw, "ZAPRET_AUTOTUNE_DOH_BOOTSTRAP_IPS": "8.8.8.8"})
|
||||||
|
}
|
||||||
|
for _, raw := range []string{"dns.google", "1.1.1.1,", " 1.1.1.1", "1.1.1.1:443", "[::1]", "fe80::1%eth0", "1.1.1.1,1.1.1.1,1.1.1.1,1.1.1.1,1.1.1.1"} {
|
||||||
|
cases = append(cases, map[string]string{"ZAPRET_AUTOTUNE_DOH_BOOTSTRAP_IPS": raw})
|
||||||
|
}
|
||||||
|
for _, raw := range []string{"", "0", "31", "-1", "05", "1\n"} {
|
||||||
|
cases = append(cases, map[string]string{"ZAPRET_AUTOTUNE_DOH_TIMEOUT_SEC": raw})
|
||||||
|
}
|
||||||
|
for i, env := range cases {
|
||||||
|
_, err := parseManagedDNSConfig(func(k string) (string, bool) { v, ok := env[k]; return v, ok })
|
||||||
|
if err != errDNSConfig {
|
||||||
|
t.Errorf("case %d: %v", i, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDNSForwarderCLI(t *testing.T) {
|
||||||
|
t.Setenv("ZAPRET_AUTOTUNE_DNS_MODE", "doh")
|
||||||
|
t.Setenv("ZAPRET_AUTOTUNE_DOH_URL", defaultDoHURL)
|
||||||
|
t.Setenv("ZAPRET_AUTOTUNE_DOH_BOOTSTRAP_IPS", "8.8.8.8")
|
||||||
|
t.Setenv("ZAPRET_AUTOTUNE_DOH_TIMEOUT_SEC", "5")
|
||||||
|
t.Setenv("SS_DNS_SERVER", "secret-invalid-legacy")
|
||||||
|
for cmd, want := range map[string]string{"mode": "doh\n", "address": dnsListenAddress + "\n"} {
|
||||||
|
var out, diagnostic bytes.Buffer
|
||||||
|
if code := run([]string{"dns-forwarder", cmd}, &out, &diagnostic); code != 0 || out.String() != want || diagnostic.Len() != 0 {
|
||||||
|
t.Fatal(code, out.String(), diagnostic.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
t.Setenv("ZAPRET_AUTOTUNE_DNS_MODE", "native")
|
||||||
|
t.Setenv("SS_DNS_SERVER", "")
|
||||||
|
var out, diagnostic bytes.Buffer
|
||||||
|
if code := run([]string{"dns-forwarder", "address"}, &out, &diagnostic); code != 0 || out.String() != "\n" {
|
||||||
|
t.Fatal(code, out.String())
|
||||||
|
}
|
||||||
|
for _, args := range [][]string{{"serve"}, {"wrong"}, {"mode", "extra"}} {
|
||||||
|
out.Reset()
|
||||||
|
diagnostic.Reset()
|
||||||
|
if code := dnsForwarderCLI(args, &out, &diagnostic); code != 64 || out.Len() != 0 || !strings.HasPrefix(diagnostic.String(), "configerror:") {
|
||||||
|
t.Fatal(code, out.String(), diagnostic.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,295 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bufio"
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"syscall"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func dnsReadyAnchorsForTest(t *testing.T) (string, *os.File, *os.File, func()) {
|
||||||
|
t.Helper()
|
||||||
|
dir := t.TempDir()
|
||||||
|
if err := os.Chmod(dir, 0700); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
path := filepath.Join(dir, "dns-ready")
|
||||||
|
cleanup, err := createDNSReady(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
t.Cleanup(cleanup)
|
||||||
|
parent, err := os.Open(dir)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
t.Cleanup(func() { parent.Close() })
|
||||||
|
anchor, err := os.Open(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
t.Cleanup(func() { anchor.Close() })
|
||||||
|
return path, parent, anchor, cleanup
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDNSReadinessAnchorActions(t *testing.T) {
|
||||||
|
path, parent, anchor, childCleanup := dnsReadyAnchorsForTest(t)
|
||||||
|
pid := os.Getpid()
|
||||||
|
if _, err := anchor.Seek(1, io.SeekStart); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := dnsReadinessAction(parent, anchor, pid, false); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if offset, _ := anchor.Seek(0, io.SeekCurrent); offset != 1 {
|
||||||
|
t.Fatal("shared anchor offset changed", offset)
|
||||||
|
}
|
||||||
|
if err := dnsReadinessAction(parent, anchor, pid+1, false); err == nil {
|
||||||
|
t.Fatal("wrong PID accepted")
|
||||||
|
}
|
||||||
|
if err := dnsReadinessAction(parent, nil, pid, false); err == nil {
|
||||||
|
t.Fatal("missing file anchor accepted")
|
||||||
|
}
|
||||||
|
if err := dnsReadinessAction(parent, nil, pid, true); err == nil {
|
||||||
|
t.Fatal("unanchored entry removed")
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(path); err != nil {
|
||||||
|
t.Fatal("unanchored entry not preserved", err)
|
||||||
|
}
|
||||||
|
if err := dnsReadinessAction(parent, anchor, pid, true); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(path); !os.IsNotExist(err) {
|
||||||
|
t.Fatal("owned entry not removed", err)
|
||||||
|
}
|
||||||
|
if err := dnsReadinessAction(parent, nil, pid, true); err != nil {
|
||||||
|
t.Fatal("missing entry cleanup not idempotent", err)
|
||||||
|
}
|
||||||
|
if err := dnsReadinessAction(parent, anchor, pid, false); err == nil {
|
||||||
|
t.Fatal("unlinked entry reported ready")
|
||||||
|
}
|
||||||
|
childCleanup() // Graceful child cleanup after parent cleanup also remains safe.
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDNSReadinessForeignReplacement(t *testing.T) {
|
||||||
|
for _, kind := range []string{"same-PID-copy", "symlink", "directory", "FIFO"} {
|
||||||
|
t.Run(kind, func(t *testing.T) {
|
||||||
|
path, parent, anchor, childCleanup := dnsReadyAnchorsForTest(t)
|
||||||
|
os.Remove(path)
|
||||||
|
switch kind {
|
||||||
|
case "same-PID-copy":
|
||||||
|
if err := os.WriteFile(path, []byte(fmt.Sprintf("%d\n", os.Getpid())), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
case "symlink":
|
||||||
|
if err := os.Symlink("missing", path); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
case "directory":
|
||||||
|
if err := os.Mkdir(path, 0700); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
case "FIFO":
|
||||||
|
if err := syscall.Mkfifo(path, 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
foreign, err := os.Lstat(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for _, cleanup := range []bool{false, true} {
|
||||||
|
if err := dnsReadinessAction(parent, anchor, os.Getpid(), cleanup); err == nil {
|
||||||
|
t.Fatal("foreign entry accepted", cleanup)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
childCleanup()
|
||||||
|
current, err := os.Lstat(path)
|
||||||
|
if err != nil || !os.SameFile(foreign, current) {
|
||||||
|
t.Fatal("foreign entry deleted/replaced", err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDNSReadinessRejectsUnsafeAnchors(t *testing.T) {
|
||||||
|
path, parent, anchor, _ := dnsReadyAnchorsForTest(t)
|
||||||
|
pid := os.Getpid()
|
||||||
|
if err := dnsReadinessAction(nil, anchor, pid, true); err == nil {
|
||||||
|
t.Fatal("missing directory anchor accepted")
|
||||||
|
}
|
||||||
|
if err := dnsReadinessAction(anchor, anchor, pid, true); err == nil {
|
||||||
|
t.Fatal("regular file used as directory anchor")
|
||||||
|
}
|
||||||
|
for _, body := range []string{"", strconv.Itoa(pid), strconv.Itoa(pid) + "\nextra", "secret-content\n", strings.Repeat("x", 1024)} {
|
||||||
|
if err := os.WriteFile(path, []byte(body), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := dnsReadinessAction(parent, anchor, pid, true); err == nil {
|
||||||
|
t.Fatal("malformed PID body accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
os.WriteFile(path, []byte(fmt.Sprintf("%d\n", pid)), 0600)
|
||||||
|
os.Chmod(path, 0644)
|
||||||
|
if err := dnsReadinessAction(parent, anchor, pid, true); err == nil {
|
||||||
|
t.Fatal("public file accepted")
|
||||||
|
}
|
||||||
|
os.Chmod(path, 0600)
|
||||||
|
os.Chmod(filepath.Dir(path), 0755)
|
||||||
|
if err := dnsReadinessAction(parent, anchor, pid, true); err == nil {
|
||||||
|
t.Fatal("public directory accepted")
|
||||||
|
}
|
||||||
|
os.Chmod(filepath.Dir(path), 0700)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Subprocess helper exercises the actual inherited FD 8/9 CLI contract.
|
||||||
|
func TestDNSReadinessCLIHelper(t *testing.T) {
|
||||||
|
if os.Getenv("DNS_TEST_READINESS_CLI") != "1" {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
for i, arg := range os.Args {
|
||||||
|
if arg == "--" {
|
||||||
|
os.Exit(run(os.Args[i+1:], os.Stdout, os.Stderr))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
os.Exit(99)
|
||||||
|
}
|
||||||
|
|
||||||
|
func dnsReadinessSubprocess(t *testing.T, parent, anchor *os.File, command, pid string) (int, string) {
|
||||||
|
t.Helper()
|
||||||
|
binary, err := os.Executable()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
cmd := exec.CommandContext(ctx, binary, "-test.run=^TestDNSReadinessCLIHelper$", "--", "dns-forwarder", command, pid)
|
||||||
|
cmd.Env = append(os.Environ(), "DNS_TEST_READINESS_CLI=1", "ZAPRET_AUTOTUNE_DNS_MODE=doh", "ZAPRET_AUTOTUNE_DOH_URL="+defaultDoHURL, "ZAPRET_AUTOTUNE_DOH_BOOTSTRAP_IPS=", "ZAPRET_AUTOTUNE_DOH_TIMEOUT_SEC=5")
|
||||||
|
padding, err := os.Open(os.DevNull)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer padding.Close()
|
||||||
|
cmd.ExtraFiles = []*os.File{padding, padding, padding, padding, padding, parent}
|
||||||
|
if anchor != nil {
|
||||||
|
cmd.ExtraFiles = append(cmd.ExtraFiles, anchor)
|
||||||
|
}
|
||||||
|
output, err := cmd.CombinedOutput()
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
t.Fatal("readiness helper timed out")
|
||||||
|
}
|
||||||
|
if err == nil {
|
||||||
|
return 0, string(output)
|
||||||
|
}
|
||||||
|
if exit, ok := err.(*exec.ExitError); ok {
|
||||||
|
return exit.ExitCode(), string(output)
|
||||||
|
}
|
||||||
|
t.Fatal(err)
|
||||||
|
return -1, ""
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDNSReadinessInheritedCLI(t *testing.T) {
|
||||||
|
path, parent, anchor, _ := dnsReadyAnchorsForTest(t)
|
||||||
|
pid := strconv.Itoa(os.Getpid())
|
||||||
|
for _, bad := range []string{"0", "1", "-2", "+2", "02", "2\n", "2147483648", "secret-token"} {
|
||||||
|
code, out := dnsReadinessSubprocess(t, parent, anchor, "readiness-check", bad)
|
||||||
|
if code != 64 || out != "configerror: invalid DNS readiness command\n" {
|
||||||
|
t.Fatal(code, out)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if code, out := dnsReadinessSubprocess(t, parent, anchor, "readiness-check", pid); code != 0 || out != "" {
|
||||||
|
t.Fatal(code, out)
|
||||||
|
}
|
||||||
|
if code, out := dnsReadinessSubprocess(t, nil, anchor, "readiness-cleanup", pid); code != 78 || out != "error: DNS readiness ownership unavailable\n" {
|
||||||
|
t.Fatal("missing directory anchor accepted", code, out)
|
||||||
|
}
|
||||||
|
if offset, _ := anchor.Seek(0, io.SeekCurrent); offset != 0 {
|
||||||
|
t.Fatal("CLI changed parent's offset", offset)
|
||||||
|
}
|
||||||
|
if code, out := dnsReadinessSubprocess(t, parent, nil, "readiness-cleanup", pid); code != 78 || out != "error: DNS readiness ownership unavailable\n" {
|
||||||
|
t.Fatal(code, out)
|
||||||
|
}
|
||||||
|
if code, out := dnsReadinessSubprocess(t, parent, anchor, "readiness-cleanup", pid); code != 0 || out != "" {
|
||||||
|
t.Fatal(code, out)
|
||||||
|
}
|
||||||
|
if _, err := os.Stat(path); !os.IsNotExist(err) {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if code, out := dnsReadinessSubprocess(t, parent, nil, "readiness-cleanup", pid); code != 0 || out != "" {
|
||||||
|
t.Fatal(code, out)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDNSReadinessPublisherHelper(t *testing.T) {
|
||||||
|
dir := os.Getenv("DNS_TEST_READINESS_PUBLISH_DIR")
|
||||||
|
if dir == "" {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
cleanup, err := createDNSReady(filepath.Join(dir, "dns-ready"))
|
||||||
|
if err != nil {
|
||||||
|
os.Exit(99)
|
||||||
|
}
|
||||||
|
defer cleanup()
|
||||||
|
fmt.Fprintln(os.Stdout, "published")
|
||||||
|
time.Sleep(time.Hour)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDNSReadinessAnchorsSurviveSIGKILL(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
os.Chmod(dir, 0700)
|
||||||
|
binary, err := os.Executable()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
cmd := exec.CommandContext(ctx, binary, "-test.run=^TestDNSReadinessPublisherHelper$")
|
||||||
|
cmd.Env = append(os.Environ(), "DNS_TEST_READINESS_PUBLISH_DIR="+dir)
|
||||||
|
stdout, err := cmd.StdoutPipe()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var diagnostic bytes.Buffer
|
||||||
|
cmd.Stderr = &diagnostic
|
||||||
|
if err := cmd.Start(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer cmd.Process.Kill()
|
||||||
|
if line, err := bufio.NewReader(stdout).ReadString('\n'); err != nil || line != "published\n" {
|
||||||
|
t.Fatal("publisher failed", line, err)
|
||||||
|
}
|
||||||
|
parent, err := os.Open(dir)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer parent.Close()
|
||||||
|
path := filepath.Join(dir, "dns-ready")
|
||||||
|
anchor, err := os.Open(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer anchor.Close()
|
||||||
|
pid := cmd.Process.Pid
|
||||||
|
if err := dnsReadinessAction(parent, anchor, pid, false); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := cmd.Process.Kill(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
cmd.Wait()
|
||||||
|
if err := dnsReadinessAction(parent, anchor, pid, true); err != nil {
|
||||||
|
t.Fatal("anchored SIGKILL cleanup failed", err)
|
||||||
|
}
|
||||||
|
if _, err := os.Lstat(path); !os.IsNotExist(err) {
|
||||||
|
t.Fatal("SIGKILL left owned readiness entry", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,142 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/binary"
|
||||||
|
"io"
|
||||||
|
"net"
|
||||||
|
"net/http"
|
||||||
|
"sync/atomic"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestDNSTCPUpstreamTimeoutStillWritesServfail(t *testing.T) {
|
||||||
|
for _, slowRead := range []bool{false, true} {
|
||||||
|
name := "upstream-timeout"
|
||||||
|
if slowRead {
|
||||||
|
name = "partial-frame-then-upstream-timeout"
|
||||||
|
}
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
f := testDNSForwarder(t, func(r *http.Request) (*http.Response, error) {
|
||||||
|
<-r.Context().Done()
|
||||||
|
return nil, r.Context().Err()
|
||||||
|
})
|
||||||
|
f.cfg.Timeout = 80 * time.Millisecond
|
||||||
|
server, client := net.Pipe()
|
||||||
|
defer client.Close()
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
done := make(chan struct{})
|
||||||
|
go func() { defer close(done); f.tcpWorker(ctx, server) }()
|
||||||
|
client.SetDeadline(time.Now().Add(time.Second))
|
||||||
|
query := testDNSQuery()
|
||||||
|
var size [2]byte
|
||||||
|
binary.BigEndian.PutUint16(size[:], uint16(len(query)))
|
||||||
|
if _, err := client.Write(size[:]); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if slowRead {
|
||||||
|
if _, err := client.Write(query[:1]); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
time.Sleep(20 * time.Millisecond)
|
||||||
|
query = query[1:]
|
||||||
|
}
|
||||||
|
if _, err := client.Write(query); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := io.ReadFull(client, size[:]); err != nil {
|
||||||
|
t.Fatal("upstream timeout closed TCP instead of writing SERVFAIL", err)
|
||||||
|
}
|
||||||
|
reply := make([]byte, int(binary.BigEndian.Uint16(size[:])))
|
||||||
|
if _, err := io.ReadFull(client, reply); err != nil || !bytes.Equal(reply, dnsServfail(testDNSQuery())) {
|
||||||
|
t.Fatal("invalid framed SERVFAIL", reply, err)
|
||||||
|
}
|
||||||
|
cancel()
|
||||||
|
select {
|
||||||
|
case <-done:
|
||||||
|
case <-time.After(time.Second):
|
||||||
|
t.Fatal("worker did not join")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDNSTCPCancellationDuringUpstream(t *testing.T) {
|
||||||
|
started := make(chan struct{})
|
||||||
|
f := testDNSForwarder(t, func(r *http.Request) (*http.Response, error) {
|
||||||
|
close(started)
|
||||||
|
<-r.Context().Done()
|
||||||
|
return nil, r.Context().Err()
|
||||||
|
})
|
||||||
|
server, client := net.Pipe()
|
||||||
|
defer client.Close()
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
done := make(chan struct{})
|
||||||
|
go func() { defer close(done); f.tcpWorker(ctx, server) }()
|
||||||
|
client.SetDeadline(time.Now().Add(time.Second))
|
||||||
|
query := testDNSQuery()
|
||||||
|
var size [2]byte
|
||||||
|
binary.BigEndian.PutUint16(size[:], uint16(len(query)))
|
||||||
|
if err := writeDNSFrame(client, size[:], query); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
<-started
|
||||||
|
cancel()
|
||||||
|
select {
|
||||||
|
case <-done:
|
||||||
|
case <-time.After(time.Second):
|
||||||
|
t.Fatal("upstream cancellation did not join TCP worker")
|
||||||
|
}
|
||||||
|
if _, err := client.Read(size[:]); err == nil {
|
||||||
|
t.Fatal("cancelled TCP connection remained open")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDNSTCPTimeoutPhasesDoNotExtendLifetime(t *testing.T) {
|
||||||
|
var requests atomic.Int32
|
||||||
|
f := testDNSForwarder(t, func(r *http.Request) (*http.Response, error) {
|
||||||
|
requests.Add(1)
|
||||||
|
<-r.Context().Done()
|
||||||
|
return nil, r.Context().Err()
|
||||||
|
})
|
||||||
|
f.cfg.Timeout = 20 * time.Millisecond
|
||||||
|
server, client := net.Pipe()
|
||||||
|
defer client.Close()
|
||||||
|
done := make(chan struct{})
|
||||||
|
start := time.Now()
|
||||||
|
go func() { defer close(done); f.tcpWorker(context.Background(), server) }()
|
||||||
|
client.SetDeadline(start.Add(2 * time.Second))
|
||||||
|
query := testDNSQuery()
|
||||||
|
var size [2]byte
|
||||||
|
for frame := 0; frame < 17; frame++ {
|
||||||
|
binary.BigEndian.PutUint16(size[:], uint16(len(query)))
|
||||||
|
if err := writeDNSFrame(client, size[:], query); err != nil {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if _, err := io.ReadFull(client, size[:]); err != nil {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
reply := make([]byte, int(binary.BigEndian.Uint16(size[:])))
|
||||||
|
if _, err := io.ReadFull(client, reply); err != nil {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if !bytes.Equal(reply, dnsServfail(query)) {
|
||||||
|
t.Fatal("invalid SERVFAIL")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
select {
|
||||||
|
case <-done:
|
||||||
|
case <-time.After(time.Second):
|
||||||
|
t.Fatal("total lifetime not enforced")
|
||||||
|
}
|
||||||
|
if elapsed := time.Since(start); elapsed > 16*f.cfg.Timeout+200*time.Millisecond {
|
||||||
|
t.Fatal("phase deadlines extended total lifetime", elapsed)
|
||||||
|
}
|
||||||
|
if n := requests.Load(); n < 2 || n > 16 {
|
||||||
|
t.Fatal("unexpected request count", n)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,664 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bufio"
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"crypto/rand"
|
||||||
|
"crypto/rsa"
|
||||||
|
"crypto/tls"
|
||||||
|
"crypto/x509"
|
||||||
|
"crypto/x509/pkix"
|
||||||
|
"encoding/binary"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"math/big"
|
||||||
|
"net"
|
||||||
|
"net/http"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strconv"
|
||||||
|
"sync"
|
||||||
|
"sync/atomic"
|
||||||
|
"syscall"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
type dnsRoundTripFunc func(*http.Request) (*http.Response, error)
|
||||||
|
|
||||||
|
func (f dnsRoundTripFunc) RoundTrip(r *http.Request) (*http.Response, error) { return f(r) }
|
||||||
|
|
||||||
|
func testDNSQuery() []byte {
|
||||||
|
return []byte{0x12, 0x34, 1, 0x20, 0, 1, 0, 0, 0, 0, 0, 0, 3, 'w', 'w', 'w', 7, 'e', 'x', 'a', 'm', 'p', 'l', 'e', 0, 0, 1, 0, 1}
|
||||||
|
}
|
||||||
|
|
||||||
|
func testDNSResponse(query []byte) []byte {
|
||||||
|
response := append([]byte(nil), query...)
|
||||||
|
response[2] |= 0x80
|
||||||
|
response[3] |= 0x80
|
||||||
|
return response
|
||||||
|
}
|
||||||
|
|
||||||
|
func testDNSForwarder(t *testing.T, roundTrip dnsRoundTripFunc) *dnsForwarder {
|
||||||
|
t.Helper()
|
||||||
|
return &dnsForwarder{cfg: dnsConfigForTest(t, nil), client: &http.Client{Transport: roundTrip}, slots: make(chan struct{}, dnsConcurrency)}
|
||||||
|
}
|
||||||
|
|
||||||
|
func dnsHTTPResponse(body []byte) *http.Response {
|
||||||
|
return &http.Response{StatusCode: 200, Header: http.Header{"Content-Type": {"application/dns-message"}}, Body: io.NopCloser(bytes.NewReader(body)), ContentLength: int64(len(body))}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDNSWireValidation(t *testing.T) {
|
||||||
|
query := testDNSQuery()
|
||||||
|
if !validDNSQuery(query) || !validDNSResponse(query, testDNSResponse(query)) {
|
||||||
|
t.Fatal("valid wire rejected")
|
||||||
|
}
|
||||||
|
edns := append([]byte(nil), query...)
|
||||||
|
edns[11] = 1
|
||||||
|
edns = append(edns, 0, 0, 41, 0x10, 0, 0, 0, 0x80, 0, 0, 0)
|
||||||
|
if !validDNSQuery(edns) {
|
||||||
|
t.Fatal("EDNS/DNSSEC query rejected")
|
||||||
|
}
|
||||||
|
failure := dnsServfail(edns)
|
||||||
|
if len(failure) != len(query) || !bytes.Equal(failure[:2], query[:2]) || failure[2] != 0x81 || failure[3] != 2 || !bytes.Equal(failure[12:], query[12:]) || !bytes.Equal(failure[6:12], make([]byte, 6)) {
|
||||||
|
t.Fatal(failure)
|
||||||
|
}
|
||||||
|
for _, mutate := range []func([]byte) []byte{
|
||||||
|
func(q []byte) []byte { return q[:11] },
|
||||||
|
func(q []byte) []byte { q[2] |= 0x80; return q },
|
||||||
|
func(q []byte) []byte { q[2] |= 0x08; return q },
|
||||||
|
func(q []byte) []byte { q[2] |= 0x04; return q },
|
||||||
|
func(q []byte) []byte { q[3] |= 0x40; return q },
|
||||||
|
func(q []byte) []byte { q[5] = 2; return q },
|
||||||
|
func(q []byte) []byte { q[7] = 1; return q },
|
||||||
|
func(q []byte) []byte { q[11] = 255; return q },
|
||||||
|
func(q []byte) []byte { q[12] = 64; return q },
|
||||||
|
func(q []byte) []byte { q[12] = 0xc0; q[13] = 12; return q },
|
||||||
|
func(q []byte) []byte { q[12] = 0xc0; q[13] = 255; return q },
|
||||||
|
func(q []byte) []byte { return q[:len(q)-1] },
|
||||||
|
func(q []byte) []byte { return append(q, 0) },
|
||||||
|
func(q []byte) []byte { return append(q, make([]byte, 4096)...) },
|
||||||
|
} {
|
||||||
|
bad := mutate(append([]byte(nil), query...))
|
||||||
|
if validDNSQuery(bad) || dnsServfail(bad) != nil {
|
||||||
|
t.Fatal("malformed query accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// A backwards compression reference is legal; a self-reference is not.
|
||||||
|
if end, ok := dnsNameEnd(append(query, 0xc0, 12), len(query)); !ok || end != len(query)+2 {
|
||||||
|
t.Fatal(end, ok)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDNSExchangePassThroughAndRejects(t *testing.T) {
|
||||||
|
query := testDNSQuery()
|
||||||
|
response := testDNSResponse(query)
|
||||||
|
// Opaque record bytes (including DNSSEC and TTL) must remain untouched.
|
||||||
|
response[7] = 1
|
||||||
|
response = append(response, 0xc0, 12, 0, 1, 0, 1, 0, 0, 0, 37, 0, 4, 1, 2, 3, 4)
|
||||||
|
f := testDNSForwarder(t, func(r *http.Request) (*http.Response, error) {
|
||||||
|
body, _ := io.ReadAll(r.Body)
|
||||||
|
if r.Method != "POST" || r.URL.String() != defaultDoHURL || r.Header.Get("Content-Type") != "application/dns-message" || r.Header.Get("Accept") != "application/dns-message" || !bytes.Equal(body, query) {
|
||||||
|
t.Error("incorrect DoH request")
|
||||||
|
}
|
||||||
|
return dnsHTTPResponse(response), nil
|
||||||
|
})
|
||||||
|
if actual := f.exchange(context.Background(), query); !bytes.Equal(actual, response) {
|
||||||
|
t.Fatal(actual)
|
||||||
|
}
|
||||||
|
for name, change := range map[string]func(*http.Response){
|
||||||
|
"status": func(r *http.Response) { r.StatusCode = 503 },
|
||||||
|
"type": func(r *http.Response) { r.Header.Set("Content-Type", "text/plain") },
|
||||||
|
"encoding": func(r *http.Response) { r.Header.Set("Content-Encoding", "gzip") },
|
||||||
|
"length": func(r *http.Response) { r.ContentLength++ },
|
||||||
|
"overflow": func(r *http.Response) {
|
||||||
|
r.Body = io.NopCloser(bytes.NewReader(make([]byte, 65536)))
|
||||||
|
r.ContentLength = 65536
|
||||||
|
},
|
||||||
|
"short": func(r *http.Response) { r.Body = io.NopCloser(bytes.NewReader(response[:11])); r.ContentLength = 11 },
|
||||||
|
"id": func(r *http.Response) {
|
||||||
|
b := append([]byte(nil), response...)
|
||||||
|
b[0]++
|
||||||
|
r.Body = io.NopCloser(bytes.NewReader(b))
|
||||||
|
},
|
||||||
|
"qr": func(r *http.Response) {
|
||||||
|
b := append([]byte(nil), response...)
|
||||||
|
b[2] &= 0x7f
|
||||||
|
r.Body = io.NopCloser(bytes.NewReader(b))
|
||||||
|
},
|
||||||
|
"opcode": func(r *http.Response) {
|
||||||
|
b := append([]byte(nil), response...)
|
||||||
|
b[2] |= 0x08
|
||||||
|
r.Body = io.NopCloser(bytes.NewReader(b))
|
||||||
|
},
|
||||||
|
"question": func(r *http.Response) {
|
||||||
|
b := append([]byte(nil), response...)
|
||||||
|
b[13] = 'x'
|
||||||
|
r.Body = io.NopCloser(bytes.NewReader(b))
|
||||||
|
},
|
||||||
|
"qdcount": func(r *http.Response) {
|
||||||
|
b := append([]byte(nil), response...)
|
||||||
|
b[5] = 0
|
||||||
|
r.Body = io.NopCloser(bytes.NewReader(b))
|
||||||
|
},
|
||||||
|
"record-count": func(r *http.Response) {
|
||||||
|
b := append([]byte(nil), response...)
|
||||||
|
b[7] = 2
|
||||||
|
r.Body = io.NopCloser(bytes.NewReader(b))
|
||||||
|
},
|
||||||
|
"record-length": func(r *http.Response) {
|
||||||
|
b := append([]byte(nil), response...)
|
||||||
|
b[len(query)+11] = 255
|
||||||
|
r.Body = io.NopCloser(bytes.NewReader(b))
|
||||||
|
},
|
||||||
|
"truncated": func(r *http.Response) { r.Body = io.NopCloser(dnsBrokenReader{}) },
|
||||||
|
} {
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
f := testDNSForwarder(t, func(*http.Request) (*http.Response, error) { r := dnsHTTPResponse(response); change(r); return r, nil })
|
||||||
|
if actual := f.exchange(context.Background(), query); !bytes.Equal(actual, dnsServfail(query)) {
|
||||||
|
t.Fatal(actual)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
f = testDNSForwarder(t, func(*http.Request) (*http.Response, error) { return nil, errors.New("secret upstream") })
|
||||||
|
if !bytes.Equal(f.exchange(context.Background(), query), dnsServfail(query)) {
|
||||||
|
t.Fatal("outage not SERVFAIL")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type dnsBrokenReader struct{}
|
||||||
|
|
||||||
|
func (dnsBrokenReader) Read([]byte) (int, error) { return 0, io.ErrUnexpectedEOF }
|
||||||
|
|
||||||
|
func TestDNSBootstrapAndHTTPPolicy(t *testing.T) {
|
||||||
|
cfg := dnsConfigForTest(t, nil)
|
||||||
|
var addresses []string
|
||||||
|
dial := dnsBootstrapDial(cfg, func(ctx context.Context, network, address string) (net.Conn, error) {
|
||||||
|
if network != "tcp" {
|
||||||
|
t.Fatal(network)
|
||||||
|
}
|
||||||
|
if _, ok := ctx.Deadline(); !ok {
|
||||||
|
t.Fatal("no bounded deadline")
|
||||||
|
}
|
||||||
|
addresses = append(addresses, address)
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
})
|
||||||
|
for i := 0; i < 2; i++ {
|
||||||
|
dial(context.Background(), "tcp", "dns.google:443")
|
||||||
|
}
|
||||||
|
if len(addresses) != 4 || addresses[0] != "8.8.8.8:443" || addresses[2] != "8.8.4.4:443" {
|
||||||
|
t.Fatal(addresses)
|
||||||
|
}
|
||||||
|
for _, addr := range []string{"other:443", "dns.google:80", "8.8.8.8:443"} {
|
||||||
|
dial(context.Background(), "tcp", addr)
|
||||||
|
}
|
||||||
|
if len(addresses) != 4 {
|
||||||
|
t.Fatal("unexpected endpoint dialed")
|
||||||
|
}
|
||||||
|
cfg.BootstrapIPs = []net.IP{net.ParseIP("2001:db8::1")}
|
||||||
|
dnsBootstrapDial(cfg, func(_ context.Context, _, addr string) (net.Conn, error) {
|
||||||
|
if addr != "[2001:db8::1]:443" {
|
||||||
|
t.Fatal(addr)
|
||||||
|
}
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
})(context.Background(), "tcp", "dns.google:443")
|
||||||
|
client, transport := newDNSHTTPClient(cfg, markDNSSocket)
|
||||||
|
defer transport.CloseIdleConnections()
|
||||||
|
if transport.Proxy != nil || transport.TLSClientConfig.InsecureSkipVerify || transport.TLSClientConfig.ServerName != "dns.google" || transport.TLSClientConfig.MinVersion != tls.VersionTLS12 || !transport.DisableCompression || transport.MaxConnsPerHost != 16 {
|
||||||
|
t.Fatal("unsafe transport policy")
|
||||||
|
}
|
||||||
|
if client.CheckRedirect(&http.Request{}, nil) == nil {
|
||||||
|
t.Fatal("redirect allowed")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type dnsFakeRaw struct{ err error }
|
||||||
|
|
||||||
|
func (r dnsFakeRaw) Control(f func(uintptr)) error {
|
||||||
|
if r.err == nil {
|
||||||
|
f(42)
|
||||||
|
}
|
||||||
|
return r.err
|
||||||
|
}
|
||||||
|
func (dnsFakeRaw) Read(func(uintptr) bool) error { return nil }
|
||||||
|
func (dnsFakeRaw) Write(func(uintptr) bool) error { return nil }
|
||||||
|
|
||||||
|
func TestDNSSocketMarkStrict(t *testing.T) {
|
||||||
|
called := false
|
||||||
|
if err := setDNSSocketMark(dnsFakeRaw{}, func(fd, level, opt, mark int) error {
|
||||||
|
called = true
|
||||||
|
if fd != 42 || level != syscall.SOL_SOCKET || opt != syscall.SO_MARK || mark != 0x50000000 || mark&0x40000000 == 0 || mark&0x10000000 == 0 {
|
||||||
|
t.Fatal(fd, level, opt, mark)
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}); err != nil || !called {
|
||||||
|
t.Fatal(err, called)
|
||||||
|
}
|
||||||
|
failure := errors.New("mark denied")
|
||||||
|
if err := setDNSSocketMark(dnsFakeRaw{}, func(int, int, int, int) error { return failure }); err != failure {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := setDNSSocketMark(dnsFakeRaw{err: failure}, func(int, int, int, int) error { t.Fatal("called on raw failure"); return nil }); err != failure {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDNSTCPFramesAndCancellation(t *testing.T) {
|
||||||
|
f := testDNSForwarder(t, func(r *http.Request) (*http.Response, error) {
|
||||||
|
q, _ := io.ReadAll(r.Body)
|
||||||
|
return dnsHTTPResponse(testDNSResponse(q)), nil
|
||||||
|
})
|
||||||
|
server, client := net.Pipe()
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
done := make(chan struct{})
|
||||||
|
go func() { defer close(done); f.tcpWorker(ctx, server) }()
|
||||||
|
defer client.Close()
|
||||||
|
client.SetDeadline(time.Now().Add(3 * time.Second))
|
||||||
|
q := testDNSQuery()
|
||||||
|
for i := 0; i < 2; i++ {
|
||||||
|
q[1] = byte(i)
|
||||||
|
var size [2]byte
|
||||||
|
binary.BigEndian.PutUint16(size[:], uint16(len(q)))
|
||||||
|
if err := writeDNSFrame(client, size[:], q); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := io.ReadFull(client, size[:]); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
b := make([]byte, int(binary.BigEndian.Uint16(size[:])))
|
||||||
|
if _, err := io.ReadFull(client, b); err != nil || !bytes.Equal(b, testDNSResponse(q)) {
|
||||||
|
t.Fatal(err, b)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
cancel()
|
||||||
|
select {
|
||||||
|
case <-done:
|
||||||
|
case <-time.After(time.Second):
|
||||||
|
t.Fatal("idle TCP worker did not join")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDNSTCPLimits(t *testing.T) {
|
||||||
|
for _, n := range []int{11, 4097, 65535} {
|
||||||
|
t.Run(fmt.Sprint(n), func(t *testing.T) {
|
||||||
|
f := testDNSForwarder(t, func(*http.Request) (*http.Response, error) {
|
||||||
|
t.Error("invalid frame forwarded")
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
})
|
||||||
|
server, client := net.Pipe()
|
||||||
|
done := make(chan struct{})
|
||||||
|
go func() { defer close(done); f.tcpWorker(context.Background(), server) }()
|
||||||
|
client.SetDeadline(time.Now().Add(time.Second))
|
||||||
|
var size [2]byte
|
||||||
|
binary.BigEndian.PutUint16(size[:], uint16(n))
|
||||||
|
client.Write(size[:])
|
||||||
|
select {
|
||||||
|
case <-done:
|
||||||
|
case <-time.After(time.Second):
|
||||||
|
t.Fatal("invalid frame not closed")
|
||||||
|
}
|
||||||
|
client.Close()
|
||||||
|
})
|
||||||
|
}
|
||||||
|
f := testDNSForwarder(t, func(r *http.Request) (*http.Response, error) {
|
||||||
|
q, _ := io.ReadAll(r.Body)
|
||||||
|
return dnsHTTPResponse(testDNSResponse(q)), nil
|
||||||
|
})
|
||||||
|
server, client := net.Pipe()
|
||||||
|
defer client.Close()
|
||||||
|
client.SetDeadline(time.Now().Add(3 * time.Second))
|
||||||
|
done := make(chan struct{})
|
||||||
|
go func() { defer close(done); f.tcpWorker(context.Background(), server) }()
|
||||||
|
q := testDNSQuery()
|
||||||
|
var size [2]byte
|
||||||
|
for i := 0; i < 16; i++ {
|
||||||
|
binary.BigEndian.PutUint16(size[:], uint16(len(q)))
|
||||||
|
if err := writeDNSFrame(client, size[:], q); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := io.ReadFull(client, size[:]); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := io.CopyN(io.Discard, client, int64(binary.BigEndian.Uint16(size[:]))); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
select {
|
||||||
|
case <-done:
|
||||||
|
case <-time.After(time.Second):
|
||||||
|
t.Fatal("frame lifetime cap not enforced")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDNSRedirectNeverFollowed(t *testing.T) {
|
||||||
|
cfg := dnsConfigForTest(t, nil)
|
||||||
|
client, transport := newDNSHTTPClient(cfg, markDNSSocket)
|
||||||
|
defer transport.CloseIdleConnections()
|
||||||
|
var calls int
|
||||||
|
client.Transport = dnsRoundTripFunc(func(*http.Request) (*http.Response, error) {
|
||||||
|
calls++
|
||||||
|
return &http.Response{StatusCode: 302, Header: http.Header{"Location": {"https://secret.example/dns-query"}}, Body: io.NopCloser(bytes.NewReader(nil))}, nil
|
||||||
|
})
|
||||||
|
f := &dnsForwarder{cfg: cfg, client: client}
|
||||||
|
if !bytes.Equal(f.exchange(context.Background(), testDNSQuery()), dnsServfail(testDNSQuery())) || calls != 1 {
|
||||||
|
t.Fatal("redirect followed", calls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDNSReadinessFailureClosesListeners(t *testing.T) {
|
||||||
|
f := testDNSForwarder(t, func(*http.Request) (*http.Response, error) {
|
||||||
|
t.Error("request before ready")
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
})
|
||||||
|
p := &dnsFakePacketConn{closed: make(chan struct{})}
|
||||||
|
l := &dnsIdleListener{closed: make(chan struct{})}
|
||||||
|
if err := f.run(context.Background(), p, l, func() (func(), error) { return nil, errDNSForwarder }); err == nil {
|
||||||
|
t.Fatal("readiness error ignored")
|
||||||
|
}
|
||||||
|
select {
|
||||||
|
case <-p.closed:
|
||||||
|
default:
|
||||||
|
t.Fatal("UDP not closed")
|
||||||
|
}
|
||||||
|
select {
|
||||||
|
case <-l.closed:
|
||||||
|
default:
|
||||||
|
t.Fatal("TCP not closed")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDNSReadyOwnership(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
os.Chmod(dir, 0700)
|
||||||
|
path := filepath.Join(dir, "dns-ready")
|
||||||
|
cleanup, err := createDNSReady(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
info, _ := os.Stat(path)
|
||||||
|
body, _ := os.ReadFile(path)
|
||||||
|
if info.Mode().Perm() != 0600 || string(body) != fmt.Sprintf("%d\n", os.Getpid()) {
|
||||||
|
t.Fatal(info, string(body))
|
||||||
|
}
|
||||||
|
if _, err := createDNSReady(path); err == nil {
|
||||||
|
t.Fatal("existing readiness overwritten")
|
||||||
|
}
|
||||||
|
if entries, err := os.ReadDir(dir); err != nil || len(entries) != 1 || entries[0].Name() != "dns-ready" {
|
||||||
|
t.Fatal("readiness publication/failure left a temporary link", entries, err)
|
||||||
|
}
|
||||||
|
// The forwarder itself must hold both descriptors, not the test, so an
|
||||||
|
// unlinked readiness inode cannot be recycled for a foreign replacement.
|
||||||
|
fileFD := dnsHeldFD(t, info)
|
||||||
|
parentInfo, _ := os.Stat(dir)
|
||||||
|
parentFD := dnsHeldFD(t, parentInfo)
|
||||||
|
os.Remove(path)
|
||||||
|
var held syscall.Stat_t
|
||||||
|
if err := syscall.Fstat(fileFD, &held); err != nil || held.Nlink != 0 {
|
||||||
|
t.Fatal("readiness inode not held after unlink", err, held.Nlink)
|
||||||
|
}
|
||||||
|
os.WriteFile(path, []byte("replacement"), 0600)
|
||||||
|
cleanup()
|
||||||
|
cleanup() // Idempotent even after its descriptors have been closed.
|
||||||
|
for _, fd := range []int{fileFD, parentFD} {
|
||||||
|
if err := syscall.Fstat(fd, &held); err != syscall.EBADF {
|
||||||
|
t.Fatal("cleanup leaked descriptor", fd, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if b, err := os.ReadFile(path); err != nil || string(b) != "replacement" {
|
||||||
|
t.Fatal("foreign readiness removed", err)
|
||||||
|
}
|
||||||
|
os.Remove(path)
|
||||||
|
cleanup, err = createDNSReady(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
cleanup()
|
||||||
|
if _, err := os.Lstat(path); !os.IsNotExist(err) {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
os.Symlink("missing", path)
|
||||||
|
if _, err := createDNSReady(path); err == nil {
|
||||||
|
t.Fatal("symlink accepted")
|
||||||
|
}
|
||||||
|
os.Remove(path)
|
||||||
|
os.Chmod(dir, 0755)
|
||||||
|
if _, err := createDNSReady(path); err == nil {
|
||||||
|
t.Fatal("public parent accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func dnsHeldFD(t *testing.T, info os.FileInfo) int {
|
||||||
|
t.Helper()
|
||||||
|
entries, err := os.ReadDir("/proc/self/fd")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
want := info.Sys().(*syscall.Stat_t)
|
||||||
|
for _, entry := range entries {
|
||||||
|
fd, err := strconv.Atoi(entry.Name())
|
||||||
|
if err != nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
var stat syscall.Stat_t
|
||||||
|
if syscall.Fstat(fd, &stat) == nil && stat.Ino == want.Ino && stat.Dev == want.Dev {
|
||||||
|
return fd
|
||||||
|
}
|
||||||
|
}
|
||||||
|
t.Fatal("owned inode has no held descriptor")
|
||||||
|
return -1
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDNSReadyAtomicPublication(t *testing.T) {
|
||||||
|
dir := t.TempDir()
|
||||||
|
file, err := os.CreateTemp(dir, ".dns-ready-")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer file.Close()
|
||||||
|
path := filepath.Join(dir, "dns-ready")
|
||||||
|
if _, err := os.Lstat(path); !os.IsNotExist(err) {
|
||||||
|
t.Fatal("readiness published before PID", err)
|
||||||
|
}
|
||||||
|
if err := publishDNSReady(file, path); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if body, err := os.ReadFile(path); err != nil || string(body) != fmt.Sprintf("%d\n", os.Getpid()) {
|
||||||
|
t.Fatal("partial PID published", string(body), err)
|
||||||
|
}
|
||||||
|
other, err := os.CreateTemp(dir, "foreign-")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer other.Close()
|
||||||
|
if err := publishDNSReady(other, path); !os.IsExist(err) {
|
||||||
|
t.Fatal("publication overwrote existing inode", err)
|
||||||
|
}
|
||||||
|
if body, _ := os.ReadFile(path); string(body) != fmt.Sprintf("%d\n", os.Getpid()) {
|
||||||
|
t.Fatal("original PID changed")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Verify actual certificate validation over net.Pipe, never a TCP network listener.
|
||||||
|
func TestDNSVerifiedTLSOverPipe(t *testing.T) {
|
||||||
|
key, err := rsa.GenerateKey(rand.Reader, 2048)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
template := &x509.Certificate{SerialNumber: big.NewInt(1), Subject: pkix.Name{CommonName: "dns.google"}, DNSNames: []string{"dns.google"}, NotBefore: time.Now().Add(-time.Hour), NotAfter: time.Now().Add(time.Hour), KeyUsage: x509.KeyUsageDigitalSignature | x509.KeyUsageCertSign, ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth}, IsCA: true, BasicConstraintsValid: true}
|
||||||
|
der, err := x509.CreateCertificate(rand.Reader, template, template, &key.PublicKey, key)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
cert, err := x509.ParseCertificate(der)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for _, name := range []string{"trusted", "untrusted", "wrong-name"} {
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
cfg := dnsConfigForTest(t, nil)
|
||||||
|
client, transport := newDNSHTTPClient(cfg, markDNSSocket)
|
||||||
|
defer transport.CloseIdleConnections()
|
||||||
|
if name != "untrusted" {
|
||||||
|
roots := x509.NewCertPool()
|
||||||
|
roots.AddCert(cert)
|
||||||
|
transport.TLSClientConfig.RootCAs = roots
|
||||||
|
}
|
||||||
|
if name == "wrong-name" {
|
||||||
|
transport.TLSClientConfig.ServerName = "wrong.example"
|
||||||
|
}
|
||||||
|
var servers sync.WaitGroup
|
||||||
|
transport.DialContext = dnsBootstrapDial(cfg, func(ctx context.Context, _, addr string) (net.Conn, error) {
|
||||||
|
if addr != "8.8.8.8:443" && addr != "8.8.4.4:443" {
|
||||||
|
t.Error(addr)
|
||||||
|
}
|
||||||
|
a, b := net.Pipe()
|
||||||
|
servers.Add(1)
|
||||||
|
go func() {
|
||||||
|
defer servers.Done()
|
||||||
|
defer b.Close()
|
||||||
|
b.SetDeadline(time.Now().Add(3 * time.Second))
|
||||||
|
tlsConn := tls.Server(b, &tls.Config{Certificates: []tls.Certificate{{Certificate: [][]byte{der}, PrivateKey: key}}, MinVersion: tls.VersionTLS12})
|
||||||
|
if err := tlsConn.Handshake(); err != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if tlsConn.ConnectionState().ServerName != "dns.google" {
|
||||||
|
t.Error("SNI changed")
|
||||||
|
}
|
||||||
|
request, err := http.ReadRequest(bufio.NewReader(tlsConn))
|
||||||
|
if err != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
q, _ := io.ReadAll(request.Body)
|
||||||
|
request.Body.Close()
|
||||||
|
response := dnsHTTPResponse(testDNSResponse(q))
|
||||||
|
response.Close = true
|
||||||
|
response.Write(tlsConn)
|
||||||
|
}()
|
||||||
|
return a, nil
|
||||||
|
})
|
||||||
|
f := &dnsForwarder{cfg: cfg, client: client}
|
||||||
|
actual := f.exchange(context.Background(), testDNSQuery())
|
||||||
|
want := dnsServfail(testDNSQuery())
|
||||||
|
if name == "trusted" {
|
||||||
|
want = testDNSResponse(testDNSQuery())
|
||||||
|
}
|
||||||
|
if !bytes.Equal(actual, want) {
|
||||||
|
t.Fatal(actual)
|
||||||
|
}
|
||||||
|
transport.CloseIdleConnections()
|
||||||
|
servers.Wait()
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type dnsTestPacket struct {
|
||||||
|
data []byte
|
||||||
|
addr net.Addr
|
||||||
|
}
|
||||||
|
type dnsFakePacketConn struct {
|
||||||
|
incoming chan dnsTestPacket
|
||||||
|
outgoing chan dnsTestPacket
|
||||||
|
closed chan struct{}
|
||||||
|
once sync.Once
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *dnsFakePacketConn) ReadFrom(b []byte) (int, net.Addr, error) {
|
||||||
|
select {
|
||||||
|
case q := <-p.incoming:
|
||||||
|
return copy(b, q.data), q.addr, nil
|
||||||
|
case <-p.closed:
|
||||||
|
return 0, nil, net.ErrClosed
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func (p *dnsFakePacketConn) WriteTo(b []byte, a net.Addr) (int, error) {
|
||||||
|
select {
|
||||||
|
case p.outgoing <- dnsTestPacket{append([]byte(nil), b...), a}:
|
||||||
|
return len(b), nil
|
||||||
|
case <-p.closed:
|
||||||
|
return 0, net.ErrClosed
|
||||||
|
}
|
||||||
|
}
|
||||||
|
func (p *dnsFakePacketConn) Close() error { p.once.Do(func() { close(p.closed) }); return nil }
|
||||||
|
func (p *dnsFakePacketConn) LocalAddr() net.Addr { return &net.UDPAddr{} }
|
||||||
|
func (p *dnsFakePacketConn) SetDeadline(time.Time) error { return nil }
|
||||||
|
func (p *dnsFakePacketConn) SetReadDeadline(time.Time) error { return nil }
|
||||||
|
func (p *dnsFakePacketConn) SetWriteDeadline(time.Time) error { return nil }
|
||||||
|
|
||||||
|
type dnsIdleListener struct {
|
||||||
|
closed chan struct{}
|
||||||
|
once sync.Once
|
||||||
|
}
|
||||||
|
|
||||||
|
func (l *dnsIdleListener) Accept() (net.Conn, error) { <-l.closed; return nil, net.ErrClosed }
|
||||||
|
func (l *dnsIdleListener) Close() error { l.once.Do(func() { close(l.closed) }); return nil }
|
||||||
|
func (l *dnsIdleListener) Addr() net.Addr { return &net.TCPAddr{} }
|
||||||
|
|
||||||
|
func TestDNSUDPBoundedCopiesAndJoins(t *testing.T) {
|
||||||
|
var active, maximum atomic.Int32
|
||||||
|
started := make(chan struct{}, dnsConcurrency)
|
||||||
|
f := testDNSForwarder(t, func(r *http.Request) (*http.Response, error) {
|
||||||
|
n := active.Add(1)
|
||||||
|
defer active.Add(-1)
|
||||||
|
for old := maximum.Load(); n > old; old = maximum.Load() {
|
||||||
|
if maximum.CompareAndSwap(old, n) {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
q, _ := io.ReadAll(r.Body)
|
||||||
|
started <- struct{}{}
|
||||||
|
<-r.Context().Done()
|
||||||
|
if q[1] >= dnsConcurrency {
|
||||||
|
t.Error("UDP buffer was reused")
|
||||||
|
}
|
||||||
|
return nil, r.Context().Err()
|
||||||
|
})
|
||||||
|
p := &dnsFakePacketConn{incoming: make(chan dnsTestPacket, 32), outgoing: make(chan dnsTestPacket, 32), closed: make(chan struct{})}
|
||||||
|
l := &dnsIdleListener{closed: make(chan struct{})}
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
ready := make(chan struct{})
|
||||||
|
cleaned := make(chan struct{})
|
||||||
|
done := make(chan error, 1)
|
||||||
|
go func() {
|
||||||
|
done <- f.run(ctx, p, l, func() (func(), error) { close(ready); return func() { close(cleaned) }, nil })
|
||||||
|
}()
|
||||||
|
<-ready
|
||||||
|
for i := 0; i < dnsConcurrency; i++ {
|
||||||
|
q := testDNSQuery()
|
||||||
|
q[1] = byte(i)
|
||||||
|
p.incoming <- dnsTestPacket{q, &net.UDPAddr{}}
|
||||||
|
}
|
||||||
|
for i := 0; i < dnsConcurrency; i++ {
|
||||||
|
select {
|
||||||
|
case <-started:
|
||||||
|
case <-time.After(time.Second):
|
||||||
|
t.Fatal("worker not started")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
q := testDNSQuery()
|
||||||
|
q[1] = 99
|
||||||
|
p.incoming <- dnsTestPacket{q, &net.UDPAddr{}}
|
||||||
|
select {
|
||||||
|
case response := <-p.outgoing:
|
||||||
|
if !bytes.Equal(response.data, dnsServfail(q)) {
|
||||||
|
t.Fatal(response)
|
||||||
|
}
|
||||||
|
case <-time.After(time.Second):
|
||||||
|
t.Fatal("saturation not SERVFAIL")
|
||||||
|
}
|
||||||
|
cancel()
|
||||||
|
select {
|
||||||
|
case err := <-done:
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
case <-time.After(time.Second):
|
||||||
|
t.Fatal("workers not joined")
|
||||||
|
}
|
||||||
|
<-cleaned
|
||||||
|
if maximum.Load() != dnsConcurrency || active.Load() != 0 || len(f.slots) != 0 {
|
||||||
|
t.Fatal(maximum.Load(), active.Load(), len(f.slots))
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,169 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/binary"
|
||||||
|
"errors"
|
||||||
|
"io"
|
||||||
|
"net"
|
||||||
|
"net/http"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func dnsTestEDNSQuery(payload uint16) []byte {
|
||||||
|
q := testDNSQuery()
|
||||||
|
q[11] = 1
|
||||||
|
return append(q, 0, 0, 41, byte(payload>>8), byte(payload), 0, 0, 0x80, 0, 0, 0)
|
||||||
|
}
|
||||||
|
|
||||||
|
func dnsTestSizedResponse(query []byte, size int) []byte {
|
||||||
|
response := dnsQuestionReply(query, query)
|
||||||
|
response[2] = 0x81
|
||||||
|
response[3] = 0xa0 // RA and AD survive truncation.
|
||||||
|
response[7] = 1
|
||||||
|
dataSize := size - len(response) - 12
|
||||||
|
response = append(response, 0xc0, 12, 0, 16, 0, 1, 0, 0, 0, 37, byte(dataSize>>8), byte(dataSize))
|
||||||
|
return append(response, make([]byte, dataSize)...)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDNSUDPPayloadLimits(t *testing.T) {
|
||||||
|
for _, tc := range []struct {
|
||||||
|
name string
|
||||||
|
query []byte
|
||||||
|
size, limit int
|
||||||
|
truncated bool
|
||||||
|
}{
|
||||||
|
{"no-EDNS-full-TCP", testDNSQuery(), 65535, 512, true},
|
||||||
|
{"no-EDNS-at-limit", testDNSQuery(), 512, 512, false},
|
||||||
|
{"EDNS-at-limit", dnsTestEDNSQuery(1232), 1232, 1232, false},
|
||||||
|
{"EDNS-over-limit", dnsTestEDNSQuery(1232), 1233, 1232, true},
|
||||||
|
{"EDNS-small-clamp", dnsTestEDNSQuery(1), 513, 512, true},
|
||||||
|
{"EDNS-max-at-IPv4-limit", dnsTestEDNSQuery(65535), 65507, 65507, false},
|
||||||
|
{"EDNS-max-over-IPv4-limit", dnsTestEDNSQuery(65535), 65535, 65507, true},
|
||||||
|
} {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
response := dnsTestSizedResponse(tc.query, tc.size)
|
||||||
|
if !validDNSQuery(tc.query) || !validDNSResponse(tc.query, response) {
|
||||||
|
t.Fatal("invalid fixture")
|
||||||
|
}
|
||||||
|
actual := dnsUDPReply(tc.query, response)
|
||||||
|
if len(actual) > tc.limit || (actual[2]&2 != 0) != tc.truncated {
|
||||||
|
t.Fatal("incorrect UDP ceiling/TC", len(actual), actual[:12])
|
||||||
|
}
|
||||||
|
if !tc.truncated {
|
||||||
|
if !bytes.Equal(actual, response) {
|
||||||
|
t.Fatal("in-range answer changed")
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if !validDNSResponse(tc.query, actual) || !bytes.Equal(actual[:2], response[:2]) || actual[2] != response[2]|2 || actual[3] != response[3] || !bytes.Equal(actual[6:12], make([]byte, 6)) {
|
||||||
|
t.Fatal("invalid truncated reply", actual)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDNSOversizedUDPThenFullTCP(t *testing.T) {
|
||||||
|
q := testDNSQuery()
|
||||||
|
response := dnsTestSizedResponse(q, 65535)
|
||||||
|
f := testDNSForwarder(t, func(*http.Request) (*http.Response, error) { return dnsHTTPResponse(response), nil })
|
||||||
|
p := &dnsFakePacketConn{outgoing: make(chan dnsTestPacket, 1), closed: make(chan struct{})}
|
||||||
|
writeDNSDatagram(p, &net.UDPAddr{}, q, f.exchange(context.Background(), q))
|
||||||
|
if packet := <-p.outgoing; len(packet.data) > 512 || packet.data[2]&2 == 0 {
|
||||||
|
t.Fatal("oversized UDP did not force TCP")
|
||||||
|
}
|
||||||
|
server, client := net.Pipe()
|
||||||
|
defer client.Close()
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
done := make(chan struct{})
|
||||||
|
go func() { defer close(done); f.tcpWorker(ctx, server) }()
|
||||||
|
client.SetDeadline(time.Now().Add(3 * time.Second))
|
||||||
|
var size [2]byte
|
||||||
|
binary.BigEndian.PutUint16(size[:], uint16(len(q)))
|
||||||
|
if err := writeDNSFrame(client, size[:], q); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := io.ReadFull(client, size[:]); err != nil || binary.BigEndian.Uint16(size[:]) != 65535 {
|
||||||
|
t.Fatal("incorrect full TCP frame", size, err)
|
||||||
|
}
|
||||||
|
actual := make([]byte, 65535)
|
||||||
|
if _, err := io.ReadFull(client, actual); err != nil || !bytes.Equal(actual, response) {
|
||||||
|
t.Fatal("full TCP reply changed", err)
|
||||||
|
}
|
||||||
|
cancel()
|
||||||
|
select {
|
||||||
|
case <-done:
|
||||||
|
case <-time.After(time.Second):
|
||||||
|
t.Fatal("TCP worker did not join")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDNSCompressedQuestionReplies(t *testing.T) {
|
||||||
|
plain := testDNSQuery()
|
||||||
|
// Question points forward into an additional RR's owner; copying just the
|
||||||
|
// original question would leave a dangling pointer after dropping that RR.
|
||||||
|
q := append([]byte(nil), plain[:12]...)
|
||||||
|
q[11] = 1
|
||||||
|
q = append(q, 0xc0, 18, 0, 1, 0, 1)
|
||||||
|
q = append(q, plain[12:len(plain)-4]...)
|
||||||
|
q = append(q, 0, 1, 0, 1, 0, 0, 0, 37, 0, 4, 1, 2, 3, 4)
|
||||||
|
if !validDNSQuery(q) {
|
||||||
|
t.Fatal("compressed query rejected")
|
||||||
|
}
|
||||||
|
for _, reply := range [][]byte{dnsServfail(q), dnsUDPReply(q, dnsTestSizedResponse(q, 65535))} {
|
||||||
|
if !validDNSResponse(q, reply) || !bytes.Equal(reply[12:], plain[12:]) || reply[12]&0xc0 != 0 || !bytes.Equal(reply[6:12], make([]byte, 6)) {
|
||||||
|
t.Fatal("dangling/compressed synthesized question", reply)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
response := dnsTestSizedResponse(q, 65535)
|
||||||
|
f := testDNSForwarder(t, func(*http.Request) (*http.Response, error) { return dnsHTTPResponse(response), nil })
|
||||||
|
if !bytes.Equal(f.exchange(context.Background(), q), response) {
|
||||||
|
t.Fatal("equivalent uncompressed upstream question rejected")
|
||||||
|
}
|
||||||
|
q[18] = 0xc0
|
||||||
|
q[19] = 12 // Question -> owner -> question compression loop.
|
||||||
|
if validDNSQuery(q) || dnsServfail(q) != nil {
|
||||||
|
t.Fatal("compression loop accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDNSMalformedEDNSQueries(t *testing.T) {
|
||||||
|
q := dnsTestEDNSQuery(1232)
|
||||||
|
duplicate := append(append([]byte(nil), q...), q[len(testDNSQuery()):]...)
|
||||||
|
duplicate[11] = 2
|
||||||
|
badOwner := append([]byte(nil), q...)
|
||||||
|
start := len(testDNSQuery())
|
||||||
|
badOwner = append(badOwner[:start], append([]byte{1, 'x'}, badOwner[start:]...)...)
|
||||||
|
badOptions := append([]byte(nil), q...)
|
||||||
|
badOptions[len(badOptions)-1] = 3
|
||||||
|
badOptions = append(badOptions, 0, 1, 0)
|
||||||
|
for i, bad := range [][]byte{duplicate, badOwner, badOptions, q[:len(q)-1]} {
|
||||||
|
if validDNSQuery(bad) {
|
||||||
|
t.Fatal("malformed EDNS accepted", i)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type dnsFailingPacketConn struct {
|
||||||
|
*dnsFakePacketConn
|
||||||
|
writes int
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *dnsFailingPacketConn) WriteTo([]byte, net.Addr) (int, error) {
|
||||||
|
p.writes++
|
||||||
|
return 0, errors.New("datagram dropped")
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestDNSUDPWriteErrorDropsOnlyDatagram(t *testing.T) {
|
||||||
|
p := &dnsFailingPacketConn{}
|
||||||
|
q := testDNSQuery()
|
||||||
|
for i := 0; i < 2; i++ {
|
||||||
|
writeDNSDatagram(p, &net.UDPAddr{}, q, dnsTestSizedResponse(q, 65535))
|
||||||
|
}
|
||||||
|
if p.writes != 2 {
|
||||||
|
t.Fatal("write error stopped subsequent datagrams", p.writes)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -37,8 +37,8 @@ func TestHealthConfigDefaultsAndOverrides(t *testing.T) {
|
|||||||
if err != nil || !reflect.DeepEqual(cfg, want) {
|
if err != nil || !reflect.DeepEqual(cfg, want) {
|
||||||
t.Fatalf("overrides = %+v, %v; want %+v", cfg, err, want)
|
t.Fatalf("overrides = %+v, %v; want %+v", cfg, err, want)
|
||||||
}
|
}
|
||||||
for _, proxy := range []string{"socks5://localhost:1", "socks5://LOCALHOST:65535", "socks5://127.0.0.2:11080", "socks5://[::1]:11080"} {
|
for _, proxy := range []string{"socks5://localhost:1", "socks5://LOCALHOST:65535", "socks5://127.0.0.2:11080", "socks5://[::1]:11080", "socks5h://localhost:1", "socks5h://LOCALHOST:65535", "socks5h://127.0.0.2:11080", "socks5h://[::1]:11080"} {
|
||||||
if _, err := parseHealthConfig(configLookup(map[string]string{"ZAPRET_AUTOTUNE_SOCKS_PROXY": proxy})); err != nil {
|
if cfg, err := parseHealthConfig(configLookup(map[string]string{"ZAPRET_AUTOTUNE_SOCKS_PROXY": proxy})); err != nil || cfg.SOCKSProxy != proxy {
|
||||||
t.Errorf("valid loopback rejected: %s: %v", proxy, err)
|
t.Errorf("valid loopback rejected: %s: %v", proxy, err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -47,12 +47,14 @@ func TestHealthConfigDefaultsAndOverrides(t *testing.T) {
|
|||||||
func TestHealthConfigRejectsInvalid(t *testing.T) {
|
func TestHealthConfigRejectsInvalid(t *testing.T) {
|
||||||
invalid := map[string][]string{
|
invalid := map[string][]string{
|
||||||
"ZAPRET_AUTOTUNE_SOCKS_PROXY": {
|
"ZAPRET_AUTOTUNE_SOCKS_PROXY": {
|
||||||
"", "http://127.0.0.1:11080", "socks5h://127.0.0.1:11080",
|
"", "http://127.0.0.1:11080", "https://127.0.0.1:11080", "socks4://127.0.0.1:11080",
|
||||||
"socks5://192.168.1.1:11080", "socks5://0.0.0.0:11080", "socks5://[::]:11080",
|
"socks5://192.168.1.1:11080", "socks5://0.0.0.0:11080", "socks5://[::]:11080",
|
||||||
"socks5://proxy.example:11080", "socks5://localhost.evil:11080",
|
"socks5://proxy.example:11080", "socks5://localhost.evil:11080",
|
||||||
"socks5://user:SECRET@localhost:11080", "socks5://localhost", "socks5://localhost:0",
|
"socks5://user:SECRET@localhost:11080", "socks5://localhost", "socks5://localhost:0",
|
||||||
"socks5://localhost:65536", "socks5://localhost:bad", "socks5://localhost:11080/path",
|
"socks5://localhost:65536", "socks5://localhost:bad", "socks5://localhost:11080/path",
|
||||||
"socks5://localhost:11080?SECRET", "socks5://localhost:11080#SECRET", " socks5://localhost:11080",
|
"socks5://localhost:11080?SECRET", "socks5://localhost:11080#SECRET", " socks5://localhost:11080",
|
||||||
|
"socks5://@localhost:11080", "socks5://SECRET@localhost:11080", "socks5://localhost:11080?", "socks5://localhost:11080#",
|
||||||
|
"socks5://localhost:11080\x00", "socks5://localhost:11080\x1f", "socks5://localhost:11080\x7f",
|
||||||
},
|
},
|
||||||
"ZAPRET_AUTOTUNE_VIDEO_ID": {"", "short", "dQw4w9WgXc!"},
|
"ZAPRET_AUTOTUNE_VIDEO_ID": {"", "short", "dQw4w9WgXc!"},
|
||||||
"ZAPRET_AUTOTUNE_HEALTH_INTERVAL": {"", "0", "-5m", "5", "SECRET", "999999999999999999999h"},
|
"ZAPRET_AUTOTUNE_HEALTH_INTERVAL": {"", "0", "-5m", "5", "SECRET", "999999999999999999999h"},
|
||||||
@@ -60,6 +62,13 @@ func TestHealthConfigRejectsInvalid(t *testing.T) {
|
|||||||
"ZAPRET_AUTOTUNE_FAILURE_THRESHOLD": {"", "0", "-3", "1.5", "SECRET", "999999999999999999999"},
|
"ZAPRET_AUTOTUNE_FAILURE_THRESHOLD": {"", "0", "-3", "1.5", "SECRET", "999999999999999999999"},
|
||||||
}
|
}
|
||||||
for key, values := range invalid {
|
for key, values := range invalid {
|
||||||
|
if key == "ZAPRET_AUTOTUNE_SOCKS_PROXY" {
|
||||||
|
for _, value := range values {
|
||||||
|
if strings.HasPrefix(value, "socks5://") {
|
||||||
|
values = append(values, strings.Replace(value, "socks5://", "socks5h://", 1))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
for _, value := range values {
|
for _, value := range values {
|
||||||
if _, err := parseHealthConfig(configLookup(map[string]string{key: value})); err == nil || !strings.Contains(err.Error(), key) || strings.Contains(err.Error(), "SECRET") {
|
if _, err := parseHealthConfig(configLookup(map[string]string{key: value})); err == nil || !strings.Contains(err.Error(), key) || strings.Contains(err.Error(), "SECRET") {
|
||||||
t.Errorf("%s=%q: expected sanitized config error, got %v", key, value, err)
|
t.Errorf("%s=%q: expected sanitized config error, got %v", key, value, err)
|
||||||
@@ -105,8 +114,10 @@ func setHealthEnvironment(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func TestRunHealthYoutube(t *testing.T) {
|
func TestRunHealthYoutube(t *testing.T) {
|
||||||
|
for _, proxy := range []string{"socks5://[::1]:12080", "socks5h://[::1]:12080", "socks5h://127.0.0.1:11082"} {
|
||||||
|
t.Run(proxy, func(t *testing.T) {
|
||||||
setHealthEnvironment(t)
|
setHealthEnvironment(t)
|
||||||
t.Setenv("ZAPRET_AUTOTUNE_SOCKS_PROXY", "socks5://[::1]:12080")
|
t.Setenv("ZAPRET_AUTOTUNE_SOCKS_PROXY", proxy)
|
||||||
t.Setenv("ZAPRET_AUTOTUNE_VIDEO_ID", "abcdefghijk")
|
t.Setenv("ZAPRET_AUTOTUNE_VIDEO_ID", "abcdefghijk")
|
||||||
for _, key := range []string{"ALL_PROXY", "HTTPS_PROXY", "https_proxy", "all_proxy"} {
|
for _, key := range []string{"ALL_PROXY", "HTTPS_PROXY", "https_proxy", "all_proxy"} {
|
||||||
t.Setenv(key, "http://untrusted.example:8080")
|
t.Setenv(key, "http://untrusted.example:8080")
|
||||||
@@ -120,7 +131,7 @@ func TestRunHealthYoutube(t *testing.T) {
|
|||||||
if !ok || time.Until(deadline) <= 0 || time.Until(deadline) > 35*time.Second {
|
if !ok || time.Until(deadline) <= 0 || time.Until(deadline) > 35*time.Second {
|
||||||
t.Error("request must have bounded deadline")
|
t.Error("request must have bounded deadline")
|
||||||
}
|
}
|
||||||
prefix := []string{"--disable", "--silent", "--show-error", "--fail", "--max-time", "30", "--noproxy", "", "--proxy", "socks5://[::1]:12080", "--user-agent", playerUserAgent, "--json"}
|
prefix := []string{"--disable", "--silent", "--show-error", "--fail", "--max-time", "30", "--noproxy", "", "--proxy", proxy, "--user-agent", playerUserAgent, "--json"}
|
||||||
if len(args) != len(prefix)+2 || !reflect.DeepEqual(args[:len(prefix)], prefix) || args[len(args)-1] != playerURL {
|
if len(args) != len(prefix)+2 || !reflect.DeepEqual(args[:len(prefix)], prefix) || args[len(args)-1] != playerURL {
|
||||||
t.Fatalf("unexpected proxy/API arguments: %q", args)
|
t.Fatalf("unexpected proxy/API arguments: %q", args)
|
||||||
}
|
}
|
||||||
@@ -137,6 +148,8 @@ func TestRunHealthYoutube(t *testing.T) {
|
|||||||
if code := runWithPlayer([]string{"health", "youtube"}, &stdout, &stderr, runner); code != 0 || calls != 1 || stderr.Len() != 0 || !strings.Contains(stdout.String(), "Player API: OK (not a playback check)") {
|
if code := runWithPlayer([]string{"health", "youtube"}, &stdout, &stderr, runner); code != 0 || calls != 1 || stderr.Len() != 0 || !strings.Contains(stdout.String(), "Player API: OK (not a playback check)") {
|
||||||
t.Fatalf("code=%d calls=%d stdout=%q stderr=%q", code, calls, &stdout, &stderr)
|
t.Fatalf("code=%d calls=%d stdout=%q stderr=%q", code, calls, &stdout, &stderr)
|
||||||
}
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestHealthYoutubeFailures(t *testing.T) {
|
func TestHealthYoutubeFailures(t *testing.T) {
|
||||||
|
|||||||
@@ -0,0 +1,200 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"image"
|
||||||
|
"image/jpeg"
|
||||||
|
"io"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func rejectedImageRunner(mode string) subprocessRunner {
|
||||||
|
return func(_ context.Context, args []string, in io.Reader, out, diag io.Writer) error {
|
||||||
|
config, err := io.ReadAll(in)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
raw := strings.TrimSuffix(strings.TrimPrefix(string(config), "url = \""), "\"\n")
|
||||||
|
body, contentType := resourceFixture(), "image/png"
|
||||||
|
switch mode {
|
||||||
|
case "jpeg":
|
||||||
|
var b bytes.Buffer
|
||||||
|
if err := jpeg.Encode(&b, image.NewRGBA(image.Rect(0, 0, 8, 8)), nil); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
body, contentType = b.Bytes(), "image/jpeg"
|
||||||
|
if start := bytes.Index(body, []byte{0xff, 0xda}); start >= 0 {
|
||||||
|
body = body[:start+12]
|
||||||
|
}
|
||||||
|
case "png":
|
||||||
|
body = body[:len(body)/2]
|
||||||
|
case "magic":
|
||||||
|
body = []byte("not an image")
|
||||||
|
case "gif":
|
||||||
|
body, contentType = []byte("GIF89a\x01\x00\x01\x00\x00\x00\x00"), "image/gif"
|
||||||
|
case "dimensions":
|
||||||
|
body, contentType = []byte("GIF89a\xff\xff\xff\xff\x00\x00\x00"), "image/gif"
|
||||||
|
case "type":
|
||||||
|
contentType = "image/jpeg"
|
||||||
|
case "unknown":
|
||||||
|
return errors.New("invalid image body")
|
||||||
|
}
|
||||||
|
fmt.Fprintf(out, "HTTP/1.1 200 OK\r\nContent-Type: %s\r\nContent-Length: %d\r\n\r\n", contentType, len(body))
|
||||||
|
if _, err := out.Write(body); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
metrics, _ := json.Marshal(map[string]interface{}{"url_effective": raw, "ssl_verify_result": 0, "num_redirects": 0, "http_code": 200, "time_total": 0.1, "speed_download": 10000, "size_download": len(body)})
|
||||||
|
if mode == "footer" {
|
||||||
|
metrics = []byte("{malformed")
|
||||||
|
}
|
||||||
|
fmt.Fprint(out, resourceTrailer, string(metrics))
|
||||||
|
protocol := "TLSv1.2"
|
||||||
|
if strings.Contains(strings.Join(args, " "), "--tlsv1.3") {
|
||||||
|
protocol = "TLSv1.3"
|
||||||
|
}
|
||||||
|
fmt.Fprintln(diag, "SSL connection using "+protocol)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestImageBodyRejectionCollectorBoundaries(t *testing.T) {
|
||||||
|
for _, mode := range []string{"jpeg", "png", "magic", "gif", "dimensions", "type", "unknown", "footer"} {
|
||||||
|
t.Run(mode, func(t *testing.T) {
|
||||||
|
m, err := collectAssetMeasurement(context.Background(), "socks5://127.0.0.1:1080", "https://i.ytimg.com/image", "i.ytimg.com", "tls12", "", rejectedImageRunner(mode))
|
||||||
|
if err == nil || len(m.Transfers) != 0 {
|
||||||
|
t.Fatal("invalid image qualified", m, err)
|
||||||
|
}
|
||||||
|
ordinary := mode != "unknown" && mode != "footer"
|
||||||
|
var fatal *managedRecoveryFatal
|
||||||
|
if errors.As(err, &fatal) == ordinary || managedAssetTransferNonpass(err) != ordinary || managedAssetHealthNonpass(err) != ordinary {
|
||||||
|
t.Fatal("wrong rejection class", err)
|
||||||
|
}
|
||||||
|
if managedGooglevideoTransferNonpass(err) {
|
||||||
|
t.Fatal("image rejection became media absence")
|
||||||
|
}
|
||||||
|
for _, veto := range []error{context.Canceled, errors.New("unknown infrastructure"), &managedRecoveryFatal{Err: imageBodyRejection("invalid image body")}} {
|
||||||
|
mixed := resourceFailure(context.Background(), errors.Join(err, veto))
|
||||||
|
if !errors.As(mixed, &fatal) || managedAssetTransferNonpass(mixed) || managedAssetHealthNonpass(mixed) {
|
||||||
|
t.Fatal("mixed safety veto lost", mixed)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
_, _, metadataErr := playerMediaResource([]byte("{malformed"))
|
||||||
|
if metadataErr == nil {
|
||||||
|
t.Fatal("malformed Player JSON accepted")
|
||||||
|
}
|
||||||
|
for _, err := range []error{metadataErr, errors.New("invalid normalized image evidence")} {
|
||||||
|
failure := resourceFailure(context.Background(), err)
|
||||||
|
var fatal *managedRecoveryFatal
|
||||||
|
if !errors.As(failure, &fatal) || managedAssetTransferNonpass(failure) || managedGooglevideoTransferNonpass(failure) {
|
||||||
|
t.Fatal("local metadata damage became absence", failure)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
oversized := assetTransfer{Body: make([]byte, (5<<20)+1)}
|
||||||
|
if err := normalizeAssetBody(&oversized); err == nil || !managedAssetTransferNonpass(resourceFailure(context.Background(), err)) {
|
||||||
|
t.Fatal("image decoder budget rejection lost", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestManagedTruncatedImageMonitorRecovery(t *testing.T) {
|
||||||
|
for _, recoverable := range []bool{false, true} {
|
||||||
|
t.Run(fmt.Sprint(recoverable), func(t *testing.T) {
|
||||||
|
d, front, _ := recoveryFixture(t, true)
|
||||||
|
target, _, _ := managedCLIFixture(t)
|
||||||
|
initManagedCLI(t, target)
|
||||||
|
d.transaction.Dir = target.directory
|
||||||
|
r, events, _ := assetRuntimeFixture(t)
|
||||||
|
good, start := r.curl, r.start
|
||||||
|
attempts := 0
|
||||||
|
r.start = func(args []string) (managedAssetWorker, error) { attempts++; return start(args) }
|
||||||
|
r.curl = func(ctx context.Context, args []string, in io.Reader, out, diag io.Writer) error {
|
||||||
|
if recoverable && attempts >= 2 {
|
||||||
|
return good(ctx, args, in, out, diag)
|
||||||
|
}
|
||||||
|
return rejectedImageRunner("jpeg")(ctx, args, in, out, diag)
|
||||||
|
}
|
||||||
|
ggc := managedCandidateFixture(t, d.db, "googlevideo", "fresh.googlevideo.com", "--lua-desync=fake", true)
|
||||||
|
ids := []int64{front, ggc}
|
||||||
|
if err := d.transaction.Apply(d.db, ids); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
d.service = true
|
||||||
|
d.frontHealth = func(context.Context) (string, error) { return "OK", nil }
|
||||||
|
d.resources = func(context.Context) (managedServiceInventory, error) {
|
||||||
|
return managedServiceInventory{images: map[string]string{"i.ytimg.com": "https://i.ytimg.com/image"}, proxy: r.proxy, runner: r.curl}, nil
|
||||||
|
}
|
||||||
|
d.verifyAssets = r.verify
|
||||||
|
d.health = managedServiceHealth(r.proxy, d.frontHealth, d.resources, r.curl)
|
||||||
|
monitor := monitorRuntimeFixture(t)
|
||||||
|
monitor.db, monitor.target, monitor.threshold = d.db, target, 1
|
||||||
|
if err := wireManagedRecovery(&monitor, d, func(error) {}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
monitor.reportRecoveryError = func(err error) {
|
||||||
|
if recoverable || !errors.Is(err, errNoQualifiedStrategy) {
|
||||||
|
t.Fatal("wrong recovery result", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
waits := 0
|
||||||
|
monitor.wait = func(ctx context.Context, _ time.Duration) error {
|
||||||
|
state, err := readMonitorState(d.db, monitor.serviceID)
|
||||||
|
if err != nil || state.RecoveryRunning {
|
||||||
|
t.Fatal("recovery ownership retained", state, err)
|
||||||
|
}
|
||||||
|
if !recoverable && state.CooldownUntil.Before(monitor.now().Add(2*time.Hour)) {
|
||||||
|
t.Fatal("missing two-hour pause", state)
|
||||||
|
}
|
||||||
|
waits++
|
||||||
|
if waits == 2 {
|
||||||
|
cancel()
|
||||||
|
return ctx.Err()
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if err := runManagedMonitor(ctx, monitor); !errors.Is(err, context.Canceled) {
|
||||||
|
t.Fatal("invalid payload killed monitor", err, *events)
|
||||||
|
}
|
||||||
|
want := len(managedAssetSeeds)
|
||||||
|
if recoverable {
|
||||||
|
want = 2
|
||||||
|
}
|
||||||
|
joins := 0
|
||||||
|
for _, event := range *events {
|
||||||
|
if event == "join" {
|
||||||
|
joins++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if attempts != want || joins != want {
|
||||||
|
t.Fatal("seed fallback/worker join failed", attempts, joins, *events)
|
||||||
|
}
|
||||||
|
active, err := d.transaction.ActiveCandidates(d.db)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if recoverable {
|
||||||
|
if len(active) != 3 {
|
||||||
|
t.Fatal("fresh image not applied", active)
|
||||||
|
}
|
||||||
|
fresh, err := readStrategyCandidate(d.db, active[2])
|
||||||
|
if err != nil || !fresh.Qualified || fresh.Args != managedAssetSeeds[1] {
|
||||||
|
t.Fatal("invalid fresh evidence", fresh, err)
|
||||||
|
}
|
||||||
|
ids = append(ids, fresh.ID)
|
||||||
|
var rows int
|
||||||
|
if err := d.db.QueryRow("SELECT count(*) FROM candidate_evidence WHERE candidate_id=? AND passes_ok=3 AND passes_total=3", fresh.ID).Scan(&rows); err != nil || rows != 2 {
|
||||||
|
t.Fatal("missing both TLS 3/3 rows", rows, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
assertActiveCandidates(t, d.transaction, d.db, ids)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -3,13 +3,16 @@ package main
|
|||||||
import (
|
import (
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
"net"
|
|
||||||
"os"
|
"os"
|
||||||
)
|
)
|
||||||
|
|
||||||
const usage = `Usage: zapret-autotune <command>
|
const usage = `Usage: zapret-autotune <command>
|
||||||
|
|
||||||
Commands:
|
Commands:
|
||||||
|
dns-forwarder mode|address|serve
|
||||||
|
Managed DNS configuration or loopback DoH forwarder
|
||||||
|
dns-forwarder readiness-check|readiness-cleanup <expected-dns-pid>
|
||||||
|
Validate or clean readiness using inherited PID 1 anchors
|
||||||
status Show latest recorded tune run/proposal (not live service health)
|
status Show latest recorded tune run/proposal (not live service health)
|
||||||
history [--limit N] List recorded tune attempts (default 20; proposals not active)
|
history [--limit N] List recorded tune attempts (default 20; proposals not active)
|
||||||
strategies youtube List recorded proposals by group (not active)
|
strategies youtube List recorded proposals by group (not active)
|
||||||
@@ -18,7 +21,13 @@ Commands:
|
|||||||
health youtube One-shot Player API health through configured loopback SOCKS
|
health youtube One-shot Player API health through configured loopback SOCKS
|
||||||
managed init|reconcile|status|apply|monitor
|
managed init|reconcile|status|apply|monitor
|
||||||
Guarded activation/monitoring in the dedicated managed-test target only
|
Guarded activation/monitoring in the dedicated managed-test target only
|
||||||
|
managed monitor --resource-config FILE
|
||||||
|
Periodic configured checks only (no recovery/apply)
|
||||||
|
managed resource-status
|
||||||
|
Read-only configured-check snapshot (explicit stale/blocked)
|
||||||
check youtube Refuse checks until probe URLs/video IDs are configured and agreed
|
check youtube Refuse checks until probe URLs/video IDs are configured and agreed
|
||||||
|
check --config FILE
|
||||||
|
Check configured HTTPS resources (checks only; no strategy selection)
|
||||||
tune youtube Test youtube.com frontend only; print options without applying
|
tune youtube Test youtube.com frontend only; print options without applying
|
||||||
tune googlevideo --video-id ID --proxy socks5://HOST:PORT
|
tune googlevideo --video-id ID --proxy socks5://HOST:PORT
|
||||||
Discover and verify one exact cache host; proposal only
|
Discover and verify one exact cache host; proposal only
|
||||||
@@ -36,13 +45,30 @@ func run(args []string, stdout, stderr io.Writer) int {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func runWithPlayer(args []string, stdout, stderr io.Writer, player playerRunner) int {
|
func runWithPlayer(args []string, stdout, stderr io.Writer, player playerRunner) int {
|
||||||
|
if len(args) == 3 && args[0] == "check" && args[1] == "--config" {
|
||||||
|
return configuredResourceCheck(args[2], stdout, stderr)
|
||||||
|
}
|
||||||
|
if len(args) > 0 && args[0] == "dns-forwarder" {
|
||||||
|
return dnsForwarderCLI(args[1:], stdout, stderr)
|
||||||
|
}
|
||||||
if len(args) == 1 && args[0] == "target-config-check" {
|
if len(args) == 1 && args[0] == "target-config-check" {
|
||||||
if _, err := parseHealthConfig(os.LookupEnv); err != nil {
|
path, _ := os.LookupEnv("ZAPRET_AUTOTUNE_RESOURCE_CONFIG")
|
||||||
|
if path != "" {
|
||||||
|
port, err := managedTargetSettings(os.LookupEnv)
|
||||||
|
if err != nil || port != 1080 {
|
||||||
|
fmt.Fprintln(stderr, "configerror: invalid generic target settings")
|
||||||
|
return 64
|
||||||
|
}
|
||||||
|
if _, _, err := resourceStaticConfig(os.LookupEnv); err != nil {
|
||||||
|
fmt.Fprintln(stderr, "configerror: invalid generic DNS/proxy configuration")
|
||||||
|
return 64
|
||||||
|
}
|
||||||
|
} else if _, err := parseManagedHealthConfig(os.LookupEnv); err != nil {
|
||||||
fmt.Fprintln(stderr, "configerror: invalid target health/service configuration")
|
fmt.Fprintln(stderr, "configerror: invalid target health/service configuration")
|
||||||
return 64
|
return 64
|
||||||
}
|
}
|
||||||
if dns := os.Getenv("SS_DNS_SERVER"); dns != "" && net.ParseIP(dns) == nil {
|
if _, err := parseManagedDNSConfig(os.LookupEnv); err != nil {
|
||||||
fmt.Fprintln(stderr, "configerror: SS_DNS_SERVER requires a literal IPv4 or IPv6 address")
|
fmt.Fprintln(stderr, "configerror: invalid managed DNS configuration")
|
||||||
return 64
|
return 64
|
||||||
}
|
}
|
||||||
return 0
|
return 0
|
||||||
|
|||||||
@@ -41,8 +41,12 @@ func newManagedAssetVerifier(target managedDeps) func(context.Context, storedCan
|
|||||||
|
|
||||||
func newManagedAssetRuntime(target managedDeps) managedAssetRuntime {
|
func newManagedAssetRuntime(target managedDeps) managedAssetRuntime {
|
||||||
proxy, _ := target.lookup("ZAPRET_AUTOTUNE_SOCKS_PROXY")
|
proxy, _ := target.lookup("ZAPRET_AUTOTUNE_SOCKS_PROXY")
|
||||||
|
proxy, proxyErr := managedProbeProxy(proxy)
|
||||||
r := managedAssetRuntime{proxy: proxy, curl: runResourceCurl}
|
r := managedAssetRuntime{proxy: proxy, curl: runResourceCurl}
|
||||||
r.guard = func() error {
|
r.guard = func() error {
|
||||||
|
if proxyErr != nil {
|
||||||
|
return proxyErr
|
||||||
|
}
|
||||||
if _, err := managedRuntimeGuard(target); err != nil {
|
if _, err := managedRuntimeGuard(target); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
@@ -347,9 +351,17 @@ func managedAssetTransferNonpass(err error) bool {
|
|||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
switch e := err.(type) {
|
switch e := err.(type) {
|
||||||
|
case *managedRecoveryFatal:
|
||||||
|
return false
|
||||||
|
case *resourceError:
|
||||||
|
return managedResourceTransferNonpass(e, false, true)
|
||||||
|
case imageBodyRejection:
|
||||||
|
return true
|
||||||
|
case *resourcePeerTLSRejection:
|
||||||
|
return true
|
||||||
case *exec.ExitError:
|
case *exec.ExitError:
|
||||||
switch e.ExitCode() {
|
switch e.ExitCode() {
|
||||||
case 6, 7, 28, 35, 51, 52, 55, 56, 60:
|
case 6, 28, 35, 51, 52, 55, 56, 60:
|
||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
return false
|
return false
|
||||||
|
|||||||
@@ -38,6 +38,27 @@ func (e *managedResourceFailure) PublicError() string {
|
|||||||
}
|
}
|
||||||
func (e *managedResourceFailure) Unwrap() error { return e.Err }
|
func (e *managedResourceFailure) Unwrap() error { return e.Err }
|
||||||
|
|
||||||
|
// Only a typed collector's complete known-negative error set may lose its
|
||||||
|
// generic fatal wrapper at a live-health boundary. Missing resources remain
|
||||||
|
// DEGRADED; joined guard/infrastructure/cancellation failures retain their type.
|
||||||
|
func managedResourceHealthError(ctx context.Context, err error) error {
|
||||||
|
// Only a direct, known parser absence is missing inventory. Never flatten
|
||||||
|
// joined guard/infrastructure failures into an unrelated image absence.
|
||||||
|
if ctx.Err() == nil {
|
||||||
|
if failure, ok := err.(*managedResourceFailure); ok && failure.ProbeKind == "avatar" && failure.Host == "yt3.ggpht.com" {
|
||||||
|
if code, ok := failure.Err.(avatarMetadataError); ok && (code == "markerAbsent" || code == "jsonInvalid" || code == "noAllowedAvatar") {
|
||||||
|
return &managedResourceFailure{ProbeKind: failure.ProbeKind, Host: failure.Host}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
var failure *managedResourceFailure
|
||||||
|
var collector *resourceError
|
||||||
|
if ctx.Err() == nil && errors.As(err, &failure) && (failure.ProbeKind == "media" || assetEndpoint(failure.Host) && failure.ProbeKind == managedImageProbeKind(failure.Host)) && errors.As(failure.Err, &collector) && managedResourceTransferNonpass(err, assetEndpoint(failure.Host), assetEndpoint(failure.Host)) {
|
||||||
|
return &managedResourceFailure{ProbeKind: failure.ProbeKind, Host: failure.Host}
|
||||||
|
}
|
||||||
|
return managedRecoveryContextError(ctx, err)
|
||||||
|
}
|
||||||
|
|
||||||
// Ephemeral inventory: never serialize URLs, page bodies or Player responses.
|
// Ephemeral inventory: never serialize URLs, page bodies or Player responses.
|
||||||
type managedServiceInventory struct {
|
type managedServiceInventory struct {
|
||||||
images map[string]string
|
images map[string]string
|
||||||
@@ -50,10 +71,21 @@ type managedServiceInventory struct {
|
|||||||
func managedServiceResources(cfg healthConfig, player func(context.Context) ([]byte, error), page playerRunner) func(context.Context) (managedServiceInventory, error) {
|
func managedServiceResources(cfg healthConfig, player func(context.Context) ([]byte, error), page playerRunner) func(context.Context) (managedServiceInventory, error) {
|
||||||
return func(ctx context.Context) (managedServiceInventory, error) {
|
return func(ctx context.Context) (managedServiceInventory, error) {
|
||||||
i := managedServiceInventory{images: make(map[string]string), proxy: cfg.SOCKSProxy, runner: runResourceCurl}
|
i := managedServiceInventory{images: make(map[string]string), proxy: cfg.SOCKSProxy, runner: runResourceCurl}
|
||||||
|
if err := ctx.Err(); err != nil {
|
||||||
|
return i, managedRecoveryContextError(ctx, err)
|
||||||
|
}
|
||||||
|
proxy, err := managedProbeProxy(cfg.SOCKSProxy)
|
||||||
|
if err != nil {
|
||||||
|
return i, &managedRecoveryFatal{Err: err}
|
||||||
|
}
|
||||||
|
i.proxy = proxy
|
||||||
body, err := player(ctx)
|
body, err := player(ctx)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return i, &managedResourceFailure{"front", managedFrontendHost, err}
|
return i, &managedResourceFailure{"front", managedFrontendHost, err}
|
||||||
}
|
}
|
||||||
|
if err := ctx.Err(); err != nil {
|
||||||
|
return i, managedRecoveryContextError(ctx, err)
|
||||||
|
}
|
||||||
var p struct {
|
var p struct {
|
||||||
VideoDetails struct {
|
VideoDetails struct {
|
||||||
Thumbnail struct {
|
Thumbnail struct {
|
||||||
@@ -75,20 +107,44 @@ func managedServiceResources(cfg healthConfig, player func(context.Context) ([]b
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
i.resourceErr = &managedResourceFailure{"media", "unknown", err}
|
i.resourceErr = &managedResourceFailure{"media", "unknown", err}
|
||||||
}
|
}
|
||||||
|
for attempt := 0; attempt < 2; attempt++ {
|
||||||
|
if err := ctx.Err(); err != nil {
|
||||||
|
return i, managedRecoveryContextError(ctx, err)
|
||||||
|
}
|
||||||
|
origin := "https://www.youtube.com/"
|
||||||
|
if attempt == 1 {
|
||||||
|
if !videoIDPattern.MatchString(cfg.VideoID) {
|
||||||
|
return i, &managedRecoveryFatal{Err: errors.New("invalid video ID")}
|
||||||
|
}
|
||||||
|
origin = "https://www.youtube.com/watch?v=" + cfg.VideoID
|
||||||
|
}
|
||||||
bounded, cancel := context.WithTimeout(ctx, 35*time.Second)
|
bounded, cancel := context.WithTimeout(ctx, 35*time.Second)
|
||||||
defer cancel()
|
|
||||||
var output bytes.Buffer
|
var output bytes.Buffer
|
||||||
out := &playerBoundedWriter{dst: &output, limit: 5 << 20}
|
out := &playerBoundedWriter{dst: &output, limit: 5 << 20}
|
||||||
// Fixed root origin; no redirects, credentials, user URL or diagnostics.
|
// Fixed trusted WWW origin; no redirects, credentials or diagnostics.
|
||||||
err = page(bounded, []string{"--disable", "--silent", "--show-error", "--fail", "--noproxy", "", "--proxy", cfg.SOCKSProxy, "--proto", "=https", "--max-time", "30", "--max-filesize", "5242880", "https://www.youtube.com/"}, out, io.Discard)
|
err = page(bounded, []string{"--disable", "--silent", "--show-error", "--fail", "--noproxy", "", "--proxy", proxy, "--proto", "=https", "--max-time", "30", "--max-filesize", "5242880", origin}, out, io.Discard)
|
||||||
|
requestErr := bounded.Err()
|
||||||
|
cancel()
|
||||||
if ctx.Err() != nil {
|
if ctx.Err() != nil {
|
||||||
return i, managedRecoveryContextError(ctx, err)
|
return i, managedRecoveryContextError(ctx, err)
|
||||||
}
|
}
|
||||||
if err != nil || bounded.Err() != nil || out.exceeded {
|
if err != nil || requestErr != nil || out.exceeded {
|
||||||
return i, &managedResourceFailure{"avatar", "yt3.ggpht.com", resourceFailure(bounded, errors.Join(err, bounded.Err(), errors.New("fresh root metadata request failed")))}
|
cause := errors.Join(err, requestErr)
|
||||||
|
if out.exceeded {
|
||||||
|
cause = errors.Join(cause, errors.New("fresh metadata exceeds limit"))
|
||||||
}
|
}
|
||||||
if raw, e := avatarResourceFromPage(output.Bytes()); e == nil {
|
return i, managedResourceHealthError(ctx, &managedResourceFailure{"avatar", "yt3.ggpht.com", resourceFailure(ctx, cause)})
|
||||||
|
}
|
||||||
|
raw, parseErr := avatarResourceFromPage(output.Bytes())
|
||||||
|
if parseErr == nil {
|
||||||
i.images["yt3.ggpht.com"] = raw
|
i.images["yt3.ggpht.com"] = raw
|
||||||
|
return i, nil
|
||||||
|
}
|
||||||
|
// Only missing structural metadata permits one fresh watch request.
|
||||||
|
var missing avatarMetadataError
|
||||||
|
if attempt == 1 || !errors.As(parseErr, &missing) || (missing != "markerAbsent" && missing != "noAllowedAvatar") {
|
||||||
|
return i, managedResourceHealthError(ctx, &managedResourceFailure{"avatar", "yt3.ggpht.com", parseErr})
|
||||||
|
}
|
||||||
}
|
}
|
||||||
return i, nil
|
return i, nil
|
||||||
}
|
}
|
||||||
@@ -397,6 +453,14 @@ func managedAssetInventoryHealth(ctx context.Context, i managedServiceInventory,
|
|||||||
// Every joined leaf must be a known nonpass; mixed safety failures stay fatal.
|
// Every joined leaf must be a known nonpass; mixed safety failures stay fatal.
|
||||||
func managedAssetHealthNonpass(err error) bool {
|
func managedAssetHealthNonpass(err error) bool {
|
||||||
switch e := err.(type) {
|
switch e := err.(type) {
|
||||||
|
case *managedRecoveryFatal:
|
||||||
|
return false
|
||||||
|
case *resourceError:
|
||||||
|
cause := e.cause
|
||||||
|
if fatal, ok := cause.(*managedRecoveryFatal); ok {
|
||||||
|
cause = fatal.Err
|
||||||
|
}
|
||||||
|
return managedAssetHealthNonpass(cause)
|
||||||
case interface{ Unwrap() []error }:
|
case interface{ Unwrap() []error }:
|
||||||
children := e.Unwrap()
|
children := e.Unwrap()
|
||||||
if len(children) == 0 {
|
if len(children) == 0 {
|
||||||
@@ -414,7 +478,7 @@ func managedAssetHealthNonpass(err error) bool {
|
|||||||
if err == nil || err == context.Canceled {
|
if err == nil || err == context.Canceled {
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
return managedAssetTransferNonpass(err) || err.Error() == "image body exceeds limit" || err.Error() == "invalid image dimensions" || err.Error() == "invalid image body"
|
return managedAssetTransferNonpass(err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,173 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"io"
|
||||||
|
"os/exec"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestAvatarPageClassification(t *testing.T) {
|
||||||
|
for _, tc := range []struct{ body, code string }{
|
||||||
|
{`<html></html>`, "markerAbsent"},
|
||||||
|
{`var ytInitialData = {broken};`, "jsonInvalid"},
|
||||||
|
{`var ytInitialData = {"url":"https://yt3.ggpht.com.attacker.invalid/avatar"};`, "noAllowedAvatar"},
|
||||||
|
{`var ytInitialData = {"url":"http://yt3.ggpht.com/avatar"};`, "noAllowedAvatar"},
|
||||||
|
{`var ytInitialData = {"url":"https://user@yt3.ggpht.com/avatar"};`, "noAllowedAvatar"},
|
||||||
|
{`var ytInitialData = {"url":"https://yt3.ggpht.com/avatar?private=fixture"};`, "valid"},
|
||||||
|
} {
|
||||||
|
t.Run(tc.code, func(t *testing.T) {
|
||||||
|
raw, err := avatarResourceFromPage([]byte(tc.body))
|
||||||
|
if tc.code == "valid" {
|
||||||
|
if err != nil || raw == "" {
|
||||||
|
t.Fatal("missing validated resource", err)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
var classified avatarMetadataError
|
||||||
|
if raw != "" || !errors.As(err, &classified) || string(classified) != tc.code {
|
||||||
|
t.Fatal("unexpected classification", err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestManagedAvatarInventoryFallback(t *testing.T) {
|
||||||
|
exit := func(code string) error { return exec.Command("sh", "-c", "exit "+code).Run() }
|
||||||
|
for _, tc := range []struct {
|
||||||
|
name, home, watch, video string
|
||||||
|
pageErr, watchErr error
|
||||||
|
calls int
|
||||||
|
avatar, fatal bool
|
||||||
|
}{
|
||||||
|
{name: "homeNoAvatarWatchHasValidatedURL", home: `var ytInitialData = {};`, watch: `var ytInitialData = {"url":"https://yt3.ggpht.com/fresh?private=fixture"};`, calls: 2, avatar: true},
|
||||||
|
{name: "homeAlreadyValid", home: `var ytInitialData = {"url":"https://yt3.ggpht.com/fresh"};`, calls: 1, avatar: true},
|
||||||
|
{name: "bothMissing", home: `var ytInitialData = {};`, watch: `<html></html>`, calls: 2},
|
||||||
|
{name: "malformedHome", home: `var ytInitialData = {bad};`, calls: 1},
|
||||||
|
{name: "malformedWatch", home: `var ytInitialData = {};`, watch: `var ytInitialData = {bad};`, calls: 2},
|
||||||
|
{name: "watchHTTPMissing", home: `var ytInitialData = {};`, watchErr: exit("22"), calls: 2},
|
||||||
|
{name: "watchTimeout", home: `var ytInitialData = {};`, watchErr: exit("28"), calls: 2},
|
||||||
|
{name: "watchWrongCN", home: `var ytInitialData = {};`, watchErr: exit("60"), calls: 2},
|
||||||
|
{name: "watchMixedGuard", home: `var ytInitialData = {};`, watchErr: errors.Join(exit("28"), errors.New("owned proxy guard failed")), calls: 2, fatal: true},
|
||||||
|
{name: "untrustedWatchURL", watch: `var ytInitialData = {"url":"https://yt3.ggpht.com.evil.invalid/avatar"};`, calls: 2},
|
||||||
|
{name: "invalidVideo", video: "bad&injection", calls: 1, fatal: true},
|
||||||
|
{name: "timeoutNoFallback", pageErr: exit("28"), calls: 1},
|
||||||
|
{name: "wrongCNNoFallback", pageErr: exit("60"), calls: 1},
|
||||||
|
{name: "mixedGuardNoFallback", pageErr: errors.Join(exit("28"), errors.New("owned proxy guard failed")), calls: 1, fatal: true},
|
||||||
|
{name: "oversizedHomeNoFallback", home: strings.Repeat("x", (5<<20)+1), calls: 1, fatal: true},
|
||||||
|
} {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
cfg := healthConfig{SOCKSProxy: "socks5://127.0.0.1:11080", VideoID: "dQw4w9WgXcQ"}
|
||||||
|
if tc.video != "" {
|
||||||
|
cfg.VideoID = tc.video
|
||||||
|
}
|
||||||
|
calls := 0
|
||||||
|
resources := managedServiceResources(cfg, func(context.Context) ([]byte, error) {
|
||||||
|
return []byte(`{"playabilityStatus":{"status":"OK"},"videoDetails":{"thumbnail":{"thumbnails":[{"url":"https://i.ytimg.com/fresh"}]}},"streamingData":{"formats":[{"url":"https://fresh.googlevideo.com/private","mimeType":"video/mp4"}]}}`), nil
|
||||||
|
}, func(ctx context.Context, args []string, out, diagnostic io.Writer) error {
|
||||||
|
calls++
|
||||||
|
if _, ok := ctx.Deadline(); !ok || diagnostic != io.Discard {
|
||||||
|
t.Fatal("unbounded metadata or diagnostics")
|
||||||
|
}
|
||||||
|
joined := strings.Join(args, " ")
|
||||||
|
if strings.Contains(joined, "private=") || strings.Contains(joined, "--insecure") || strings.Contains(joined, "--location") || !strings.Contains(joined, "--proto =https") || !strings.Contains(joined, "--max-filesize 5242880") || !strings.Contains(joined, "--proxy socks5h://127.0.0.1:11080") {
|
||||||
|
t.Fatal("unsafe metadata request")
|
||||||
|
}
|
||||||
|
assertManagedDNSArgs(t, args, "socks5h://127.0.0.1:11080")
|
||||||
|
body, origin := tc.home, "https://www.youtube.com/"
|
||||||
|
if calls == 2 {
|
||||||
|
body, origin = tc.watch, "https://www.youtube.com/watch?v="+cfg.VideoID
|
||||||
|
}
|
||||||
|
if args[len(args)-1] != origin {
|
||||||
|
t.Fatal("unexpected metadata origin")
|
||||||
|
}
|
||||||
|
if tc.pageErr != nil {
|
||||||
|
return tc.pageErr
|
||||||
|
}
|
||||||
|
if calls == 2 && tc.watchErr != nil {
|
||||||
|
return tc.watchErr
|
||||||
|
}
|
||||||
|
_, err := io.WriteString(out, body)
|
||||||
|
return err
|
||||||
|
})
|
||||||
|
i, err := resources(context.Background())
|
||||||
|
var fatal *managedRecoveryFatal
|
||||||
|
if calls != tc.calls || (i.images["yt3.ggpht.com"] != "") != tc.avatar || errors.As(err, &fatal) != tc.fatal {
|
||||||
|
t.Fatal("fallback or guard contract", calls, err)
|
||||||
|
}
|
||||||
|
if (err == nil) != tc.avatar || i.images["i.ytimg.com"] == "" || i.media.url == "" || i.resourceErr != nil {
|
||||||
|
t.Fatal("missing avatar masked or partial inventory lost", err)
|
||||||
|
}
|
||||||
|
if err != nil && strings.Contains(err.Error(), "private") {
|
||||||
|
t.Fatal("private metadata leaked")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestManagedAvatarMetadataHealthClassification(t *testing.T) {
|
||||||
|
for _, code := range []avatarMetadataError{"markerAbsent", "jsonInvalid", "noAllowedAvatar"} {
|
||||||
|
failure := &managedResourceFailure{"avatar", "yt3.ggpht.com", code}
|
||||||
|
err := managedResourceHealthError(context.Background(), failure)
|
||||||
|
var missing *managedResourceFailure
|
||||||
|
if !errors.As(err, &missing) || missing.Err != nil {
|
||||||
|
t.Fatal("known metadata absence not normalized", code)
|
||||||
|
}
|
||||||
|
for _, mixed := range []error{
|
||||||
|
errors.Join(failure, errors.New("owned proxy guard failed")),
|
||||||
|
&managedResourceFailure{"avatar", "yt3.ggpht.com", errors.Join(code, errors.New("collector unavailable"))},
|
||||||
|
&managedResourceFailure{"avatar", "yt3.ggpht.com", errors.Join(code, context.Canceled)},
|
||||||
|
} {
|
||||||
|
err := managedResourceHealthError(context.Background(), mixed)
|
||||||
|
if !errors.As(err, &missing) || missing.Err == nil {
|
||||||
|
t.Fatal("mixed safety failure normalized", code)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
unknown := avatarMetadataError("unknown")
|
||||||
|
err := managedResourceHealthError(context.Background(), &managedResourceFailure{"avatar", "yt3.ggpht.com", unknown})
|
||||||
|
if !errors.Is(err, unknown) {
|
||||||
|
t.Fatal("unknown parser error normalized")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestManagedAvatarInventoryCancellationNoFallback(t *testing.T) {
|
||||||
|
for _, timing := range []string{"before-player", "after-player", "during-home"} {
|
||||||
|
t.Run(timing, func(t *testing.T) {
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
if timing == "before-player" {
|
||||||
|
cancel()
|
||||||
|
}
|
||||||
|
players, pages := 0, 0
|
||||||
|
resources := managedServiceResources(healthConfig{VideoID: "dQw4w9WgXcQ", SOCKSProxy: "socks5h://127.0.0.1:11080"}, func(context.Context) ([]byte, error) {
|
||||||
|
players++
|
||||||
|
if timing == "after-player" {
|
||||||
|
cancel()
|
||||||
|
}
|
||||||
|
return []byte(`{"playabilityStatus":{"status":"OK"}}`), nil
|
||||||
|
}, func(ctx context.Context, args []string, out, _ io.Writer) error {
|
||||||
|
pages++
|
||||||
|
assertManagedDNSArgs(t, args, "socks5h://127.0.0.1:11080")
|
||||||
|
if args[len(args)-1] != "https://www.youtube.com/" {
|
||||||
|
t.Fatal("watch fallback after cancellation", args)
|
||||||
|
}
|
||||||
|
cancel()
|
||||||
|
_, err := io.WriteString(out, `var ytInitialData = {};`)
|
||||||
|
return err
|
||||||
|
})
|
||||||
|
i, err := resources(ctx)
|
||||||
|
wantPlayers, wantPages := 1, 0
|
||||||
|
if timing == "before-player" {
|
||||||
|
wantPlayers = 0
|
||||||
|
} else if timing == "during-home" {
|
||||||
|
wantPages = 1
|
||||||
|
}
|
||||||
|
if players != wantPlayers || pages != wantPages || !errors.Is(err, context.Canceled) || i.images["yt3.ggpht.com"] != "" {
|
||||||
|
t.Fatal("cancellation lost or request after cancellation", players, pages, err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -272,6 +272,7 @@ func managedReport(m *managedTransaction, out io.Writer, openDB func() (*sql.DB,
|
|||||||
return errManagedDrift
|
return errManagedDrift
|
||||||
}
|
}
|
||||||
groups := map[string]bool{}
|
groups := map[string]bool{}
|
||||||
|
operatorCoverage := map[string]bool{}
|
||||||
assetHosts := map[string]bool{}
|
assetHosts := map[string]bool{}
|
||||||
known := j.State == "committed" && j.Version >= 2 && string(b) == managedBaseline && len(j.Candidates) == 0
|
known := j.State == "committed" && j.Version >= 2 && string(b) == managedBaseline && len(j.Candidates) == 0
|
||||||
if j.State == "committed" && j.Version >= 2 && len(j.Candidates) != 0 {
|
if j.State == "committed" && j.Version >= 2 && len(j.Candidates) != 0 {
|
||||||
@@ -290,6 +291,7 @@ func managedReport(m *managedTransaction, out io.Writer, openDB func() (*sql.DB,
|
|||||||
break
|
break
|
||||||
}
|
}
|
||||||
groups[c.Group] = true
|
groups[c.Group] = true
|
||||||
|
operatorCoverage[c.Group] = true
|
||||||
if experimentalGooglevideo(j.Scope, c) {
|
if experimentalGooglevideo(j.Scope, c) {
|
||||||
groups[c.Group] = false
|
groups[c.Group] = false
|
||||||
}
|
}
|
||||||
@@ -304,6 +306,7 @@ func managedReport(m *managedTransaction, out io.Writer, openDB func() (*sql.DB,
|
|||||||
}
|
}
|
||||||
if !known {
|
if !known {
|
||||||
groups = map[string]bool{}
|
groups = map[string]bool{}
|
||||||
|
operatorCoverage = map[string]bool{}
|
||||||
// Unvalidated journal references are not active recovery candidates.
|
// Unvalidated journal references are not active recovery candidates.
|
||||||
j.Candidates = nil
|
j.Candidates = nil
|
||||||
}
|
}
|
||||||
@@ -321,6 +324,19 @@ func managedReport(m *managedTransaction, out io.Writer, openDB func() (*sql.DB,
|
|||||||
if (groups["assets"] || (len(requestAssets) != 0 && requestAssets[0])) && (!known || len(assetHosts) != 2) {
|
if (groups["assets"] || (len(requestAssets) != 0 && requestAssets[0])) && (!known || len(assetHosts) != 2) {
|
||||||
missing = append(missing, "assets")
|
missing = append(missing, "assets")
|
||||||
}
|
}
|
||||||
|
// Coverage labels active profiles, not qualification or the full host inventory.
|
||||||
|
covered := []string{}
|
||||||
|
for _, group := range []string{"frontend", "assets", "googlevideo"} {
|
||||||
|
if operatorCoverage[group] {
|
||||||
|
covered = append(covered, group)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
scope := j.Scope
|
||||||
|
if !known {
|
||||||
|
scope = "unknown"
|
||||||
|
} else if operatorCoverage["googlevideo"] && scope == "" {
|
||||||
|
scope = GooglevideoScopeExact
|
||||||
|
}
|
||||||
if activeOnly {
|
if activeOnly {
|
||||||
if !known {
|
if !known {
|
||||||
return fmt.Errorf("active candidate IDs unresolved; deployment preserved by hash")
|
return fmt.Errorf("active candidate IDs unresolved; deployment preserved by hash")
|
||||||
@@ -328,7 +344,9 @@ func managedReport(m *managedTransaction, out io.Writer, openDB func() (*sql.DB,
|
|||||||
return json.NewEncoder(out).Encode(struct {
|
return json.NewEncoder(out).Encode(struct {
|
||||||
IDs []int64 `json:"active_candidate_ids"`
|
IDs []int64 `json:"active_candidate_ids"`
|
||||||
Missing []string `json:"unresolved_groups"`
|
Missing []string `json:"unresolved_groups"`
|
||||||
}{append([]int64{}, j.Candidates...), missing})
|
Covered []string `json:"covered_groups"`
|
||||||
|
Scope GooglevideoScope `json:"googlevideo_scope"`
|
||||||
|
}{append([]int64{}, j.Candidates...), missing, covered, scope})
|
||||||
}
|
}
|
||||||
groupText := strings.Join(active, ",")
|
groupText := strings.Join(active, ",")
|
||||||
if groupText == "" {
|
if groupText == "" {
|
||||||
@@ -341,33 +359,29 @@ func managedReport(m *managedTransaction, out io.Writer, openDB func() (*sql.DB,
|
|||||||
if known {
|
if known {
|
||||||
reportIDs = j.Candidates
|
reportIDs = j.Candidates
|
||||||
}
|
}
|
||||||
_, err = fmt.Fprintf(out, "managed-test generation=%d hash=%s state=%s groups=%s active_candidate_ids=%v unresolved_groups=%s\nLive health: NOT CHECKED (Player API is not GGC/video playback)\n", j.Generation, j.Hash, j.State, groupText, reportIDs, strings.Join(missing, ","))
|
coverageText := strings.Join(covered, ",")
|
||||||
|
if coverageText == "" {
|
||||||
|
coverageText = "none"
|
||||||
|
if !known {
|
||||||
|
coverageText = "unknown"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
_, err = fmt.Fprintf(out, "managed-test generation=%d hash=%s state=%s groups=%s active_candidate_ids=%v unresolved_groups=%s covered=%s\nLive health: NOT CHECKED (Player API is not GGC/video playback)\nCoverage: active config profiles only, not qualification or all-host/asset-inventory success.\n", j.Generation, j.Hash, j.State, groupText, reportIDs, strings.Join(missing, ","), coverageText)
|
||||||
if err == nil && j.Version == 3 {
|
if err == nil && j.Version == 3 {
|
||||||
_, err = fmt.Fprintf(out, "Googlevideo scope: %s (suffix is authorized extrapolation, not all-host qualification)\n", j.Scope)
|
if known && scope == GooglevideoScopeExperimental {
|
||||||
|
_, err = fmt.Fprintf(out, "Googlevideo scope: %s (experimental/unqualified; one authorized composite profile, not all hosts checked)\n", scope)
|
||||||
|
} else {
|
||||||
|
_, err = fmt.Fprintf(out, "Googlevideo scope: %s (suffix is authorized extrapolation, not all-host qualification)\n", scope)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
return err
|
return err
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
func managedRuntimeGuard(d managedDeps) (int, error) {
|
func managedRuntimeGuard(d managedDeps) (int, error) {
|
||||||
role, _ := d.lookup("ZAPRET_AUTOTUNE_TARGET_ROLE")
|
port, err := managedTargetSettings(d.lookup)
|
||||||
if role != "managed-test" {
|
if err != nil {
|
||||||
return 2, fmt.Errorf("ZAPRET_AUTOTUNE_TARGET_ROLE must be managed-test")
|
return 2, err
|
||||||
}
|
|
||||||
for key, expected := range map[string]string{
|
|
||||||
"ZAPRET_AUTOTUNE_TARGET_ID": managedTargetID,
|
|
||||||
"ZAPRET_AUTOTUNE_TARGET_DIR": managedTargetDirectory,
|
|
||||||
"ZAPRET_CONFIG": managedTargetConfig,
|
|
||||||
} {
|
|
||||||
if value, _ := d.lookup(key); value != expected {
|
|
||||||
return 2, fmt.Errorf("%s does not match the fresh managed target contract", key)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
portText, _ := d.lookup("SOCKS_PORT")
|
|
||||||
port, portErr := strconv.Atoi(portText)
|
|
||||||
proxy, _ := d.lookup("ZAPRET_AUTOTUNE_SOCKS_PROXY")
|
|
||||||
if portErr != nil || port < 1 || port > 65535 || strconv.Itoa(port) != portText || proxy != "socks5://127.0.0.1:"+portText {
|
|
||||||
return 2, fmt.Errorf("managed SOCKS proxy must match the target SOCKS_PORT listener")
|
|
||||||
}
|
}
|
||||||
if d.verifyRuntime == nil || d.verifySOCKS == nil {
|
if d.verifyRuntime == nil || d.verifySOCKS == nil {
|
||||||
return 1, fmt.Errorf("managed runtime validators unavailable")
|
return 1, fmt.Errorf("managed runtime validators unavailable")
|
||||||
@@ -381,7 +395,34 @@ func managedRuntimeGuard(d managedDeps) (int, error) {
|
|||||||
return 0, nil
|
return 0, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func managedTargetSettings(lookup func(string) (string, bool)) (int, error) {
|
||||||
|
role, _ := lookup("ZAPRET_AUTOTUNE_TARGET_ROLE")
|
||||||
|
if role != "managed-test" {
|
||||||
|
return 2, fmt.Errorf("ZAPRET_AUTOTUNE_TARGET_ROLE must be managed-test")
|
||||||
|
}
|
||||||
|
for key, expected := range map[string]string{
|
||||||
|
"ZAPRET_AUTOTUNE_TARGET_ID": managedTargetID,
|
||||||
|
"ZAPRET_AUTOTUNE_TARGET_DIR": managedTargetDirectory,
|
||||||
|
"ZAPRET_CONFIG": managedTargetConfig,
|
||||||
|
} {
|
||||||
|
if value, _ := lookup(key); value != expected {
|
||||||
|
return 2, fmt.Errorf("%s does not match the fresh managed target contract", key)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
portText, _ := lookup("SOCKS_PORT")
|
||||||
|
port, portErr := strconv.Atoi(portText)
|
||||||
|
proxy, _ := lookup("ZAPRET_AUTOTUNE_SOCKS_PROXY")
|
||||||
|
proxy, proxyErr := managedProbeProxy(proxy)
|
||||||
|
if portErr != nil || port < 1 || port > 65535 || strconv.Itoa(port) != portText || proxyErr != nil || proxy != "socks5h://127.0.0.1:"+portText {
|
||||||
|
return 2, fmt.Errorf("managed SOCKS proxy must match the target SOCKS_PORT listener")
|
||||||
|
}
|
||||||
|
return port, nil
|
||||||
|
}
|
||||||
|
|
||||||
func managedCLI(args []string, stdout, stderr io.Writer, d managedDeps) int {
|
func managedCLI(args []string, stdout, stderr io.Writer, d managedDeps) int {
|
||||||
|
if len(args) == 1 && args[0] == "resource-status" {
|
||||||
|
return resourceStatusCLI(stdout, d)
|
||||||
|
}
|
||||||
if len(args) > 0 && args[0] == "monitor" {
|
if len(args) > 0 && args[0] == "monitor" {
|
||||||
return managedMonitorCLI(args[1:], stdout, stderr, d)
|
return managedMonitorCLI(args[1:], stdout, stderr, d)
|
||||||
}
|
}
|
||||||
@@ -506,7 +547,7 @@ func managedActivate(m *managedTransaction, d managedDeps, ids []int64, expected
|
|||||||
if err != nil || !info.Mode().IsRegular() || info.Mode().Perm()&0111 == 0 {
|
if err != nil || !info.Mode().IsRegular() || info.Mode().Perm()&0111 == 0 {
|
||||||
return fmt.Errorf("target zapret daemon init executable unavailable")
|
return fmt.Errorf("target zapret daemon init executable unavailable")
|
||||||
}
|
}
|
||||||
cfg, err := parseHealthConfig(d.lookup)
|
cfg, err := parseManagedHealthConfig(d.lookup)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -47,7 +47,7 @@ func managedCLIFixture(t *testing.T) (managedDeps, map[string]string, *int) {
|
|||||||
return nil
|
return nil
|
||||||
},
|
},
|
||||||
player: func(_ context.Context, args []string, stdout, _ io.Writer) error {
|
player: func(_ context.Context, args []string, stdout, _ io.Writer) error {
|
||||||
if !strings.Contains(strings.Join(args, "|"), "--noproxy||--proxy|socks5://127.0.0.1:11080|") {
|
if !strings.Contains(strings.Join(args, "|"), "--noproxy||--proxy|socks5h://127.0.0.1:11080|") {
|
||||||
t.Fatalf("unsafe health argv: %q", args)
|
t.Fatalf("unsafe health argv: %q", args)
|
||||||
}
|
}
|
||||||
_, err := io.WriteString(stdout, `{"playabilityStatus":{"status":"OK"}}`)
|
_, err := io.WriteString(stdout, `{"playabilityStatus":{"status":"OK"}}`)
|
||||||
@@ -458,6 +458,144 @@ func TestManagedCLILegacyIDsUnresolved(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestManagedStatusOperatorCoverage(t *testing.T) {
|
||||||
|
for _, mode := range []string{"qualified", "experimental", "missing-candidate", "missing-db", "render-mismatch", "invalid-hash", "legacy", "pending"} {
|
||||||
|
t.Run(mode, func(t *testing.T) {
|
||||||
|
db := testStore(t)
|
||||||
|
front := managedCandidateFixture(t, db, "frontend", "www.youtube.com", "--lua-desync=fake", true)
|
||||||
|
ids := []int64{front}
|
||||||
|
scope := GooglevideoScopeExact
|
||||||
|
if mode == "qualified" {
|
||||||
|
ids = append(ids, managedCandidateFixture(t, db, "googlevideo", "rr1.googlevideo.com", "--lua-desync=fake", true))
|
||||||
|
} else {
|
||||||
|
asset := unitAssetCandidate(t, db, "i.ytimg.com", "--lua-desync=fake", true)
|
||||||
|
tx, err := db.Begin()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
ggc, err := insertStrategyCandidate(tx, "googlevideo", []string{"rr1---sn-alpha.googlevideo.com"}, managedGooglevideoArgs, nil)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := tx.Commit(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
ids = append(ids, asset, ggc)
|
||||||
|
scope = GooglevideoScopeExperimental
|
||||||
|
c, err := readStrategyCandidate(db, ggc)
|
||||||
|
if err != nil || c.Qualified || c.Mode != QualificationCold {
|
||||||
|
t.Fatalf("experimental candidate: %+v %v", c, err)
|
||||||
|
}
|
||||||
|
var evidence int
|
||||||
|
if err := db.QueryRow("SELECT count(*) FROM candidate_evidence WHERE candidate_id=?", ggc).Scan(&evidence); err != nil || evidence != 0 {
|
||||||
|
t.Fatalf("fabricated evidence: %d %v", evidence, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
m, _ := managedFixture(t)
|
||||||
|
if err := m.Apply(db, ids, scope); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
j, err := m.load()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
openDB := func() (*sql.DB, error) { return openHistoryStore(databasePath()) }
|
||||||
|
switch mode {
|
||||||
|
case "missing-candidate":
|
||||||
|
j.Candidates[len(ids)-1] = 999999
|
||||||
|
case "missing-db":
|
||||||
|
openDB = func() (*sql.DB, error) { return nil, errors.New("unavailable") }
|
||||||
|
case "render-mismatch":
|
||||||
|
b := []byte("# not the rendered candidates\n" + managedBaseline)
|
||||||
|
if err := os.WriteFile(filepath.Join(m.Dir, m.Name), b, 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
j.Hash = managedHash(b)
|
||||||
|
case "invalid-hash":
|
||||||
|
j.Hash = managedHash([]byte("wrong hash"))
|
||||||
|
case "legacy":
|
||||||
|
j.Version, j.Scope = 2, ""
|
||||||
|
case "pending":
|
||||||
|
j.State, j.Next = "prepared", j.Hash
|
||||||
|
j.PendingCandidates, j.PendingScope = append([]int64{}, ids...), scope
|
||||||
|
}
|
||||||
|
if err := m.journal(j); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
before, err := os.ReadFile(filepath.Join(m.Dir, m.Name))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
journalBefore, err := os.ReadFile(filepath.Join(m.Dir, managedJournalName))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
m.Health = func() (string, error) { t.Fatal("status checked live health"); return "", nil }
|
||||||
|
m.Reload = func() error { t.Fatal("status reloaded target"); return nil }
|
||||||
|
var out bytes.Buffer
|
||||||
|
err = managedReport(m, &out, openDB, false)
|
||||||
|
if mode == "invalid-hash" {
|
||||||
|
if !errors.Is(err, errManagedDrift) || out.Len() != 0 {
|
||||||
|
t.Fatalf("drift trusted: %s %v", out.String(), err)
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
if err != nil || !strings.Contains(out.String(), "Live health: NOT CHECKED") || strings.Contains(out.String(), "READY") {
|
||||||
|
t.Fatalf("status: %s %v", out.String(), err)
|
||||||
|
}
|
||||||
|
want := []string{"groups=unknown ", "covered=unknown", "active_candidate_ids=[] ", "unresolved_groups=frontend,googlevideo"}
|
||||||
|
if mode == "qualified" {
|
||||||
|
want = []string{"groups=frontend,googlevideo ", "covered=frontend,googlevideo", "unresolved_groups= covered="}
|
||||||
|
} else if mode == "experimental" {
|
||||||
|
want = []string{"groups=frontend,assets ", "covered=frontend,assets,googlevideo", "unresolved_groups=googlevideo,assets ", "experimental/unqualified", "one authorized composite profile", "not all hosts checked"}
|
||||||
|
}
|
||||||
|
for _, text := range want {
|
||||||
|
if !strings.Contains(out.String(), text) {
|
||||||
|
t.Fatalf("missing %q: %s", text, out.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
out.Reset()
|
||||||
|
err = managedReport(m, &out, openDB, true)
|
||||||
|
if mode == "qualified" || mode == "experimental" {
|
||||||
|
var state struct {
|
||||||
|
IDs []int64 `json:"active_candidate_ids"`
|
||||||
|
Missing []string `json:"unresolved_groups"`
|
||||||
|
Covered []string `json:"covered_groups"`
|
||||||
|
Scope GooglevideoScope `json:"googlevideo_scope"`
|
||||||
|
}
|
||||||
|
if err != nil || json.Unmarshal(out.Bytes(), &state) != nil || !reflect.DeepEqual(state.IDs, ids) || state.Scope != scope {
|
||||||
|
t.Fatalf("JSON: %s %v", out.String(), err)
|
||||||
|
}
|
||||||
|
covered, missing := []string{"frontend", "googlevideo"}, []string(nil)
|
||||||
|
if mode == "experimental" {
|
||||||
|
covered, missing = []string{"frontend", "assets", "googlevideo"}, []string{"googlevideo", "assets"}
|
||||||
|
}
|
||||||
|
if !reflect.DeepEqual(state.Covered, covered) || !reflect.DeepEqual(state.Missing, missing) {
|
||||||
|
t.Fatalf("coverage changed qualification: %+v", state)
|
||||||
|
}
|
||||||
|
encoded, err := json.Marshal(state)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var roundtrip map[string]json.RawMessage
|
||||||
|
if err := json.Unmarshal(encoded, &roundtrip); err != nil || len(roundtrip) != 4 {
|
||||||
|
t.Fatalf("JSON roundtrip: %s %v", encoded, err)
|
||||||
|
}
|
||||||
|
} else if err == nil || out.Len() != 0 {
|
||||||
|
t.Fatalf("invalid IDs trusted: %s %v", out.String(), err)
|
||||||
|
}
|
||||||
|
after, err := os.ReadFile(filepath.Join(m.Dir, m.Name))
|
||||||
|
if err != nil || !bytes.Equal(before, after) {
|
||||||
|
t.Fatal("status changed config", err)
|
||||||
|
}
|
||||||
|
journalAfter, err := os.ReadFile(filepath.Join(m.Dir, managedJournalName))
|
||||||
|
if err != nil || !bytes.Equal(journalBefore, journalAfter) {
|
||||||
|
t.Fatal("status changed journal", err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestManagedMountpoint(t *testing.T) {
|
func TestManagedMountpoint(t *testing.T) {
|
||||||
for _, path := range []string{"/", filepath.Dir(managedTargetDirectory), managedTargetDirectory + "-other"} {
|
for _, path := range []string{"/", filepath.Dir(managedTargetDirectory), managedTargetDirectory + "-other"} {
|
||||||
if managedMountpoint("1 2 0:3 / "+path+" rw - tmpfs tmpfs rw\n", managedTargetDirectory) {
|
if managedMountpoint("1 2 0:3 / "+path+" rw - tmpfs tmpfs rw\n", managedTargetDirectory) {
|
||||||
|
|||||||
@@ -0,0 +1,65 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"os"
|
||||||
|
"reflect"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
// check_config.py provides sanitized actual `docker compose config` output.
|
||||||
|
// This bridges deployment interpolation to the production config parser offline.
|
||||||
|
func TestManagedGooglevideoComposeForwarding(t *testing.T) {
|
||||||
|
path := os.Getenv("AUTOTUNE_COMPOSE_CONFIG_FIXTURE")
|
||||||
|
if path == "" {
|
||||||
|
t.Skip("run deploy/managed-target/check_config.py for Compose fixture")
|
||||||
|
}
|
||||||
|
data, err := os.ReadFile(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal("compose fixture unavailable")
|
||||||
|
}
|
||||||
|
var cases []struct {
|
||||||
|
Environment map[string]string `json:"environment"`
|
||||||
|
IDs []string `json:"ids"`
|
||||||
|
}
|
||||||
|
if json.Unmarshal(data, &cases) != nil || len(cases) < 6 {
|
||||||
|
t.Fatal("invalid compose fixture")
|
||||||
|
}
|
||||||
|
for i, c := range cases {
|
||||||
|
lookup := func(key string) (string, bool) { value, ok := c.Environment[key]; return value, ok }
|
||||||
|
mode, ids, err := managedGooglevideoConfig(lookup, c.Environment["ZAPRET_AUTOTUNE_VIDEO_ID"])
|
||||||
|
if err != nil || string(mode) != c.Environment["ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE"] || !reflect.DeepEqual(ids, c.IDs) {
|
||||||
|
t.Fatalf("compose factory case %d mismatch", i)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestManagedGooglevideoBlankInventoryDefault(t *testing.T) {
|
||||||
|
for _, value := range []string{"", " ", "\n", "[]", "bad", `["aaaaaaaaaaa"]`} {
|
||||||
|
lookup := func(key string) (string, bool) {
|
||||||
|
switch key {
|
||||||
|
case "ZAPRET_AUTOTUNE_YOUTUBE_SERVICE":
|
||||||
|
return "1", true
|
||||||
|
case "ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE":
|
||||||
|
return "adaptive", true
|
||||||
|
case "ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS":
|
||||||
|
return value, true
|
||||||
|
}
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
mode, ids, err := managedGooglevideoConfig(lookup, "dQw4w9WgXcQ")
|
||||||
|
if value != "" && value != `["aaaaaaaaaaa"]` {
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("invalid nonempty inventory accepted")
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
want := []string{"dQw4w9WgXcQ", "aqz-KE-bpKQ", "jNQXAC9IVRw"}
|
||||||
|
if value != "" {
|
||||||
|
want = []string{"aaaaaaaaaaa"}
|
||||||
|
}
|
||||||
|
if err != nil || mode != QualificationAdaptive || !reflect.DeepEqual(ids, want) {
|
||||||
|
t.Fatal("inventory default/override mismatch")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -64,6 +64,7 @@ func renderManagedConfigPolicy(db *sql.DB, ids []int64, scope GooglevideoScope,
|
|||||||
seen := map[int64]bool{}
|
seen := map[int64]bool{}
|
||||||
assets := map[string]bool{}
|
assets := map[string]bool{}
|
||||||
var otherScopes []string
|
var otherScopes []string
|
||||||
|
nativeSuffix := false
|
||||||
for _, id := range ids {
|
for _, id := range ids {
|
||||||
if id <= 0 || seen[id] {
|
if id <= 0 || seen[id] {
|
||||||
return nil, fmt.Errorf("duplicate candidate %d", id)
|
return nil, fmt.Errorf("duplicate candidate %d", id)
|
||||||
@@ -110,6 +111,7 @@ func renderManagedConfigPolicy(db *sql.DB, ids []int64, scope GooglevideoScope,
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
if c.Group == "googlevideo" && scope == GooglevideoScopeSuffix {
|
if c.Group == "googlevideo" && scope == GooglevideoScopeSuffix {
|
||||||
|
nativeSuffix = c.Mode == QualificationAdaptive
|
||||||
if version < 7 || validateGooglevideoSuffixEvidencePolicy(db, c, fresh) != nil {
|
if version < 7 || validateGooglevideoSuffixEvidencePolicy(db, c, fresh) != nil {
|
||||||
return nil, fmt.Errorf("suffix requires representative qualified hosts and verified media evidence")
|
return nil, fmt.Errorf("suffix requires representative qualified hosts and verified media evidence")
|
||||||
}
|
}
|
||||||
@@ -125,8 +127,8 @@ func renderManagedConfigPolicy(db *sql.DB, ids []int64, scope GooglevideoScope,
|
|||||||
groups[c.Group] = true
|
groups[c.Group] = true
|
||||||
profiles = append(profiles, "--filter-tcp=443 --filter-l7=tls --hostlist-domains="+strings.Join(domains, ",")+" "+c.Args)
|
profiles = append(profiles, "--filter-tcp=443 --filter-l7=tls --hostlist-domains="+strings.Join(domains, ",")+" "+c.Args)
|
||||||
}
|
}
|
||||||
if scope == GooglevideoScopeSuffix && (!groups["googlevideo"] || len(assets) != 2) {
|
if scope == GooglevideoScopeSuffix && (!groups["googlevideo"] || nativeSuffix && !groups["frontend"] || !nativeSuffix && len(assets) != 2) {
|
||||||
return nil, fmt.Errorf("suffix requires googlevideo and both qualified asset profiles")
|
return nil, fmt.Errorf("suffix requires qualified googlevideo and frontend (legacy cold scope requires both assets)")
|
||||||
}
|
}
|
||||||
if scope == GooglevideoScopeExperimental && (!groups["googlevideo"] || !groups["frontend"] || len(assets) == 0) {
|
if scope == GooglevideoScopeExperimental && (!groups["googlevideo"] || !groups["frontend"] || len(assets) == 0) {
|
||||||
return nil, fmt.Errorf("experimental suffix requires frontend and existing qualified assets")
|
return nil, fmt.Errorf("experimental suffix requires frontend and existing qualified assets")
|
||||||
|
|||||||
@@ -0,0 +1,87 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"io"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func assertManagedDNSArgs(t *testing.T, args []string, proxy string) {
|
||||||
|
t.Helper()
|
||||||
|
joined := strings.Join(args, "|")
|
||||||
|
if !strings.Contains(joined, "--noproxy||--proxy|"+proxy+"|") {
|
||||||
|
t.Fatalf("missing managed remote DNS proxy: %q", args)
|
||||||
|
}
|
||||||
|
for _, arg := range args {
|
||||||
|
if arg == "-k" || arg == "--insecure" || strings.HasPrefix(arg, "--resolve") || strings.HasPrefix(arg, "--connect-to") {
|
||||||
|
t.Fatalf("DNS pin or TLS bypass: %q", args)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestManagedProbeProxy(t *testing.T) {
|
||||||
|
for _, raw := range []string{"socks5://127.0.0.1:1080", "socks5h://127.0.0.1:1080", "socks5://localhost:1080", "socks5h://[::1]:1080"} {
|
||||||
|
got, err := managedProbeProxy(raw)
|
||||||
|
if err != nil || got != strings.Replace(raw, "socks5://", "socks5h://", 1) {
|
||||||
|
t.Fatal(raw, got, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, raw := range []string{"", "http://127.0.0.1:1080", "socks5h://proxy.example:1080", "socks5h://user:secret@127.0.0.1:1080", "socks5h://127.0.0.1:0", "socks5h://127.0.0.1:65536", "socks5h://127.0.0.1:1080/", "socks5h://127.0.0.1:1080?", "socks5h://127.0.0.1:1080#", "socks5h://127.0.0.1:1080\n", "socks5h://127.0.0.1:1080\x7f"} {
|
||||||
|
if got, err := managedProbeProxy(raw); err == nil || got != "" || strings.Contains(err.Error(), "secret") {
|
||||||
|
t.Fatal("unsafe proxy accepted or leaked", got, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestManagedDNSProductionPlayerAndGuard(t *testing.T) {
|
||||||
|
for _, scheme := range []string{"socks5", "socks5h"} {
|
||||||
|
t.Run(scheme, func(t *testing.T) {
|
||||||
|
target, env, _ := managedCLIFixture(t)
|
||||||
|
env["ZAPRET_AUTOTUNE_SOCKS_PROXY"] = scheme + "://127.0.0.1:11080"
|
||||||
|
owned := 0
|
||||||
|
target.verifySOCKS = func(port int) error {
|
||||||
|
if port != 11080 {
|
||||||
|
t.Fatal(port)
|
||||||
|
}
|
||||||
|
owned++
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
requests := 0
|
||||||
|
target.player = func(_ context.Context, args []string, out, _ io.Writer) error {
|
||||||
|
assertManagedDNSArgs(t, args, "socks5h://127.0.0.1:11080")
|
||||||
|
if args[len(args)-1] != playerURL {
|
||||||
|
t.Fatal("Player hostname changed", args)
|
||||||
|
}
|
||||||
|
requests++
|
||||||
|
_, err := io.WriteString(out, `{"playabilityStatus":{"status":"OK"},"streamingData":{"serverAbrStreamingUrl":"https://fresh.googlevideo.com/video?private=secret"}}`)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
r, _, _ := recoveryFixture(t, false)
|
||||||
|
d, err := managedRecoveryDependencies(target, r.db, r.scan, r.check, r.run)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if status, err := d.health(context.Background()); err != nil || status != "OK" {
|
||||||
|
t.Fatal(status, err)
|
||||||
|
}
|
||||||
|
if host, err := d.discover(context.Background()); err != nil || host != "fresh.googlevideo.com" {
|
||||||
|
t.Fatal(host, err)
|
||||||
|
}
|
||||||
|
if requests != 2 || owned != 3 {
|
||||||
|
t.Fatal(requests, owned)
|
||||||
|
}
|
||||||
|
for _, proxy := range []string{scheme + "://127.0.0.1:1080", scheme + "://localhost:11080", scheme + "://[::1]:11080", scheme + "://proxy.example:11080"} {
|
||||||
|
env["ZAPRET_AUTOTUNE_SOCKS_PROXY"] = proxy
|
||||||
|
if _, err := managedRuntimeGuard(target); err == nil {
|
||||||
|
t.Fatal("unowned endpoint accepted", proxy)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
lookup := configLookup(map[string]string{"ZAPRET_AUTOTUNE_SOCKS_PROXY": "socks5://127.0.0.1:11080"})
|
||||||
|
legacy, err := parseHealthConfig(lookup)
|
||||||
|
if err != nil || legacy.SOCKSProxy != "socks5://127.0.0.1:11080" || validDiscoveryProxy("socks5h://127.0.0.1:11080") {
|
||||||
|
t.Fatal("legacy proxy contract changed", legacy, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -95,7 +95,7 @@ func managedExperimentalSuffix(m *managedTransaction, d managedDeps) error {
|
|||||||
if err != nil || fmt.Sprintf("%x", sha256.Sum256(lua)) != managedGooglevideoLuaHash {
|
if err != nil || fmt.Sprintf("%x", sha256.Sum256(lua)) != managedGooglevideoLuaHash {
|
||||||
return fmt.Errorf("native Lua identity mismatch")
|
return fmt.Errorf("native Lua identity mismatch")
|
||||||
}
|
}
|
||||||
cfg, err := parseHealthConfig(d.lookup)
|
cfg, err := parseManagedHealthConfig(d.lookup)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,222 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"os/exec"
|
||||||
|
"reflect"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestExperimentalProductionProbeCollectorClassification(t *testing.T) {
|
||||||
|
for _, mode := range []string{"bad-range", "healthy", "unknown", "infra", "mixed", "guard", "cancel", "metadata-timeout", "metadata-http", "metadata-malformed", "metadata-mixed", "metadata-cancel", "metadata-infra", "image-timeout", "image-mixed", "media-peer35", "media-peer60", "image-peer35", "image-peer60", "image-local58", "image-proxy7"} {
|
||||||
|
t.Run(mode, func(t *testing.T) {
|
||||||
|
d, front, _ := recoveryFixture(t, true)
|
||||||
|
target, _, _ := managedCLIFixture(t)
|
||||||
|
initManagedCLI(t, target)
|
||||||
|
d.transaction.Dir = target.directory
|
||||||
|
asset := unitAssetCandidate(t, d.db, "i.ytimg.com", "--lua-desync=fake", true)
|
||||||
|
id := managedCandidateFixture(t, d.db, "googlevideo", "rr1---sn-alpha.googlevideo.com", managedGooglevideoArgs, false)
|
||||||
|
ids := []int64{front, asset, id}
|
||||||
|
if err := d.transaction.Apply(d.db, ids, GooglevideoScopeExperimental); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
before, err := d.transaction.load()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
d.service = true
|
||||||
|
d.frontHealth = func(context.Context) (string, error) { return "OK", nil }
|
||||||
|
d.resources = func(context.Context) (managedServiceInventory, error) {
|
||||||
|
r := experimentalResourceFixture()
|
||||||
|
if mode == "unknown" {
|
||||||
|
r = mediaResource{}
|
||||||
|
}
|
||||||
|
return managedServiceInventory{images: map[string]string{"i.ytimg.com": "https://i.ytimg.com/test", "yt3.ggpht.com": "https://yt3.ggpht.com/test"}, media: r}, nil
|
||||||
|
}
|
||||||
|
knownTimeout := exec.Command("sh", "-c", "exit 28").Run()
|
||||||
|
if strings.HasPrefix(mode, "metadata-") {
|
||||||
|
d.resources = managedServiceResources(healthConfig{SOCKSProxy: "socks5://127.0.0.1:11080", VideoID: "dQw4w9WgXcQ"}, func(context.Context) ([]byte, error) {
|
||||||
|
return []byte(fmt.Sprintf(`{"playabilityStatus":{"status":"OK"},"streamingData":{"formats":[{"url":%q,"mimeType":"video/mp4"}]}}`, experimentalResourceFixture().url)), nil
|
||||||
|
}, func(_ context.Context, args []string, out, diagnostic io.Writer) error {
|
||||||
|
origin := args[len(args)-1]
|
||||||
|
if (origin != "https://www.youtube.com/" && !(mode == "metadata-malformed" && origin == "https://www.youtube.com/watch?v=dQw4w9WgXcQ")) || diagnostic != io.Discard {
|
||||||
|
t.Fatal("metadata origin/diagnostics changed")
|
||||||
|
}
|
||||||
|
switch mode {
|
||||||
|
case "metadata-malformed":
|
||||||
|
_, err := io.WriteString(out, "<html>not playable metadata</html>")
|
||||||
|
return err
|
||||||
|
case "metadata-http":
|
||||||
|
return exec.Command("sh", "-c", "exit 22").Run()
|
||||||
|
case "metadata-mixed":
|
||||||
|
return errors.Join(knownTimeout, &managedRecoveryFatal{Err: errors.New("guard failed")})
|
||||||
|
case "metadata-cancel":
|
||||||
|
return errors.Join(knownTimeout, context.Canceled)
|
||||||
|
case "metadata-infra":
|
||||||
|
return errors.Join(knownTimeout, errors.New("collector unavailable"))
|
||||||
|
default:
|
||||||
|
return knownTimeout
|
||||||
|
}
|
||||||
|
})
|
||||||
|
i, _ := d.resources(context.Background())
|
||||||
|
if !freshExperimentalMedia(i.media) {
|
||||||
|
t.Fatal("metadata failure discarded fresh media")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
calls := 0
|
||||||
|
d.recheckExperimental = func(_ context.Context, c storedCandidate, r mediaResource, _ *managedTransaction) error {
|
||||||
|
if c.ID != id || !freshExperimentalMedia(r) {
|
||||||
|
t.Fatal("recovery changed candidate or lost fresh media")
|
||||||
|
}
|
||||||
|
calls++
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
health := managedServiceHealth("socks5://127.0.0.1:11080", d.frontHealth, d.resources, func(_ context.Context, args []string, in io.Reader, out, diagnostic io.Writer) error {
|
||||||
|
if strings.HasPrefix(mode, "metadata-") {
|
||||||
|
t.Fatal("missing images must not become transfer evidence")
|
||||||
|
}
|
||||||
|
if mode == "image-timeout" {
|
||||||
|
return knownTimeout
|
||||||
|
}
|
||||||
|
if mode == "image-mixed" {
|
||||||
|
return errors.Join(knownTimeout, &managedRecoveryFatal{Err: errors.New("guard failed")})
|
||||||
|
}
|
||||||
|
config, _ := io.ReadAll(in)
|
||||||
|
raw := strings.TrimSuffix(strings.TrimPrefix(string(config), "url = \""), "\"\n")
|
||||||
|
isMedia := strings.Contains(raw, ".googlevideo.com/")
|
||||||
|
if strings.HasPrefix(mode, "media-peer") && isMedia || strings.HasPrefix(mode, "image-peer") && strings.Contains(raw, "yt3.ggpht.com/") {
|
||||||
|
return exec.Command("sh", "-c", "exit "+mode[len(mode)-2:]).Run()
|
||||||
|
}
|
||||||
|
if mode == "image-local58" {
|
||||||
|
return exec.Command("sh", "-c", "exit 58").Run()
|
||||||
|
}
|
||||||
|
if mode == "image-proxy7" {
|
||||||
|
return exec.Command("sh", "-c", "exit 7").Run()
|
||||||
|
}
|
||||||
|
var wire, diag bytes.Buffer
|
||||||
|
wireMode := ""
|
||||||
|
if strings.Contains(raw, ".googlevideo.com/") {
|
||||||
|
wireMode = "media"
|
||||||
|
if mode != "healthy" {
|
||||||
|
wireMode = "bad-range"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
resourceWire(raw, wireMode, &wire, &diag)
|
||||||
|
if strings.Contains(strings.Join(args, " "), "--tlsv1.3") {
|
||||||
|
diag = *bytes.NewBufferString(strings.ReplaceAll(diag.String(), "TLSv1.2", "TLSv1.3"))
|
||||||
|
}
|
||||||
|
_, _ = out.Write(wire.Bytes())
|
||||||
|
_, _ = diagnostic.Write(diag.Bytes())
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
d.health = func(ctx context.Context) (string, error) {
|
||||||
|
status, err := health(ctx)
|
||||||
|
if mode != "healthy" && status != "DEGRADED" {
|
||||||
|
t.Fatalf("resource failure became healthy: %s", status)
|
||||||
|
}
|
||||||
|
switch mode {
|
||||||
|
case "infra":
|
||||||
|
err = &managedResourceFailure{"media", experimentalResourceFixture().host, resourceFailure(ctx, errors.New("collector unavailable"))}
|
||||||
|
case "mixed":
|
||||||
|
err = errors.Join(err, &managedRecoveryFatal{Err: errors.New("guard failed")})
|
||||||
|
case "guard":
|
||||||
|
err = errors.Join(err, &managedRecoveryFatal{Err: errors.New("invalid media range or container")})
|
||||||
|
case "cancel":
|
||||||
|
err = errors.Join(err, context.Canceled)
|
||||||
|
}
|
||||||
|
return status, err
|
||||||
|
}
|
||||||
|
monitor := monitorRuntimeFixture(t)
|
||||||
|
monitor.db, monitor.target = d.db, target
|
||||||
|
monitor.threshold = 1
|
||||||
|
if err := wireManagedRecovery(&monitor, d, func(error) {}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
err = monitor.probe(context.Background())
|
||||||
|
var fatal *managedRecoveryFatal
|
||||||
|
switch mode {
|
||||||
|
case "healthy":
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
case "bad-range", "metadata-timeout", "metadata-http", "metadata-malformed", "image-timeout", "media-peer35", "media-peer60", "image-peer35", "image-peer60":
|
||||||
|
if err == nil || errors.As(err, &fatal) {
|
||||||
|
t.Fatalf("media damage killed monitor: %v", err)
|
||||||
|
}
|
||||||
|
case "unknown":
|
||||||
|
if err == nil || errors.Is(err, errNoQualifiedStrategy) {
|
||||||
|
t.Fatalf("unknown became green/absence: %v", err)
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
if !errors.As(err, &fatal) {
|
||||||
|
t.Fatalf("safety failure normalized: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if mode != "healthy" && mode != "unknown" {
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
monitor.wait = func(ctx context.Context, _ time.Duration) error { cancel(); return ctx.Err() }
|
||||||
|
err = runManagedMonitor(ctx, monitor)
|
||||||
|
if mode == "bad-range" || mode == "metadata-timeout" || mode == "metadata-http" || mode == "metadata-malformed" || mode == "image-timeout" || strings.Contains(mode, "-peer") {
|
||||||
|
if !errors.Is(err, context.Canceled) || errors.As(err, &fatal) || calls != 1 {
|
||||||
|
t.Fatalf("monitor recovery unreachable: %v calls=%d", err, calls)
|
||||||
|
}
|
||||||
|
} else if !errors.As(err, &fatal) || calls != 0 {
|
||||||
|
t.Fatalf("fatal probe entered recovery: %v calls=%d", err, calls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
after, err := d.transaction.load()
|
||||||
|
if err != nil || !reflect.DeepEqual(before, after) {
|
||||||
|
t.Fatalf("probe/recovery changed journal: %v", err)
|
||||||
|
}
|
||||||
|
assertActiveCandidates(t, d.transaction, d.db, ids)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestManagedResourceHealthClassificationKeepsSafetyAndEvidenceBoundaries(t *testing.T) {
|
||||||
|
timeout := exec.Command("sh", "-c", "exit 28").Run()
|
||||||
|
httpFailure := exec.Command("sh", "-c", "exit 22").Run()
|
||||||
|
if managedGooglevideoTransferNonpass(resourceFailure(context.Background(), httpFailure)) {
|
||||||
|
t.Fatal("metadata HTTP nonpass became media rejection evidence")
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
cancel()
|
||||||
|
for _, tc := range []struct {
|
||||||
|
ctx context.Context
|
||||||
|
err error
|
||||||
|
}{
|
||||||
|
{context.Background(), &managedRecoveryFatal{Err: timeout}},
|
||||||
|
{context.Background(), errors.Join(resourceFailure(context.Background(), timeout), &managedRecoveryFatal{Err: timeout})},
|
||||||
|
{context.Background(), errors.Join(resourceFailure(context.Background(), timeout), errors.New("unknown collector failure"))},
|
||||||
|
{ctx, resourceFailure(context.Background(), timeout)},
|
||||||
|
} {
|
||||||
|
err := managedResourceHealthError(tc.ctx, &managedResourceFailure{"avatar", "yt3.ggpht.com", tc.err})
|
||||||
|
var failure *managedResourceFailure
|
||||||
|
if !errors.As(err, &failure) || failure.Err == nil {
|
||||||
|
t.Fatalf("safety failure normalized: %v", err)
|
||||||
|
}
|
||||||
|
if tc.ctx.Err() != nil && !errors.Is(err, context.Canceled) {
|
||||||
|
t.Fatal("parent cancellation lost")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExperimentalProductionRecheckAdapterInstalled(t *testing.T) {
|
||||||
|
d, _, _ := recoveryFixture(t, true)
|
||||||
|
target, env, _ := managedCLIFixture(t)
|
||||||
|
env["ZAPRET_AUTOTUNE_YOUTUBE_SERVICE"] = "1"
|
||||||
|
adapted, err := managedMonitorRecoveryAdapters(target, d.db, d.scan, d.check, d.run)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if !adapted.service || adapted.resources == nil || adapted.recheckExperimental == nil || adapted.run == nil || adapted.transaction == nil {
|
||||||
|
t.Fatal("production recheck dependency missing")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,218 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/sha256"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
var errManagedExperimentalUnqualified = errors.New("experimental googlevideo suffix: operational health is not family qualification")
|
||||||
|
|
||||||
|
// URLs are trusted Player inventory, never rewritten or persisted.
|
||||||
|
func freshExperimentalMedia(r mediaResource) bool {
|
||||||
|
u, err := privateResourceURL(r.url, r.host)
|
||||||
|
if err != nil || !strings.HasSuffix(r.host, ".googlevideo.com") || u.Path != "/videoplayback" || (r.kind != "video/mp4" && r.kind != "audio/mp4") {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
q := u.Query()
|
||||||
|
expires, err := strconv.ParseInt(q.Get("expire"), 10, 64)
|
||||||
|
return err == nil && expires > time.Now().Unix()+120 && (q.Get("sig") != "" || q.Get("signature") != "" || q.Get("lsig") != "")
|
||||||
|
}
|
||||||
|
|
||||||
|
func experimentalMedia(ctx context.Context, d managedRecoveryDeps) (mediaResource, error) {
|
||||||
|
i, err := d.resources(ctx)
|
||||||
|
var fatal *managedRecoveryFatal
|
||||||
|
var failure *managedResourceFailure
|
||||||
|
// An unrelated metadata/image failure cannot conceal usable media, but a
|
||||||
|
// mixed safety/infrastructure error must not be turned into transfer absence.
|
||||||
|
unrelated := errors.As(err, &failure) && assetEndpoint(failure.Host) && (err == failure && failure.Err == nil || managedGooglevideoTransferNonpass(err))
|
||||||
|
if ctx.Err() != nil || errors.As(err, &fatal) && !unrelated || errors.Is(err, context.Canceled) || err != nil && !unrelated {
|
||||||
|
return mediaResource{}, managedRecoveryContextError(ctx, err)
|
||||||
|
}
|
||||||
|
if !freshExperimentalMedia(i.media) {
|
||||||
|
return mediaResource{}, &managedResourceFailure{"media", "unknown", nil}
|
||||||
|
}
|
||||||
|
return i.media, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func recoverManagedExperimental(ctx context.Context, d managedRecoveryDeps) (int64, error) {
|
||||||
|
if !d.service || d.resources == nil || d.recheckExperimental == nil {
|
||||||
|
return 0, errManagedGGCUnresolved
|
||||||
|
}
|
||||||
|
ids, err := d.transaction.ActiveCandidates(d.db)
|
||||||
|
if err != nil {
|
||||||
|
return 0, &managedRecoveryFatal{Err: err}
|
||||||
|
}
|
||||||
|
j, err := d.transaction.load()
|
||||||
|
if err != nil {
|
||||||
|
return 0, &managedRecoveryFatal{Err: err}
|
||||||
|
}
|
||||||
|
for _, id := range ids {
|
||||||
|
c, err := readStrategyCandidate(d.db, id)
|
||||||
|
if err != nil {
|
||||||
|
return 0, &managedRecoveryFatal{Err: err}
|
||||||
|
}
|
||||||
|
if experimentalGooglevideo(j.Scope, c) {
|
||||||
|
r, err := experimentalMedia(ctx, d)
|
||||||
|
if err == nil {
|
||||||
|
err = d.recheckExperimental(ctx, c, r, d.transaction)
|
||||||
|
}
|
||||||
|
if err == nil && d.mediaSamples != nil && d.verifyGooglevideo != nil {
|
||||||
|
// One-host operational health never promotes the experiment. Only
|
||||||
|
// fresh representative inventory and a completed owned proof may.
|
||||||
|
qualified, err := qualifyManagedGooglevideo(ctx, d)
|
||||||
|
if err != nil {
|
||||||
|
return id, err
|
||||||
|
}
|
||||||
|
return qualified, nil
|
||||||
|
}
|
||||||
|
return id, managedRecoveryContextError(ctx, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return 0, &managedRecoveryFatal{Err: errors.New("owned experimental candidate missing")}
|
||||||
|
}
|
||||||
|
|
||||||
|
func newManagedExperimentalRecheck(target managedDeps, d managedRecoveryDeps) func(context.Context, storedCandidate, mediaResource, *managedTransaction) error {
|
||||||
|
runtime := newManagedAssetRuntime(target)
|
||||||
|
return func(ctx context.Context, c storedCandidate, resource mediaResource, transaction *managedTransaction) error {
|
||||||
|
// The adapter is constructed with path-only dependencies. Use this job's
|
||||||
|
// bound hooks for ownership checks and reconciliation, not that stale copy.
|
||||||
|
d := d
|
||||||
|
d.transaction = transaction
|
||||||
|
guard := func() error {
|
||||||
|
if err := runtime.guard(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
lua, err := os.ReadFile("/opt/zapret2/lua/zapret-auto.lua")
|
||||||
|
if err != nil || fmt.Sprintf("%x", sha256.Sum256(lua)) != managedGooglevideoLuaHash {
|
||||||
|
return errors.New("unverified native automation Lua source")
|
||||||
|
}
|
||||||
|
_, err = d.transaction.ActiveCandidates(d.db)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
restart := func(ctx context.Context) (err error) {
|
||||||
|
// Existing daemon protocol stops/joins before starting the same full set.
|
||||||
|
// Never activate a pending or drifted deployment after a partial stop.
|
||||||
|
defer func() {
|
||||||
|
if err != nil {
|
||||||
|
cleanup, cancel := context.WithTimeout(context.WithoutCancel(ctx), 30*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
if e := d.transaction.Reconcile(d.db); e != nil {
|
||||||
|
err = errors.Join(err, e)
|
||||||
|
} else {
|
||||||
|
err = errors.Join(err, d.run(cleanup, managedZapretScript, "start"))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
if err = d.run(ctx, managedZapretScript, "stop"); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err = d.transaction.Reconcile(d.db); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return d.run(ctx, managedZapretScript, "start")
|
||||||
|
}
|
||||||
|
if !experimentalGooglevideo(GooglevideoScopeExperimental, c) {
|
||||||
|
return &managedRecoveryFatal{Err: errors.New("unaudited experimental candidate")}
|
||||||
|
}
|
||||||
|
ownedGuard := func() error {
|
||||||
|
if err := guard(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
j, err := d.transaction.load()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if j.Scope == GooglevideoScopeExperimental {
|
||||||
|
for _, id := range j.Candidates {
|
||||||
|
if id == c.ID {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return errors.New("experimental ownership changed")
|
||||||
|
}
|
||||||
|
return recheckDeployedExperimental(ctx, resource, ownedGuard, func(ctx context.Context) error {
|
||||||
|
if err := ownedGuard(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return restart(ctx)
|
||||||
|
}, func(ctx context.Context) (mediaResource, error) {
|
||||||
|
return experimentalMedia(ctx, d)
|
||||||
|
}, func(ctx context.Context, r mediaResource, protocol string) error {
|
||||||
|
_, _, err := collectMediaTransfer(ctx, runtime.proxy, r, protocol, runtime.curl)
|
||||||
|
return err
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// No temporary NFQ worker: successful learning stays in the deployed daemon RAM.
|
||||||
|
func recheckDeployedExperimental(ctx context.Context, resource mediaResource, guard func() error, restart func(context.Context) error, fresh func(context.Context) (mediaResource, error), transfer func(context.Context, mediaResource, string) error) error {
|
||||||
|
check := func(ctx context.Context, protocol string) error {
|
||||||
|
if err := ctx.Err(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := guard(); err != nil {
|
||||||
|
return &managedRecoveryFatal{Err: err}
|
||||||
|
}
|
||||||
|
if !freshExperimentalMedia(resource) {
|
||||||
|
var err error
|
||||||
|
resource, err = fresh(ctx)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if !freshExperimentalMedia(resource) {
|
||||||
|
return &managedResourceFailure{"media", "unknown", nil}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return managedRecoveryContextError(ctx, transfer(ctx, resource, protocol))
|
||||||
|
}
|
||||||
|
sample := func() error {
|
||||||
|
for _, protocol := range []string{"tls12", "tls13"} {
|
||||||
|
for n := 0; n < 3; n++ {
|
||||||
|
if err := check(ctx, protocol); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
err := sample()
|
||||||
|
if err == nil || !managedGooglevideoTransferNonpass(err) {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := restart(ctx); err != nil {
|
||||||
|
return &managedRecoveryFatal{Err: managedRecoveryContextError(ctx, err)}
|
||||||
|
}
|
||||||
|
warm, cancel := context.WithTimeout(ctx, 120*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
learned := map[string]bool{}
|
||||||
|
for n := 0; n < 10 && len(learned) < 2; n++ {
|
||||||
|
protocol := []string{"tls12", "tls13"}[n%2]
|
||||||
|
err = check(warm, protocol)
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
return managedRecoveryContextError(ctx, err)
|
||||||
|
}
|
||||||
|
if warm.Err() != nil && managedRecoveryTimeoutSafe(err) {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
if err != nil && !managedGooglevideoTransferNonpass(err) {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err == nil {
|
||||||
|
learned[protocol] = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(learned) == 2 {
|
||||||
|
err = sample()
|
||||||
|
if err == nil || !managedGooglevideoTransferNonpass(err) {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return fmt.Errorf("experimental media relearning exhausted: %w", errNoQualifiedStrategy)
|
||||||
|
}
|
||||||
@@ -0,0 +1,177 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"reflect"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func experimentalResourceFixture() mediaResource {
|
||||||
|
return mediaResource{url: fmt.Sprintf("https://rr2---sn-new.googlevideo.com/videoplayback?expire=%d&sig=private", time.Now().Add(time.Hour).Unix()), host: "rr2---sn-new.googlevideo.com", kind: "video/mp4"}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExperimentalDeployedRecheck(t *testing.T) {
|
||||||
|
for _, mode := range []string{"healthy", "learn", "exhausted", "mixed", "cancel", "guard"} {
|
||||||
|
t.Run(mode, func(t *testing.T) {
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
r := experimentalResourceFixture()
|
||||||
|
calls, restarts := 0, 0
|
||||||
|
protocols := []string{}
|
||||||
|
bad := errors.New("invalid media range or container")
|
||||||
|
infra := errors.New("collector unavailable")
|
||||||
|
err := recheckDeployedExperimental(ctx, r, func() error {
|
||||||
|
if mode == "guard" {
|
||||||
|
return infra
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}, func(context.Context) error { restarts++; return nil }, func(context.Context) (mediaResource, error) {
|
||||||
|
t.Fatal("unexpected refresh")
|
||||||
|
return mediaResource{}, infra
|
||||||
|
}, func(_ context.Context, got mediaResource, protocol string) error {
|
||||||
|
if got != r {
|
||||||
|
t.Fatal("signed resource rewritten")
|
||||||
|
}
|
||||||
|
calls++
|
||||||
|
protocols = append(protocols, protocol)
|
||||||
|
switch mode {
|
||||||
|
case "learn":
|
||||||
|
if calls == 1 {
|
||||||
|
return resourceFailure(ctx, bad)
|
||||||
|
}
|
||||||
|
case "exhausted":
|
||||||
|
return resourceFailure(ctx, bad)
|
||||||
|
case "mixed":
|
||||||
|
return errors.Join(resourceFailure(ctx, bad), infra)
|
||||||
|
case "cancel":
|
||||||
|
cancel()
|
||||||
|
return context.Canceled
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
switch mode {
|
||||||
|
case "healthy":
|
||||||
|
if err != nil || restarts != 0 || !reflect.DeepEqual(protocols, []string{"tls12", "tls12", "tls12", "tls13", "tls13", "tls13"}) {
|
||||||
|
t.Fatalf("%v %d %v", err, restarts, protocols)
|
||||||
|
}
|
||||||
|
case "learn":
|
||||||
|
if err != nil || restarts != 1 || calls != 9 {
|
||||||
|
t.Fatalf("%v restarts=%d calls=%d", err, restarts, calls)
|
||||||
|
}
|
||||||
|
case "exhausted":
|
||||||
|
if !errors.Is(err, errNoQualifiedStrategy) || restarts != 1 || calls != 11 {
|
||||||
|
t.Fatalf("%v restarts=%d calls=%d", err, restarts, calls)
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
if err == nil || errors.Is(err, errNoQualifiedStrategy) || restarts != 0 {
|
||||||
|
t.Fatalf("unsafe classification: %v restarts=%d", err, restarts)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExperimentalServiceRecoveryPreservesOwnership(t *testing.T) {
|
||||||
|
d, front, _ := recoveryFixture(t, true)
|
||||||
|
target, _, _ := managedCLIFixture(t)
|
||||||
|
initManagedCLI(t, target)
|
||||||
|
d.transaction.Dir = target.directory
|
||||||
|
asset := unitAssetCandidate(t, d.db, "i.ytimg.com", "--lua-desync=fake", true)
|
||||||
|
id := managedCandidateFixture(t, d.db, "googlevideo", "rr1---sn-alpha.googlevideo.com", managedGooglevideoArgs, false)
|
||||||
|
ids := []int64{front, asset, id}
|
||||||
|
if err := d.transaction.Apply(d.db, ids, GooglevideoScopeExperimental); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
before, err := d.transaction.load()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
d.service = true
|
||||||
|
d.frontHealth = func(context.Context) (string, error) { return "OK", nil }
|
||||||
|
d.health = func(context.Context) (string, error) { return "OK", nil }
|
||||||
|
d.resources = func(context.Context) (managedServiceInventory, error) {
|
||||||
|
return managedServiceInventory{media: experimentalResourceFixture()}, &managedResourceFailure{"avatar", "yt3.ggpht.com", nil}
|
||||||
|
}
|
||||||
|
calls := 0
|
||||||
|
var boundTransaction *managedTransaction
|
||||||
|
d.recheckExperimental = func(_ context.Context, c storedCandidate, r mediaResource, transaction *managedTransaction) error {
|
||||||
|
calls++
|
||||||
|
boundTransaction = transaction
|
||||||
|
if transaction.Reload == nil || transaction.Health == nil {
|
||||||
|
t.Fatal("recheck retained path-only adapter transaction")
|
||||||
|
}
|
||||||
|
assertActiveCandidates(t, transaction, d.db, ids)
|
||||||
|
if c.ID != id || !freshExperimentalMedia(r) {
|
||||||
|
t.Fatal("wrong committed candidate/resource")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
// Production adapters carry paths only until recoverManagedTarget binds the
|
||||||
|
// job's reload/health hooks. The recheck must receive that bound transaction.
|
||||||
|
d.transaction.Reload, d.transaction.Health = nil, nil
|
||||||
|
for _, adaptive := range []bool{false, true} {
|
||||||
|
d.adaptive = adaptive
|
||||||
|
result, err := recoverManagedTarget(context.Background(), d)
|
||||||
|
if err != nil || result.GooglevideoID != id || result.GGCError != nil {
|
||||||
|
t.Fatalf("%+v %v", result, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
after, err := d.transaction.load()
|
||||||
|
if err != nil || !reflect.DeepEqual(before, after) || calls != 2 {
|
||||||
|
t.Fatalf("journal changed: %v calls=%d", err, calls)
|
||||||
|
}
|
||||||
|
c, err := readStrategyCandidate(d.db, id)
|
||||||
|
if err != nil || c.Qualified {
|
||||||
|
t.Fatalf("qualification forged: %+v %v", c, err)
|
||||||
|
}
|
||||||
|
monitor := managedMonitorDeps{db: d.db, target: target}
|
||||||
|
reports := 0
|
||||||
|
if err := wireManagedRecovery(&monitor, d, func(err error) {
|
||||||
|
if errors.Is(err, errManagedExperimentalUnqualified) {
|
||||||
|
reports++
|
||||||
|
}
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := monitor.probe(context.Background()); err != nil || reports != 1 {
|
||||||
|
t.Fatalf("operational probe: %v reports=%d", err, reports)
|
||||||
|
}
|
||||||
|
assertActiveCandidates(t, boundTransaction, d.db, ids)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExperimentalRefreshAndMixedFatalPrecedence(t *testing.T) {
|
||||||
|
bad := errors.New("invalid media range or container")
|
||||||
|
for _, err := range []error{
|
||||||
|
&managedRecoveryFatal{Err: bad},
|
||||||
|
fmt.Errorf("wrapped: %w", &managedRecoveryFatal{Err: bad}),
|
||||||
|
errors.Join(resourceFailure(context.Background(), bad), &managedRecoveryFatal{Err: bad}),
|
||||||
|
errors.Join(resourceFailure(context.Background(), bad), context.Canceled),
|
||||||
|
} {
|
||||||
|
if managedGooglevideoTransferNonpass(err) {
|
||||||
|
t.Fatalf("fatal accepted: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
refreshes, transfers := 0, 0
|
||||||
|
err := recheckDeployedExperimental(context.Background(), mediaResource{}, func() error { return nil }, func(context.Context) error {
|
||||||
|
t.Fatal("unknown media caused restart")
|
||||||
|
return nil
|
||||||
|
}, func(context.Context) (mediaResource, error) {
|
||||||
|
refreshes++
|
||||||
|
return experimentalResourceFixture(), nil
|
||||||
|
}, func(context.Context, mediaResource, string) error { transfers++; return nil })
|
||||||
|
if err != nil || refreshes != 1 || transfers != 6 {
|
||||||
|
t.Fatalf("refresh: %v %d %d", err, refreshes, transfers)
|
||||||
|
}
|
||||||
|
err = recheckDeployedExperimental(context.Background(), mediaResource{}, func() error { return nil }, func(context.Context) error {
|
||||||
|
t.Fatal("unknown media caused restart")
|
||||||
|
return nil
|
||||||
|
}, func(context.Context) (mediaResource, error) {
|
||||||
|
return mediaResource{}, &managedResourceFailure{"media", "unknown", nil}
|
||||||
|
}, func(context.Context, mediaResource, string) error { t.Fatal("unknown media transferred"); return nil })
|
||||||
|
if err == nil || errors.Is(err, errNoQualifiedStrategy) {
|
||||||
|
t.Fatalf("unknown treated as absence: %v", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,87 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/sha256"
|
||||||
|
"database/sql"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"reflect"
|
||||||
|
)
|
||||||
|
|
||||||
|
func newManagedGooglevideoApplyRecheck(target managedDeps, db *sql.DB) func(context.Context, storedCandidate, []mediaResource, *managedTransaction) error {
|
||||||
|
r := newManagedAssetRuntime(target)
|
||||||
|
guard := r.guard
|
||||||
|
r.guard = func() error {
|
||||||
|
if err := guard(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
lua, err := os.ReadFile("/opt/zapret2/lua/zapret-auto.lua")
|
||||||
|
if err != nil || fmt.Sprintf("%x", sha256.Sum256(lua)) != managedGooglevideoLuaHash {
|
||||||
|
return errors.New("unverified native automation Lua source")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return managedGooglevideoApplyRecheck(r, db)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Runs inside Apply's lock: never ActiveCandidates/Reconcile/restart here.
|
||||||
|
// The newly reloaded daemon learns in-place; a failure rolls back via Apply.
|
||||||
|
func managedGooglevideoApplyRecheck(r managedAssetRuntime, db *sql.DB) func(context.Context, storedCandidate, []mediaResource, *managedTransaction) error {
|
||||||
|
return func(ctx context.Context, c storedCandidate, resources []mediaResource, m *managedTransaction) error {
|
||||||
|
j, err := m.load()
|
||||||
|
if err != nil {
|
||||||
|
return &managedRecoveryFatal{Err: err}
|
||||||
|
}
|
||||||
|
found := false
|
||||||
|
for _, id := range j.PendingCandidates {
|
||||||
|
found = found || id == c.ID
|
||||||
|
}
|
||||||
|
var hosts []string
|
||||||
|
for _, resource := range resources {
|
||||||
|
hosts = append(hosts, resource.host)
|
||||||
|
}
|
||||||
|
if j.State != "prepared" || j.PendingScope != GooglevideoScopeSuffix || !found || c.Mode != QualificationAdaptive || !reflect.DeepEqual(hosts, c.Endpoints) || validateGooglevideoSuffixEvidence(db, c) != nil {
|
||||||
|
return &managedRecoveryFatal{Err: errors.New("deployed media candidate ownership mismatch")}
|
||||||
|
}
|
||||||
|
guard := func() error {
|
||||||
|
if err := r.guard(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := m.owns(j, false); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := validateManagedRefs(db, j.PendingCandidates, j.Next, j.PendingScope); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
b, err := m.read(m.Name)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if managedHash(b) != j.Next {
|
||||||
|
return errManagedDrift
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
for _, resource := range resources {
|
||||||
|
err := recheckDeployedExperimental(ctx, resource, guard,
|
||||||
|
func(context.Context) error { return guard() }, // already reloaded; warm current daemon only
|
||||||
|
func(context.Context) (mediaResource, error) { return mediaResource{}, errCandidateNonpass },
|
||||||
|
func(ctx context.Context, resource mediaResource, protocol string) error {
|
||||||
|
_, _, err := collectMediaTransfer(ctx, r.proxy, resource, protocol, r.curl)
|
||||||
|
return err
|
||||||
|
})
|
||||||
|
if errors.Is(err, errNoQualifiedStrategy) && !managedRecoveryTimeoutSafe(err) {
|
||||||
|
var fatal *managedRecoveryFatal
|
||||||
|
if !errors.As(err, &fatal) {
|
||||||
|
return errCandidateNonpass
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,191 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func samplePlayerBody(host string, expires time.Time) []byte {
|
||||||
|
u, _ := json.Marshal(fmt.Sprintf("https://%s/videoplayback?expire=%d&sig=private-secret", host, expires.Unix()))
|
||||||
|
return []byte(fmt.Sprintf(`{"playabilityStatus":{"status":"OK"},"streamingData":{"formats":[{"mimeType":"video/mp4","url":%s}]}}`, u))
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestManagedGooglevideoDiscoveryRotatesFreshInventory(t *testing.T) {
|
||||||
|
ids := []string{"dQw4w9WgXcQ", "aqz-KE-bpKQ", "jNQXAC9IVRw"}
|
||||||
|
hosts := []string{"rr1---sn-alpha.googlevideo.com", "rr2---sn-beta.googlevideo.com", "rr5---sn-gamma.googlevideo.com"}
|
||||||
|
for _, scenario := range []string{"rotating", "unavailable", "expired", "same-root", "unsafe-origin", "invalid-host", "control", "unknown", "mixed-safety", "cancel"} {
|
||||||
|
t.Run(scenario, func(t *testing.T) {
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
calls := 0
|
||||||
|
collect := managedGooglevideoSamples(ids, func(ctx context.Context, id string) ([]byte, error) {
|
||||||
|
if _, ok := ctx.Deadline(); !ok {
|
||||||
|
t.Fatal("unbounded discovery")
|
||||||
|
}
|
||||||
|
if id != ids[calls%3] {
|
||||||
|
t.Fatal("not round robin")
|
||||||
|
}
|
||||||
|
calls++
|
||||||
|
host := hosts[(calls-1)%3]
|
||||||
|
expires := time.Now().Add(time.Hour)
|
||||||
|
if calls <= 3 {
|
||||||
|
switch scenario {
|
||||||
|
case "rotating":
|
||||||
|
host = hosts[0]
|
||||||
|
case "unavailable":
|
||||||
|
return nil, context.DeadlineExceeded
|
||||||
|
case "expired":
|
||||||
|
expires = time.Now().Add(5 * time.Minute)
|
||||||
|
case "unsafe-origin":
|
||||||
|
host += ":444"
|
||||||
|
case "invalid-host":
|
||||||
|
host = "not-a-ggc.googlevideo.com"
|
||||||
|
case "control":
|
||||||
|
host += "\n"
|
||||||
|
case "unknown":
|
||||||
|
return nil, errors.New("runner unavailable")
|
||||||
|
case "mixed-safety":
|
||||||
|
return nil, errors.Join(context.DeadlineExceeded, errors.New("cleanup failed"))
|
||||||
|
case "cancel":
|
||||||
|
cancel()
|
||||||
|
return nil, ctx.Err()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if scenario == "same-root" {
|
||||||
|
host = fmt.Sprintf("rr%d---sn-alpha-x.googlevideo.com", calls)
|
||||||
|
}
|
||||||
|
return samplePlayerBody(host, expires), nil
|
||||||
|
})
|
||||||
|
resources, err := collect(ctx)
|
||||||
|
if scenario == "unknown" || scenario == "mixed-safety" {
|
||||||
|
var fatal *managedRecoveryFatal
|
||||||
|
if !errors.As(err, &fatal) || calls != 1 || len(resources) != 0 {
|
||||||
|
t.Fatal(err, calls)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if scenario == "cancel" {
|
||||||
|
if !errors.Is(err, context.Canceled) || calls != 1 || len(resources) != 0 {
|
||||||
|
t.Fatal(err, calls)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if scenario == "same-root" {
|
||||||
|
var insufficient *InsufficientAdaptiveSamples
|
||||||
|
if !errors.As(err, &insufficient) || insufficient.Attempts != 9 || len(resources) != 0 || strings.Contains(err.Error(), "private-secret") {
|
||||||
|
t.Fatal(err, calls)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err != nil || len(resources) != 3 || calls != 6 {
|
||||||
|
t.Fatal(err, calls, len(resources))
|
||||||
|
}
|
||||||
|
for i, r := range resources {
|
||||||
|
if r.host != hosts[i] || !strings.Contains(r.url, "private-secret") {
|
||||||
|
t.Fatal("changed signed inventory")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestManagedGooglevideoDiscoveryBoundsAndLateCancellation(t *testing.T) {
|
||||||
|
ids := []string{"dQw4w9WgXcQ", "aqz-KE-bpKQ", "jNQXAC9IVRw", "aaaaaaaaaaa", "bbbbbbbbbbb"}
|
||||||
|
calls := 0
|
||||||
|
resources, err := managedGooglevideoSamples(ids, func(context.Context, string) ([]byte, error) {
|
||||||
|
calls++
|
||||||
|
return []byte(`{"playabilityStatus":{"status":"OK"},"streamingData":{"serverAbrStreamingUrl":"private-secret"}}`), nil
|
||||||
|
})(context.Background())
|
||||||
|
var insufficient *InsufficientAdaptiveSamples
|
||||||
|
if !errors.As(err, &insufficient) || calls != 15 || insufficient.Unsupported != 15 || len(resources) != 0 {
|
||||||
|
t.Fatal(err, calls)
|
||||||
|
}
|
||||||
|
for _, invalid := range [][]string{nil, {"bad"}, append(ids, "ccccccccccc")} {
|
||||||
|
_, err := managedGooglevideoSamples(invalid, func(context.Context, string) ([]byte, error) { t.Fatal("invalid config queried"); return nil, nil })(context.Background())
|
||||||
|
var fatal *managedRecoveryFatal
|
||||||
|
if !errors.As(err, &fatal) {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
hosts := []string{"rr1---sn-alpha.googlevideo.com", "rr2---sn-beta.googlevideo.com", "rr5---sn-gamma.googlevideo.com"}
|
||||||
|
calls = 0
|
||||||
|
resources, err = managedGooglevideoSamples(ids[:3], func(context.Context, string) ([]byte, error) {
|
||||||
|
host := hosts[calls]
|
||||||
|
calls++
|
||||||
|
if calls == 3 {
|
||||||
|
cancel()
|
||||||
|
}
|
||||||
|
return samplePlayerBody(host, time.Now().Add(time.Hour)), nil
|
||||||
|
})(ctx)
|
||||||
|
if !errors.Is(err, context.Canceled) || len(resources) != 0 || calls != 3 {
|
||||||
|
t.Fatal(err, calls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExperimentalPromotionRequiresRepresentativeOwnedProof(t *testing.T) {
|
||||||
|
for _, enough := range []bool{false, true} {
|
||||||
|
t.Run(fmt.Sprint(enough), func(t *testing.T) {
|
||||||
|
d, front, _ := recoveryFixture(t, true)
|
||||||
|
r, _, resources, _ := googlevideoRuntimeFixture(t)
|
||||||
|
d.service = true
|
||||||
|
d.frontHealth = d.health
|
||||||
|
thumb := unitAssetCandidate(t, d.db, "i.ytimg.com", "--lua-desync=fake", true)
|
||||||
|
avatar := unitAssetCandidate(t, d.db, "yt3.ggpht.com", "--lua-desync=fake", true)
|
||||||
|
tx, err := d.db.Begin()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
experiment, err := insertStrategyCandidate(tx, "googlevideo", []string{resources[0].host}, managedGooglevideoArgs, nil, QualificationCold)
|
||||||
|
if err != nil {
|
||||||
|
_ = tx.Rollback()
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := tx.Commit(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
old := []int64{front, thumb, avatar, experiment}
|
||||||
|
if err := d.transaction.Apply(d.db, old, GooglevideoScopeExperimental); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
d.resources = func(context.Context) (managedServiceInventory, error) {
|
||||||
|
return managedServiceInventory{media: resources[0], images: map[string]string{"i.ytimg.com": "https://i.ytimg.com/sample.jpg", "yt3.ggpht.com": "https://yt3.ggpht.com/sample.jpg"}, proxy: r.proxy, runner: adaptiveImageRunner}, nil
|
||||||
|
}
|
||||||
|
d.recheckExperimental = func(context.Context, storedCandidate, mediaResource, *managedTransaction) error { return nil }
|
||||||
|
d.mediaSamples = func(context.Context) ([]mediaResource, error) {
|
||||||
|
if enough {
|
||||||
|
return resources, nil
|
||||||
|
}
|
||||||
|
return resources[:1], nil
|
||||||
|
}
|
||||||
|
d.verifyGooglevideo = r.verifyGooglevideo
|
||||||
|
d.recheckGooglevideo = managedGooglevideoApplyRecheck(r, d.db)
|
||||||
|
id, err := recoverManagedExperimental(context.Background(), d)
|
||||||
|
if !enough {
|
||||||
|
if !errors.Is(err, errGooglevideoInsufficientSamples) || id != experiment {
|
||||||
|
t.Fatal(id, err)
|
||||||
|
}
|
||||||
|
assertActiveCandidates(t, d.transaction, d.db, old)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err != nil || id == experiment {
|
||||||
|
t.Fatal(id, err)
|
||||||
|
}
|
||||||
|
assertActiveCandidates(t, d.transaction, d.db, []int64{front, thumb, avatar, id})
|
||||||
|
p, err := readGooglevideoProof(d.db, id)
|
||||||
|
if err != nil || p.Version != 1 || len(p.Hosts) != 3 {
|
||||||
|
t.Fatal(p, err)
|
||||||
|
}
|
||||||
|
c, err := readStrategyCandidate(d.db, experiment)
|
||||||
|
if err != nil || c.Qualified || c.Mode != QualificationCold {
|
||||||
|
t.Fatal("old row changed", c, err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -3,111 +3,162 @@ package main
|
|||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"errors"
|
"errors"
|
||||||
|
"fmt"
|
||||||
"sort"
|
"sort"
|
||||||
"strconv"
|
"strconv"
|
||||||
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Ordinary unresolved state, not a target/ownership guard failure.
|
// Ordinary unresolved state, not a target/ownership guard failure.
|
||||||
type InsufficientAdaptiveSamples struct{}
|
type InsufficientAdaptiveSamples struct {
|
||||||
|
Attempts, DistinctHosts, Unavailable, Unsupported, Rejected int
|
||||||
|
}
|
||||||
|
|
||||||
func (*InsufficientAdaptiveSamples) Error() string {
|
func (e *InsufficientAdaptiveSamples) Error() string {
|
||||||
return "DEGRADED: insufficient fresh ordinary signed GGC samples"
|
return fmt.Sprintf("DEGRADED: insufficient fresh ordinary signed GGC samples (attempts=%d distinct_hosts=%d unavailable=%d unsupported=%d rejected=%d)", e.Attempts, e.DistinctHosts, e.Unavailable, e.Unsupported, e.Rejected)
|
||||||
}
|
}
|
||||||
func (*InsufficientAdaptiveSamples) Unwrap() []error {
|
func (*InsufficientAdaptiveSamples) Unwrap() []error {
|
||||||
return []error{errGooglevideoInsufficientSamples, errNoQualifiedStrategy}
|
return []error{errGooglevideoInsufficientSamples}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Sequential round-robin: at most 15 fresh requests, with no child goroutines.
|
||||||
|
const managedGooglevideoSampleRounds = 3
|
||||||
|
const managedGooglevideoDiscoveryTimeout = 3 * time.Minute
|
||||||
|
|
||||||
|
// Three hosts each need up to 120s warmup + six 35s transfers. Leave margin
|
||||||
|
// for daemon lifecycle as well; short-lived URLs are refetched, never rewritten.
|
||||||
|
const managedGooglevideoSampleTTL = 20 * time.Minute
|
||||||
|
|
||||||
|
func freshGooglevideoSample(r mediaResource) bool {
|
||||||
|
// The same exact GGC hostname syntax as the representative gate; reject bad
|
||||||
|
// inventory before counting it, without fetching any returned URL here.
|
||||||
|
prefix, root, ok := strings.Cut(strings.TrimSuffix(r.host, ".googlevideo.com"), "---sn-")
|
||||||
|
if !ok || !strings.HasSuffix(r.host, ".googlevideo.com") || !validDomain(r.host) || r.host != strings.ToLower(r.host) || root == "" || len(prefix) < 3 || !strings.HasPrefix(prefix, "rr") || strings.Trim(prefix[2:], "0123456789") != "" {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
u, err := privateResourceURL(r.url, r.host)
|
||||||
|
if err != nil || u.Path != "/videoplayback" || (r.kind != "video/mp4" && r.kind != "audio/mp4") {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
q := u.Query()
|
||||||
|
expires, err := strconv.ParseInt(q.Get("expire"), 10, 64)
|
||||||
|
return err == nil && expires > time.Now().Add(managedGooglevideoSampleTTL).Unix() && (q.Get("sig") != "" || q.Get("signature") != "" || q.Get("lsig") != "")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Choose exactly three actual hosts passing the unchanged representative gate.
|
||||||
|
func representativeGooglevideoSamples(byHost map[string]mediaResource) []mediaResource {
|
||||||
|
hosts := make([]string, 0, len(byHost))
|
||||||
|
for host, r := range byHost {
|
||||||
|
if freshGooglevideoSample(r) {
|
||||||
|
hosts = append(hosts, host)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sort.Strings(hosts)
|
||||||
|
for i := 0; i < len(hosts); i++ {
|
||||||
|
for j := i + 1; j < len(hosts); j++ {
|
||||||
|
for k := j + 1; k < len(hosts); k++ {
|
||||||
|
if validateGooglevideoHosts([]string{hosts[i], hosts[j], hosts[k]}) == nil {
|
||||||
|
return []mediaResource{byHost[hosts[i]], byHost[hosts[j]], byHost[hosts[k]]}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func managedGooglevideoSamples(ids []string, request func(context.Context, string) ([]byte, error)) func(context.Context) ([]mediaResource, error) {
|
func managedGooglevideoSamples(ids []string, request func(context.Context, string) ([]byte, error)) func(context.Context) ([]mediaResource, error) {
|
||||||
|
ids = append([]string(nil), ids...)
|
||||||
return func(ctx context.Context) ([]mediaResource, error) {
|
return func(ctx context.Context) ([]mediaResource, error) {
|
||||||
byHost := map[string]mediaResource{}
|
if len(ids) == 0 || len(ids) > 5 || request == nil {
|
||||||
|
return nil, &managedRecoveryFatal{Err: errors.New("invalid sample collector configuration")}
|
||||||
|
}
|
||||||
for _, id := range ids {
|
for _, id := range ids {
|
||||||
body, err := request(ctx, id)
|
if !videoIDPattern.MatchString(id) {
|
||||||
|
return nil, &managedRecoveryFatal{Err: errors.New("invalid sample video ID")}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
bounded, cancel := context.WithTimeout(ctx, managedGooglevideoDiscoveryTimeout)
|
||||||
|
defer cancel()
|
||||||
|
stats := &InsufficientAdaptiveSamples{}
|
||||||
|
byHost := map[string]mediaResource{}
|
||||||
|
for round := 0; round < managedGooglevideoSampleRounds; round++ {
|
||||||
|
for _, id := range ids {
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
return nil, managedRecoveryContextError(ctx, ctx.Err())
|
||||||
|
}
|
||||||
|
if bounded.Err() != nil {
|
||||||
|
return nil, stats
|
||||||
|
}
|
||||||
|
stats.Attempts++
|
||||||
|
body, err := request(bounded, id)
|
||||||
var fatal *managedRecoveryFatal
|
var fatal *managedRecoveryFatal
|
||||||
if ctx.Err() != nil || errors.As(err, &fatal) {
|
if ctx.Err() != nil || errors.As(err, &fatal) {
|
||||||
return nil, managedRecoveryContextError(ctx, err)
|
return nil, managedRecoveryContextError(ctx, err)
|
||||||
}
|
}
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
if !managedResourceTransferNonpass(err, true, false) && !(bounded.Err() != nil && managedRecoveryTimeoutSafe(err)) {
|
||||||
|
return nil, &managedRecoveryFatal{Err: err}
|
||||||
|
}
|
||||||
|
stats.Unavailable++
|
||||||
|
if bounded.Err() != nil {
|
||||||
|
return nil, stats
|
||||||
|
}
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
r, _, err := playerMediaResource(body)
|
r, _, err := playerMediaResource(body)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
stats.Unavailable++
|
||||||
continue
|
continue
|
||||||
} // Missing/SABR is unknown, never fabricated.
|
} // Missing/SABR is unknown, never fabricated.
|
||||||
u, err := privateResourceURL(r.url, r.host)
|
if r.url == "" {
|
||||||
if err != nil || u.Path != "/videoplayback" {
|
stats.Unsupported++
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
q := u.Query()
|
if !freshGooglevideoSample(r) {
|
||||||
expires, err := strconv.ParseInt(q.Get("expire"), 10, 64)
|
stats.Rejected++
|
||||||
if err != nil || expires <= time.Now().Unix()+120 || (q.Get("sig") == "" && q.Get("signature") == "" && q.Get("lsig") == "") {
|
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
byHost[r.host] = r
|
byHost[r.host] = r
|
||||||
|
stats.DistinctHosts = len(byHost)
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
return nil, managedRecoveryContextError(ctx, ctx.Err())
|
||||||
}
|
}
|
||||||
hosts := make([]string, 0, len(byHost))
|
if bounded.Err() != nil {
|
||||||
for host := range byHost {
|
return nil, stats
|
||||||
hosts = append(hosts, host)
|
|
||||||
}
|
|
||||||
sort.Strings(hosts)
|
|
||||||
if validateGooglevideoHosts(hosts) != nil {
|
|
||||||
return nil, &InsufficientAdaptiveSamples{}
|
|
||||||
}
|
|
||||||
resources := make([]mediaResource, 0, len(hosts))
|
|
||||||
for _, host := range hosts {
|
|
||||||
resources = append(resources, byHost[host])
|
|
||||||
}
|
}
|
||||||
|
if resources := representativeGooglevideoSamples(byHost); resources != nil {
|
||||||
return resources, nil
|
return resources, nil
|
||||||
}
|
}
|
||||||
}
|
|
||||||
|
|
||||||
func managedAdaptiveAssetsQualified(d managedRecoveryDeps, ids []int64) (bool, error) {
|
|
||||||
covered := map[string]bool{}
|
|
||||||
for _, id := range ids {
|
|
||||||
c, err := readStrategyCandidate(d.db, id)
|
|
||||||
if err != nil {
|
|
||||||
return false, err
|
|
||||||
}
|
|
||||||
if c.Group == "assets" && c.Qualified {
|
|
||||||
for _, host := range c.Endpoints {
|
|
||||||
covered[host] = true
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
return nil, stats
|
||||||
}
|
}
|
||||||
return covered["i.ytimg.com"] && covered["yt3.ggpht.com"], nil
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func recoverManagedGooglevideo(ctx context.Context, d managedRecoveryDeps) (int64, error) {
|
func recoverManagedGooglevideo(ctx context.Context, d managedRecoveryDeps) (int64, error) {
|
||||||
if !d.service || d.mediaSamples == nil || d.verifyGooglevideo == nil {
|
|
||||||
return 0, &InsufficientAdaptiveSamples{}
|
|
||||||
}
|
|
||||||
m := *d.transaction
|
m := *d.transaction
|
||||||
journal, err := m.load()
|
journal, err := m.load()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return 0, &managedRecoveryFatal{Err: err}
|
return 0, &managedRecoveryFatal{Err: err}
|
||||||
}
|
}
|
||||||
if journal.Scope == GooglevideoScopeExperimental {
|
if journal.Scope == GooglevideoScopeExperimental {
|
||||||
// Explicit operator policy remains owned, never auto-qualified/replaced.
|
return recoverManagedExperimental(ctx, d)
|
||||||
|
}
|
||||||
|
return qualifyManagedGooglevideo(ctx, d)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Both autonomous adaptive recovery and an operationally rechecked experiment
|
||||||
|
// use the same unchanged three-host owned proof and transactional apply gate.
|
||||||
|
func qualifyManagedGooglevideo(ctx context.Context, d managedRecoveryDeps) (int64, error) {
|
||||||
|
m := *d.transaction
|
||||||
|
if !d.service || d.mediaSamples == nil || d.verifyGooglevideo == nil {
|
||||||
return 0, &InsufficientAdaptiveSamples{}
|
return 0, &InsufficientAdaptiveSamples{}
|
||||||
}
|
}
|
||||||
current, err := m.ActiveCandidates(d.db)
|
current, err := m.ActiveCandidates(d.db)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return 0, &managedRecoveryFatal{Err: err}
|
return 0, &managedRecoveryFatal{Err: err}
|
||||||
}
|
}
|
||||||
ok, err := managedAdaptiveAssetsQualified(d, current)
|
|
||||||
if err != nil {
|
|
||||||
return 0, &managedRecoveryFatal{Err: err}
|
|
||||||
}
|
|
||||||
if !ok {
|
|
||||||
return 0, &managedResourceFailure{"thumbnail", "unknown", nil}
|
|
||||||
}
|
|
||||||
status, err := managedAssetApplyHealth(ctx, d, []string{"i.ytimg.com", "yt3.ggpht.com"})
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
if status != "OK" {
|
|
||||||
return 0, &managedHealthRejection{Status: status}
|
|
||||||
}
|
|
||||||
saved, err := listQualifiedCandidates(d.db, "googlevideo", 10)
|
saved, err := listQualifiedCandidates(d.db, "googlevideo", 10)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return 0, &managedRecoveryFatal{Err: err}
|
return 0, &managedRecoveryFatal{Err: err}
|
||||||
@@ -190,16 +241,31 @@ func recoverManagedGooglevideo(ctx context.Context, d managedRecoveryDeps) (int6
|
|||||||
return d.run(cleanup, managedZapretScript, "restart")
|
return d.run(cleanup, managedZapretScript, "restart")
|
||||||
}
|
}
|
||||||
m.Health = func() (string, error) {
|
m.Health = func() (string, error) {
|
||||||
status, err := d.health(ctx) // Fresh media byte health, not hostname-only.
|
if d.frontHealth == nil || d.recheckGooglevideo == nil {
|
||||||
var failure *managedResourceFailure
|
return "DEGRADED", errors.New("frontend and deployed media health required")
|
||||||
|
}
|
||||||
|
status, err := d.frontHealth(ctx)
|
||||||
|
if err != nil || status != "OK" {
|
||||||
|
return status, managedRecoveryContextError(ctx, err)
|
||||||
|
}
|
||||||
|
deployed, err := readStrategyCandidate(d.db, id)
|
||||||
|
if err == nil {
|
||||||
|
err = d.recheckGooglevideo(ctx, deployed, resources, &m)
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
var fatal *managedRecoveryFatal
|
var fatal *managedRecoveryFatal
|
||||||
if ctx.Err() != nil {
|
if ctx.Err() != nil || errors.As(err, &fatal) || errors.Is(err, context.Canceled) {
|
||||||
return "DEGRADED", managedRecoveryContextError(ctx, err)
|
return "DEGRADED", managedRecoveryContextError(ctx, err)
|
||||||
}
|
}
|
||||||
if errors.As(err, &failure) && !errors.As(err, &fatal) && !errors.Is(err, context.Canceled) && !errors.Is(err, context.DeadlineExceeded) {
|
if managedGooglevideoTransferNonpass(err) || errors.Is(err, errCandidateNonpass) {
|
||||||
return "DEGRADED", nil
|
return "DEGRADED", nil
|
||||||
}
|
}
|
||||||
return status, err
|
return "DEGRADED", managedRecoveryContextError(ctx, err)
|
||||||
|
}
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
return "DEGRADED", managedRecoveryContextError(ctx, ctx.Err())
|
||||||
|
}
|
||||||
|
return "OK", nil // Stage readiness only; whole-service monitor is unchanged.
|
||||||
}
|
}
|
||||||
err = m.Apply(d.db, desired, GooglevideoScopeSuffix)
|
err = m.Apply(d.db, desired, GooglevideoScopeSuffix)
|
||||||
var rollback *managedRollbackError
|
var rollback *managedRollbackError
|
||||||
|
|||||||
@@ -13,6 +13,7 @@ import (
|
|||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
|
"time"
|
||||||
)
|
)
|
||||||
|
|
||||||
func TestManagedAdaptiveConfig(t *testing.T) {
|
func TestManagedAdaptiveConfig(t *testing.T) {
|
||||||
@@ -42,7 +43,7 @@ func TestManagedAdaptiveConfig(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func TestManagedAdaptiveRecoveryProductionPath(t *testing.T) {
|
func TestManagedAdaptiveRecoveryProductionPath(t *testing.T) {
|
||||||
for _, scenario := range []string{"qualified", "one-host", "unqualified-assets", "rollback"} {
|
for _, scenario := range []string{"qualified", "one-host", "unqualified-assets", "avatar-missing", "rollback", "apply-bad-body", "apply-mixed-guard", "apply-learns"} {
|
||||||
t.Run(scenario, func(t *testing.T) {
|
t.Run(scenario, func(t *testing.T) {
|
||||||
d, front, events := recoveryFixture(t, true)
|
d, front, events := recoveryFixture(t, true)
|
||||||
r, _, resources, _ := googlevideoRuntimeFixture(t)
|
r, _, resources, _ := googlevideoRuntimeFixture(t)
|
||||||
@@ -66,16 +67,71 @@ func TestManagedAdaptiveRecoveryProductionPath(t *testing.T) {
|
|||||||
return resources, nil
|
return resources, nil
|
||||||
}
|
}
|
||||||
d.verifyGooglevideo = r.verifyGooglevideo
|
d.verifyGooglevideo = r.verifyGooglevideo
|
||||||
if scenario == "rollback" {
|
d.recheckGooglevideo = managedGooglevideoApplyRecheck(r, d.db)
|
||||||
|
applyCalls := 0
|
||||||
|
if strings.HasPrefix(scenario, "apply-") {
|
||||||
|
deployed := r
|
||||||
|
if scenario == "apply-mixed-guard" {
|
||||||
|
deployed.guard = func() error { return errors.Join(context.Canceled, errors.New("guard failed")) }
|
||||||
|
}
|
||||||
|
deployed.curl = func(ctx context.Context, args []string, stdin io.Reader, out, diagnostic io.Writer) error {
|
||||||
|
applyCalls++
|
||||||
|
if scenario == "apply-learns" && applyCalls <= 2 {
|
||||||
|
return context.DeadlineExceeded
|
||||||
|
}
|
||||||
|
var body bytes.Buffer
|
||||||
|
err := r.curl(ctx, args, stdin, &body, diagnostic)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
b := body.Bytes()
|
||||||
|
if scenario == "apply-bad-body" {
|
||||||
|
b = bytes.ReplaceAll(b, []byte("ftyp"), []byte("oops"))
|
||||||
|
}
|
||||||
|
_, err = out.Write(b)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
d.recheckGooglevideo = managedGooglevideoApplyRecheck(deployed, d.db)
|
||||||
|
}
|
||||||
|
if scenario == "avatar-missing" {
|
||||||
d.health = func(context.Context) (string, error) {
|
d.health = func(context.Context) (string, error) {
|
||||||
return "DEGRADED", &managedResourceFailure{"media", resources[0].host, nil}
|
return "DEGRADED", &managedResourceFailure{"avatar", "yt3.ggpht.com", nil}
|
||||||
|
}
|
||||||
|
_, ids, err := managedGooglevideoConfig(func(key string) (string, bool) { return "1", key == "ZAPRET_AUTOTUNE_YOUTUBE_SERVICE" }, "dQw4w9WgXcQ")
|
||||||
|
if err != nil || len(ids) != 3 {
|
||||||
|
t.Fatal(ids, err)
|
||||||
|
}
|
||||||
|
calls := 0
|
||||||
|
d.mediaSamples = managedGooglevideoSamples(ids, func(_ context.Context, id string) ([]byte, error) {
|
||||||
|
if id != ids[calls%3] {
|
||||||
|
t.Fatal("wrong default video")
|
||||||
|
}
|
||||||
|
index := calls % 3
|
||||||
|
if calls < 3 {
|
||||||
|
index = 0
|
||||||
|
}
|
||||||
|
calls++
|
||||||
|
return samplePlayerBody(resources[index].host, time.Now().Add(time.Hour)), nil
|
||||||
|
})
|
||||||
|
}
|
||||||
|
if scenario == "rollback" {
|
||||||
|
d.recheckGooglevideo = func(context.Context, storedCandidate, []mediaResource, *managedTransaction) error {
|
||||||
|
return errCandidateNonpass
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
result, err := recoverManagedTarget(context.Background(), d)
|
result, err := recoverManagedTarget(context.Background(), d)
|
||||||
|
if scenario == "apply-mixed-guard" {
|
||||||
|
var fatal *managedRecoveryFatal
|
||||||
|
if !errors.As(err, &fatal) {
|
||||||
|
t.Fatal("mixed guard downgraded", err)
|
||||||
|
}
|
||||||
|
assertActiveCandidates(t, d.transaction, d.db, old)
|
||||||
|
return
|
||||||
|
}
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
}
|
}
|
||||||
if scenario != "qualified" {
|
if scenario == "one-host" || scenario == "rollback" || scenario == "apply-bad-body" {
|
||||||
if result.GGCError == nil {
|
if result.GGCError == nil {
|
||||||
t.Fatal("unqualified stage succeeded")
|
t.Fatal("unqualified stage succeeded")
|
||||||
}
|
}
|
||||||
@@ -102,6 +158,15 @@ func TestManagedAdaptiveRecoveryProductionPath(t *testing.T) {
|
|||||||
t.Fatal(j, err)
|
t.Fatal(j, err)
|
||||||
}
|
}
|
||||||
assertActiveCandidates(t, d.transaction, d.db, append(old, c.ID))
|
assertActiveCandidates(t, d.transaction, d.db, append(old, c.ID))
|
||||||
|
if scenario == "apply-learns" && applyCalls < 20 {
|
||||||
|
t.Fatal("deployed warmup or full TLS transfers skipped", applyCalls)
|
||||||
|
}
|
||||||
|
if scenario == "avatar-missing" {
|
||||||
|
status, err := d.health(context.Background())
|
||||||
|
if status != "DEGRADED" || err == nil {
|
||||||
|
t.Fatal("whole-service health concealed")
|
||||||
|
}
|
||||||
|
}
|
||||||
if strings.Join(*events, ",") != "stop,start,restart" {
|
if strings.Join(*events, ",") != "stop,start,restart" {
|
||||||
t.Fatal("restore must precede apply", *events)
|
t.Fatal("restore must precede apply", *events)
|
||||||
}
|
}
|
||||||
@@ -136,7 +201,39 @@ func adaptiveImageRunner(_ context.Context, args []string, stdin io.Reader, out,
|
|||||||
|
|
||||||
func TestInsufficientAdaptiveSamplesCooldown(t *testing.T) {
|
func TestInsufficientAdaptiveSamplesCooldown(t *testing.T) {
|
||||||
err := &InsufficientAdaptiveSamples{}
|
err := &InsufficientAdaptiveSamples{}
|
||||||
if !errors.Is(err, errNoQualifiedStrategy) || !errors.Is(err, errGooglevideoInsufficientSamples) {
|
if errors.Is(err, errNoQualifiedStrategy) || !errors.Is(err, errGooglevideoInsufficientSamples) || managedRecoveryCooldown(err, time.Minute) != time.Minute {
|
||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestGenuineNoQualifiedCooldown(t *testing.T) {
|
||||||
|
if managedRecoveryCooldown(errNoQualifiedStrategy, time.Minute) != 2*time.Hour {
|
||||||
|
t.Fatal("lost exhausted-search pause")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestManagedYouTubeDefaultSampleInventory(t *testing.T) {
|
||||||
|
for _, primary := range []string{"dQw4w9WgXcQ", "aqz-KE-bpKQ"} {
|
||||||
|
lookup := func(key string) (string, bool) { return "1", key == "ZAPRET_AUTOTUNE_YOUTUBE_SERVICE" }
|
||||||
|
_, ids, err := managedGooglevideoConfig(lookup, primary)
|
||||||
|
want := []string{primary, "aqz-KE-bpKQ", "jNQXAC9IVRw"}
|
||||||
|
if primary == "aqz-KE-bpKQ" {
|
||||||
|
want = []string{primary, "jNQXAC9IVRw"}
|
||||||
|
}
|
||||||
|
if err != nil || !reflect.DeepEqual(ids, want) {
|
||||||
|
t.Fatal(ids, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
_, ids, err := managedGooglevideoConfig(func(key string) (string, bool) {
|
||||||
|
switch key {
|
||||||
|
case "ZAPRET_AUTOTUNE_YOUTUBE_SERVICE":
|
||||||
|
return "1", true
|
||||||
|
case "ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS":
|
||||||
|
return `["aaaaaaaaaaa"]`, true
|
||||||
|
}
|
||||||
|
return "", false
|
||||||
|
}, "dQw4w9WgXcQ")
|
||||||
|
if err != nil || !reflect.DeepEqual(ids, []string{"aaaaaaaaaaa"}) {
|
||||||
|
t.Fatal(ids, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -7,6 +7,7 @@ import (
|
|||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"os"
|
"os"
|
||||||
|
"os/exec"
|
||||||
"reflect"
|
"reflect"
|
||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
@@ -185,6 +186,46 @@ func (r managedAssetRuntime) verifyGooglevideo(ctx context.Context, c storedCand
|
|||||||
}
|
}
|
||||||
|
|
||||||
func managedGooglevideoTransferNonpass(err error) bool {
|
func managedGooglevideoTransferNonpass(err error) bool {
|
||||||
|
return managedResourceTransferNonpass(err, false, false)
|
||||||
|
}
|
||||||
|
|
||||||
|
func managedResourceTransferNonpass(err error, metadata, image bool) bool {
|
||||||
|
// Collector normalization wraps known transfer failures as fatal for generic
|
||||||
|
// consumers. Only that boundary may unwrap the wrapper; guard/cleanup failures
|
||||||
|
// and mixed error sets must never authorize absence or a daemon restart.
|
||||||
|
if e, ok := err.(*resourceError); ok {
|
||||||
|
cause := e.cause
|
||||||
|
if fatal, ok := cause.(*managedRecoveryFatal); ok {
|
||||||
|
cause = fatal.Err
|
||||||
|
}
|
||||||
|
return managedResourceTransferNonpass(cause, metadata, image)
|
||||||
|
}
|
||||||
|
if _, ok := err.(*managedRecoveryFatal); ok {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if joined, ok := err.(interface{ Unwrap() []error }); ok {
|
||||||
|
children := joined.Unwrap()
|
||||||
|
if len(children) == 0 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for _, child := range children {
|
||||||
|
if !managedResourceTransferNonpass(child, metadata, image) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
if wrapped, ok := err.(interface{ Unwrap() error }); ok {
|
||||||
|
return managedResourceTransferNonpass(wrapped.Unwrap(), metadata, image)
|
||||||
|
}
|
||||||
|
if _, ok := err.(imageBodyRejection); ok {
|
||||||
|
return image
|
||||||
|
}
|
||||||
|
// Metadata curl uses --fail. A non-2xx response is missing inventory,
|
||||||
|
// never media rejection or candidate-absence evidence.
|
||||||
|
if exit, ok := err.(*exec.ExitError); metadata && ok && exit.ExitCode() == 22 {
|
||||||
|
return true
|
||||||
|
}
|
||||||
if managedAssetTransferNonpass(err) {
|
if managedAssetTransferNonpass(err) {
|
||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
@@ -194,9 +235,6 @@ func managedGooglevideoTransferNonpass(err error) bool {
|
|||||||
if err.Error() == "invalid media range or container" {
|
if err.Error() == "invalid media range or container" {
|
||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
if e, ok := err.(interface{ Unwrap() error }); ok {
|
|
||||||
return managedGooglevideoTransferNonpass(e.Unwrap())
|
|
||||||
}
|
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -56,6 +56,7 @@ func TestManagedGooglevideoOwnedPostWarmupProducer(t *testing.T) {
|
|||||||
r, c, resources, events := googlevideoRuntimeFixture(t)
|
r, c, resources, events := googlevideoRuntimeFixture(t)
|
||||||
runner, calls := r.curl, 0
|
runner, calls := r.curl, 0
|
||||||
r.curl = func(ctx context.Context, args []string, stdin io.Reader, out, diagnostic io.Writer) error {
|
r.curl = func(ctx context.Context, args []string, stdin io.Reader, out, diagnostic io.Writer) error {
|
||||||
|
assertManagedDNSArgs(t, args, "socks5h://127.0.0.1:1080")
|
||||||
calls++
|
calls++
|
||||||
if calls <= 2 {
|
if calls <= 2 {
|
||||||
return context.DeadlineExceeded
|
return context.DeadlineExceeded
|
||||||
|
|||||||
@@ -57,7 +57,11 @@ func TestManagedGooglevideoFreshPrivateSamples(t *testing.T) {
|
|||||||
t.Fatal(actual, err)
|
t.Fatal(actual, err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if calls != 2*len(ids) {
|
want := 2 * len(ids)
|
||||||
|
if scenario != "diverse" {
|
||||||
|
want *= managedGooglevideoSampleRounds
|
||||||
|
}
|
||||||
|
if calls != want {
|
||||||
t.Fatal("samples were reused", calls)
|
t.Fatal("samples were reused", calls)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -16,7 +16,7 @@ import (
|
|||||||
|
|
||||||
// Provisioning owns executable, database, proxy and target paths; no path flags.
|
// Provisioning owns executable, database, proxy and target paths; no path flags.
|
||||||
func managedMonitorConfig(args []string, lookup func(string) (string, bool)) (healthConfig, error) {
|
func managedMonitorConfig(args []string, lookup func(string) (string, bool)) (healthConfig, error) {
|
||||||
cfg, err := parseHealthConfig(lookup)
|
cfg, err := parseManagedHealthConfig(lookup)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return cfg, err
|
return cfg, err
|
||||||
}
|
}
|
||||||
@@ -57,6 +57,17 @@ func managedMonitorCLI(args []string, stdout, stderr io.Writer, target managedDe
|
|||||||
}
|
}
|
||||||
|
|
||||||
func managedMonitorCLIContext(ctx context.Context, args []string, stdout, stderr io.Writer, target managedDeps, supervise func(context.Context, managedMonitorDeps) error) (code int) {
|
func managedMonitorCLIContext(ctx context.Context, args []string, stdout, stderr io.Writer, target managedDeps, supervise func(context.Context, managedMonitorDeps) error) (code int) {
|
||||||
|
resourcePath, _ := target.lookup("ZAPRET_AUTOTUNE_RESOURCE_CONFIG")
|
||||||
|
if len(args) == 2 && args[0] == "--resource-config" && args[1] != "" {
|
||||||
|
return resourceMonitorCLI(ctx, args[1], stdout, stderr, target)
|
||||||
|
}
|
||||||
|
if resourcePath != "" {
|
||||||
|
if len(args) != 0 {
|
||||||
|
fmt.Fprintln(stderr, "configerror: generic monitor accepts only --resource-config FILE")
|
||||||
|
return 64
|
||||||
|
}
|
||||||
|
return resourceMonitorCLI(ctx, resourcePath, stdout, stderr, target)
|
||||||
|
}
|
||||||
cfg, err := managedMonitorConfig(args, target.lookup)
|
cfg, err := managedMonitorConfig(args, target.lookup)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
fmt.Fprintln(stderr, "error: invalid managed monitor configuration")
|
fmt.Fprintln(stderr, "error: invalid managed monitor configuration")
|
||||||
@@ -191,9 +202,9 @@ func managedMonitorRecoveryAdapters(target managedDeps, db *sql.DB, scanCommand
|
|||||||
d, err := managedRecoveryDependencies(target, db, scan, check, run)
|
d, err := managedRecoveryDependencies(target, db, scan, check, run)
|
||||||
if err == nil && d.service {
|
if err == nil && d.service {
|
||||||
d.verifyAssets = newManagedAssetVerifier(target)
|
d.verifyAssets = newManagedAssetVerifier(target)
|
||||||
if d.adaptive {
|
d.recheckExperimental = newManagedExperimentalRecheck(target, d)
|
||||||
d.verifyGooglevideo = newManagedGooglevideoVerifier(target)
|
d.verifyGooglevideo = newManagedGooglevideoVerifier(target)
|
||||||
}
|
d.recheckGooglevideo = newManagedGooglevideoApplyRecheck(target, db)
|
||||||
}
|
}
|
||||||
return d, err
|
return d, err
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -34,6 +34,9 @@ func TestManagedPrestartHelper(t *testing.T) {
|
|||||||
// Use the production validator, not a fixture-only success response.
|
// Use the production validator, not a fixture-only success response.
|
||||||
os.Exit(run(args, os.Stdout, os.Stderr))
|
os.Exit(run(args, os.Stdout, os.Stderr))
|
||||||
}
|
}
|
||||||
|
if len(args) == 2 && args[0] == "dns-forwarder" && (args[1] == "mode" || args[1] == "address") {
|
||||||
|
os.Exit(run(args, os.Stdout, os.Stderr))
|
||||||
|
}
|
||||||
if len(args) < 2 || args[0] != "managed" {
|
if len(args) < 2 || args[0] != "managed" {
|
||||||
os.Exit(1)
|
os.Exit(1)
|
||||||
}
|
}
|
||||||
@@ -165,7 +168,8 @@ func TestManagedEntrypointPrestartReconciliation(t *testing.T) {
|
|||||||
// use the fixture config, via its own script rather than CLI path flags.
|
// use the fixture config, via its own script rather than CLI path flags.
|
||||||
cmd.Env = append(cmd.Env, "FIXTURE_CONFIG="+filepath.Join(d.directory, "config"))
|
cmd.Env = append(cmd.Env, "FIXTURE_CONFIG="+filepath.Join(d.directory, "config"))
|
||||||
childStarted := filepath.Join(root, "child-started")
|
childStarted := filepath.Join(root, "child-started")
|
||||||
cmd.Env = append(cmd.Env, "FIXTURE_CHILD_STARTED="+childStarted, "SS_DNS_SERVER=8.8.8.8")
|
// This legacy daemon fixture deliberately exercises explicit native DNS.
|
||||||
|
cmd.Env = append(cmd.Env, "FIXTURE_CHILD_STARTED="+childStarted, "ZAPRET_AUTOTUNE_DNS_MODE=native", "SS_DNS_SERVER=8.8.8.8")
|
||||||
wantExit := 78
|
wantExit := 78
|
||||||
if mode == "invalid-dns" {
|
if mode == "invalid-dns" {
|
||||||
cmd.Env = append(cmd.Env, "SS_DNS_SERVER=8.8.8.8\n--invalid")
|
cmd.Env = append(cmd.Env, "SS_DNS_SERVER=8.8.8.8\n--invalid")
|
||||||
@@ -352,7 +356,7 @@ func TestManagedMonitorProductionAdapterBinding(t *testing.T) {
|
|||||||
if ctx.Err() != nil {
|
if ctx.Err() != nil {
|
||||||
t.Fatal(ctx.Err())
|
t.Fatal(ctx.Err())
|
||||||
}
|
}
|
||||||
if !strings.Contains(strings.Join(args, "|"), "--noproxy||--proxy|socks5://127.0.0.1:11080|") {
|
if !strings.Contains(strings.Join(args, "|"), "--noproxy||--proxy|socks5h://127.0.0.1:11080|") {
|
||||||
t.Fatal(args)
|
t.Fatal(args)
|
||||||
}
|
}
|
||||||
_, err := io.WriteString(out, `{"playabilityStatus":{"status":"OK"},"streamingData":{"serverAbrStreamingUrl":"https://fresh.googlevideo.com/video?secret=never-print"}}`)
|
_, err := io.WriteString(out, `{"playabilityStatus":{"status":"OK"},"streamingData":{"serverAbrStreamingUrl":"https://fresh.googlevideo.com/video?secret=never-print"}}`)
|
||||||
@@ -488,7 +492,7 @@ func TestManagedMonitorProductionAbsenceDiagnostics(t *testing.T) {
|
|||||||
var fatal *managedRecoveryFatal
|
var fatal *managedRecoveryFatal
|
||||||
wantFatal := strings.Contains(mode, "nfq")
|
wantFatal := strings.Contains(mode, "nfq")
|
||||||
if errors.As(err, &fatal) != wantFatal || (!wantFatal && !errors.Is(err, context.Canceled)) {
|
if errors.As(err, &fatal) != wantFatal || (!wantFatal && !errors.Is(err, context.Canceled)) {
|
||||||
t.Fatalf("mode=%s err=%v", mode, err)
|
t.Fatalf("mode=%s err=%v reported=%v", mode, err, reported)
|
||||||
}
|
}
|
||||||
if strings.Contains(errors.Join(err, reported).Error(), "secret-never-print") {
|
if strings.Contains(errors.Join(err, reported).Error(), "secret-never-print") {
|
||||||
t.Fatal("diagnostics leaked")
|
t.Fatal("diagnostics leaked")
|
||||||
@@ -578,7 +582,7 @@ func TestManagedMonitorCLIBindingAndCancellation(t *testing.T) {
|
|||||||
requests := 0
|
requests := 0
|
||||||
d.player = func(ctx context.Context, args []string, out, _ io.Writer) error {
|
d.player = func(ctx context.Context, args []string, out, _ io.Writer) error {
|
||||||
requests++
|
requests++
|
||||||
if !strings.Contains(strings.Join(args, "|"), "--noproxy||--proxy|socks5://127.0.0.1:11080|") {
|
if !strings.Contains(strings.Join(args, "|"), "--noproxy||--proxy|socks5h://127.0.0.1:11080|") {
|
||||||
t.Fatal(args)
|
t.Fatal(args)
|
||||||
}
|
}
|
||||||
if requests == 1 {
|
if requests == 1 {
|
||||||
@@ -672,6 +676,14 @@ func TestManagedEntrypointSupervision(t *testing.T) {
|
|||||||
write("ss-local", child)
|
write("ss-local", child)
|
||||||
zapret := write("zapret", "#!/bin/sh\necho zapret-$1 >> \"$LOG\"\nstage=$1\n"+startupFailure+"if [ \"$MODE:$1\" = ss-early:start ] && [ \"$(grep -c zapret-start \"$LOG\")\" -gt 1 ]; then\nuntil [ \"$(grep -c 'stopped-.*/ss-local' \"$LOG\")\" -ge \"$(grep -c zapret-start \"$LOG\")\" ]; do sleep 0.02; done\nsleep 0.02\nfi\nif [ \"$MODE:$1\" = stop-failure:stop ]; then exit 1; fi\nif [ \"$MODE:$1\" = stop-hang:stop ]; then trap '' TERM INT; exec sleep 300; fi\n")
|
zapret := write("zapret", "#!/bin/sh\necho zapret-$1 >> \"$LOG\"\nstage=$1\n"+startupFailure+"if [ \"$MODE:$1\" = ss-early:start ] && [ \"$(grep -c zapret-start \"$LOG\")\" -gt 1 ]; then\nuntil [ \"$(grep -c 'stopped-.*/ss-local' \"$LOG\")\" -ge \"$(grep -c zapret-start \"$LOG\")\" ]; do sleep 0.02; done\nsleep 0.02\nfi\nif [ \"$MODE:$1\" = stop-failure:stop ]; then exit 1; fi\nif [ \"$MODE:$1\" = stop-hang:stop ]; then trap '' TERM INT; exec sleep 300; fi\n")
|
||||||
monitor := write("autotune", "#!/bin/sh\nif [ \"$2\" != monitor ]; then\necho managed-$2 >> \"$LOG\"\nstage=$2\n"+startupFailure+"exit 0\nfi\nsleeper=\ntrap '"+joinSleeper+"echo monitor-stopped >> \"$LOG\"; case \"$MODE\" in *unsafe) exit 78;; *unknown) exit 7;; ss-technical) exit 75;; esac; exit 0' TERM INT\necho managed-monitor >> \"$LOG\"\nif [ \"$MODE\" = term-hang ]; then trap '' TERM INT; exec sleep 300; fi\ncase \"$MODE\" in fatal) exit 78;; technical|stop-failure|stop-hang) exit 75;; panic) exit 2;; killed) kill -KILL $$;; segv) exit 139;; zero) exit 0;; unknown) exit 7;; esac\nwhile :; do sleep 0.1 & sleeper=$!; wait \"$sleeper\"; done\n")
|
monitor := write("autotune", "#!/bin/sh\nif [ \"$2\" != monitor ]; then\necho managed-$2 >> \"$LOG\"\nstage=$2\n"+startupFailure+"exit 0\nfi\nsleeper=\ntrap '"+joinSleeper+"echo monitor-stopped >> \"$LOG\"; case \"$MODE\" in *unsafe) exit 78;; *unknown) exit 7;; ss-technical) exit 75;; esac; exit 0' TERM INT\necho managed-monitor >> \"$LOG\"\nif [ \"$MODE\" = term-hang ]; then trap '' TERM INT; exec sleep 300; fi\ncase \"$MODE\" in fatal) exit 78;; technical|stop-failure|stop-hang) exit 75;; panic) exit 2;; killed) kill -KILL $$;; segv) exit 139;; zero) exit 0;; unknown) exit 7;; esac\nwhile :; do sleep 0.1 & sleeper=$!; wait \"$sleeper\"; done\n")
|
||||||
|
// This legacy lifecycle fixture has no DNS service. Implement only
|
||||||
|
// the explicit native getters; dedicated Python fixtures own DoH ACKs.
|
||||||
|
monitorData, err := os.ReadFile(monitor)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
getters := "if [ \"$1\" = dns-forwarder ]; then\ncase \"$2\" in mode) echo native;; address) echo;; *) exit 64;; esac\nexit 0\nfi\n"
|
||||||
|
write("autotune", strings.Replace(string(monitorData), "#!/bin/sh\n", "#!/bin/sh\n"+getters, 1))
|
||||||
data, err := os.ReadFile("../../managed-target-entrypoint.sh")
|
data, err := os.ReadFile("../../managed-target-entrypoint.sh")
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
@@ -690,13 +702,17 @@ func TestManagedEntrypointSupervision(t *testing.T) {
|
|||||||
text = shortened
|
text = shortened
|
||||||
}
|
}
|
||||||
if mode == "stop-hang" {
|
if mode == "stop-hang" {
|
||||||
text = strings.Replace(text, "timeout -k 5 30", "timeout -k 1 1", 1)
|
shortened := strings.Replace(text, "cleanup_command 30 "+zapret+" stop", "timeout -k 1 1 "+zapret+" stop", 1)
|
||||||
|
if shortened == text {
|
||||||
|
t.Fatal("fixture did not shorten stop watchdog")
|
||||||
|
}
|
||||||
|
text = shortened
|
||||||
}
|
}
|
||||||
entry := write("entrypoint", text)
|
entry := write("entrypoint", text)
|
||||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||||
defer cancel()
|
defer cancel()
|
||||||
cmd := exec.CommandContext(ctx, "/bin/sh", entry)
|
cmd := exec.CommandContext(ctx, "/bin/sh", entry)
|
||||||
cmd.Env = append(os.Environ(), "PATH="+root+":"+os.Getenv("PATH"), "LOG="+log, "MODE="+mode, "ZAPRET_CONFIG="+filepath.Join(root, "config"), "SS_PASSWORD="+strings.Repeat("a", 32))
|
cmd.Env = append(os.Environ(), "PATH="+root+":"+os.Getenv("PATH"), "LOG="+log, "MODE="+mode, "ZAPRET_CONFIG="+filepath.Join(root, "config"), "SS_PASSWORD="+strings.Repeat("a", 32), "ZAPRET_AUTOTUNE_DNS_MODE=native")
|
||||||
cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true}
|
cmd.SysProcAttr = &syscall.SysProcAttr{Setpgid: true}
|
||||||
var shutdownSnapshot string
|
var shutdownSnapshot string
|
||||||
stopMonitorSession := func() {
|
stopMonitorSession := func() {
|
||||||
|
|||||||
@@ -27,7 +27,7 @@ func TestManagedSingleGroupCLI(t *testing.T) {
|
|||||||
t.Fatalf("apply: %d %s %s", code, out, diagnostic)
|
t.Fatalf("apply: %d %s %s", code, out, diagnostic)
|
||||||
}
|
}
|
||||||
code, out, diagnostic := callManaged(t, d, "status")
|
code, out, diagnostic := callManaged(t, d, "status")
|
||||||
if code != 0 || !strings.Contains(out, "groups="+group+" ") || !strings.Contains(out, "unresolved_groups="+missing+"\n") || !strings.Contains(out, "Live health: NOT CHECKED") {
|
if code != 0 || !strings.Contains(out, "groups="+group+" ") || !strings.Contains(out, "unresolved_groups="+missing+" covered="+group+"\n") || !strings.Contains(out, "Live health: NOT CHECKED") {
|
||||||
t.Fatalf("partial status: %d %s %s", code, out, diagnostic)
|
t.Fatalf("partial status: %d %s %s", code, out, diagnostic)
|
||||||
}
|
}
|
||||||
code, out, diagnostic = callManaged(t, d, "active-ids")
|
code, out, diagnostic = callManaged(t, d, "active-ids")
|
||||||
|
|||||||
@@ -39,6 +39,8 @@ type managedRecoveryDeps struct {
|
|||||||
adaptive bool
|
adaptive bool
|
||||||
mediaSamples func(context.Context) ([]mediaResource, error)
|
mediaSamples func(context.Context) ([]mediaResource, error)
|
||||||
verifyGooglevideo func(context.Context, storedCandidate, []mediaResource) (GooglevideoQualificationProof, error)
|
verifyGooglevideo func(context.Context, storedCandidate, []mediaResource) (GooglevideoQualificationProof, error)
|
||||||
|
recheckExperimental func(context.Context, storedCandidate, mediaResource, *managedTransaction) error
|
||||||
|
recheckGooglevideo func(context.Context, storedCandidate, []mediaResource, *managedTransaction) error
|
||||||
}
|
}
|
||||||
|
|
||||||
// Target-local adapter: reuses bounded Player requests without invoking a CLI.
|
// Target-local adapter: reuses bounded Player requests without invoking a CLI.
|
||||||
@@ -51,7 +53,7 @@ func managedRecoveryDependencies(target managedDeps, db *sql.DB, scan cacheScan,
|
|||||||
if _, err := managedRuntimeGuard(target); err != nil {
|
if _, err := managedRuntimeGuard(target); err != nil {
|
||||||
return d, err
|
return d, err
|
||||||
}
|
}
|
||||||
cfg, err := parseHealthConfig(target.lookup)
|
cfg, err := parseManagedHealthConfig(target.lookup)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return d, err
|
return d, err
|
||||||
}
|
}
|
||||||
@@ -119,6 +121,9 @@ func managedRecoveryDependencies(target managedDeps, db *sql.DB, scan cacheScan,
|
|||||||
}
|
}
|
||||||
if d.service {
|
if d.service {
|
||||||
d.frontHealth = d.health
|
d.frontHealth = d.health
|
||||||
|
// Construction is side-effect free; experiments may seek representative
|
||||||
|
// qualification without changing the legacy default exact-host mode.
|
||||||
|
d.mediaSamples = managedGooglevideoSamples(videoIDs, requestVideo)
|
||||||
d.resources = managedServiceResources(cfg, request, target.player)
|
d.resources = managedServiceResources(cfg, request, target.player)
|
||||||
d.health = managedServiceHealth(cfg.SOCKSProxy, d.frontHealth, d.resources, runResourceCurl)
|
d.health = managedServiceHealth(cfg.SOCKSProxy, d.frontHealth, d.resources, runResourceCurl)
|
||||||
d.verifyAssets = unwiredManagedAssetVerifier
|
d.verifyAssets = unwiredManagedAssetVerifier
|
||||||
@@ -267,11 +272,32 @@ func recoverManagedTarget(ctx context.Context, d managedRecoveryDeps) (result ma
|
|||||||
if d.service {
|
if d.service {
|
||||||
_, healthErr := d.health(ctx)
|
_, healthErr := d.health(ctx)
|
||||||
var fatal *managedRecoveryFatal
|
var fatal *managedRecoveryFatal
|
||||||
|
var failure *managedResourceFailure
|
||||||
|
j, e := m.load()
|
||||||
|
if e != nil {
|
||||||
|
return result, &managedRecoveryFatal{Err: e}
|
||||||
|
}
|
||||||
|
if j.Scope == GooglevideoScopeExperimental {
|
||||||
|
healthErr = managedResourceHealthError(ctx, healthErr)
|
||||||
|
}
|
||||||
if ctx.Err() != nil || errors.As(healthErr, &fatal) || errors.Is(healthErr, context.Canceled) || errors.Is(healthErr, context.DeadlineExceeded) {
|
if ctx.Err() != nil || errors.As(healthErr, &fatal) || errors.Is(healthErr, context.Canceled) || errors.Is(healthErr, context.DeadlineExceeded) {
|
||||||
return result, managedRecoveryContextError(ctx, healthErr)
|
return result, managedRecoveryContextError(ctx, healthErr)
|
||||||
}
|
}
|
||||||
var failure *managedResourceFailure
|
// A previously authorized native suffix stays adaptive after promotion,
|
||||||
if errors.As(healthErr, &failure) && assetEndpoint(failure.Host) {
|
// even if the operator's legacy environment still says "cold". Stored
|
||||||
|
// validation permits reconstruction, never fresh proof authorization.
|
||||||
|
if j.Scope == GooglevideoScopeSuffix {
|
||||||
|
for _, id := range j.Candidates {
|
||||||
|
c, err := readStrategyCandidatePolicy(d.db, id, false)
|
||||||
|
if err != nil {
|
||||||
|
return result, &managedRecoveryFatal{Err: err}
|
||||||
|
}
|
||||||
|
if c.Group == "googlevideo" && c.Mode == QualificationAdaptive && c.Qualified {
|
||||||
|
d.adaptive = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if errors.As(healthErr, &failure) && assetEndpoint(failure.Host) && j.Scope != GooglevideoScopeExperimental && !d.adaptive {
|
||||||
return result, repairManagedAssets(ctx, d, failure.Host)
|
return result, repairManagedAssets(ctx, d, failure.Host)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -440,9 +466,7 @@ func recoverManagedTarget(ctx context.Context, d managedRecoveryDeps) (result ma
|
|||||||
return result, e
|
return result, e
|
||||||
}
|
}
|
||||||
if experimental {
|
if experimental {
|
||||||
// Operator extrapolation survives recovery in either configured mode.
|
result.GooglevideoID, result.GGCError = recoverManagedExperimental(ctx, d)
|
||||||
// Frontend/asset repairs remain available, but cannot qualify this policy.
|
|
||||||
result.GGCError = errManagedGGCUnresolved
|
|
||||||
return result, result.GGCError
|
return result, result.GGCError
|
||||||
}
|
}
|
||||||
if !d.service && (len(current) != 1 || current[0] != result.FrontendID) {
|
if !d.service && (len(current) != 1 || current[0] != result.FrontendID) {
|
||||||
@@ -716,14 +740,6 @@ func wireManagedRecovery(monitor *managedMonitorDeps, recovery managedRecoveryDe
|
|||||||
if err := guard(); err != nil {
|
if err := guard(); err != nil {
|
||||||
return &managedRecoveryFatal{Err: err}
|
return &managedRecoveryFatal{Err: err}
|
||||||
}
|
}
|
||||||
status, err := recovery.health(ctx)
|
|
||||||
err = managedRecoveryContextError(ctx, err)
|
|
||||||
if err != nil {
|
|
||||||
return err
|
|
||||||
}
|
|
||||||
if status != "OK" {
|
|
||||||
return &managedHealthRejection{Status: status}
|
|
||||||
}
|
|
||||||
transaction := *recovery.transaction
|
transaction := *recovery.transaction
|
||||||
transaction.Health = func() (string, error) { return recovery.health(ctx) }
|
transaction.Health = func() (string, error) { return recovery.health(ctx) }
|
||||||
// ActiveCandidates is read-only; never permit a probe to reload config.
|
// ActiveCandidates is read-only; never permit a probe to reload config.
|
||||||
@@ -737,9 +753,26 @@ func wireManagedRecovery(monitor *managedMonitorDeps, recovery managedRecoveryDe
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return &managedRecoveryFatal{Err: err}
|
return &managedRecoveryFatal{Err: err}
|
||||||
}
|
}
|
||||||
|
status, err := recovery.health(ctx)
|
||||||
|
if journal.Scope == GooglevideoScopeExperimental && recovery.service && recovery.recheckExperimental != nil {
|
||||||
|
err = managedResourceHealthError(ctx, err)
|
||||||
|
} else {
|
||||||
|
err = managedRecoveryContextError(ctx, err)
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if status != "OK" {
|
||||||
|
return &managedHealthRejection{Status: status}
|
||||||
|
}
|
||||||
if journal.Scope == GooglevideoScopeExperimental {
|
if journal.Scope == GooglevideoScopeExperimental {
|
||||||
|
if !recovery.service || recovery.recheckExperimental == nil {
|
||||||
return errManagedGGCUnresolved
|
return errManagedGGCUnresolved
|
||||||
}
|
}
|
||||||
|
// Operational service health is not family qualification.
|
||||||
|
reportGGC(errManagedExperimentalUnqualified)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
// Two bounded Player requests reuse the existing health/discovery adapters.
|
// Two bounded Player requests reuse the existing health/discovery adapters.
|
||||||
host, err := recovery.discover(ctx)
|
host, err := recovery.discover(ctx)
|
||||||
err = managedRecoveryContextError(ctx, err)
|
err = managedRecoveryContextError(ctx, err)
|
||||||
@@ -758,7 +791,7 @@ func wireManagedRecovery(monitor *managedMonitorDeps, recovery managedRecoveryDe
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return &managedRecoveryFatal{Err: err}
|
return &managedRecoveryFatal{Err: err}
|
||||||
}
|
}
|
||||||
if recovery.adaptive && candidate.Group == "googlevideo" && candidate.Mode == QualificationAdaptive && candidate.Qualified {
|
if candidate.Group == "googlevideo" && candidate.Mode == QualificationAdaptive && candidate.Qualified {
|
||||||
journal, e := transaction.load()
|
journal, e := transaction.load()
|
||||||
if e != nil {
|
if e != nil {
|
||||||
return &managedRecoveryFatal{Err: e}
|
return &managedRecoveryFatal{Err: e}
|
||||||
|
|||||||
@@ -34,17 +34,18 @@ func TestManagedRecoveryProductionTransactionHooks(t *testing.T) {
|
|||||||
d.transaction = &managedTransaction{Dir: d.transaction.Dir, Name: "config"}
|
d.transaction = &managedTransaction{Dir: d.transaction.Dir, Name: "config"}
|
||||||
d.service, d.adaptive = true, true
|
d.service, d.adaptive = true, true
|
||||||
d.frontHealth = d.health
|
d.frontHealth = d.health
|
||||||
|
samples := 0
|
||||||
d.mediaSamples = func(context.Context) ([]mediaResource, error) {
|
d.mediaSamples = func(context.Context) ([]mediaResource, error) {
|
||||||
t.Fatal("asset qualification must precede private media discovery")
|
samples++
|
||||||
return nil, nil
|
return nil, &InsufficientAdaptiveSamples{}
|
||||||
}
|
}
|
||||||
d.verifyGooglevideo = func(context.Context, storedCandidate, []mediaResource) (GooglevideoQualificationProof, error) {
|
d.verifyGooglevideo = func(context.Context, storedCandidate, []mediaResource) (GooglevideoQualificationProof, error) {
|
||||||
t.Fatal("unexpected verification")
|
t.Fatal("unexpected verification")
|
||||||
return GooglevideoQualificationProof{}, nil
|
return GooglevideoQualificationProof{}, nil
|
||||||
}
|
}
|
||||||
result, err := recoverManagedTarget(context.Background(), d)
|
result, err := recoverManagedTarget(context.Background(), d)
|
||||||
var failure *managedResourceFailure
|
var failure *InsufficientAdaptiveSamples
|
||||||
if err != nil || result.FrontendID != front || !errors.As(result.GGCError, &failure) {
|
if err != nil || result.FrontendID != front || samples != 1 || !errors.As(result.GGCError, &failure) {
|
||||||
// Never expose private DB contents or error causes.
|
// Never expose private DB contents or error causes.
|
||||||
t.Fatalf("frontend_preserved=%t recovery_error_type=%T ggc_error_type=%T", result.FrontendID == front, err, result.GGCError)
|
t.Fatalf("frontend_preserved=%t recovery_error_type=%T ggc_error_type=%T", result.FrontendID == front, err, result.GGCError)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -93,9 +93,10 @@ func TestManagedExpiredAdaptiveJournalRecovery(t *testing.T) {
|
|||||||
verifies++
|
verifies++
|
||||||
return r.verifyGooglevideo(ctx, c, resources)
|
return r.verifyGooglevideo(ctx, c, resources)
|
||||||
}
|
}
|
||||||
|
d.recheckGooglevideo = managedGooglevideoApplyRecheck(r, d.db)
|
||||||
if scenario == "rollback" {
|
if scenario == "rollback" {
|
||||||
d.health = func(context.Context) (string, error) {
|
d.recheckGooglevideo = func(context.Context, storedCandidate, []mediaResource, *managedTransaction) error {
|
||||||
return "DEGRADED", &managedResourceFailure{"media", resources[0].host, nil}
|
return errCandidateNonpass
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
result, err := recoverManagedTarget(context.Background(), d)
|
result, err := recoverManagedTarget(context.Background(), d)
|
||||||
|
|||||||
@@ -305,12 +305,14 @@ func TestManagedServiceAssetLocalHealthFailureRollsBack(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func TestManagedAssetHealthNonpassPreservesMixedSafetyFailures(t *testing.T) {
|
func TestManagedAssetHealthNonpassPreservesMixedSafetyFailures(t *testing.T) {
|
||||||
badBody := &managedRecoveryFatal{Err: errors.New("invalid image body")}
|
badBody := resourceFailure(context.Background(), imageBodyRejection("invalid image body"))
|
||||||
for _, tc := range []struct {
|
for _, tc := range []struct {
|
||||||
err error
|
err error
|
||||||
nonpass bool
|
nonpass bool
|
||||||
}{
|
}{
|
||||||
{badBody, true},
|
{badBody, true},
|
||||||
|
{resourceFailure(context.Background(), errors.New("invalid image body")), false},
|
||||||
|
{&managedRecoveryFatal{Err: errors.New("invalid image body")}, false},
|
||||||
{errors.Join(badBody, &managedRecoveryFatal{Err: errors.New("guard failed")}), false},
|
{errors.Join(badBody, &managedRecoveryFatal{Err: errors.New("guard failed")}), false},
|
||||||
{errors.Join(badBody, context.Canceled), false},
|
{errors.Join(badBody, context.Canceled), false},
|
||||||
{&managedAssetCleanupFailure{Err: errors.New("cleanup failed")}, false},
|
{&managedAssetCleanupFailure{Err: errors.New("cleanup failed")}, false},
|
||||||
|
|||||||
@@ -141,7 +141,7 @@ func runManagedMonitor(ctx context.Context, d managedMonitorDeps) (err error) {
|
|||||||
if ctx.Err() != nil {
|
if ctx.Err() != nil {
|
||||||
return ctx.Err()
|
return ctx.Err()
|
||||||
}
|
}
|
||||||
if errors.Is(err, errNoQualifiedStrategy) {
|
if errors.Is(err, errNoQualifiedStrategy) || errors.Is(err, errGooglevideoInsufficientSamples) {
|
||||||
if err := pauseMonitorRecovery(d.db, d.serviceID, d.now(), managedRecoveryCooldown(err, d.cooldown)); err != nil {
|
if err := pauseMonitorRecovery(d.db, d.serviceID, d.now(), managedRecoveryCooldown(err, d.cooldown)); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,464 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"mime"
|
||||||
|
"net"
|
||||||
|
"net/netip"
|
||||||
|
"net/url"
|
||||||
|
"os"
|
||||||
|
"os/signal"
|
||||||
|
"regexp"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
"syscall"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
var errResourceConfig = errors.New("invalid resource configuration")
|
||||||
|
var resourceID = regexp.MustCompile(`^[a-z][a-z0-9_-]{0,47}$`)
|
||||||
|
var resourceEnv = regexp.MustCompile(`^[A-Z_][A-Z0-9_]{0,127}$`)
|
||||||
|
var resourceHeader = regexp.MustCompile("^[!#$%&'*+.^_`|~0-9A-Za-z-]{1,64}$")
|
||||||
|
var resourceNumericHost = regexp.MustCompile(`(?i)^(?:[0-9]+|0x[0-9a-f]+)(?:\.(?:[0-9]+|0x[0-9a-f]+))*$`)
|
||||||
|
|
||||||
|
type resourceCheckConfig struct {
|
||||||
|
Version int `json:"version"`
|
||||||
|
Services []resourceCheckService `json:"services"`
|
||||||
|
}
|
||||||
|
type resourceCheckService struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
Resources []resourceCheckSpec `json:"resources"`
|
||||||
|
}
|
||||||
|
type resourceCheckSpec struct {
|
||||||
|
ID string `json:"id"`
|
||||||
|
Group string `json:"group"`
|
||||||
|
Kind string `json:"kind"`
|
||||||
|
URL string `json:"url,omitempty"`
|
||||||
|
URLEnv string `json:"url_env,omitempty"`
|
||||||
|
Method string `json:"method"`
|
||||||
|
TLS string `json:"tls"`
|
||||||
|
TimeoutMS int `json:"timeout_ms"`
|
||||||
|
IntervalMS *int `json:"interval_ms,omitempty"`
|
||||||
|
CooldownMS *int `json:"cooldown_ms,omitempty"`
|
||||||
|
Expect resourceCheckExpect `json:"expect"`
|
||||||
|
HeadersEnv map[string]string `json:"headers_env,omitempty"`
|
||||||
|
AllowPrivate bool `json:"allow_private,omitempty"`
|
||||||
|
AllowedNetworks []string `json:"allowed_networks,omitempty"`
|
||||||
|
host, privateConfig string
|
||||||
|
networks []netip.Prefix
|
||||||
|
}
|
||||||
|
type resourceCheckExpect struct {
|
||||||
|
Statuses []int `json:"statuses"`
|
||||||
|
ContentType string `json:"content_type,omitempty"`
|
||||||
|
MinBytes int `json:"min_bytes"`
|
||||||
|
MaxBytes int `json:"max_bytes"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// encoding/json otherwise silently accepts duplicate keys and null scalars.
|
||||||
|
func uniqueResourceJSON(d *json.Decoder, depth int) error {
|
||||||
|
if depth > 16 {
|
||||||
|
return errResourceConfig
|
||||||
|
}
|
||||||
|
t, err := d.Token()
|
||||||
|
if err != nil || t == nil {
|
||||||
|
return errResourceConfig
|
||||||
|
}
|
||||||
|
if delim, ok := t.(json.Delim); ok {
|
||||||
|
switch delim {
|
||||||
|
case '{':
|
||||||
|
seen := map[string]bool{}
|
||||||
|
for d.More() {
|
||||||
|
key, err := d.Token()
|
||||||
|
if err != nil {
|
||||||
|
return errResourceConfig
|
||||||
|
}
|
||||||
|
s, ok := key.(string)
|
||||||
|
if !ok || seen[s] {
|
||||||
|
return errResourceConfig
|
||||||
|
}
|
||||||
|
seen[s] = true
|
||||||
|
if err := uniqueResourceJSON(d, depth+1); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
case '[':
|
||||||
|
for d.More() {
|
||||||
|
if err := uniqueResourceJSON(d, depth+1); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
return errResourceConfig
|
||||||
|
}
|
||||||
|
if _, err := d.Token(); err != nil {
|
||||||
|
return errResourceConfig
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func privateCurlValue(v string) string {
|
||||||
|
return "\"" + strings.NewReplacer("\\", "\\\\", "\"", "\\\"").Replace(v) + "\""
|
||||||
|
}
|
||||||
|
func safeResourceValue(v string) bool {
|
||||||
|
return len(v) <= 16<<10 && strings.IndexFunc(v, func(c rune) bool { return c < 32 || c == 127 }) < 0
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseResourceChecks(data []byte, lookup func(string) (string, bool)) (resourceCheckConfig, error) {
|
||||||
|
var cfg resourceCheckConfig
|
||||||
|
fail := func() (resourceCheckConfig, error) { return resourceCheckConfig{}, errResourceConfig }
|
||||||
|
if len(data) > 1<<20 || lookup == nil {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
d := json.NewDecoder(bytes.NewReader(data))
|
||||||
|
if uniqueResourceJSON(d, 0) != nil {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
if _, err := d.Token(); err != io.EOF {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
d = json.NewDecoder(bytes.NewReader(data))
|
||||||
|
d.DisallowUnknownFields()
|
||||||
|
if d.Decode(&cfg) != nil || cfg.Version != 1 || len(cfg.Services) == 0 || len(cfg.Services) > 16 {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
services, total := map[string]bool{}, 0
|
||||||
|
for si := range cfg.Services {
|
||||||
|
s := &cfg.Services[si]
|
||||||
|
if !resourceID.MatchString(s.ID) || services[s.ID] || len(s.Resources) == 0 {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
services[s.ID] = true
|
||||||
|
ids := map[string]bool{}
|
||||||
|
for ri := range s.Resources {
|
||||||
|
r := &s.Resources[ri]
|
||||||
|
total++
|
||||||
|
if total > 64 || !resourceID.MatchString(r.ID) || ids[r.ID] || !resourceID.MatchString(r.Group) || r.Kind != "http" {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
ids[r.ID] = true
|
||||||
|
for _, scheduling := range []struct {
|
||||||
|
value **int
|
||||||
|
fallback int
|
||||||
|
}{{&r.IntervalMS, 300000}, {&r.CooldownMS, 30000}} {
|
||||||
|
if *scheduling.value == nil {
|
||||||
|
v := scheduling.fallback
|
||||||
|
*scheduling.value = &v
|
||||||
|
}
|
||||||
|
if **scheduling.value < 100 || **scheduling.value > 86400000 {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (r.URL == "") == (r.URLEnv == "") {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
if r.URLEnv != "" {
|
||||||
|
if !resourceEnv.MatchString(r.URLEnv) {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
var ok bool
|
||||||
|
r.URL, ok = lookup(r.URLEnv)
|
||||||
|
if !ok || r.URL == "" {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
u, err := url.Parse(r.URL)
|
||||||
|
if err != nil {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
r.host = u.Hostname()
|
||||||
|
// curl normalizes legacy IPv4 spellings before matching --connect-to.
|
||||||
|
// Only canonical literals may bypass DNS without defeating that match.
|
||||||
|
if resourceNumericHost.MatchString(r.host) {
|
||||||
|
ip := net.ParseIP(r.host)
|
||||||
|
if ip == nil || ip.To4() == nil || ip.String() != r.host {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if _, err := privateResourceURL(r.URL, r.host); err != nil {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
// Reject encoded controls too, without ever displaying the decoded path.
|
||||||
|
decoded, err := url.PathUnescape(r.URL)
|
||||||
|
if err != nil || !safeResourceValue(decoded) {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
if r.Method != "GET" && r.Method != "HEAD" {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
if r.TLS != "tls12" && r.TLS != "tls13" && r.TLS != "both" {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
if r.TimeoutMS < 100 || r.TimeoutMS > 30000 || r.Expect.MaxBytes < 1 || r.Expect.MaxBytes > 5<<20 || r.Expect.MinBytes < 0 || r.Expect.MinBytes > r.Expect.MaxBytes || (r.Method == "HEAD" && r.Expect.MinBytes != 0) {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
if len(r.Expect.Statuses) == 0 || len(r.Expect.Statuses) > 16 {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
statuses := map[int]bool{}
|
||||||
|
for _, status := range r.Expect.Statuses {
|
||||||
|
if status < 200 || status > 599 || status >= 300 && status < 400 || statuses[status] {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
statuses[status] = true
|
||||||
|
}
|
||||||
|
if r.Expect.ContentType != "" {
|
||||||
|
mt, params, err := mime.ParseMediaType(r.Expect.ContentType)
|
||||||
|
if err != nil || len(params) != 0 || mt != r.Expect.ContentType {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(r.HeadersEnv) > 16 {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
headers := map[string]bool{}
|
||||||
|
names := make([]string, 0, len(r.HeadersEnv))
|
||||||
|
for name := range r.HeadersEnv {
|
||||||
|
names = append(names, name)
|
||||||
|
}
|
||||||
|
sort.Strings(names)
|
||||||
|
for _, name := range names {
|
||||||
|
env := r.HeadersEnv[name]
|
||||||
|
lower := strings.ToLower(name)
|
||||||
|
if !resourceHeader.MatchString(name) || headers[lower] || !resourceEnv.MatchString(env) {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
headers[lower] = true
|
||||||
|
switch lower {
|
||||||
|
case "host", "connection", "transfer-encoding", "content-length", "expect", "upgrade", "proxy-authorization", "proxy-connection", "te", "trailer", "range", "accept-encoding":
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
value, ok := lookup(env)
|
||||||
|
if !ok || value == "" || !safeResourceValue(value) {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
r.privateConfig += "header = " + privateCurlValue(name+": "+value) + "\n"
|
||||||
|
}
|
||||||
|
if len(r.AllowedNetworks) > 16 || r.AllowPrivate && len(r.AllowedNetworks) == 0 {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
for _, raw := range r.AllowedNetworks {
|
||||||
|
prefix, err := netip.ParsePrefix(raw)
|
||||||
|
if err != nil || prefix != prefix.Masked() || prefix.Addr().Is4In6() {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
r.networks = append(r.networks, prefix)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return cfg, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func readResourceChecks(path string, lookup func(string) (string, bool)) (resourceCheckConfig, error) {
|
||||||
|
return readResourceChecksMode(path, lookup, false)
|
||||||
|
}
|
||||||
|
|
||||||
|
func readResourceChecksMode(path string, lookup func(string) (string, bool), owned bool) (resourceCheckConfig, error) {
|
||||||
|
fd, err := syscall.Open(path, syscall.O_RDONLY|syscall.O_NOFOLLOW|syscall.O_NONBLOCK|syscall.O_CLOEXEC, 0)
|
||||||
|
if err != nil {
|
||||||
|
return resourceCheckConfig{}, errResourceConfig
|
||||||
|
}
|
||||||
|
f := os.NewFile(uintptr(fd), "resource config")
|
||||||
|
defer f.Close()
|
||||||
|
info, err := f.Stat()
|
||||||
|
if err != nil || !info.Mode().IsRegular() || info.Size() > 1<<20 {
|
||||||
|
return resourceCheckConfig{}, errResourceConfig
|
||||||
|
}
|
||||||
|
if owned {
|
||||||
|
st, ok := info.Sys().(*syscall.Stat_t)
|
||||||
|
if !ok || (st.Uid != 0 && st.Uid != uint32(os.Geteuid())) || st.Nlink != 1 || info.Mode().Perm()&0022 != 0 {
|
||||||
|
return resourceCheckConfig{}, errResourceConfig
|
||||||
|
}
|
||||||
|
}
|
||||||
|
data, err := io.ReadAll(io.LimitReader(f, (1<<20)+1))
|
||||||
|
if err != nil {
|
||||||
|
return resourceCheckConfig{}, errResourceConfig
|
||||||
|
}
|
||||||
|
if owned {
|
||||||
|
current, err := os.Lstat(path)
|
||||||
|
if err != nil || !os.SameFile(info, current) {
|
||||||
|
return resourceCheckConfig{}, errResourceConfig
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return parseResourceChecks(data, lookup)
|
||||||
|
}
|
||||||
|
|
||||||
|
var resourceDeniedNetworks = []string{"0.0.0.0/8", "100.64.0.0/10", "192.0.0.0/24", "192.0.2.0/24", "198.18.0.0/15", "198.51.100.0/24", "203.0.113.0/24", "240.0.0.0/4", "2001:db8::/32", "2001::/23", "2002::/16", "64:ff9b::/96", "64:ff9b:1::/48", "100::/64"}
|
||||||
|
|
||||||
|
func approvedResourceIP(r resourceCheckSpec, ip net.IP) bool {
|
||||||
|
a, ok := netip.AddrFromSlice(ip)
|
||||||
|
if !ok {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
a = a.Unmap()
|
||||||
|
// Link-local and metadata are never enabled by allow_private.
|
||||||
|
if !a.IsGlobalUnicast() || a.IsLoopback() || a.IsLinkLocalUnicast() || a.IsLinkLocalMulticast() || a.String() == "100.100.100.200" || a.String() == "fd00:ec2::254" || a.String() == "168.63.129.16" {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for _, raw := range resourceDeniedNetworks {
|
||||||
|
p := netip.MustParsePrefix(raw)
|
||||||
|
if p.Contains(a) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if a.Is6() && !a.IsPrivate() && !netip.MustParsePrefix("2000::/3").Contains(a) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if a.IsPrivate() || a.IsLoopback() {
|
||||||
|
if !r.AllowPrivate {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(r.networks) != 0 {
|
||||||
|
for _, p := range r.networks {
|
||||||
|
if p.Contains(a) {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
type resourceCheckDeps struct {
|
||||||
|
lookup func(string) (string, bool)
|
||||||
|
guard func() error
|
||||||
|
resolve func(context.Context, string, string) ([]net.IP, error)
|
||||||
|
runner subprocessRunner
|
||||||
|
}
|
||||||
|
|
||||||
|
func resolveResourceIPs(ctx context.Context, server, host string) ([]net.IP, error) {
|
||||||
|
return resolveResourceIPsWithDial(ctx, server, host, (&net.Dialer{}).DialContext)
|
||||||
|
}
|
||||||
|
|
||||||
|
func productionResourceCheckDeps() resourceCheckDeps {
|
||||||
|
d := managedDependencies(runPlayerCurl)
|
||||||
|
return resourceCheckDeps{lookup: os.LookupEnv, guard: func() error {
|
||||||
|
if _, err := managedRuntimeGuard(d); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
// The provisioned local.json and owned SS chain use the target's fixed
|
||||||
|
// internal port, not an additional ss-local listener in this namespace.
|
||||||
|
if port, _ := d.lookup("SOCKS_PORT"); port != "1080" {
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
cfg, err := parseManagedDNSConfig(d.lookup)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return verifyResourceDNSRuntime(cfg)
|
||||||
|
}, resolve: resolveResourceIPs, runner: runResourceCurl}
|
||||||
|
}
|
||||||
|
|
||||||
|
type resourceCheckResult struct {
|
||||||
|
Service string `json:"service"`
|
||||||
|
Resource string `json:"resource"`
|
||||||
|
Group string `json:"group"`
|
||||||
|
Outcome string `json:"outcome"`
|
||||||
|
TLS string `json:"tls"`
|
||||||
|
Bytes int `json:"bytes"`
|
||||||
|
TimeMS float64 `json:"time_ms"`
|
||||||
|
Status int `json:"status"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// A synchronous runner: subprocess cancellation joins before this returns.
|
||||||
|
// The guard is also called after DNS, immediately before the pinned transfer.
|
||||||
|
func runResourceProtocol(ctx context.Context, service string, r resourceCheckSpec, protocol, server, proxy string, d resourceCheckDeps) (resourceCheckResult, error) {
|
||||||
|
result := resourceCheckResult{Service: service, Resource: r.ID, Group: r.Group, TLS: protocol, Outcome: "dns_failed"}
|
||||||
|
if err := d.guard(); err != nil {
|
||||||
|
return result, err
|
||||||
|
}
|
||||||
|
probeCtx, cancel := context.WithTimeout(ctx, time.Duration(r.TimeoutMS)*time.Millisecond)
|
||||||
|
defer cancel()
|
||||||
|
ips, err := d.resolve(probeCtx, server, r.host)
|
||||||
|
if err == nil && len(ips) > 0 && len(ips) <= 64 {
|
||||||
|
approved := true
|
||||||
|
for _, ip := range ips {
|
||||||
|
if !approvedResourceIP(r, ip) {
|
||||||
|
approved = false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
result.Outcome = "destination_denied"
|
||||||
|
if approved {
|
||||||
|
if err := d.guard(); err != nil {
|
||||||
|
return result, err
|
||||||
|
}
|
||||||
|
pin := r.host + ":443:" + net.JoinHostPort(ips[0].String(), "443")
|
||||||
|
options := resourceTransferOptions{generic: true, method: r.Method, limit: r.Expect.MaxBytes, timeout: time.Duration(r.TimeoutMS) * time.Millisecond, config: r.privateConfig + "connect-to = " + privateCurlValue(pin) + "\n"}
|
||||||
|
transfer, _, err := collectResourceTransferOptions(probeCtx, proxy, r.URL, r.host, protocol, false, d.runner, options)
|
||||||
|
result.Outcome = "transfer_failed"
|
||||||
|
if err == nil {
|
||||||
|
result.Bytes, result.TimeMS, result.Status = len(transfer.Body), transfer.LatencyMS, transfer.Status
|
||||||
|
result.Outcome = "expectation_failed"
|
||||||
|
for _, status := range r.Expect.Statuses {
|
||||||
|
if status == transfer.Status && len(transfer.Body) >= r.Expect.MinBytes && (r.Expect.ContentType == "" || transfer.ContentType == r.Expect.ContentType) {
|
||||||
|
result.Outcome = "pass"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if probeCtx.Err() != nil {
|
||||||
|
result.Outcome = "timeout"
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
result.Outcome = "cancelled"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return result, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func resourceCheckCLI(ctx context.Context, path string, stdout, stderr io.Writer, d resourceCheckDeps) int {
|
||||||
|
if d.lookup == nil {
|
||||||
|
fmt.Fprintln(stderr, "configerror: invalid resource check configuration")
|
||||||
|
return 2
|
||||||
|
}
|
||||||
|
cfg, err := readResourceChecks(path, d.lookup)
|
||||||
|
dns, dnsErr := parseManagedDNSConfig(d.lookup)
|
||||||
|
proxy, _ := d.lookup("ZAPRET_AUTOTUNE_SOCKS_PROXY")
|
||||||
|
proxy, proxyErr := managedProbeProxy(proxy)
|
||||||
|
if err != nil || dnsErr != nil || proxyErr != nil || d.guard == nil || d.resolve == nil || d.runner == nil || (dns.Mode == "native" && dns.NativeServer == "") {
|
||||||
|
fmt.Fprintln(stderr, "configerror: invalid resource check configuration")
|
||||||
|
return 2
|
||||||
|
}
|
||||||
|
server := dns.serverAddress()
|
||||||
|
if dns.Mode == "native" {
|
||||||
|
server = net.JoinHostPort(server, "53")
|
||||||
|
}
|
||||||
|
code := 0
|
||||||
|
for _, s := range cfg.Services {
|
||||||
|
for _, r := range s.Resources {
|
||||||
|
protocols := []string{r.TLS}
|
||||||
|
if r.TLS == "both" {
|
||||||
|
protocols = []string{"tls12", "tls13"}
|
||||||
|
}
|
||||||
|
for _, protocol := range protocols {
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
result, err := runResourceProtocol(ctx, s.ID, r, protocol, server, proxy, d)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintln(stderr, "blocked: resource check ownership unavailable")
|
||||||
|
return 2
|
||||||
|
}
|
||||||
|
if result.Outcome != "pass" {
|
||||||
|
code = 1
|
||||||
|
}
|
||||||
|
if json.NewEncoder(stdout).Encode(result) != nil {
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return code
|
||||||
|
}
|
||||||
|
|
||||||
|
func configuredResourceCheck(path string, stdout, stderr io.Writer) int {
|
||||||
|
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||||
|
defer stop()
|
||||||
|
return resourceCheckCLI(ctx, path, stdout, stderr, productionResourceCheckDeps())
|
||||||
|
}
|
||||||
@@ -0,0 +1,496 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/binary"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"net"
|
||||||
|
"os"
|
||||||
|
"os/exec"
|
||||||
|
"path/filepath"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
const resourceCheckExample = `{"version":1,"services":[{"id":"first","resources":[{"id":"page","group":"web","kind":"http","url":"https://example.com/private?secret=value","method":"GET","tls":"tls12","timeout_ms":1000,"expect":{"statuses":[200],"content_type":"text/plain","min_bytes":1,"max_bytes":100}}]},{"id":"second","resources":[{"id":"headers","group":"web","kind":"http","url_env":"CHECK_URL","method":"HEAD","tls":"both","timeout_ms":1000,"expect":{"statuses":[200],"min_bytes":0,"max_bytes":100},"headers_env":{"Authorization":"CHECK_TOKEN"}}]}]}`
|
||||||
|
|
||||||
|
func resourceCheckLookup(key string) (string, bool) {
|
||||||
|
values := map[string]string{"CHECK_URL": "https://example.org/secret", "CHECK_TOKEN": "Bearer private-token", "ZAPRET_AUTOTUNE_SOCKS_PROXY": "socks5h://127.0.0.1:1080"}
|
||||||
|
v, ok := values[key]
|
||||||
|
return v, ok
|
||||||
|
}
|
||||||
|
|
||||||
|
func resourceCheckFile(t *testing.T, data string) string {
|
||||||
|
t.Helper()
|
||||||
|
path := filepath.Join(t.TempDir(), "checks.json")
|
||||||
|
if err := os.WriteFile(path, []byte(data), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return path
|
||||||
|
}
|
||||||
|
|
||||||
|
func configuredWire(raw, method, protocol, mode string, out, diag io.Writer) {
|
||||||
|
body := "hello"
|
||||||
|
length := len(body)
|
||||||
|
if method == "HEAD" {
|
||||||
|
body, length = "", 999999999
|
||||||
|
}
|
||||||
|
status, verify, redirects, speed := 200, 0, 0, 50
|
||||||
|
if method == "HEAD" {
|
||||||
|
speed = 0
|
||||||
|
}
|
||||||
|
if mode == "redirect" {
|
||||||
|
status = 302
|
||||||
|
}
|
||||||
|
if mode == "redirect-count" {
|
||||||
|
redirects = 1
|
||||||
|
}
|
||||||
|
if mode == "verify" {
|
||||||
|
verify = 60
|
||||||
|
}
|
||||||
|
if mode == "origin" {
|
||||||
|
raw = "https://evil.example/"
|
||||||
|
}
|
||||||
|
if mode == "overflow" {
|
||||||
|
body, length = strings.Repeat("a", 101), 101
|
||||||
|
}
|
||||||
|
if mode == "truncated" {
|
||||||
|
length++
|
||||||
|
}
|
||||||
|
if mode == "extra" {
|
||||||
|
body += "extra"
|
||||||
|
}
|
||||||
|
if mode == "chunked" {
|
||||||
|
fmt.Fprint(out, "HTTP/1.1 200 OK\r\nContent-Type: text/plain\r\nTransfer-Encoding: chunked\r\n\r\n")
|
||||||
|
body = "5\r\nhello\r\n0\r\n\r\n"
|
||||||
|
} else {
|
||||||
|
fmt.Fprintf(out, "HTTP/1.1 %d OK\r\nContent-Type: text/plain\r\nContent-Length: %d\r\n\r\n", status, length)
|
||||||
|
}
|
||||||
|
fmt.Fprint(out, body)
|
||||||
|
metrics, _ := json.Marshal(map[string]interface{}{"url_effective": raw, "ssl_verify_result": verify, "num_redirects": redirects, "http_code": status, "time_total": 0.1, "speed_download": speed, "size_download": len(body)})
|
||||||
|
fmt.Fprint(out, resourceTrailer, string(metrics))
|
||||||
|
if mode == "tls" {
|
||||||
|
protocol = "tls13"
|
||||||
|
}
|
||||||
|
version := "1.2"
|
||||||
|
if protocol == "tls13" {
|
||||||
|
version = "1.3"
|
||||||
|
}
|
||||||
|
fmt.Fprintln(diag, "SSL connection using TLSv"+version)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceCheckSchema(t *testing.T) {
|
||||||
|
if _, err := parseResourceChecks([]byte(resourceCheckExample), resourceCheckLookup); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for _, data := range []string{
|
||||||
|
resourceCheckExample + ` {}`, strings.Replace(resourceCheckExample, `"version":1`, `"version":2`, 1),
|
||||||
|
strings.Replace(resourceCheckExample, `"version":1`, `"version":1,"version":1`, 1),
|
||||||
|
strings.Replace(resourceCheckExample, `"version":1`, `"unknown":1,"version":1`, 1),
|
||||||
|
strings.Replace(resourceCheckExample, `"id":"first"`, `"id":"First /"`, 1),
|
||||||
|
strings.Replace(resourceCheckExample, `"id":"second"`, `"id":"first"`, 1),
|
||||||
|
strings.Replace(resourceCheckExample, `"max_bytes":100`, `"max_bytes":0`, 1),
|
||||||
|
strings.Replace(resourceCheckExample, `"method":"GET"`, `"method":"POST"`, 1),
|
||||||
|
strings.Replace(resourceCheckExample, `"kind":"http"`, `"kind":"image"`, 1),
|
||||||
|
strings.Replace(resourceCheckExample, `"statuses":[200]`, `"statuses":[302]`, 1),
|
||||||
|
strings.Replace(resourceCheckExample, `"statuses":[200]`, `"statuses":[200,200]`, 1),
|
||||||
|
strings.Replace(resourceCheckExample, `"tls":"tls12"`, `"tls":"auto"`, 1),
|
||||||
|
strings.Replace(resourceCheckExample, `"timeout_ms":1000`, `"timeout_ms":30001`, 1),
|
||||||
|
strings.Replace(resourceCheckExample, `"headers_env":{"Authorization":"CHECK_TOKEN"}`, `"headers_env":{"Host":"CHECK_TOKEN"}`, 1),
|
||||||
|
strings.Replace(resourceCheckExample, `"CHECK_TOKEN"`, `"missing"`, 1),
|
||||||
|
strings.Replace(resourceCheckExample, `"CHECK_TOKEN"`, `"MISSING"`, 1),
|
||||||
|
strings.Replace(resourceCheckExample, `"CHECK_URL"`, `"MISSING"`, 1),
|
||||||
|
strings.Replace(resourceCheckExample, `"method":"GET"`, `"method":null`, 1),
|
||||||
|
strings.Replace(resourceCheckExample, `"kind":"http"`, `"allow_private":true,"kind":"http"`, 1),
|
||||||
|
strings.Replace(resourceCheckExample, `"kind":"http"`, `"allowed_networks":["10.0.0.1/24"],"kind":"http"`, 1),
|
||||||
|
} {
|
||||||
|
if _, err := parseResourceChecks([]byte(data), resourceCheckLookup); err == nil {
|
||||||
|
t.Fatal("invalid schema accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, raw := range []string{"http://example.com/", "https://user:password@example.com/", "https://example.com:444/", "https://example.com/a%0d%0aHeader", "https://example.com/a\n", "https://example.com/#fragment", "https://example.com:/", "https://[::1]/", "https://2130706433/", "https://127.1/", "https://0x7f000001/", "https://0177.0.0.1/"} {
|
||||||
|
lookup := func(key string) (string, bool) {
|
||||||
|
if key == "CHECK_URL" {
|
||||||
|
return raw, true
|
||||||
|
}
|
||||||
|
return resourceCheckLookup(key)
|
||||||
|
}
|
||||||
|
if _, err := parseResourceChecks([]byte(resourceCheckExample), lookup); err == nil {
|
||||||
|
t.Fatal("invalid origin accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
lookup := func(key string) (string, bool) {
|
||||||
|
if key == "CHECK_TOKEN" {
|
||||||
|
return "secret\r\nHost: evil", true
|
||||||
|
}
|
||||||
|
return resourceCheckLookup(key)
|
||||||
|
}
|
||||||
|
if _, err := parseResourceChecks([]byte(resourceCheckExample), lookup); err == nil {
|
||||||
|
t.Fatal("header injection")
|
||||||
|
}
|
||||||
|
for _, name := range []string{"Range", "rAnGe", "Accept-Encoding", "Host", "Connection", "Content-Length", "Proxy-Authorization"} {
|
||||||
|
data := strings.Replace(resourceCheckExample, `"headers_env":{"Authorization":"CHECK_TOKEN"}`, `"headers_env":{"`+name+`":"CHECK_TOKEN"}`, 1)
|
||||||
|
if _, err := parseResourceChecks([]byte(data), resourceCheckLookup); err == nil {
|
||||||
|
t.Fatal("transport header accepted", name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceCheckFiles(t *testing.T) {
|
||||||
|
path := resourceCheckFile(t, resourceCheckExample)
|
||||||
|
link := filepath.Join(t.TempDir(), "link")
|
||||||
|
if err := os.Symlink(path, link); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for _, p := range []string{link, t.TempDir(), resourceCheckFile(t, strings.Repeat("x", (1<<20)+1))} {
|
||||||
|
if _, err := readResourceChecks(p, resourceCheckLookup); err == nil {
|
||||||
|
t.Fatal("unsafe file accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if _, err := readResourceChecks(path, resourceCheckLookup); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
info, _ := os.Stat(path)
|
||||||
|
if info.Mode().Perm() != 0600 {
|
||||||
|
t.Fatal("changed permissions")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceCheckResolverPinnedServer(t *testing.T) {
|
||||||
|
var workers sync.WaitGroup
|
||||||
|
dial := func(ctx context.Context, network, address string) (net.Conn, error) {
|
||||||
|
if network != "tcp" || address != dnsListenAddress {
|
||||||
|
t.Error("DNS fallback", network, address)
|
||||||
|
}
|
||||||
|
client, server := net.Pipe()
|
||||||
|
workers.Add(1)
|
||||||
|
go func() {
|
||||||
|
defer workers.Done()
|
||||||
|
defer server.Close()
|
||||||
|
server.SetDeadline(time.Now().Add(time.Second))
|
||||||
|
var length [2]byte
|
||||||
|
if _, err := io.ReadFull(server, length[:]); err != nil {
|
||||||
|
t.Error(err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
query := make([]byte, int(binary.BigEndian.Uint16(length[:])))
|
||||||
|
if _, err := io.ReadFull(server, query); err != nil {
|
||||||
|
t.Error(err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
end, name, ok := dnsQuestion(query)
|
||||||
|
if !ok {
|
||||||
|
t.Error("invalid query")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if !bytes.Equal(name[:len(name)-4], []byte{7, 'e', 'x', 'a', 'm', 'p', 'l', 'e', 3, 'c', 'o', 'm', 0}) {
|
||||||
|
t.Error("search-domain expansion")
|
||||||
|
}
|
||||||
|
response := append([]byte(nil), query[:end]...)
|
||||||
|
response[2], response[3] = 0x81, 0x80
|
||||||
|
response[10], response[11] = 0, 0 // strip query EDNS
|
||||||
|
if binary.BigEndian.Uint16(query[end-4:end-2]) == 1 {
|
||||||
|
response[7] = 2
|
||||||
|
for _, ip := range [][]byte{{93, 184, 216, 34}, {10, 0, 0, 1}} {
|
||||||
|
response = append(response, 0xc0, 12, 0, 1, 0, 1, 0, 0, 0, 1, 0, 4)
|
||||||
|
response = append(response, ip...)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
binary.BigEndian.PutUint16(length[:], uint16(len(response)))
|
||||||
|
server.Write(append(length[:], response...))
|
||||||
|
}()
|
||||||
|
return client, nil
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
ips, err := resolveResourceIPsWithDial(ctx, dnsListenAddress, "example.com", dial)
|
||||||
|
workers.Wait()
|
||||||
|
if err != nil || len(ips) != 2 {
|
||||||
|
t.Fatal("lost mixed DNS answers", ips, err)
|
||||||
|
}
|
||||||
|
for _, ip := range ips {
|
||||||
|
if ip.String() != "93.184.216.34" && ip.String() != "10.0.0.1" {
|
||||||
|
t.Fatal("unexpected answer", ip)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
_, err = resolveResourceIPsWithDial(ctx, dnsListenAddress, "example.com", func(context.Context, string, string) (net.Conn, error) {
|
||||||
|
return nil, errors.New("forwarder unavailable")
|
||||||
|
})
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("DNS outage fell back")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceCheckDestinationPolicy(t *testing.T) {
|
||||||
|
cfg, _ := parseResourceChecks([]byte(resourceCheckExample), resourceCheckLookup)
|
||||||
|
r := cfg.Services[0].Resources[0]
|
||||||
|
for _, raw := range []string{"127.0.0.1", "10.0.0.1", "172.16.0.1", "192.168.0.1", "169.254.169.254", "0.0.0.0", "224.0.0.1", "100.64.0.1", "100.100.100.200", "198.18.0.1", "::1", "::2", "fe80::1", "fc00::1", "fd00:ec2::254", "168.63.129.16", "::ffff:127.0.0.1", "2002:7f00:1::1"} {
|
||||||
|
if approvedResourceIP(r, net.ParseIP(raw)) {
|
||||||
|
t.Fatal("unsafe destination", raw)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !approvedResourceIP(r, net.ParseIP("93.184.216.34")) {
|
||||||
|
t.Fatal("public denied")
|
||||||
|
}
|
||||||
|
data := strings.Replace(resourceCheckExample, `"kind":"http"`, `"allow_private":true,"allowed_networks":["10.20.0.0/16"],"kind":"http"`, 1)
|
||||||
|
cfg, err := parseResourceChecks([]byte(data), resourceCheckLookup)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
r = cfg.Services[0].Resources[0]
|
||||||
|
if !approvedResourceIP(r, net.ParseIP("10.20.1.2")) || approvedResourceIP(r, net.ParseIP("10.21.1.2")) || approvedResourceIP(r, net.ParseIP("93.184.216.34")) || approvedResourceIP(r, net.ParseIP("169.254.169.254")) {
|
||||||
|
t.Fatal("explicit subnet policy")
|
||||||
|
}
|
||||||
|
data = strings.Replace(resourceCheckExample, `"kind":"http"`, `"allowed_networks":["93.184.216.0/24"],"kind":"http"`, 1)
|
||||||
|
cfg, err = parseResourceChecks([]byte(data), resourceCheckLookup)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
r = cfg.Services[0].Resources[0]
|
||||||
|
if !approvedResourceIP(r, net.ParseIP("93.184.216.34")) || approvedResourceIP(r, net.ParseIP("8.8.8.8")) {
|
||||||
|
t.Fatal("public CIDR did not restrict")
|
||||||
|
}
|
||||||
|
data = strings.Replace(resourceCheckExample, `"kind":"http"`, `"allow_private":true,"allowed_networks":["0.0.0.0/0","::/0"],"kind":"http"`, 1)
|
||||||
|
cfg, err = parseResourceChecks([]byte(data), resourceCheckLookup)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
r = cfg.Services[0].Resources[0]
|
||||||
|
for _, raw := range []string{"127.0.0.1", "::1", "::ffff:127.0.0.1", "169.254.169.254", "fd00:ec2::254", "100.100.100.200", "168.63.129.16", "198.18.0.1", "fe80::1"} {
|
||||||
|
if approvedResourceIP(r, net.ParseIP(raw)) {
|
||||||
|
t.Fatal("private opt-in enabled metadata/reserved", raw)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceCheckIsolationPrivacyAndPinning(t *testing.T) {
|
||||||
|
for _, mode := range []string{"pass", "mixed", "rebind", "runner", "expect", "guard"} {
|
||||||
|
t.Run(mode, func(t *testing.T) {
|
||||||
|
calls, resolves, guards := 0, 0, 0
|
||||||
|
var output, diagnostic bytes.Buffer
|
||||||
|
d := resourceCheckDeps{lookup: resourceCheckLookup, guard: func() error {
|
||||||
|
guards++
|
||||||
|
if mode == "guard" {
|
||||||
|
return errors.New("private-secret")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}, resolve: func(_ context.Context, server, host string) ([]net.IP, error) {
|
||||||
|
resolves++
|
||||||
|
if server != dnsListenAddress {
|
||||||
|
t.Fatal("wrong resolver")
|
||||||
|
}
|
||||||
|
if host == "example.com" && mode == "mixed" {
|
||||||
|
return []net.IP{net.ParseIP("93.184.216.34"), net.ParseIP("127.0.0.1")}, nil
|
||||||
|
}
|
||||||
|
if resolves == 3 && mode == "rebind" {
|
||||||
|
return []net.IP{net.ParseIP("10.0.0.1")}, nil
|
||||||
|
}
|
||||||
|
return []net.IP{net.ParseIP("93.184.216.34")}, nil
|
||||||
|
}, runner: func(ctx context.Context, args []string, stdin io.Reader, out, diag io.Writer) error {
|
||||||
|
calls++
|
||||||
|
argv := strings.Join(args, " ")
|
||||||
|
if strings.Contains(argv, "example.") || strings.Contains(argv, "secret") || strings.Contains(argv, "private-token") || strings.Contains(argv, "93.184.") {
|
||||||
|
t.Fatal("private argv", args)
|
||||||
|
}
|
||||||
|
config, _ := io.ReadAll(stdin)
|
||||||
|
if !strings.Contains(string(config), `connect-to = "example.`) || !strings.Contains(string(config), `:443:93.184.216.34:443"`) || strings.Contains(argv, "--resolve") {
|
||||||
|
t.Fatal("not pinning CONNECT")
|
||||||
|
}
|
||||||
|
lines := strings.Split(string(config), "\n")
|
||||||
|
raw, err := strconv.Unquote(strings.TrimPrefix(lines[0], "url = "))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
method, protocol := "GET", "tls12"
|
||||||
|
if strings.Contains(argv, "--head") {
|
||||||
|
method = "HEAD"
|
||||||
|
if !strings.Contains(string(config), "Bearer private-token") {
|
||||||
|
t.Fatal("missing private header")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if strings.Contains(argv, "--tlsv1.3") {
|
||||||
|
protocol = "tls13"
|
||||||
|
}
|
||||||
|
if method == "GET" && mode == "runner" {
|
||||||
|
return errors.New("private-secret")
|
||||||
|
}
|
||||||
|
wireMode := ""
|
||||||
|
if mode == "expect" && method == "GET" {
|
||||||
|
wireMode = "redirect"
|
||||||
|
}
|
||||||
|
configuredWire(raw, method, protocol, wireMode, out, diag)
|
||||||
|
return nil
|
||||||
|
}}
|
||||||
|
code := resourceCheckCLI(context.Background(), resourceCheckFile(t, resourceCheckExample), &output, &diagnostic, d)
|
||||||
|
if mode == "pass" && (code != 0 || calls != 3 || resolves != 3) {
|
||||||
|
t.Fatalf("code=%d calls=%d output=%s", code, calls, output.String())
|
||||||
|
}
|
||||||
|
if mode != "pass" && code == 0 {
|
||||||
|
t.Fatal("false pass")
|
||||||
|
}
|
||||||
|
if mode == "rebind" && (calls != 2 || !strings.Contains(output.String(), `"outcome":"destination_denied"`)) {
|
||||||
|
t.Fatal("rebinding was not revalidated", output.String())
|
||||||
|
}
|
||||||
|
if mode != "guard" && strings.Count(output.String(), `"outcome":"pass"`) < 2 {
|
||||||
|
t.Fatal("failure stopped second service", output.String())
|
||||||
|
}
|
||||||
|
if mode == "guard" && (calls != 0 || resolves != 0 || guards != 1) {
|
||||||
|
t.Fatal("unguarded probe")
|
||||||
|
}
|
||||||
|
if strings.Contains(output.String()+diagnostic.String(), "secret") || strings.Contains(output.String(), "example.") || strings.Contains(output.String(), "Bearer") {
|
||||||
|
t.Fatal("private output")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceCheckTransferContract(t *testing.T) {
|
||||||
|
for _, method := range []string{"GET", "HEAD"} {
|
||||||
|
for _, mode := range []string{"pass", "chunked", "redirect", "redirect-count", "verify", "origin", "tls", "overflow", "truncated", "extra"} {
|
||||||
|
t.Run(method+"/"+mode, func(t *testing.T) {
|
||||||
|
if method == "HEAD" && mode == "chunked" {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
runner := func(_ context.Context, _ []string, _ io.Reader, out, diag io.Writer) error {
|
||||||
|
configuredWire("https://example.com/", method, "tls12", mode, out, diag)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
transfer, _, err := collectResourceTransferOptions(context.Background(), "socks5h://127.0.0.1:1080", "https://example.com/", "example.com", "tls12", false, runner, resourceTransferOptions{generic: true, method: method, limit: 100, timeout: time.Second})
|
||||||
|
valid := mode == "pass" || mode == "chunked" || method == "HEAD" && mode == "truncated"
|
||||||
|
if (err == nil) != valid {
|
||||||
|
t.Fatalf("err=%v", err)
|
||||||
|
}
|
||||||
|
if method == "HEAD" && err == nil && len(transfer.Body) != 0 {
|
||||||
|
t.Fatal("HEAD body")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceCheckCancellationAndPrevalidation(t *testing.T) {
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
joined := false
|
||||||
|
d := resourceCheckDeps{lookup: resourceCheckLookup, guard: func() error { return nil }, resolve: func(context.Context, string, string) ([]net.IP, error) {
|
||||||
|
return []net.IP{net.ParseIP("93.184.216.34")}, nil
|
||||||
|
}, runner: func(ctx context.Context, _ []string, _ io.Reader, _, _ io.Writer) error {
|
||||||
|
cancel()
|
||||||
|
<-ctx.Done()
|
||||||
|
joined = true
|
||||||
|
return ctx.Err()
|
||||||
|
}}
|
||||||
|
var output, diagnostic bytes.Buffer
|
||||||
|
if resourceCheckCLI(ctx, resourceCheckFile(t, resourceCheckExample), &output, &diagnostic, d) != 1 || !joined || strings.Count(output.String(), "cancelled") != 1 {
|
||||||
|
t.Fatal("cancel/join")
|
||||||
|
}
|
||||||
|
d.guard = func() error { t.Fatal("guard before all config validated"); return nil }
|
||||||
|
bad := strings.Replace(resourceCheckExample, `"CHECK_URL"`, `"MISSING"`, 1)
|
||||||
|
if resourceCheckCLI(context.Background(), resourceCheckFile(t, bad), &output, &diagnostic, d) != 2 {
|
||||||
|
t.Fatal("prevalidation")
|
||||||
|
}
|
||||||
|
// Runtime child deadlines are bounded and returned only after the child joins.
|
||||||
|
d.guard = func() error { return nil }
|
||||||
|
d.runner = func(ctx context.Context, _ []string, _ io.Reader, _, _ io.Writer) error {
|
||||||
|
<-ctx.Done()
|
||||||
|
return ctx.Err()
|
||||||
|
}
|
||||||
|
short := strings.ReplaceAll(resourceCheckExample, `"timeout_ms":1000`, `"timeout_ms":100`)
|
||||||
|
output.Reset()
|
||||||
|
if resourceCheckCLI(context.Background(), resourceCheckFile(t, short), &output, &diagnostic, d) != 1 || strings.Count(output.String(), `"outcome":"timeout"`) != 3 {
|
||||||
|
t.Fatal("timeouts", output.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// This loopback-only SOCKS fixture observes real curl's CONNECT destination.
|
||||||
|
// It rejects CONNECT immediately: no TLS or resource traffic leaves the fixture.
|
||||||
|
func TestResourceCheckCurlSOCKSPinnedDestination(t *testing.T) {
|
||||||
|
if _, err := exec.LookPath("curl"); err != nil {
|
||||||
|
t.Skip("curl unavailable")
|
||||||
|
}
|
||||||
|
listener, err := net.Listen("tcp", "127.0.0.1:0")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer listener.Close()
|
||||||
|
target := make(chan string, 1)
|
||||||
|
go func() {
|
||||||
|
conn, err := listener.Accept()
|
||||||
|
if err != nil {
|
||||||
|
target <- "accept failed"
|
||||||
|
return
|
||||||
|
}
|
||||||
|
defer conn.Close()
|
||||||
|
conn.SetDeadline(time.Now().Add(3 * time.Second))
|
||||||
|
fail := func() { target <- "invalid SOCKS framing" }
|
||||||
|
var hello [2]byte
|
||||||
|
if _, err := io.ReadFull(conn, hello[:]); err != nil || hello[0] != 5 {
|
||||||
|
fail()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
methods := make([]byte, int(hello[1]))
|
||||||
|
if _, err := io.ReadFull(conn, methods); err != nil {
|
||||||
|
fail()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
conn.Write([]byte{5, 0})
|
||||||
|
var header [4]byte
|
||||||
|
if _, err := io.ReadFull(conn, header[:]); err != nil || header[0] != 5 || header[1] != 1 {
|
||||||
|
fail()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
var host string
|
||||||
|
switch header[3] {
|
||||||
|
case 1:
|
||||||
|
b := make([]byte, 4)
|
||||||
|
if _, err := io.ReadFull(conn, b); err != nil {
|
||||||
|
fail()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
host = net.IP(b).String()
|
||||||
|
case 3:
|
||||||
|
var size [1]byte
|
||||||
|
if _, err := io.ReadFull(conn, size[:]); err != nil {
|
||||||
|
fail()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
b := make([]byte, int(size[0]))
|
||||||
|
if _, err := io.ReadFull(conn, b); err != nil {
|
||||||
|
fail()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
host = string(b)
|
||||||
|
default:
|
||||||
|
fail()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
var port [2]byte
|
||||||
|
if _, err := io.ReadFull(conn, port[:]); err != nil || port != [2]byte{1, 187} {
|
||||||
|
fail()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
target <- host
|
||||||
|
conn.Write([]byte{5, 5, 0, 1, 0, 0, 0, 0, 0, 0})
|
||||||
|
}()
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 4*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
_, _, err = collectResourceTransferOptions(ctx, "socks5h://"+listener.Addr().String(), "https://example.com/private?secret=value", "example.com", "tls12", false, runResourceCurl, resourceTransferOptions{generic: true, method: "GET", limit: 100, timeout: 3 * time.Second, config: "connect-to = \"example.com:443:93.184.216.34:443\"\n"})
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("fixture unexpectedly transferred")
|
||||||
|
}
|
||||||
|
select {
|
||||||
|
case host := <-target:
|
||||||
|
if host != "93.184.216.34" {
|
||||||
|
t.Fatal("CONNECT not pinned", host)
|
||||||
|
}
|
||||||
|
case <-ctx.Done():
|
||||||
|
t.Fatal("SOCKS child not joined")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -33,6 +33,41 @@ func runResourceCurl(ctx context.Context, args []string, stdin io.Reader, out, d
|
|||||||
|
|
||||||
type resourceError struct{ cause error }
|
type resourceError struct{ cause error }
|
||||||
|
|
||||||
|
// Created only after proxy, URL and TLS inputs have passed collector validation.
|
||||||
|
// Rejection is never qualification, nor evidence that local ownership failed.
|
||||||
|
type resourcePeerTLSRejection struct{ error }
|
||||||
|
|
||||||
|
func (e *resourcePeerTLSRejection) Error() string { return "resource peer TLS rejected" }
|
||||||
|
func (e *resourcePeerTLSRejection) Unwrap() error { return e.error }
|
||||||
|
|
||||||
|
func resourceRejectionOnly(err error) bool {
|
||||||
|
switch err.(type) {
|
||||||
|
case *resourcePeerTLSRejection, imageBodyRejection:
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
if e, ok := err.(*resourceError); ok {
|
||||||
|
return resourceRejectionOnly(e.cause)
|
||||||
|
}
|
||||||
|
if e, ok := err.(interface{ Unwrap() []error }); ok {
|
||||||
|
children := e.Unwrap()
|
||||||
|
if len(children) == 0 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for _, child := range children {
|
||||||
|
if !resourceRejectionOnly(child) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
// Safe metadata classifications contain neither page content nor resource URLs.
|
||||||
|
type avatarMetadataError string
|
||||||
|
|
||||||
|
func (e avatarMetadataError) Error() string { return string(e) }
|
||||||
|
|
||||||
func (e *resourceError) PublicError() string { return "resource qualification failed" }
|
func (e *resourceError) PublicError() string { return "resource qualification failed" }
|
||||||
func (e *resourceError) Error() string { return e.PublicError() }
|
func (e *resourceError) Error() string { return e.PublicError() }
|
||||||
func (e *resourceError) Unwrap() error { return e.cause }
|
func (e *resourceError) Unwrap() error { return e.cause }
|
||||||
@@ -41,7 +76,7 @@ func resourceFailure(ctx context.Context, err error) error {
|
|||||||
if ctx.Err() != nil {
|
if ctx.Err() != nil {
|
||||||
err = errors.Join(err, ctx.Err())
|
err = errors.Join(err, ctx.Err())
|
||||||
}
|
}
|
||||||
if !managedRecoveryTimeoutSafe(err) {
|
if !managedRecoveryTimeoutSafe(err) && !resourceRejectionOnly(err) {
|
||||||
err = &managedRecoveryFatal{Err: err}
|
err = &managedRecoveryFatal{Err: err}
|
||||||
}
|
}
|
||||||
return &resourceError{cause: err}
|
return &resourceError{cause: err}
|
||||||
@@ -78,7 +113,7 @@ func avatarResource(metadata []byte) (string, error) {
|
|||||||
}
|
}
|
||||||
var root interface{}
|
var root interface{}
|
||||||
if json.Unmarshal(metadata, &root) != nil {
|
if json.Unmarshal(metadata, &root) != nil {
|
||||||
return "", errors.New("invalid metadata")
|
return "", avatarMetadataError("jsonInvalid")
|
||||||
}
|
}
|
||||||
var visit func(interface{}, int) string
|
var visit func(interface{}, int) string
|
||||||
visit = func(v interface{}, depth int) string {
|
visit = func(v interface{}, depth int) string {
|
||||||
@@ -109,7 +144,7 @@ func avatarResource(metadata []byte) (string, error) {
|
|||||||
if raw := visit(root, 0); raw != "" {
|
if raw := visit(root, 0); raw != "" {
|
||||||
return raw, nil
|
return raw, nil
|
||||||
}
|
}
|
||||||
return "", errors.New("avatar unavailable")
|
return "", avatarMetadataError("noAllowedAvatar")
|
||||||
}
|
}
|
||||||
|
|
||||||
// The WWW page is transient; decode the embedded JSON structurally and never
|
// The WWW page is transient; decode the embedded JSON structurally and never
|
||||||
@@ -118,15 +153,20 @@ func avatarResourceFromPage(page []byte) (string, error) {
|
|||||||
if len(page) > 5<<20 {
|
if len(page) > 5<<20 {
|
||||||
return "", errors.New("page exceeds limit")
|
return "", errors.New("page exceeds limit")
|
||||||
}
|
}
|
||||||
|
found := false
|
||||||
for _, marker := range []string{"var ytInitialData = ", "ytInitialData = ", "window[\"ytInitialData\"] = "} {
|
for _, marker := range []string{"var ytInitialData = ", "ytInitialData = ", "window[\"ytInitialData\"] = "} {
|
||||||
if i := bytes.Index(page, []byte(marker)); i >= 0 {
|
if i := bytes.Index(page, []byte(marker)); i >= 0 {
|
||||||
|
found = true
|
||||||
var object json.RawMessage
|
var object json.RawMessage
|
||||||
if json.NewDecoder(bytes.NewReader(page[i+len(marker):])).Decode(&object) == nil {
|
if json.NewDecoder(bytes.NewReader(page[i+len(marker):])).Decode(&object) == nil {
|
||||||
return avatarResource(object)
|
return avatarResource(object)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return "", errors.New("structural avatar metadata unavailable")
|
if found {
|
||||||
|
return "", avatarMetadataError("jsonInvalid")
|
||||||
|
}
|
||||||
|
return "", avatarMetadataError("markerAbsent")
|
||||||
}
|
}
|
||||||
|
|
||||||
var resourceContentRange = regexp.MustCompile(`^bytes 0-([0-9]+)/([0-9]+)$`)
|
var resourceContentRange = regexp.MustCompile(`^bytes 0-([0-9]+)/([0-9]+)$`)
|
||||||
@@ -180,10 +220,24 @@ func playerMediaResource(body []byte) (mediaResource, mediaCapability, error) {
|
|||||||
const resourceTrailer = "\nAUTOTUNE_RESOURCE_METRICS:"
|
const resourceTrailer = "\nAUTOTUNE_RESOURCE_METRICS:"
|
||||||
|
|
||||||
func collectResourceTransfer(ctx context.Context, proxy, raw, host, protocol string, media bool, runner subprocessRunner) (assetTransfer, http.Header, error) {
|
func collectResourceTransfer(ctx context.Context, proxy, raw, host, protocol string, media bool, runner subprocessRunner) (assetTransfer, http.Header, error) {
|
||||||
|
return collectResourceTransferOptions(ctx, proxy, raw, host, protocol, media, runner, resourceTransferOptions{})
|
||||||
|
}
|
||||||
|
|
||||||
|
type resourceTransferOptions struct {
|
||||||
|
method string
|
||||||
|
limit int
|
||||||
|
timeout time.Duration
|
||||||
|
// Private curl config only: never placed in argv or public evidence.
|
||||||
|
config string
|
||||||
|
generic bool
|
||||||
|
}
|
||||||
|
|
||||||
|
func collectResourceTransferOptions(ctx context.Context, proxy, raw, host, protocol string, media bool, runner subprocessRunner, options resourceTransferOptions) (assetTransfer, http.Header, error) {
|
||||||
fail := func(err error) (assetTransfer, http.Header, error) {
|
fail := func(err error) (assetTransfer, http.Header, error) {
|
||||||
return assetTransfer{}, nil, resourceFailure(ctx, err)
|
return assetTransfer{}, nil, resourceFailure(ctx, err)
|
||||||
}
|
}
|
||||||
if !validDiscoveryProxy(proxy) || runner == nil {
|
proxy, proxyErr := managedProbeProxy(proxy)
|
||||||
|
if proxyErr != nil || runner == nil {
|
||||||
return fail(errors.New("invalid owned proxy or runner"))
|
return fail(errors.New("invalid owned proxy or runner"))
|
||||||
}
|
}
|
||||||
if _, err := privateResourceURL(raw, host); err != nil {
|
if _, err := privateResourceURL(raw, host); err != nil {
|
||||||
@@ -199,15 +253,32 @@ func collectResourceTransfer(ctx context.Context, proxy, raw, host, protocol str
|
|||||||
if media {
|
if media {
|
||||||
limit = 65536
|
limit = 65536
|
||||||
}
|
}
|
||||||
|
if options.generic {
|
||||||
|
limit = options.limit
|
||||||
|
}
|
||||||
// Qualification uses HTTP/1.1, not browser HTTP/2. --raw preserves native
|
// Qualification uses HTTP/1.1, not browser HTTP/2. --raw preserves native
|
||||||
// transfer framing so net/http, rather than curl and Go twice, decodes it.
|
// transfer framing so net/http, rather than curl and Go twice, decodes it.
|
||||||
args := []string{"--disable", "--silent", "--show-error", "--verbose", "--http1.1", "--raw", "--noproxy", "", "--proxy", proxy, "--proto", "=https", tlsFlag, "--tls-max", maxTLS, "--max-time", "30", "--max-filesize", strconv.Itoa(limit), "--suppress-connect-headers", "--include", "--write-out", resourceTrailer + "%{json}", "--config", "-"}
|
args := []string{"--disable", "--silent", "--show-error", "--verbose", "--http1.1", "--raw", "--noproxy", "", "--proxy", proxy, "--proto", "=https", tlsFlag, "--tls-max", maxTLS, "--max-time", "30", "--max-filesize", strconv.Itoa(limit), "--suppress-connect-headers", "--include", "--write-out", resourceTrailer + "%{json}", "--config", "-"}
|
||||||
if media {
|
if media {
|
||||||
args = append(args, "--range", "0-65535")
|
args = append(args, "--range", "0-65535")
|
||||||
}
|
}
|
||||||
|
if options.generic {
|
||||||
|
if (options.method != "GET" && options.method != "HEAD") || options.limit < 1 || options.limit > 5<<20 || options.timeout < 100*time.Millisecond || options.timeout > 30*time.Second {
|
||||||
|
return fail(errors.New("invalid transfer options"))
|
||||||
|
}
|
||||||
|
args = append(args, "--max-time", strconv.FormatFloat(options.timeout.Seconds(), 'f', 3, 64))
|
||||||
|
if options.method == "HEAD" {
|
||||||
|
args = append(args, "--head", "--max-filesize", "0")
|
||||||
|
}
|
||||||
|
}
|
||||||
// curl config quotes are not JSON quotes; reject controls above, escape only these.
|
// curl config quotes are not JSON quotes; reject controls above, escape only these.
|
||||||
config := "url = \"" + strings.NewReplacer("\\", "\\\\", "\"", "\\\"").Replace(raw) + "\"\n"
|
config := "url = \"" + strings.NewReplacer("\\", "\\\\", "\"", "\\\"").Replace(raw) + "\"\n"
|
||||||
childCtx, cancel := context.WithTimeout(ctx, 35*time.Second)
|
config += options.config
|
||||||
|
timeout := 35 * time.Second
|
||||||
|
if options.generic {
|
||||||
|
timeout = options.timeout
|
||||||
|
}
|
||||||
|
childCtx, cancel := context.WithTimeout(ctx, timeout)
|
||||||
defer cancel()
|
defer cancel()
|
||||||
var output, diagnostic bytes.Buffer
|
var output, diagnostic bytes.Buffer
|
||||||
out := &playerBoundedWriter{dst: &output, limit: limit + (96 << 10)}
|
out := &playerBoundedWriter{dst: &output, limit: limit + (96 << 10)}
|
||||||
@@ -217,6 +288,14 @@ func collectResourceTransfer(ctx context.Context, proxy, raw, host, protocol str
|
|||||||
err = errors.Join(err, errors.New("resource output exceeds limit"))
|
err = errors.Join(err, errors.New("resource output exceeds limit"))
|
||||||
}
|
}
|
||||||
if err != nil || childCtx.Err() != nil {
|
if err != nil || childCtx.Err() != nil {
|
||||||
|
// Only actual curl peer negatives become typed resource damage. Mixed
|
||||||
|
// cancellation, output overflow and explicit safety errors veto this path.
|
||||||
|
if exit, ok := err.(*exec.ExitError); ok && childCtx.Err() == nil {
|
||||||
|
switch exit.ExitCode() {
|
||||||
|
case 35, 51, 60:
|
||||||
|
err = &resourcePeerTLSRejection{error: err}
|
||||||
|
}
|
||||||
|
}
|
||||||
return fail(errors.Join(err, childCtx.Err()))
|
return fail(errors.Join(err, childCtx.Err()))
|
||||||
}
|
}
|
||||||
i := bytes.LastIndex(output.Bytes(), []byte(resourceTrailer))
|
i := bytes.LastIndex(output.Bytes(), []byte(resourceTrailer))
|
||||||
@@ -268,31 +347,41 @@ func collectResourceTransfer(ctx context.Context, proxy, raw, host, protocol str
|
|||||||
return fail(errors.New("ambiguous response framing"))
|
return fail(errors.New("ambiguous response framing"))
|
||||||
}
|
}
|
||||||
r := bufio.NewReader(bytes.NewReader(wire))
|
r := bufio.NewReader(bytes.NewReader(wire))
|
||||||
response, err := http.ReadResponse(r, nil)
|
var request *http.Request
|
||||||
|
if options.generic {
|
||||||
|
request = &http.Request{Method: options.method}
|
||||||
|
}
|
||||||
|
response, err := http.ReadResponse(r, request)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fail(errors.New("invalid response headers"))
|
return fail(errors.New("invalid response headers"))
|
||||||
}
|
}
|
||||||
defer response.Body.Close()
|
defer response.Body.Close()
|
||||||
// CONNECT headers are suppressed by curl. Interim/duplicate responses and
|
// CONNECT headers are suppressed by curl. Interim/duplicate responses and
|
||||||
// unrequested versions are outside this contract, never heuristically skipped.
|
// unrequested versions are outside this contract, never heuristically skipped.
|
||||||
if response.ProtoMajor != 1 || response.ProtoMinor != 1 || response.StatusCode < 200 || response.Header.Get("Content-Type") == "" {
|
if response.ProtoMajor != 1 || response.ProtoMinor != 1 || response.StatusCode < 200 || (!options.generic && response.Header.Get("Content-Type") == "") {
|
||||||
return fail(errors.New("unsupported response contract"))
|
return fail(errors.New("unsupported response contract"))
|
||||||
}
|
}
|
||||||
|
if options.generic && response.StatusCode >= 300 && response.StatusCode < 400 {
|
||||||
|
return fail(errors.New("redirect rejected"))
|
||||||
|
}
|
||||||
body, err := io.ReadAll(io.LimitReader(response.Body, int64(limit+1)))
|
body, err := io.ReadAll(io.LimitReader(response.Body, int64(limit+1)))
|
||||||
// With --raw curl counts the wire body, including chunk framing; the image
|
// With --raw curl counts the wire body, including chunk framing; the image
|
||||||
// limit and durable byte identity apply to the fully decoded payload instead.
|
// limit and durable byte identity apply to the fully decoded payload instead.
|
||||||
wireSize := len(wire) - headerEnd - 4
|
wireSize := len(wire) - headerEnd - 4
|
||||||
_, remaining := r.Peek(1)
|
_, remaining := r.Peek(1)
|
||||||
if err != nil || remaining != io.EOF || len(body) == 0 || len(body) > limit || metrics.Size != float64(wireSize) || response.StatusCode != metrics.Status {
|
if err != nil || remaining != io.EOF || (!options.generic && len(body) == 0) || len(body) > limit || metrics.Size != float64(wireSize) || response.StatusCode != metrics.Status {
|
||||||
return fail(errors.New("incomplete or oversized resource body"))
|
return fail(errors.New("incomplete or oversized resource body"))
|
||||||
}
|
}
|
||||||
for _, n := range []float64{metrics.Time, metrics.Speed} {
|
for _, n := range []float64{metrics.Time, metrics.Speed} {
|
||||||
if n <= 0 || math.IsNaN(n) || math.IsInf(n, 0) {
|
if n < 0 || (!options.generic && n == 0) || math.IsNaN(n) || math.IsInf(n, 0) {
|
||||||
return fail(errors.New("invalid resource metrics"))
|
return fail(errors.New("invalid resource metrics"))
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
if options.generic && metrics.Time > options.timeout.Seconds()+1 {
|
||||||
|
return fail(errors.New("resource timing exceeds budget"))
|
||||||
|
}
|
||||||
contentType, _, err := mime.ParseMediaType(response.Header.Get("Content-Type"))
|
contentType, _, err := mime.ParseMediaType(response.Header.Get("Content-Type"))
|
||||||
if err != nil {
|
if err != nil && (!options.generic || response.Header.Get("Content-Type") != "") {
|
||||||
return fail(errors.New("invalid content type"))
|
return fail(errors.New("invalid content type"))
|
||||||
}
|
}
|
||||||
t := assetTransfer{EffectiveHost: host, SSLVerifyResult: metrics.Verify, Redirects: *metrics.Redirects, TLSVersion: version, Status: metrics.Status, ContentType: contentType, Body: body, LatencyMS: metrics.Time * 1000, SpeedKBPS: metrics.Speed / 1024}
|
t := assetTransfer{EffectiveHost: host, SSLVerifyResult: metrics.Verify, Redirects: *metrics.Redirects, TLSVersion: version, Status: metrics.Status, ContentType: contentType, Body: body, LatencyMS: metrics.Time * 1000, SpeedKBPS: metrics.Speed / 1024}
|
||||||
|
|||||||
@@ -63,6 +63,7 @@ func TestResourceChild(t *testing.T) {
|
|||||||
if os.Getenv("AUTOTUNE_RESOURCE_CHILD") != "1" {
|
if os.Getenv("AUTOTUNE_RESOURCE_CHILD") != "1" {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
assertManagedDNSArgs(t, os.Args[3:], "socks5h://127.0.0.1:1080")
|
||||||
b, err := io.ReadAll(io.LimitReader(os.Stdin, 32<<10))
|
b, err := io.ReadAll(io.LimitReader(os.Stdin, 32<<10))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
os.Exit(3)
|
os.Exit(3)
|
||||||
@@ -84,9 +85,10 @@ func TestResourcePrivateChildAndEvidence(t *testing.T) {
|
|||||||
if strings.Contains(joined, "private-secret") || strings.Contains(joined, "https://") || strings.Contains(joined, "-k") || args[0] != "--disable" {
|
if strings.Contains(joined, "private-secret") || strings.Contains(joined, "https://") || strings.Contains(joined, "-k") || args[0] != "--disable" {
|
||||||
t.Fatalf("unsafe argv: %v", args)
|
t.Fatalf("unsafe argv: %v", args)
|
||||||
}
|
}
|
||||||
if !strings.Contains(joined, "--noproxy --proxy socks5://127.0.0.1:1080") || !strings.Contains(joined, "--tlsv1.2 --tls-max 1.2") {
|
if !strings.Contains(joined, "--noproxy --proxy socks5h://127.0.0.1:1080") || !strings.Contains(joined, "--tlsv1.2 --tls-max 1.2") {
|
||||||
t.Fatal("missing explicit routing/TLS")
|
t.Fatal("missing explicit routing/TLS")
|
||||||
}
|
}
|
||||||
|
assertManagedDNSArgs(t, args, "socks5h://127.0.0.1:1080")
|
||||||
if !strings.Contains(joined, "--http1.1 --raw") || !strings.Contains(joined, "--suppress-connect-headers") {
|
if !strings.Contains(joined, "--http1.1 --raw") || !strings.Contains(joined, "--suppress-connect-headers") {
|
||||||
t.Fatal("missing native HTTP/1.1 raw contract")
|
t.Fatal("missing native HTTP/1.1 raw contract")
|
||||||
}
|
}
|
||||||
@@ -216,7 +218,7 @@ func TestResourceRejectionsAndCancellation(t *testing.T) {
|
|||||||
t.Fatal("accepted failure or exposed secret", err)
|
t.Fatal("accepted failure or exposed secret", err)
|
||||||
}
|
}
|
||||||
var fatal *managedRecoveryFatal
|
var fatal *managedRecoveryFatal
|
||||||
if errors.As(err, &fatal) != (mode != "cancel") {
|
if errors.As(err, &fatal) != (mode != "cancel" && mode != "truncated" && mode != "type") {
|
||||||
t.Fatal("unsafe cancellation classification", err)
|
t.Fatal("unsafe cancellation classification", err)
|
||||||
}
|
}
|
||||||
if (mode == "cancel" || mode == "mixed") && !errors.Is(err, context.Canceled) {
|
if (mode == "cancel" || mode == "mixed") && !errors.Is(err, context.Canceled) {
|
||||||
@@ -247,6 +249,7 @@ func TestResourceDiscoveryAndMedia(t *testing.T) {
|
|||||||
}
|
}
|
||||||
for _, mode := range []string{"media", "bad-range", "root"} {
|
for _, mode := range []string{"media", "bad-range", "root"} {
|
||||||
runner := func(_ context.Context, args []string, _ io.Reader, out, diagnostic io.Writer) error {
|
runner := func(_ context.Context, args []string, _ io.Reader, out, diagnostic io.Writer) error {
|
||||||
|
assertManagedDNSArgs(t, args, "socks5h://127.0.0.1:1080")
|
||||||
if !strings.Contains(strings.Join(args, " "), "--range 0-65535") {
|
if !strings.Contains(strings.Join(args, " "), "--range 0-65535") {
|
||||||
t.Fatal("missing bounded range")
|
t.Fatal("missing bounded range")
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,245 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/rand"
|
||||||
|
"encoding/binary"
|
||||||
|
"io"
|
||||||
|
"net"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func resourceDNSQuery(host string, kind uint16) ([]byte, error) {
|
||||||
|
if !discoveryHostname(host) {
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
}
|
||||||
|
query := make([]byte, 12)
|
||||||
|
if _, err := rand.Read(query[:2]); err != nil {
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
}
|
||||||
|
query[2], query[5] = 1, 1
|
||||||
|
for _, label := range strings.Split(strings.ToLower(host), ".") {
|
||||||
|
query = append(query, byte(len(label)))
|
||||||
|
query = append(query, label...)
|
||||||
|
}
|
||||||
|
query = append(query, 0, byte(kind>>8), byte(kind), 0, 1)
|
||||||
|
if !validDNSQuery(query) {
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
}
|
||||||
|
return query, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func resourceDNSExchange(ctx context.Context, server string, query []byte, dial dnsDialFunc) ([]byte, error) {
|
||||||
|
conn, err := dial(ctx, "tcp", server)
|
||||||
|
if err != nil {
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
}
|
||||||
|
defer conn.Close()
|
||||||
|
deadline := time.Now().Add(5 * time.Second)
|
||||||
|
if d, ok := ctx.Deadline(); ok && d.Before(deadline) {
|
||||||
|
deadline = d
|
||||||
|
}
|
||||||
|
if conn.SetDeadline(deadline) != nil {
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
}
|
||||||
|
// Closing the connection interrupts framed I/O, including contexts cancelled
|
||||||
|
// without a deadline. The callback is joined before returning.
|
||||||
|
done := make(chan struct{})
|
||||||
|
stop := context.AfterFunc(ctx, func() { conn.Close(); close(done) })
|
||||||
|
defer func() {
|
||||||
|
if !stop() {
|
||||||
|
<-done
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
var size [2]byte
|
||||||
|
binary.BigEndian.PutUint16(size[:], uint16(len(query)))
|
||||||
|
if writeDNSFrame(conn, size[:], query) != nil {
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
}
|
||||||
|
if _, err := io.ReadFull(conn, size[:]); err != nil {
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
}
|
||||||
|
n := int(binary.BigEndian.Uint16(size[:]))
|
||||||
|
if n < 12 {
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
}
|
||||||
|
response := make([]byte, n)
|
||||||
|
if _, err := io.ReadFull(conn, response); err != nil {
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
}
|
||||||
|
if ctx.Err() != nil || !validDNSResponse(query, response) || response[2]&2 != 0 || response[3]&15 != 0 {
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
}
|
||||||
|
return response, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func resourceDNSHost(name []byte) (string, bool) {
|
||||||
|
var labels []string
|
||||||
|
for i := 0; i < len(name); {
|
||||||
|
n := int(name[i])
|
||||||
|
i++
|
||||||
|
if n == 0 {
|
||||||
|
host := strings.ToLower(strings.Join(labels, "."))
|
||||||
|
return host, i == len(name) && discoveryHostname(host)
|
||||||
|
}
|
||||||
|
if n > 63 || i+n > len(name) {
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
labels = append(labels, string(name[i:i+n]))
|
||||||
|
i += n
|
||||||
|
}
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
|
||||||
|
// Only Answer records on a complete, unambiguous chain from the original
|
||||||
|
// question contribute addresses. Authority/Additional glue is never a target.
|
||||||
|
func resourceDNSAnswers(query, wire []byte, priorAliases map[string]bool) ([]net.IP, string, error) {
|
||||||
|
if !validDNSResponse(query, wire) {
|
||||||
|
return nil, "", errDNSForwarder
|
||||||
|
}
|
||||||
|
_, question, ok := dnsQuestion(query)
|
||||||
|
if !ok {
|
||||||
|
return nil, "", errDNSForwarder
|
||||||
|
}
|
||||||
|
host, ok := resourceDNSHost(question[:len(question)-4])
|
||||||
|
if !ok {
|
||||||
|
return nil, "", errDNSForwarder
|
||||||
|
}
|
||||||
|
end, ok := dnsQuestionEnd(wire)
|
||||||
|
if !ok {
|
||||||
|
return nil, "", errDNSForwarder
|
||||||
|
}
|
||||||
|
count := int(binary.BigEndian.Uint16(wire[6:8]))
|
||||||
|
if count > 128 {
|
||||||
|
return nil, "", errDNSForwarder
|
||||||
|
}
|
||||||
|
aliases := map[string]string{}
|
||||||
|
addresses := map[string][]net.IP{}
|
||||||
|
for i := 0; i < count; i++ {
|
||||||
|
header, name, ok := dnsName(wire, end)
|
||||||
|
if !ok || header+10 > len(wire) {
|
||||||
|
return nil, "", errDNSForwarder
|
||||||
|
}
|
||||||
|
owner, valid := resourceDNSHost(name)
|
||||||
|
if !valid {
|
||||||
|
return nil, "", errDNSForwarder
|
||||||
|
}
|
||||||
|
kind, class := binary.BigEndian.Uint16(wire[header:]), binary.BigEndian.Uint16(wire[header+2:])
|
||||||
|
start := header + 10
|
||||||
|
end = start + int(binary.BigEndian.Uint16(wire[header+8:]))
|
||||||
|
if end > len(wire) || class != 1 {
|
||||||
|
return nil, "", errDNSForwarder
|
||||||
|
}
|
||||||
|
switch kind {
|
||||||
|
case 1, 28:
|
||||||
|
size := 4
|
||||||
|
if kind == 28 {
|
||||||
|
size = 16
|
||||||
|
}
|
||||||
|
if end-start != size {
|
||||||
|
return nil, "", errDNSForwarder
|
||||||
|
}
|
||||||
|
addresses[owner] = append(addresses[owner], append(net.IP(nil), wire[start:end]...))
|
||||||
|
case 5:
|
||||||
|
last, target, ok := dnsName(wire, start)
|
||||||
|
if !ok || last != end {
|
||||||
|
return nil, "", errDNSForwarder
|
||||||
|
}
|
||||||
|
alias, ok := resourceDNSHost(target)
|
||||||
|
if !ok || aliases[owner] != "" {
|
||||||
|
return nil, "", errDNSForwarder
|
||||||
|
}
|
||||||
|
aliases[owner] = alias
|
||||||
|
}
|
||||||
|
}
|
||||||
|
seen := map[string]bool{}
|
||||||
|
for steps := 0; steps < 16; steps++ {
|
||||||
|
if seen[host] || priorAliases[host] {
|
||||||
|
return nil, "", errDNSForwarder
|
||||||
|
}
|
||||||
|
seen[host] = true
|
||||||
|
if alias := aliases[host]; alias != "" {
|
||||||
|
if len(addresses[host]) != 0 {
|
||||||
|
return nil, "", errDNSForwarder
|
||||||
|
}
|
||||||
|
priorAliases[host] = true
|
||||||
|
if len(priorAliases) > 16 {
|
||||||
|
return nil, "", errDNSForwarder
|
||||||
|
}
|
||||||
|
host = alias
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
// Unrelated answer addresses/aliases are ambiguous, not a safe partial set.
|
||||||
|
for owner := range addresses {
|
||||||
|
if !seen[owner] {
|
||||||
|
return nil, "", errDNSForwarder
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for owner := range aliases {
|
||||||
|
if !seen[owner] {
|
||||||
|
return nil, "", errDNSForwarder
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(addresses[host]) > 0 {
|
||||||
|
return addresses[host], "", nil
|
||||||
|
}
|
||||||
|
if len(aliases) > 0 {
|
||||||
|
return nil, host, nil
|
||||||
|
}
|
||||||
|
return nil, "", nil // authenticated framing with NOERROR/NODATA
|
||||||
|
}
|
||||||
|
return nil, "", errDNSForwarder
|
||||||
|
}
|
||||||
|
|
||||||
|
func resolveResourceIPsWithDial(ctx context.Context, server, host string, dial dnsDialFunc) ([]net.IP, error) {
|
||||||
|
serverHost, port, err := net.SplitHostPort(server)
|
||||||
|
if err != nil || net.ParseIP(serverHost) == nil || (port != "53" && server != dnsListenAddress) || dial == nil {
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
}
|
||||||
|
if ip := net.ParseIP(host); ip != nil {
|
||||||
|
return []net.IP{ip}, nil
|
||||||
|
}
|
||||||
|
var ips []net.IP
|
||||||
|
unique := map[string]bool{}
|
||||||
|
for _, kind := range []uint16{1, 28} {
|
||||||
|
name := host
|
||||||
|
seen := map[string]bool{}
|
||||||
|
priorAliases := map[string]bool{}
|
||||||
|
for steps := 0; ; steps++ {
|
||||||
|
if steps >= 16 || seen[strings.ToLower(name)] || ctx.Err() != nil {
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
}
|
||||||
|
seen[strings.ToLower(name)] = true
|
||||||
|
query, err := resourceDNSQuery(name, kind)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
wire, err := resourceDNSExchange(ctx, server, query, dial)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
answers, alias, err := resourceDNSAnswers(query, wire, priorAliases)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
for _, ip := range answers {
|
||||||
|
if !unique[ip.String()] {
|
||||||
|
unique[ip.String()] = true
|
||||||
|
ips = append(ips, ip)
|
||||||
|
}
|
||||||
|
if len(ips) > 64 {
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if alias == "" {
|
||||||
|
if steps > 0 && len(answers) == 0 {
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
}
|
||||||
|
break
|
||||||
|
}
|
||||||
|
name = alias
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ips, nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,229 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"syscall"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Generic checks do not inherit PID 1's anchors. Read them through procfs,
|
||||||
|
// without changing the stricter inherited-FD readiness/cleanup protocol.
|
||||||
|
func verifyResourceDNSRuntime(cfg managedDNSConfig) error {
|
||||||
|
if os.Geteuid() != 0 {
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
return verifyResourceDNSRuntimeAt(cfg, "/proc", filepath.Dir(dnsReadyPath), "/usr/local/bin/zapret-autotune")
|
||||||
|
}
|
||||||
|
|
||||||
|
func resourcePrivateFile(dir *os.File, name string) (*os.File, error) {
|
||||||
|
fd, err := syscall.Openat(int(dir.Fd()), name, syscall.O_RDONLY|syscall.O_NOFOLLOW|syscall.O_NONBLOCK|syscall.O_CLOEXEC, 0)
|
||||||
|
if err != nil {
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
}
|
||||||
|
f := os.NewFile(uintptr(fd), name)
|
||||||
|
info, err := f.Stat()
|
||||||
|
if err != nil || !dnsOwnedAnchor(info, false) || info.Size() > 64<<10 {
|
||||||
|
f.Close()
|
||||||
|
return nil, errDNSForwarder
|
||||||
|
}
|
||||||
|
return f, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func resourceProcessOwned(proc string, pid int, exe string, args []string) bool {
|
||||||
|
base := filepath.Join(proc, strconv.Itoa(pid))
|
||||||
|
actual, err := os.Readlink(filepath.Join(base, "exe"))
|
||||||
|
if err != nil || actual != exe {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
cmd, err := os.ReadFile(filepath.Join(base, "cmdline"))
|
||||||
|
if err != nil || string(cmd) != strings.Join(args, "\x00")+"\x00" {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
status, err := os.ReadFile(filepath.Join(base, "status"))
|
||||||
|
if err != nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
uid := strconv.Itoa(os.Geteuid())
|
||||||
|
parent, owner := false, false
|
||||||
|
for _, line := range strings.Split(string(status), "\n") {
|
||||||
|
fields := strings.Fields(line)
|
||||||
|
if len(fields) == 2 && fields[0] == "PPid:" && fields[1] == "1" {
|
||||||
|
parent = true
|
||||||
|
}
|
||||||
|
if len(fields) == 5 && fields[0] == "Uid:" && fields[1] == uid && fields[2] == uid && fields[3] == uid && fields[4] == uid {
|
||||||
|
owner = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !parent || !owner {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
for _, ns := range []string{"pid", "net"} {
|
||||||
|
self, err := os.Readlink(filepath.Join(proc, "self/ns", ns))
|
||||||
|
if err != nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
other, err := os.Readlink(filepath.Join(base, "ns", ns))
|
||||||
|
if err != nil || other != self {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
// All sockets matching a binding must be held by this process; finding just
|
||||||
|
// one matching inode is insufficient when another process owns UDP/TCP.
|
||||||
|
func resourceProcessSockets(proc string, pid, port int, udp, wildcard bool) bool {
|
||||||
|
fds, err := os.ReadDir(filepath.Join(proc, strconv.Itoa(pid), "fd"))
|
||||||
|
if err != nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
owned := map[string]bool{}
|
||||||
|
for _, fd := range fds {
|
||||||
|
link, err := os.Readlink(filepath.Join(proc, strconv.Itoa(pid), "fd", fd.Name()))
|
||||||
|
if err == nil {
|
||||||
|
owned[link] = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
tables := []string{"tcp"}
|
||||||
|
if udp {
|
||||||
|
tables = append(tables, "udp")
|
||||||
|
}
|
||||||
|
for _, table := range tables {
|
||||||
|
data, err := os.ReadFile(filepath.Join(proc, "net", table))
|
||||||
|
if err != nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
found := false
|
||||||
|
for _, line := range strings.Split(string(data), "\n") {
|
||||||
|
f := strings.Fields(line)
|
||||||
|
if len(f) < 10 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
address := fmt.Sprintf("0100007F:%04X", port)
|
||||||
|
if !wildcard && f[1] == fmt.Sprintf("00000000:%04X", port) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
match := f[1] == address || wildcard && f[1] == fmt.Sprintf("00000000:%04X", port)
|
||||||
|
if !match {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if table == "tcp" && f[3] != "0A" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if table == "udp" && (f[3] != "07" || f[2] != "00000000:0000") {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
if !owned["socket:["+f[9]+"]"] {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
found = true
|
||||||
|
}
|
||||||
|
if !found {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
|
||||||
|
func verifyResourceDNSRuntimeAt(cfg managedDNSConfig, proc, dir, executable string) error {
|
||||||
|
if cfg.Mode != "doh" && (cfg.Mode != "native" || cfg.NativeServer == "") {
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
if managedDirectory(dir) != nil {
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
fd, err := syscall.Open(dir, syscall.O_RDONLY|syscall.O_DIRECTORY|syscall.O_NOFOLLOW|syscall.O_CLOEXEC, 0)
|
||||||
|
if err != nil {
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
parent := os.NewFile(uintptr(fd), "DNS private directory")
|
||||||
|
defer parent.Close()
|
||||||
|
info, err := parent.Stat()
|
||||||
|
if err != nil || !dnsOwnedAnchor(info, true) {
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
if cfg.Mode == "doh" {
|
||||||
|
ready, err := resourcePrivateFile(parent, "dns-ready")
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer ready.Close()
|
||||||
|
data, err := io.ReadAll(io.LimitReader(ready, 32))
|
||||||
|
if err != nil {
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
pid, err := strconv.Atoi(strings.TrimSuffix(string(data), "\n"))
|
||||||
|
if err != nil || pid <= 1 || string(data) != strconv.Itoa(pid)+"\n" {
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
anchorDir, err := os.Open(filepath.Join(proc, "1/fd/8"))
|
||||||
|
if err != nil {
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
defer anchorDir.Close()
|
||||||
|
anchor, err := os.Open(filepath.Join(proc, "1/fd/9"))
|
||||||
|
if err != nil {
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
defer anchor.Close()
|
||||||
|
anchorInfo, err := anchorDir.Stat()
|
||||||
|
if err != nil || !os.SameFile(info, anchorInfo) || dnsReadinessAction(anchorDir, anchor, pid, false) != nil {
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
if !resourceProcessOwned(proc, pid, executable, []string{"/usr/local/bin/zapret-autotune", "dns-forwarder", "serve"}) || !resourceProcessSockets(proc, pid, 5353, true, false) {
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
self, err := os.Stat(filepath.Join(proc, "self/exe"))
|
||||||
|
if err != nil {
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
peer, err := os.Stat(filepath.Join(proc, strconv.Itoa(pid), "exe"))
|
||||||
|
if err != nil || !os.SameFile(self, peer) {
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
}
|
||||||
|
local, err := resourcePrivateFile(parent, "local.json")
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer local.Close()
|
||||||
|
var localConfig struct {
|
||||||
|
Server string `json:"server"`
|
||||||
|
ServerPort int `json:"server_port"`
|
||||||
|
LocalPort int `json:"local_port"`
|
||||||
|
}
|
||||||
|
decoder := json.NewDecoder(io.LimitReader(local, 64<<10))
|
||||||
|
if decoder.Decode(&localConfig) != nil || localConfig.Server != "127.0.0.1" || localConfig.ServerPort != 8388 || localConfig.LocalPort != 1080 {
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
var extra interface{}
|
||||||
|
if decoder.Decode(&extra) != io.EOF {
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
processes, err := os.ReadDir(proc)
|
||||||
|
if err != nil {
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
localOwned, serverOwned := false, false
|
||||||
|
for _, entry := range processes {
|
||||||
|
pid, err := strconv.Atoi(entry.Name())
|
||||||
|
if err != nil || pid <= 1 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if resourceProcessOwned(proc, pid, "/usr/bin/ss-local", []string{"ss-local", "-c", dir + "/local.json"}) && resourceProcessSockets(proc, pid, 1080, false, true) {
|
||||||
|
localOwned = true
|
||||||
|
}
|
||||||
|
if resourceProcessOwned(proc, pid, "/usr/bin/ss-server", []string{"ss-server", "-c", dir + "/server.json", "-d", cfg.serverAddress()}) && resourceProcessSockets(proc, pid, 8388, false, true) {
|
||||||
|
serverOwned = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !localOwned || !serverOwned {
|
||||||
|
return errDNSForwarder
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,316 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/binary"
|
||||||
|
"errors"
|
||||||
|
"io"
|
||||||
|
"net"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func resourceDNSNameForTest(host string) []byte {
|
||||||
|
var name []byte
|
||||||
|
for _, s := range strings.Split(host, ".") {
|
||||||
|
name = append(name, byte(len(s)))
|
||||||
|
name = append(name, s...)
|
||||||
|
}
|
||||||
|
return append(name, 0)
|
||||||
|
}
|
||||||
|
func resourceDNSRecordForTest(owner string, kind uint16, data []byte) []byte {
|
||||||
|
r := resourceDNSNameForTest(owner)
|
||||||
|
r = append(r, byte(kind>>8), byte(kind), 0, 1, 0, 0, 0, 1, byte(len(data)>>8), byte(len(data)))
|
||||||
|
return append(r, data...)
|
||||||
|
}
|
||||||
|
func resourceDNSResponseForTest(q []byte, answers, additional [][]byte) []byte {
|
||||||
|
r := append([]byte(nil), q...)
|
||||||
|
r[2], r[3] = 0x81, 0x80
|
||||||
|
binary.BigEndian.PutUint16(r[6:8], uint16(len(answers)))
|
||||||
|
binary.BigEndian.PutUint16(r[10:12], uint16(len(additional)))
|
||||||
|
for _, rr := range answers {
|
||||||
|
r = append(r, rr...)
|
||||||
|
}
|
||||||
|
for _, rr := range additional {
|
||||||
|
r = append(r, rr...)
|
||||||
|
}
|
||||||
|
return r
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceDNSWireOnlyAndAliases(t *testing.T) {
|
||||||
|
for _, mode := range []string{"hosts", "alias", "follow", "loop", "cross-loop", "incomplete", "additional", "unrelated", "mixed", "mapped", "linklocal", "servfail", "wrong-id", "wrong-question", "bad-cname", "duplicate-cname", "alias-address", "bad-address", "truncated", "cancel"} {
|
||||||
|
t.Run(mode, func(t *testing.T) {
|
||||||
|
calls := 0
|
||||||
|
var workers sync.WaitGroup
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
|
||||||
|
defer cancel()
|
||||||
|
dial := func(ctx context.Context, network, server string) (net.Conn, error) {
|
||||||
|
calls++
|
||||||
|
if network != "tcp" || server != dnsListenAddress {
|
||||||
|
t.Fatal("wrong DNS route")
|
||||||
|
}
|
||||||
|
client, peer := net.Pipe()
|
||||||
|
workers.Add(1)
|
||||||
|
go func() {
|
||||||
|
defer workers.Done()
|
||||||
|
defer peer.Close()
|
||||||
|
peer.SetDeadline(time.Now().Add(time.Second))
|
||||||
|
var size [2]byte
|
||||||
|
if _, err := io.ReadFull(peer, size[:]); err != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
q := make([]byte, int(binary.BigEndian.Uint16(size[:])))
|
||||||
|
if _, err := io.ReadFull(peer, q); err != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
end, question, ok := dnsQuestion(q)
|
||||||
|
if !ok {
|
||||||
|
t.Error("bad query")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
host, _ := resourceDNSHost(question[:len(question)-4])
|
||||||
|
kind := binary.BigEndian.Uint16(q[end-4:])
|
||||||
|
if mode == "cancel" {
|
||||||
|
cancel()
|
||||||
|
var b [1]byte
|
||||||
|
peer.Read(b[:])
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if mode == "truncated" {
|
||||||
|
peer.Write([]byte{0, 20, 1})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
address := resourceDNSRecordForTest(host, 1, []byte{93, 184, 216, 34})
|
||||||
|
var answer, additional [][]byte
|
||||||
|
if kind == 1 {
|
||||||
|
answer = [][]byte{address}
|
||||||
|
}
|
||||||
|
switch mode {
|
||||||
|
case "alias":
|
||||||
|
if kind == 1 {
|
||||||
|
answer = [][]byte{resourceDNSRecordForTest(host, 5, resourceDNSNameForTest("cdn.example")), resourceDNSRecordForTest("cdn.example", 1, []byte{93, 184, 216, 34})}
|
||||||
|
}
|
||||||
|
case "follow", "incomplete":
|
||||||
|
if host != "cdn.example" {
|
||||||
|
answer = [][]byte{resourceDNSRecordForTest(host, 5, resourceDNSNameForTest("cdn.example"))}
|
||||||
|
} else if mode == "incomplete" {
|
||||||
|
answer = nil
|
||||||
|
} else {
|
||||||
|
answer = [][]byte{resourceDNSRecordForTest(host, kind, net.ParseIP("2606:4700::1111").To16())}
|
||||||
|
if kind == 1 {
|
||||||
|
answer = [][]byte{address}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
case "loop":
|
||||||
|
answer = [][]byte{resourceDNSRecordForTest(host, 5, resourceDNSNameForTest("cdn.example")), resourceDNSRecordForTest("cdn.example", 5, resourceDNSNameForTest(host))}
|
||||||
|
case "cross-loop":
|
||||||
|
if host == "example.com" {
|
||||||
|
answer = [][]byte{resourceDNSRecordForTest(host, 5, resourceDNSNameForTest("cdn.example"))}
|
||||||
|
} else {
|
||||||
|
answer = [][]byte{resourceDNSRecordForTest(host, 5, resourceDNSNameForTest("example.com")), resourceDNSRecordForTest("example.com", 1, []byte{93, 184, 216, 34})}
|
||||||
|
}
|
||||||
|
case "bad-cname":
|
||||||
|
answer = [][]byte{resourceDNSRecordForTest(host, 5, []byte{0xc0, 12, 0})}
|
||||||
|
case "duplicate-cname":
|
||||||
|
answer = [][]byte{resourceDNSRecordForTest(host, 5, resourceDNSNameForTest("one.example")), resourceDNSRecordForTest(host, 5, resourceDNSNameForTest("two.example"))}
|
||||||
|
case "alias-address":
|
||||||
|
answer = [][]byte{address, resourceDNSRecordForTest(host, 5, resourceDNSNameForTest("cdn.example"))}
|
||||||
|
case "bad-address":
|
||||||
|
answer = [][]byte{resourceDNSRecordForTest(host, 1, []byte{1, 2, 3})}
|
||||||
|
case "additional":
|
||||||
|
additional = [][]byte{resourceDNSRecordForTest("other.example", 1, []byte{127, 0, 0, 1})}
|
||||||
|
case "unrelated":
|
||||||
|
answer = append(answer, resourceDNSRecordForTest("other.example", 1, []byte{127, 0, 0, 1}))
|
||||||
|
case "mixed":
|
||||||
|
if kind == 1 {
|
||||||
|
answer = append(answer, resourceDNSRecordForTest(host, 1, []byte{10, 0, 0, 1}))
|
||||||
|
}
|
||||||
|
case "mapped":
|
||||||
|
if kind == 28 {
|
||||||
|
answer = [][]byte{resourceDNSRecordForTest(host, 28, net.ParseIP("::ffff:10.0.0.1").To16())}
|
||||||
|
}
|
||||||
|
case "linklocal":
|
||||||
|
if kind == 28 {
|
||||||
|
answer = [][]byte{resourceDNSRecordForTest(host, 28, net.ParseIP("fe80::1").To16())}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
r := resourceDNSResponseForTest(q, answer, additional)
|
||||||
|
if mode == "servfail" {
|
||||||
|
r[3] = 0x82
|
||||||
|
}
|
||||||
|
if mode == "wrong-id" {
|
||||||
|
r[0] ^= 1
|
||||||
|
}
|
||||||
|
if mode == "wrong-question" {
|
||||||
|
r[13] ^= 1
|
||||||
|
}
|
||||||
|
binary.BigEndian.PutUint16(size[:], uint16(len(r)))
|
||||||
|
writeDNSFrame(peer, size[:], r)
|
||||||
|
}()
|
||||||
|
return client, nil
|
||||||
|
}
|
||||||
|
host := "example.com"
|
||||||
|
if mode == "hosts" {
|
||||||
|
host = "localhost"
|
||||||
|
}
|
||||||
|
ips, err := resolveResourceIPsWithDial(ctx, dnsListenAddress, host, dial)
|
||||||
|
workers.Wait()
|
||||||
|
valid := mode == "hosts" || mode == "alias" || mode == "follow" || mode == "additional" || mode == "mixed" || mode == "mapped" || mode == "linklocal"
|
||||||
|
if (err == nil) != valid || calls == 0 {
|
||||||
|
t.Fatalf("calls=%d ips=%v err=%v", calls, ips, err)
|
||||||
|
}
|
||||||
|
if mode == "hosts" && (calls != 2 || len(ips) != 1 || ips[0].String() != "93.184.216.34") {
|
||||||
|
t.Fatal("hosts file used", ips)
|
||||||
|
}
|
||||||
|
if mode == "additional" && len(ips) != 1 {
|
||||||
|
t.Fatal("additional glue collected", ips)
|
||||||
|
}
|
||||||
|
if mode == "mixed" || mode == "mapped" || mode == "linklocal" {
|
||||||
|
denied := false
|
||||||
|
for _, ip := range ips {
|
||||||
|
if !approvedResourceIP(resourceCheckSpec{}, ip) {
|
||||||
|
denied = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !denied {
|
||||||
|
t.Fatal("unsafe answer lost", ips)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
for _, host := range []string{"localhost", "dev.home.zeusina.ru"} {
|
||||||
|
calls := 0
|
||||||
|
_, err := resolveResourceIPsWithDial(context.Background(), dnsListenAddress, host, func(context.Context, string, string) (net.Conn, error) { calls++; return nil, errors.New("offline") })
|
||||||
|
if err == nil || calls != 1 {
|
||||||
|
t.Fatal("system hosts/DNS bypass")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func resourceRuntimeFixture(t *testing.T) (managedDNSConfig, string, string, string) {
|
||||||
|
t.Helper()
|
||||||
|
root := t.TempDir()
|
||||||
|
proc, dir := filepath.Join(root, "proc"), filepath.Join(root, "private")
|
||||||
|
mkdir := func(p string) {
|
||||||
|
t.Helper()
|
||||||
|
if err := os.MkdirAll(p, 0700); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
write := func(p, s string) {
|
||||||
|
t.Helper()
|
||||||
|
if err := os.WriteFile(p, []byte(s), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
link := func(target, path string) {
|
||||||
|
t.Helper()
|
||||||
|
if err := os.Symlink(target, path); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
mkdir(dir)
|
||||||
|
mkdir(filepath.Join(proc, "net"))
|
||||||
|
mkdir(filepath.Join(proc, "1/fd"))
|
||||||
|
mkdir(filepath.Join(proc, "self/ns"))
|
||||||
|
executable := filepath.Join(root, "binary")
|
||||||
|
write(executable, "binary")
|
||||||
|
link(executable, filepath.Join(proc, "self/exe"))
|
||||||
|
for _, ns := range []string{"pid", "net"} {
|
||||||
|
link(ns+"[1]", filepath.Join(proc, "self/ns", ns))
|
||||||
|
}
|
||||||
|
write(filepath.Join(dir, "dns-ready"), "42\n")
|
||||||
|
link(dir, filepath.Join(proc, "1/fd/8"))
|
||||||
|
link(filepath.Join(dir, "dns-ready"), filepath.Join(proc, "1/fd/9"))
|
||||||
|
write(filepath.Join(dir, "local.json"), `{"server":"127.0.0.1","server_port":8388,"local_port":1080}`)
|
||||||
|
for _, p := range []struct {
|
||||||
|
pid int
|
||||||
|
exe string
|
||||||
|
args []string
|
||||||
|
sockets []string
|
||||||
|
}{
|
||||||
|
{42, executable, []string{"/usr/local/bin/zapret-autotune", "dns-forwarder", "serve"}, []string{"11", "12"}},
|
||||||
|
{43, "/usr/bin/ss-local", []string{"ss-local", "-c", dir + "/local.json"}, []string{"13"}},
|
||||||
|
{44, "/usr/bin/ss-server", []string{"ss-server", "-c", dir + "/server.json", "-d", dnsListenAddress}, []string{"14"}},
|
||||||
|
} {
|
||||||
|
base := filepath.Join(proc, strconv.Itoa(p.pid))
|
||||||
|
mkdir(filepath.Join(base, "fd"))
|
||||||
|
mkdir(filepath.Join(base, "ns"))
|
||||||
|
link(p.exe, filepath.Join(base, "exe"))
|
||||||
|
write(filepath.Join(base, "cmdline"), strings.Join(p.args, "\x00")+"\x00")
|
||||||
|
uid := strconv.Itoa(os.Geteuid())
|
||||||
|
write(filepath.Join(base, "status"), "PPid:\t1\nUid:\t"+strings.Join([]string{uid, uid, uid, uid}, "\t")+"\n")
|
||||||
|
for _, ns := range []string{"pid", "net"} {
|
||||||
|
link(ns+"[1]", filepath.Join(base, "ns", ns))
|
||||||
|
}
|
||||||
|
for i, inode := range p.sockets {
|
||||||
|
link("socket:["+inode+"]", filepath.Join(base, "fd", strconv.Itoa(i)))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
write(filepath.Join(proc, "net/tcp"), "0: 0100007F:14E9 00000000:0000 0A 0 0 0 0 0 11\n1: 00000000:0438 00000000:0000 0A 0 0 0 0 0 13\n2: 00000000:20C4 00000000:0000 0A 0 0 0 0 0 14\n")
|
||||||
|
write(filepath.Join(proc, "net/udp"), "0: 0100007F:14E9 00000000:0000 07 0 0 0 0 0 12\n")
|
||||||
|
return managedDNSConfig{Mode: "doh"}, proc, dir, executable
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceDNSOwnedRuntime(t *testing.T) {
|
||||||
|
for _, mode := range []string{"valid", "missing", "symlink", "permissions", "stale", "unanchored", "foreign-udp", "foreign-tcp", "foreign-exe", "wrong-parent", "namespace", "ss-dns", "ss-remote"} {
|
||||||
|
t.Run(mode, func(t *testing.T) {
|
||||||
|
cfg, proc, dir, exe := resourceRuntimeFixture(t)
|
||||||
|
write := func(path, data string) {
|
||||||
|
if err := os.WriteFile(path, []byte(data), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
switch mode {
|
||||||
|
case "missing":
|
||||||
|
os.Remove(filepath.Join(dir, "dns-ready"))
|
||||||
|
case "symlink":
|
||||||
|
os.Rename(filepath.Join(dir, "dns-ready"), filepath.Join(dir, "other"))
|
||||||
|
os.Symlink("other", filepath.Join(dir, "dns-ready"))
|
||||||
|
case "permissions":
|
||||||
|
os.Chmod(filepath.Join(dir, "dns-ready"), 0644)
|
||||||
|
case "stale":
|
||||||
|
write(filepath.Join(dir, "dns-ready"), "45\n")
|
||||||
|
case "unanchored":
|
||||||
|
os.Remove(filepath.Join(proc, "1/fd/9"))
|
||||||
|
case "foreign-udp":
|
||||||
|
write(filepath.Join(proc, "net/udp"), "0: 0100007F:14E9 00000000:0000 07 0 0 0 0 0 99\n")
|
||||||
|
case "foreign-tcp":
|
||||||
|
write(filepath.Join(proc, "net/tcp"), "0: 0100007F:14E9 00000000:0000 0A 0 0 0 0 0 99\n")
|
||||||
|
case "foreign-exe":
|
||||||
|
os.Remove(filepath.Join(proc, "42/exe"))
|
||||||
|
os.Symlink("/usr/bin/other", filepath.Join(proc, "42/exe"))
|
||||||
|
case "wrong-parent":
|
||||||
|
write(filepath.Join(proc, "42/status"), "PPid: 45\nUid: 0 0 0 0\n")
|
||||||
|
case "namespace":
|
||||||
|
os.Remove(filepath.Join(proc, "42/ns/net"))
|
||||||
|
os.Symlink("net[other]", filepath.Join(proc, "42/ns/net"))
|
||||||
|
case "ss-dns":
|
||||||
|
write(filepath.Join(proc, "44/cmdline"), "ss-server\x00-c\x00"+dir+"/server.json\x00-d\x008.8.8.8\x00")
|
||||||
|
case "ss-remote":
|
||||||
|
write(filepath.Join(dir, "local.json"), `{"server":"8.8.8.8","server_port":8388,"local_port":1080}`)
|
||||||
|
}
|
||||||
|
err := verifyResourceDNSRuntimeAt(cfg, proc, dir, exe)
|
||||||
|
if (err == nil) != (mode == "valid") {
|
||||||
|
t.Fatalf("err=%v", err)
|
||||||
|
}
|
||||||
|
if mode != "valid" {
|
||||||
|
d := resourceCheckDeps{lookup: resourceCheckLookup, guard: func() error { return verifyResourceDNSRuntimeAt(cfg, proc, dir, exe) }, resolve: func(context.Context, string, string) ([]net.IP, error) {
|
||||||
|
t.Fatal("DNS probe before ownership")
|
||||||
|
return nil, nil
|
||||||
|
}, runner: func(context.Context, []string, io.Reader, io.Writer, io.Writer) error {
|
||||||
|
t.Fatal("transfer before ownership")
|
||||||
|
return nil
|
||||||
|
}}
|
||||||
|
if code := resourceCheckCLI(context.Background(), resourceCheckFile(t, resourceCheckExample), io.Discard, io.Discard, d); code != 2 {
|
||||||
|
t.Fatal("ownership failure did not block", code)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,461 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"net"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"reflect"
|
||||||
|
"syscall"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// HEALTHY means only that the configured checks passed, not production READY.
|
||||||
|
type resourceMonitorSource struct {
|
||||||
|
config resourceCheckConfig
|
||||||
|
server, proxy, fingerprint string
|
||||||
|
}
|
||||||
|
|
||||||
|
func resourceStaticConfig(lookup func(string) (string, bool)) (string, string, error) {
|
||||||
|
dns, err := parseManagedDNSConfig(lookup)
|
||||||
|
raw, _ := lookup("ZAPRET_AUTOTUNE_SOCKS_PROXY")
|
||||||
|
proxy, proxyErr := managedProbeProxy(raw)
|
||||||
|
if err != nil || proxyErr != nil || dns.Mode == "native" && dns.NativeServer == "" {
|
||||||
|
return "", "", errResourceConfig
|
||||||
|
}
|
||||||
|
server := dns.serverAddress()
|
||||||
|
if dns.Mode == "native" {
|
||||||
|
server = net.JoinHostPort(server, "53")
|
||||||
|
}
|
||||||
|
return server, proxy, nil
|
||||||
|
}
|
||||||
|
func readResourceMonitorSource(path string, lookup func(string) (string, bool), key []byte) (resourceMonitorSource, error) {
|
||||||
|
// Resolve each referenced environment value once in this immutable snapshot.
|
||||||
|
values := map[string]struct {
|
||||||
|
value string
|
||||||
|
ok bool
|
||||||
|
}{}
|
||||||
|
snapshot := func(name string) (string, bool) {
|
||||||
|
if v, ok := values[name]; ok {
|
||||||
|
return v.value, v.ok
|
||||||
|
}
|
||||||
|
v, ok := lookup(name)
|
||||||
|
if len(v) > 16384 {
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
values[name] = struct {
|
||||||
|
value string
|
||||||
|
ok bool
|
||||||
|
}{v, ok}
|
||||||
|
return v, ok
|
||||||
|
}
|
||||||
|
cfg, err := readResourceChecksMode(path, snapshot, true)
|
||||||
|
server, proxy, staticErr := resourceStaticConfig(snapshot)
|
||||||
|
if err != nil || staticErr != nil {
|
||||||
|
return resourceMonitorSource{}, errResourceConfig
|
||||||
|
}
|
||||||
|
source := resourceMonitorSource{config: cfg, server: server, proxy: proxy}
|
||||||
|
b, _ := json.Marshal(cfg)
|
||||||
|
// Marshal omits the private curl headers; bind them explicitly as well.
|
||||||
|
parts := []string{string(b), server, proxy}
|
||||||
|
for _, s := range cfg.Services {
|
||||||
|
for _, r := range s.Resources {
|
||||||
|
parts = append(parts, r.privateConfig)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
b, _ = json.Marshal(parts)
|
||||||
|
source.fingerprint = resourceMAC(key, b)
|
||||||
|
return source, nil
|
||||||
|
}
|
||||||
|
func resourceRowFingerprint(key []byte, source resourceMonitorSource, r resourceCheckSpec) string {
|
||||||
|
b, _ := json.Marshal(r)
|
||||||
|
b, _ = json.Marshal([]string{string(b), r.privateConfig, source.server, source.proxy})
|
||||||
|
return resourceMAC(key, b)
|
||||||
|
}
|
||||||
|
func reconcileResourceRows(key []byte, source resourceMonitorSource, prior []resourceMonitorRow) []resourceMonitorRow {
|
||||||
|
old := map[string]resourceMonitorRow{}
|
||||||
|
for _, r := range prior {
|
||||||
|
old[r.Service+"/"+r.Resource] = r
|
||||||
|
}
|
||||||
|
rows := []resourceMonitorRow{}
|
||||||
|
for _, s := range source.config.Services {
|
||||||
|
for _, r := range s.Resources {
|
||||||
|
fp := resourceRowFingerprint(key, source, r)
|
||||||
|
row, ok := old[s.ID+"/"+r.ID]
|
||||||
|
if !ok || row.Fingerprint != fp {
|
||||||
|
row = resourceMonitorRow{Service: s.ID, Resource: r.ID, Fingerprint: fp, Health: "UNKNOWN", Outcome: "unknown"}
|
||||||
|
}
|
||||||
|
rows = append(rows, row)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return rows
|
||||||
|
}
|
||||||
|
|
||||||
|
type resourceMonitorDeps struct {
|
||||||
|
load func() (resourceMonitorState, error)
|
||||||
|
save func(resourceMonitorState) error
|
||||||
|
source func() (resourceMonitorSource, error)
|
||||||
|
guard func() error
|
||||||
|
probe func(context.Context, string, resourceCheckSpec, resourceMonitorSource, func() error) (string, error)
|
||||||
|
key []byte
|
||||||
|
now func() time.Time
|
||||||
|
wait func(context.Context, time.Duration) error
|
||||||
|
report func(bool)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Sequential round robin with independent due times; reload at least every
|
||||||
|
// second while idle/blocked. No recovery leases or strategy mutations.
|
||||||
|
func runResourceMonitor(ctx context.Context, d resourceMonitorDeps) error {
|
||||||
|
cursor := 0
|
||||||
|
persistenceBlocked, failedMAC, attemptedMAC := false, "", ""
|
||||||
|
var shutdownError error
|
||||||
|
for ctx.Err() == nil {
|
||||||
|
state, stateErr := d.load()
|
||||||
|
if persistenceBlocked && stateErr == nil && state.MAC != failedMAC && state.MAC != attemptedMAC {
|
||||||
|
persistenceBlocked = false
|
||||||
|
}
|
||||||
|
source, sourceErr := d.source()
|
||||||
|
blocked := persistenceBlocked || stateErr != nil || sourceErr != nil || d.guard() != nil
|
||||||
|
dirty := false
|
||||||
|
if !blocked {
|
||||||
|
rows := reconcileResourceRows(d.key, source, state.Rows)
|
||||||
|
dirty = !reflect.DeepEqual(rows, state.Rows)
|
||||||
|
state.Rows = rows
|
||||||
|
if len(state.Rows) > 0 {
|
||||||
|
cursor %= len(state.Rows)
|
||||||
|
for n := 0; n < len(state.Rows); n++ {
|
||||||
|
i := (cursor + n) % len(state.Rows)
|
||||||
|
row := &state.Rows[i]
|
||||||
|
if row.NextDue > d.now().UnixMilli() {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
var spec resourceCheckSpec
|
||||||
|
for _, s := range source.config.Services {
|
||||||
|
if s.ID == row.Service {
|
||||||
|
for _, r := range s.Resources {
|
||||||
|
if r.ID == row.Resource {
|
||||||
|
spec = r
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
guard := func() error {
|
||||||
|
if d.guard() != nil {
|
||||||
|
return errResourceConfig
|
||||||
|
}
|
||||||
|
fresh, err := d.source()
|
||||||
|
if err != nil || fresh.fingerprint != source.fingerprint {
|
||||||
|
return errResourceConfig
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
outcome, err := d.probe(ctx, row.Service, spec, source, guard)
|
||||||
|
cursor = (i + 1) % len(state.Rows)
|
||||||
|
if err != nil {
|
||||||
|
if ctx.Err() == nil || !managedCancellationOnly(err, ctx.Err()) {
|
||||||
|
blocked = true
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
shutdownError = err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
break
|
||||||
|
}
|
||||||
|
// Cancellation joins the probe but is not completed evidence.
|
||||||
|
// Keep the previous health, failure count and due time intact.
|
||||||
|
if ctx.Err() != nil || outcome == "cancelled" {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
dirty = true
|
||||||
|
row.Outcome, row.LastCompleted = outcome, d.now().UnixMilli()
|
||||||
|
delay := *spec.CooldownMS
|
||||||
|
if outcome == "pass" {
|
||||||
|
row.Health, row.Failures, delay = "HEALTHY", 0, *spec.IntervalMS
|
||||||
|
} else {
|
||||||
|
row.Health = "DEGRADED"
|
||||||
|
if row.Failures < ^uint64(0) {
|
||||||
|
row.Failures++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
row.NextDue = row.LastCompleted + int64(delay)
|
||||||
|
break // persist each completed (and joined) check before another probe
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
dirty = dirty || state.Blocked != blocked
|
||||||
|
state.Blocked = blocked
|
||||||
|
if dirty {
|
||||||
|
state.Updated = d.now().UnixMilli()
|
||||||
|
}
|
||||||
|
if dirty && stateErr == nil && !persistenceBlocked {
|
||||||
|
if d.save(state) != nil {
|
||||||
|
blocked, persistenceBlocked, failedMAC, attemptedMAC = true, true, state.MAC, resourceStateMAC(d.key, state)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
d.report(blocked)
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
if persistenceBlocked {
|
||||||
|
return errors.Join(shutdownError, errResourceState)
|
||||||
|
}
|
||||||
|
return shutdownError
|
||||||
|
}
|
||||||
|
// Bound reloads, but don't delay an already due sibling by a cooldown.
|
||||||
|
delay := time.Second
|
||||||
|
if !blocked {
|
||||||
|
for _, r := range state.Rows {
|
||||||
|
milliseconds := r.NextDue - d.now().UnixMilli()
|
||||||
|
if milliseconds > 1000 {
|
||||||
|
milliseconds = 1000
|
||||||
|
}
|
||||||
|
remaining := time.Duration(milliseconds) * time.Millisecond
|
||||||
|
if remaining < 0 {
|
||||||
|
remaining = 0
|
||||||
|
}
|
||||||
|
if remaining < delay {
|
||||||
|
delay = remaining
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := d.wait(ctx, delay); err != nil {
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
if persistenceBlocked {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if persistenceBlocked {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func probeResourceMonitor(ctx context.Context, service string, spec resourceCheckSpec, source resourceMonitorSource, guard func() error, check resourceCheckDeps) (string, error) {
|
||||||
|
check.guard = guard
|
||||||
|
protocols := []string{spec.TLS}
|
||||||
|
if spec.TLS == "both" {
|
||||||
|
protocols = []string{"tls12", "tls13"}
|
||||||
|
}
|
||||||
|
outcome := "pass"
|
||||||
|
for _, protocol := range protocols {
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
return "cancelled", nil
|
||||||
|
}
|
||||||
|
result, err := runResourceProtocol(ctx, service, spec, protocol, source.server, source.proxy, check)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
if result.Outcome != "pass" {
|
||||||
|
outcome = result.Outcome
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return outcome, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func resourceMonitorCLI(ctx context.Context, path string, stdout, stderr io.Writer, target managedDeps) (code int) {
|
||||||
|
return resourceMonitorCLIWithCheck(ctx, path, stdout, stderr, target, productionResourceCheckDeps())
|
||||||
|
}
|
||||||
|
|
||||||
|
func resourceMonitorCLIWithCheck(ctx context.Context, path string, stdout, stderr io.Writer, target managedDeps, check resourceCheckDeps) (code int) {
|
||||||
|
port, settingsErr := managedTargetSettings(target.lookup)
|
||||||
|
if settingsErr != nil || port != 1080 {
|
||||||
|
fmt.Fprintln(stderr, "configerror: invalid generic target settings")
|
||||||
|
return 64
|
||||||
|
}
|
||||||
|
if _, _, err := resourceStaticConfig(target.lookup); err != nil {
|
||||||
|
fmt.Fprintln(stderr, "configerror: invalid generic DNS/proxy configuration")
|
||||||
|
return 64
|
||||||
|
}
|
||||||
|
var lock *managedMonitorSingleton
|
||||||
|
var store *resourceMonitorStore
|
||||||
|
defer func() {
|
||||||
|
if store != nil && store.close() != nil {
|
||||||
|
code = 78
|
||||||
|
}
|
||||||
|
if lock != nil && lock.close() != nil {
|
||||||
|
code = 78
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
lastBlocked := false
|
||||||
|
report := func(blocked bool) {
|
||||||
|
if blocked && !lastBlocked {
|
||||||
|
fmt.Fprintln(stderr, "BLOCKED: generic resource monitor guard/state unavailable")
|
||||||
|
}
|
||||||
|
lastBlocked = blocked
|
||||||
|
}
|
||||||
|
fmt.Fprintln(stdout, "generic resource monitor: configured checks only; no recovery/apply; HEALTHY is not production READY")
|
||||||
|
// Guard failures before store/lock acquisition remain cancellable and alive.
|
||||||
|
for ctx.Err() == nil {
|
||||||
|
if lock == nil {
|
||||||
|
var err error
|
||||||
|
lock, err = acquireManagedMonitorSingleton(target)
|
||||||
|
if err != nil {
|
||||||
|
if errors.Is(err, syscall.EWOULDBLOCK) {
|
||||||
|
fmt.Fprintln(stderr, "BLOCKED: monitor singleton already held")
|
||||||
|
return 78
|
||||||
|
}
|
||||||
|
report(true)
|
||||||
|
if waitManagedMonitor(ctx, time.Second) != nil {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if store == nil {
|
||||||
|
var err error
|
||||||
|
store, err = openResourceMonitorStore(target.directory, true)
|
||||||
|
if err != nil {
|
||||||
|
report(true)
|
||||||
|
if waitManagedMonitor(ctx, time.Second) != nil {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
start, err := readMonitorProcessStarttime(os.Getpid())
|
||||||
|
state, loadErr := store.load()
|
||||||
|
if err != nil || loadErr != nil {
|
||||||
|
fmt.Fprintln(stderr, "BLOCKED: generic monitor ownership unavailable")
|
||||||
|
return 78
|
||||||
|
}
|
||||||
|
state.Controller = &resourceMonitorOwner{PID: os.Getpid(), Start: start}
|
||||||
|
if store.save(state) != nil {
|
||||||
|
fmt.Fprintln(stderr, "BLOCKED: generic monitor ownership publication failed")
|
||||||
|
return 78
|
||||||
|
}
|
||||||
|
}
|
||||||
|
err := runResourceMonitor(ctx, resourceMonitorDeps{
|
||||||
|
load: store.load, save: store.save, key: store.key,
|
||||||
|
source: func() (resourceMonitorSource, error) {
|
||||||
|
return readResourceMonitorSource(path, target.lookup, store.key)
|
||||||
|
},
|
||||||
|
guard: func() error {
|
||||||
|
if resourceSingletonGuard(lock, target.directory) != nil {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
// Probe guards validate the held snapshot, never reload/adopt a new
|
||||||
|
// authenticated operator edit while an older check is in flight.
|
||||||
|
if store.snapshotGuard() != nil {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
if resourceManagedIdentity(target.directory) != nil {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
return check.guard()
|
||||||
|
},
|
||||||
|
probe: func(ctx context.Context, service string, spec resourceCheckSpec, source resourceMonitorSource, guard func() error) (string, error) {
|
||||||
|
return probeResourceMonitor(ctx, service, spec, source, guard, check)
|
||||||
|
},
|
||||||
|
now: time.Now, wait: waitManagedMonitor, report: report,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintln(stderr, "BLOCKED: generic monitor safety/finalization failure")
|
||||||
|
return 78
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
func resourceSingletonGuard(lock *managedMonitorSingleton, dir string) error {
|
||||||
|
path := filepath.Join(dir, managedMonitorLockName)
|
||||||
|
if managedPrivateFile(path) != nil {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
current, err := os.Lstat(path)
|
||||||
|
held, heldErr := lock.file.Stat()
|
||||||
|
if err != nil || heldErr != nil || !os.SameFile(current, held) {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func resourceStatusCLI(stdout io.Writer, target managedDeps) int {
|
||||||
|
return resourceStatusCLIAt(stdout, target, "/proc")
|
||||||
|
}
|
||||||
|
|
||||||
|
func resourceStatusCLIAt(stdout io.Writer, target managedDeps, proc string) int {
|
||||||
|
blocked := func() int { fmt.Fprintln(stdout, `{"health":"BLOCKED","stale":true}`); return 1 }
|
||||||
|
store, err := openResourceMonitorStore(target.directory, false)
|
||||||
|
if err != nil {
|
||||||
|
return blocked()
|
||||||
|
}
|
||||||
|
defer store.close()
|
||||||
|
state, err := store.load()
|
||||||
|
path, _ := target.lookup("ZAPRET_AUTOTUNE_RESOURCE_CONFIG")
|
||||||
|
if err != nil || resourceMonitorLiveOwner(store, state, path, proc) != nil {
|
||||||
|
return blocked()
|
||||||
|
}
|
||||||
|
source, err := readResourceMonitorSource(path, target.lookup, store.key)
|
||||||
|
check := productionResourceCheckDeps()
|
||||||
|
if err != nil || check.guard() != nil || resourceManagedIdentity(target.directory) != nil {
|
||||||
|
return blocked()
|
||||||
|
}
|
||||||
|
rows := reconcileResourceRows(store.key, source, state.Rows)
|
||||||
|
b1, _ := json.Marshal(rows)
|
||||||
|
b2, _ := json.Marshal(state.Rows)
|
||||||
|
stale := resourceSnapshotStale(state, time.Now().UnixMilli()) || string(b1) != string(b2)
|
||||||
|
health := "CHECKS_ONLY"
|
||||||
|
if stale {
|
||||||
|
health = "STALE"
|
||||||
|
}
|
||||||
|
if state.Blocked {
|
||||||
|
health = "BLOCKED"
|
||||||
|
}
|
||||||
|
// Source/runtime validation may take time; don't emit a cache whose live
|
||||||
|
// ownership disappeared or whose inode changed during those checks.
|
||||||
|
if resourceMonitorLiveOwner(store, state, path, proc) != nil {
|
||||||
|
return blocked()
|
||||||
|
}
|
||||||
|
if json.NewEncoder(stdout).Encode(struct {
|
||||||
|
Health string `json:"health"`
|
||||||
|
Stale bool `json:"stale"`
|
||||||
|
State resourceMonitorState `json:"state"`
|
||||||
|
}{health, stale, state}) != nil {
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
if stale || state.Blocked {
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
// Updated is a semantic-change timestamp, not a disk heartbeat. Evidence is
|
||||||
|
// stale only once its next scheduled check plus the maximum run budget is past.
|
||||||
|
func resourceSnapshotStale(state resourceMonitorState, now int64) bool {
|
||||||
|
if state.Updated > now {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
if len(state.Rows) == 0 {
|
||||||
|
return now-state.Updated > 65000
|
||||||
|
}
|
||||||
|
for _, row := range state.Rows {
|
||||||
|
if row.NextDue == 0 {
|
||||||
|
if now-state.Updated > 65000 {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
} else if now > row.NextDue && now-row.NextDue > 65000 {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
func resourceManagedIdentity(dir string) error {
|
||||||
|
m := &managedTransaction{Dir: dir, Name: "config"}
|
||||||
|
if managedCheckIdentity(m) != nil {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
j, err := m.load()
|
||||||
|
if err != nil || j.State != "committed" {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
b, err := m.read("config")
|
||||||
|
if err != nil || managedHash(b) != j.Hash {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,326 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"io"
|
||||||
|
"net"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"reflect"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func resourceCompletedFixture(t *testing.T) (resourceMonitorSource, []byte, resourceMonitorState) {
|
||||||
|
t.Helper()
|
||||||
|
source, key := resourceMonitorFixture(t)
|
||||||
|
state := resourceMonitorState{Version: 1, Updated: 5000, Rows: reconcileResourceRows(key, source, nil)}
|
||||||
|
for i := range state.Rows {
|
||||||
|
state.Rows[i].Health, state.Rows[i].Outcome, state.Rows[i].LastCompleted = "HEALTHY", "pass", 5000
|
||||||
|
state.Rows[i].NextDue = 9000
|
||||||
|
}
|
||||||
|
return source, key, state
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceMonitorCancelledRowsUnchangedAndJoined(t *testing.T) {
|
||||||
|
for _, mixed := range []bool{false, true} {
|
||||||
|
t.Run(map[bool]string{false: "curl-cancel", true: "cancel-and-guard"}[mixed], func(t *testing.T) {
|
||||||
|
source, key, state := resourceCompletedFixture(t)
|
||||||
|
prior := append([]resourceMonitorRow(nil), state.Rows...)
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
joined, writes := false, 0
|
||||||
|
check := resourceCheckDeps{resolve: func(context.Context, string, string) ([]net.IP, error) {
|
||||||
|
return []net.IP{net.ParseIP("93.184.216.34")}, nil
|
||||||
|
}, runner: func(ctx context.Context, _ []string, _ io.Reader, _ io.Writer, _ io.Writer) error {
|
||||||
|
cancel()
|
||||||
|
<-ctx.Done()
|
||||||
|
joined = true
|
||||||
|
return ctx.Err()
|
||||||
|
}}
|
||||||
|
d := resourceMonitorDeps{key: key, now: func() time.Time { return time.UnixMilli(10000) }, load: func() (resourceMonitorState, error) { return state, nil }, save: func(s resourceMonitorState) error {
|
||||||
|
if !joined {
|
||||||
|
t.Error("persisted before joining curl")
|
||||||
|
}
|
||||||
|
writes++
|
||||||
|
state = s
|
||||||
|
return nil
|
||||||
|
}, source: func() (resourceMonitorSource, error) { return source, nil }, guard: func() error { return nil }, report: func(bool) {}, wait: func(context.Context, time.Duration) error { t.Error("wait after cancellation"); return nil }}
|
||||||
|
d.probe = func(ctx context.Context, service string, spec resourceCheckSpec, source resourceMonitorSource, guard func() error) (string, error) {
|
||||||
|
outcome, err := probeResourceMonitor(ctx, service, spec, source, guard, check)
|
||||||
|
if mixed {
|
||||||
|
return outcome, errors.Join(ctx.Err(), errResourceConfig)
|
||||||
|
}
|
||||||
|
return outcome, err
|
||||||
|
}
|
||||||
|
err := runResourceMonitor(ctx, d)
|
||||||
|
if mixed {
|
||||||
|
if !errors.Is(err, errResourceConfig) || !errors.Is(err, context.Canceled) || !state.Blocked || writes != 1 {
|
||||||
|
t.Fatal("lost mixed shutdown guard error", state, writes, err)
|
||||||
|
}
|
||||||
|
} else if err != nil || writes != 0 {
|
||||||
|
t.Fatal("cancel published evidence", writes, err)
|
||||||
|
}
|
||||||
|
if !joined || !reflect.DeepEqual(prior, state.Rows) {
|
||||||
|
t.Fatal("cancel changed completed rows or failed to join", state.Rows)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceMonitorIdleNoPersistenceHeartbeat(t *testing.T) {
|
||||||
|
source, key, state := resourceCompletedFixture(t)
|
||||||
|
for i := range state.Rows {
|
||||||
|
state.Rows[i].NextDue = 305000
|
||||||
|
}
|
||||||
|
now := time.UnixMilli(10000)
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
writes, waits := 0, 0
|
||||||
|
d := resourceMonitorDeps{key: key, now: func() time.Time { return now }, load: func() (resourceMonitorState, error) { return state, nil }, save: func(s resourceMonitorState) error { writes++; state = s; return nil }, source: func() (resourceMonitorSource, error) { return source, nil }, guard: func() error { return nil }, report: func(bool) {}, probe: func(context.Context, string, resourceCheckSpec, resourceMonitorSource, func() error) (string, error) {
|
||||||
|
t.Error("probed resource before due")
|
||||||
|
return "pass", nil
|
||||||
|
}, wait: func(_ context.Context, d time.Duration) error {
|
||||||
|
now = now.Add(d)
|
||||||
|
waits++
|
||||||
|
if waits == 5 {
|
||||||
|
cancel()
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}}
|
||||||
|
if err := runResourceMonitor(ctx, d); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if writes != 0 || state.Updated != 5000 {
|
||||||
|
t.Fatal("idle disk heartbeat", writes, state.Updated)
|
||||||
|
}
|
||||||
|
if resourceSnapshotStale(state, 100000) || !resourceSnapshotStale(state, 370001) {
|
||||||
|
t.Fatal("staleness still depends on idle heartbeat")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceMonitorBlockedTransitionsOnly(t *testing.T) {
|
||||||
|
source, key, state := resourceCompletedFixture(t)
|
||||||
|
for i := range state.Rows {
|
||||||
|
state.Rows[i].NextDue = 305000
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
tick, writes := 0, 0
|
||||||
|
d := resourceMonitorDeps{key: key, now: func() time.Time { return time.UnixMilli(10000 + int64(tick)*1000) }, load: func() (resourceMonitorState, error) { return state, nil }, save: func(s resourceMonitorState) error { writes++; state = s; return nil }, source: func() (resourceMonitorSource, error) {
|
||||||
|
if tick < 3 {
|
||||||
|
return resourceMonitorSource{}, errResourceConfig
|
||||||
|
}
|
||||||
|
return source, nil
|
||||||
|
}, guard: func() error { return nil }, report: func(bool) {}, probe: func(context.Context, string, resourceCheckSpec, resourceMonitorSource, func() error) (string, error) {
|
||||||
|
t.Error("unexpected probe")
|
||||||
|
return "pass", nil
|
||||||
|
}, wait: func(context.Context, time.Duration) error {
|
||||||
|
tick++
|
||||||
|
if tick == 5 {
|
||||||
|
cancel()
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}}
|
||||||
|
if err := runResourceMonitor(ctx, d); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if writes != 2 || state.Blocked {
|
||||||
|
t.Fatal("blocked transition heartbeat or missing resume", writes, state)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func resourceBoundaryStore(t *testing.T, createState bool) (*resourceMonitorStore, resourceMonitorState) {
|
||||||
|
t.Helper()
|
||||||
|
dir := t.TempDir()
|
||||||
|
if err := os.Chmod(dir, 0700); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
s, err := openResourceMonitorStore(dir, true)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
t.Cleanup(func() { s.close() })
|
||||||
|
v, err := s.load()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if createState {
|
||||||
|
if err := s.save(v); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return s, v
|
||||||
|
}
|
||||||
|
|
||||||
|
func replaceResourceEntry(t *testing.T, path string, data []byte) {
|
||||||
|
t.Helper()
|
||||||
|
if err := os.WriteFile(path+"-replacement", data, 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.Rename(path+"-replacement", path); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceMonitorHeldEntriesRejectReplacement(t *testing.T) {
|
||||||
|
for _, name := range []string{resourceStateName, resourceKeyName} {
|
||||||
|
for _, point := range []string{"before-load", "loaded", "staged", "publish"} {
|
||||||
|
t.Run(name+"/"+point, func(t *testing.T) {
|
||||||
|
s, v := resourceBoundaryStore(t, true)
|
||||||
|
path := filepath.Join(s.path, name)
|
||||||
|
data, err := os.ReadFile(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
var replacement os.FileInfo
|
||||||
|
mutate := func() {
|
||||||
|
replaceResourceEntry(t, path, data)
|
||||||
|
replacement, err = os.Stat(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if point == "before-load" {
|
||||||
|
mutate()
|
||||||
|
} else {
|
||||||
|
s.fault = func(p, _ string) {
|
||||||
|
if p == point {
|
||||||
|
mutate()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if point == "before-load" || point == "loaded" {
|
||||||
|
if _, err := s.load(); err == nil {
|
||||||
|
t.Fatal("load adopted identical replaced inode")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := s.save(v); err == nil {
|
||||||
|
t.Fatal("save clobbered identical replacement")
|
||||||
|
}
|
||||||
|
current, err := os.Stat(path)
|
||||||
|
if err != nil || replacement == nil || !os.SameFile(current, replacement) {
|
||||||
|
t.Fatal("foreign inode changed", err)
|
||||||
|
}
|
||||||
|
got, err := os.ReadFile(path)
|
||||||
|
if err != nil || !bytes.Equal(got, data) {
|
||||||
|
t.Fatal("foreign bytes changed", err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceMonitorMidStageCorruptionPreserved(t *testing.T) {
|
||||||
|
for _, name := range []string{resourceStateName, resourceKeyName, "stage", "stage-in-place"} {
|
||||||
|
t.Run(name, func(t *testing.T) {
|
||||||
|
s, v := resourceBoundaryStore(t, true)
|
||||||
|
bad := []byte("untrusted corrupt operator bytes")
|
||||||
|
path := ""
|
||||||
|
s.fault = func(point, stage string) {
|
||||||
|
if point != "staged" {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
path = filepath.Join(s.path, name)
|
||||||
|
if name == "stage" || name == "stage-in-place" {
|
||||||
|
path = stage
|
||||||
|
}
|
||||||
|
if name == "stage-in-place" {
|
||||||
|
if err := os.WriteFile(path, bad, 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
replaceResourceEntry(t, path, bad)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := s.save(v); err == nil {
|
||||||
|
t.Fatal("corruption overwritten")
|
||||||
|
}
|
||||||
|
got, err := os.ReadFile(path)
|
||||||
|
if err != nil || !bytes.Equal(got, bad) {
|
||||||
|
t.Fatal("unknown stage/file removed or clobbered", err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceMonitorInitialPublicationExclusive(t *testing.T) {
|
||||||
|
s, v := resourceBoundaryStore(t, false)
|
||||||
|
path := filepath.Join(s.path, resourceStateName)
|
||||||
|
bad := []byte("new foreign entry created just before initial publication")
|
||||||
|
s.fault = func(point, _ string) {
|
||||||
|
if point == "publish" {
|
||||||
|
if err := os.WriteFile(path, bad, 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := s.save(v); err == nil {
|
||||||
|
t.Fatal("exclusive initial publish overwrote raced entry")
|
||||||
|
}
|
||||||
|
got, err := os.ReadFile(path)
|
||||||
|
if err != nil || !bytes.Equal(got, bad) {
|
||||||
|
t.Fatal("new entry clobbered", err)
|
||||||
|
}
|
||||||
|
if _, err := s.load(); err == nil {
|
||||||
|
t.Fatal("new entry silently adopted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceMonitorPublicationRetainsNewAnchor(t *testing.T) {
|
||||||
|
s, v := resourceBoundaryStore(t, true)
|
||||||
|
old := s.stateFile
|
||||||
|
v.Updated = 10000
|
||||||
|
if err := s.save(v); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if old == s.stateFile {
|
||||||
|
t.Fatal("old inode retained instead of published inode")
|
||||||
|
}
|
||||||
|
if _, err := old.Stat(); err == nil {
|
||||||
|
t.Fatal("old anchor not closed after verified publication")
|
||||||
|
}
|
||||||
|
if s.entryMatches(resourceStateName, s.stateFile) != nil {
|
||||||
|
t.Fatal("published state not anchored")
|
||||||
|
}
|
||||||
|
if loaded, err := s.load(); err != nil || loaded.Updated != 10000 {
|
||||||
|
t.Fatal(loaded, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceMonitorOriginalIdentityRepair(t *testing.T) {
|
||||||
|
s, v := resourceBoundaryStore(t, true)
|
||||||
|
path := filepath.Join(s.path, resourceStateName)
|
||||||
|
valid, err := os.ReadFile(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
original, err := os.Stat(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(path, []byte(`{"version":99}`), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := s.load(); err == nil {
|
||||||
|
t.Fatal("corrupt original trusted")
|
||||||
|
}
|
||||||
|
if err := s.save(v); err == nil {
|
||||||
|
t.Fatal("corrupt original overwritten")
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(path, valid, 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := s.load(); err != nil {
|
||||||
|
t.Fatal("valid original-identity repair blocked", err)
|
||||||
|
}
|
||||||
|
current, err := os.Stat(path)
|
||||||
|
if err != nil || !os.SameFile(current, original) {
|
||||||
|
t.Fatal("repair changed inode", err)
|
||||||
|
}
|
||||||
|
if err := s.save(v); err != nil {
|
||||||
|
t.Fatal("repaired original failed publish", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,159 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestResourceMonitorFinalPublicationStageGuard(t *testing.T) {
|
||||||
|
for _, existing := range []bool{false, true} {
|
||||||
|
for _, mutation := range []string{"byte", "truncate", "append", "growth", "path-replace", "key-replace", "key-in-place", "state-replace"} {
|
||||||
|
name := "initial/"
|
||||||
|
if existing {
|
||||||
|
name = "existing/"
|
||||||
|
}
|
||||||
|
t.Run(name+mutation, func(t *testing.T) {
|
||||||
|
s, v := resourceBoundaryStore(t, existing)
|
||||||
|
path := filepath.Join(s.path, resourceStateName)
|
||||||
|
oldBytes, oldErr := os.ReadFile(path)
|
||||||
|
oldInfo, _ := os.Stat(path)
|
||||||
|
if existing && oldErr != nil {
|
||||||
|
t.Fatal(oldErr)
|
||||||
|
}
|
||||||
|
v.Updated = 10000
|
||||||
|
var changedPath string
|
||||||
|
var changedBytes []byte
|
||||||
|
var changedInfo os.FileInfo
|
||||||
|
s.fault = func(point, stage string) {
|
||||||
|
if point != "publish" {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
data, err := os.ReadFile(stage)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
changedPath = stage
|
||||||
|
switch mutation {
|
||||||
|
case "byte":
|
||||||
|
data[0] = '['
|
||||||
|
case "truncate":
|
||||||
|
data = data[:len(data)-1]
|
||||||
|
case "append":
|
||||||
|
data = append(data, ' ')
|
||||||
|
case "growth":
|
||||||
|
data = append(data, bytes.Repeat([]byte{'x'}, 65537)...)
|
||||||
|
case "key-replace", "key-in-place":
|
||||||
|
changedPath = filepath.Join(s.path, resourceKeyName)
|
||||||
|
data, err = os.ReadFile(changedPath)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if mutation == "key-in-place" {
|
||||||
|
data[0] ^= 1
|
||||||
|
}
|
||||||
|
case "state-replace":
|
||||||
|
changedPath = path
|
||||||
|
if existing {
|
||||||
|
data = append([]byte(nil), oldBytes...)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
changedBytes = append([]byte(nil), data...)
|
||||||
|
if mutation == "path-replace" || mutation == "key-replace" || mutation == "state-replace" {
|
||||||
|
replaceResourceEntry(t, changedPath, data)
|
||||||
|
} else if err := os.WriteFile(changedPath, data, 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
changedInfo, err = os.Stat(changedPath)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err := s.save(v); !errors.Is(err, errResourceState) {
|
||||||
|
t.Fatal("unsafe stage published", err)
|
||||||
|
}
|
||||||
|
current, err := os.ReadFile(changedPath)
|
||||||
|
info, infoErr := os.Stat(changedPath)
|
||||||
|
if err != nil || infoErr != nil || !bytes.Equal(current, changedBytes) || !os.SameFile(info, changedInfo) {
|
||||||
|
t.Fatal("unknown artifact changed/deleted", err, infoErr)
|
||||||
|
}
|
||||||
|
if mutation != "state-replace" {
|
||||||
|
if existing {
|
||||||
|
current, err := os.ReadFile(path)
|
||||||
|
info, infoErr := os.Stat(path)
|
||||||
|
if err != nil || infoErr != nil || !bytes.Equal(current, oldBytes) || !os.SameFile(info, oldInfo) {
|
||||||
|
t.Fatal("trusted old state lost before final guard", err, infoErr)
|
||||||
|
}
|
||||||
|
} else if _, err := os.Lstat(path); !errors.Is(err, os.ErrNotExist) {
|
||||||
|
t.Fatal("initial corrupt state published", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if existing && s.entryMatches(resourceStateName, s.stateFile) != nil && mutation != "state-replace" {
|
||||||
|
t.Fatal("old trusted anchor lost")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceMonitorPublicationFailureBlocksFurtherProbes(t *testing.T) {
|
||||||
|
source, _, _ := resourceCompletedFixture(t)
|
||||||
|
s, _ := resourceBoundaryStore(t, true)
|
||||||
|
state, err := s.load()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
state.Updated = 5000
|
||||||
|
state.Rows = reconcileResourceRows(s.key, source, nil)
|
||||||
|
for i := range state.Rows {
|
||||||
|
state.Rows[i].Health, state.Rows[i].Outcome = "HEALTHY", "pass"
|
||||||
|
state.Rows[i].LastCompleted, state.Rows[i].NextDue = 5000, 9000
|
||||||
|
}
|
||||||
|
if err := s.save(state); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
path := filepath.Join(s.path, resourceStateName)
|
||||||
|
old, err := os.ReadFile(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
oldInfo, err := os.Stat(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
s.fault = func(point, stage string) {
|
||||||
|
if point == "publish" {
|
||||||
|
if err := os.WriteFile(stage, []byte("unknown stage corruption"), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
probes, waits, blocked := 0, 0, false
|
||||||
|
d := resourceMonitorDeps{key: s.key, now: func() time.Time { return time.UnixMilli(10000) }, load: s.load, save: s.save, source: func() (resourceMonitorSource, error) { return source, nil }, guard: s.snapshotGuard, report: func(v bool) { blocked = v }, probe: func(context.Context, string, resourceCheckSpec, resourceMonitorSource, func() error) (string, error) {
|
||||||
|
probes++
|
||||||
|
return "pass", nil
|
||||||
|
}, wait: func(context.Context, time.Duration) error {
|
||||||
|
waits++
|
||||||
|
if waits == 3 {
|
||||||
|
cancel()
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}}
|
||||||
|
if err := runResourceMonitor(ctx, d); !errors.Is(err, errResourceState) {
|
||||||
|
t.Fatal("lost unsafe publication failure", err)
|
||||||
|
}
|
||||||
|
if probes != 1 || !blocked {
|
||||||
|
t.Fatal("continued probing after failed publication", probes, blocked)
|
||||||
|
}
|
||||||
|
current, err := os.ReadFile(path)
|
||||||
|
currentInfo, infoErr := os.Stat(path)
|
||||||
|
if err != nil || infoErr != nil || !bytes.Equal(current, old) || !os.SameFile(currentInfo, oldInfo) {
|
||||||
|
t.Fatal("old completed snapshot lost", err, infoErr)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,63 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strconv"
|
||||||
|
)
|
||||||
|
|
||||||
|
// A signed cache is not a live ownership claim. Inspect only procfs metadata,
|
||||||
|
// never the controller's private FD contents, and fail closed on uncertainty.
|
||||||
|
func resourceMonitorLiveOwner(store *resourceMonitorStore, state resourceMonitorState, config, proc string) error {
|
||||||
|
owner := state.Controller
|
||||||
|
if owner == nil || owner.PID <= 1 || owner.Start <= 0 || config == "" {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
base := filepath.Join(proc, strconv.Itoa(owner.PID))
|
||||||
|
identity := func() bool {
|
||||||
|
stat, err := os.ReadFile(filepath.Join(base, "stat"))
|
||||||
|
start, parseErr := parseMonitorProcessStat(owner.PID, string(stat))
|
||||||
|
if err != nil || parseErr != nil || start != owner.Start {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
exe, err := os.Readlink(filepath.Join(proc, "self/exe"))
|
||||||
|
self, selfErr := os.Stat(filepath.Join(proc, "self/exe"))
|
||||||
|
other, otherErr := os.Stat(filepath.Join(base, "exe"))
|
||||||
|
if err != nil || selfErr != nil || otherErr != nil || !os.SameFile(self, other) {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
return resourceProcessOwned(proc, owner.PID, exe, []string{exe, "managed", "monitor", "--resource-config", config}) ||
|
||||||
|
resourceProcessOwned(proc, owner.PID, exe, []string{exe, "managed", "monitor"})
|
||||||
|
}
|
||||||
|
if !identity() || store.snapshotGuard() != nil {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
// Include the singleton and directory so another target's controller cannot
|
||||||
|
// lend apparent liveness to a copied authenticated snapshot.
|
||||||
|
anchors := []string{store.path, store.entryPath(resourceKeyName), store.entryPath(resourceStateName), store.entryPath(managedMonitorLockName)}
|
||||||
|
fds, err := os.ReadDir(filepath.Join(base, "fd"))
|
||||||
|
if err != nil {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
for _, anchor := range anchors {
|
||||||
|
current, err := os.Lstat(anchor)
|
||||||
|
if err != nil {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
matched := false
|
||||||
|
for _, fd := range fds {
|
||||||
|
held, err := os.Stat(filepath.Join(base, "fd", fd.Name()))
|
||||||
|
if err == nil && os.SameFile(current, held) {
|
||||||
|
matched = true
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !matched {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !identity() || store.snapshotGuard() != nil || managedPrivateFile(store.entryPath(managedMonitorLockName)) != nil {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,149 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestResourceStatusLiveOwnership(t *testing.T) {
|
||||||
|
for _, change := range []string{"state", "key", "directory", "lock", "dead", "pid-reuse", "exe", "argv", "uid", "parent", "namespace", "missing-owner"} {
|
||||||
|
t.Run(change, func(t *testing.T) {
|
||||||
|
dir := filepath.Join(t.TempDir(), "state")
|
||||||
|
if err := os.Mkdir(dir, 0700); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
store, err := openResourceMonitorStore(dir, true)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer store.close()
|
||||||
|
write := func(path, contents string) {
|
||||||
|
t.Helper()
|
||||||
|
if err := os.WriteFile(path, []byte(contents), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
link := func(path, target string) {
|
||||||
|
t.Helper()
|
||||||
|
if err := os.Symlink(target, path); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
lockPath := filepath.Join(dir, managedMonitorLockName)
|
||||||
|
write(lockPath, "")
|
||||||
|
lock, err := os.Open(lockPath)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer lock.Close()
|
||||||
|
source, _ := resourceMonitorFixture(t)
|
||||||
|
state := resourceMonitorState{Version: 1, Updated: time.Now().UnixMilli(), Controller: &resourceMonitorOwner{PID: 42, Start: 123}, Rows: reconcileResourceRows(store.key, source, nil)}
|
||||||
|
for i := range state.Rows {
|
||||||
|
state.Rows[i].Health, state.Rows[i].Outcome = "HEALTHY", "pass"
|
||||||
|
state.Rows[i].LastCompleted, state.Rows[i].NextDue = state.Updated, state.Updated+60000
|
||||||
|
}
|
||||||
|
if err := store.save(state); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
state, err = store.load()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
proc := t.TempDir()
|
||||||
|
for _, sub := range []string{"self/ns", "42/ns", "42/fd"} {
|
||||||
|
if err := os.MkdirAll(filepath.Join(proc, sub), 0700); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
exe, err := os.Executable()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
link(filepath.Join(proc, "self/exe"), exe)
|
||||||
|
link(filepath.Join(proc, "42/exe"), exe)
|
||||||
|
for _, ns := range []string{"pid", "net"} {
|
||||||
|
link(filepath.Join(proc, "self/ns", ns), ns+"[1]")
|
||||||
|
link(filepath.Join(proc, "42/ns", ns), ns+"[1]")
|
||||||
|
}
|
||||||
|
fields := append([]string{"S"}, strings.Fields(strings.Repeat("0 ", 18))...)
|
||||||
|
fields = append(fields, "123")
|
||||||
|
stat := "42 (monitor) " + strings.Join(fields, " ")
|
||||||
|
write(filepath.Join(proc, "42/stat"), stat)
|
||||||
|
uid := strconv.Itoa(os.Geteuid())
|
||||||
|
write(filepath.Join(proc, "42/status"), "PPid:\t1\nUid:\t"+strings.Join([]string{uid, uid, uid, uid}, "\t")+"\n")
|
||||||
|
config := "/private/checks.json"
|
||||||
|
write(filepath.Join(proc, "42/cmdline"), strings.Join([]string{exe, "managed", "monitor", "--resource-config", config}, "\x00")+"\x00")
|
||||||
|
for i, f := range []*os.File{store.dir, store.keyFile, store.stateFile, lock} {
|
||||||
|
link(filepath.Join(proc, "42/fd", strconv.Itoa(i)), "/proc/self/fd/"+strconv.FormatUint(uint64(f.Fd()), 10))
|
||||||
|
}
|
||||||
|
if err := resourceMonitorLiveOwner(store, state, config, proc); err != nil {
|
||||||
|
t.Fatal("valid live owner refused", err)
|
||||||
|
}
|
||||||
|
if resourceSnapshotStale(state, time.Now().UnixMilli()) {
|
||||||
|
t.Fatal("fixture already stale")
|
||||||
|
}
|
||||||
|
var preservedPath string
|
||||||
|
var preserved []byte
|
||||||
|
switch change {
|
||||||
|
case "state", "key", "lock":
|
||||||
|
name := map[string]string{"state": resourceStateName, "key": resourceKeyName, "lock": managedMonitorLockName}[change]
|
||||||
|
preservedPath = filepath.Join(dir, name)
|
||||||
|
preserved, err = os.ReadFile(preservedPath)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
write(preservedPath+".replacement", string(preserved))
|
||||||
|
if err := os.Rename(preservedPath+".replacement", preservedPath); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
case "directory":
|
||||||
|
if err := os.Rename(dir, dir+".old"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.Mkdir(dir, 0700); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
case "dead":
|
||||||
|
os.Remove(filepath.Join(proc, "42/stat"))
|
||||||
|
case "pid-reuse":
|
||||||
|
write(filepath.Join(proc, "42/stat"), strings.TrimSuffix(stat, "123")+"124")
|
||||||
|
case "exe":
|
||||||
|
os.Remove(filepath.Join(proc, "42/exe"))
|
||||||
|
link(filepath.Join(proc, "42/exe"), "/bin/sh")
|
||||||
|
case "argv":
|
||||||
|
write(filepath.Join(proc, "42/cmdline"), exe+"\x00managed\x00resource-status\x00")
|
||||||
|
case "uid":
|
||||||
|
write(filepath.Join(proc, "42/status"), "PPid:\t1\nUid:\t9999\t9999\t9999\t9999\n")
|
||||||
|
case "parent":
|
||||||
|
write(filepath.Join(proc, "42/status"), "PPid:\t2\nUid:\t"+strings.Join([]string{uid, uid, uid, uid}, "\t")+"\n")
|
||||||
|
case "namespace":
|
||||||
|
os.Remove(filepath.Join(proc, "42/ns/net"))
|
||||||
|
link(filepath.Join(proc, "42/ns/net"), "net[2]")
|
||||||
|
case "missing-owner":
|
||||||
|
state.Controller = nil
|
||||||
|
if err := store.save(state); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if resourceMonitorLiveOwner(store, state, config, proc) == nil {
|
||||||
|
t.Fatal("invalid live ownership accepted")
|
||||||
|
}
|
||||||
|
target := managedDeps{directory: dir, lookup: func(name string) (string, bool) { return config, name == "ZAPRET_AUTOTUNE_RESOURCE_CONFIG" }}
|
||||||
|
var out bytes.Buffer
|
||||||
|
if resourceStatusCLIAt(&out, target, proc) != 1 || out.String() != "{\"health\":\"BLOCKED\",\"stale\":true}\n" {
|
||||||
|
t.Fatal("cached status hid live failure", out.String())
|
||||||
|
}
|
||||||
|
if preservedPath != "" {
|
||||||
|
got, err := os.ReadFile(preservedPath)
|
||||||
|
if err != nil || !bytes.Equal(got, preserved) {
|
||||||
|
t.Fatal("foreign entry modified", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,408 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"crypto/hmac"
|
||||||
|
"crypto/rand"
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"io"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strconv"
|
||||||
|
"syscall"
|
||||||
|
)
|
||||||
|
|
||||||
|
const resourceStateName = ".resource-monitor-state.json"
|
||||||
|
const resourceKeyName = ".resource-monitor-key"
|
||||||
|
|
||||||
|
var errResourceState = errors.New("resource state blocked")
|
||||||
|
|
||||||
|
type resourceMonitorRow struct {
|
||||||
|
Service string `json:"service"`
|
||||||
|
Resource string `json:"resource"`
|
||||||
|
Fingerprint string `json:"fingerprint"`
|
||||||
|
Health string `json:"health"`
|
||||||
|
Outcome string `json:"outcome"`
|
||||||
|
Failures uint64 `json:"failures"`
|
||||||
|
LastCompleted int64 `json:"last_completed_ms"`
|
||||||
|
NextDue int64 `json:"next_due_ms"`
|
||||||
|
}
|
||||||
|
type resourceMonitorState struct {
|
||||||
|
Version int `json:"version"`
|
||||||
|
Updated int64 `json:"updated_ms"`
|
||||||
|
Blocked bool `json:"blocked"`
|
||||||
|
Rows []resourceMonitorRow `json:"rows"`
|
||||||
|
Controller *resourceMonitorOwner `json:"controller,omitempty"`
|
||||||
|
MAC string `json:"mac"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type resourceMonitorOwner struct {
|
||||||
|
PID int `json:"pid"`
|
||||||
|
Start int64 `json:"starttime"`
|
||||||
|
}
|
||||||
|
type resourceMonitorStore struct {
|
||||||
|
dir *os.File
|
||||||
|
path string
|
||||||
|
key []byte
|
||||||
|
previous []byte
|
||||||
|
existed bool
|
||||||
|
keyFile, stateFile *os.File
|
||||||
|
// Deterministic filesystem interleavings in tests; nil in production.
|
||||||
|
fault func(point, stage string)
|
||||||
|
}
|
||||||
|
|
||||||
|
func resourceMAC(key, data []byte) string {
|
||||||
|
m := hmac.New(sha256.New, key)
|
||||||
|
m.Write(data)
|
||||||
|
return hex.EncodeToString(m.Sum(nil))
|
||||||
|
}
|
||||||
|
func resourceStateMAC(key []byte, s resourceMonitorState) string {
|
||||||
|
s.MAC = ""
|
||||||
|
b, _ := json.Marshal(s)
|
||||||
|
return resourceMAC(key, b)
|
||||||
|
}
|
||||||
|
func (s *resourceMonitorStore) guard() error {
|
||||||
|
if managedDirectory(s.path) != nil {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
current, err := os.Stat(s.path)
|
||||||
|
held, heldErr := s.dir.Stat()
|
||||||
|
if err != nil || heldErr != nil || !os.SameFile(current, held) {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
func (s *resourceMonitorStore) entryPath(name string) string {
|
||||||
|
return filepath.Join("/proc/self/fd", strconv.FormatUint(uint64(s.dir.Fd()), 10), name)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *resourceMonitorStore) close() error {
|
||||||
|
var errs []error
|
||||||
|
for _, f := range []*os.File{s.stateFile, s.keyFile, s.dir} {
|
||||||
|
if f != nil {
|
||||||
|
errs = append(errs, f.Close())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
s.stateFile, s.keyFile, s.dir = nil, nil, nil
|
||||||
|
return errors.Join(errs...)
|
||||||
|
}
|
||||||
|
|
||||||
|
func resourcePrivateEntry(f *os.File) error {
|
||||||
|
info, err := f.Stat()
|
||||||
|
if err != nil {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
st, ok := info.Sys().(*syscall.Stat_t)
|
||||||
|
if !ok || !info.Mode().IsRegular() || info.Mode().Perm() != 0600 || st.Uid != uint32(os.Geteuid()) || st.Nlink != 1 || info.Size() > 65536 {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *resourceMonitorStore) entryMatches(name string, held *os.File) error {
|
||||||
|
current, err := os.Lstat(s.entryPath(name))
|
||||||
|
anchored, heldErr := held.Stat()
|
||||||
|
if err != nil || heldErr != nil || !os.SameFile(current, anchored) || resourcePrivateEntry(held) != nil {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *resourceMonitorStore) openEntry(name string) (*os.File, []byte, error) {
|
||||||
|
if s.guard() != nil {
|
||||||
|
return nil, nil, errResourceState
|
||||||
|
}
|
||||||
|
fd, err := syscall.Openat(int(s.dir.Fd()), name, syscall.O_RDONLY|syscall.O_NOFOLLOW|syscall.O_NONBLOCK|syscall.O_CLOEXEC, 0)
|
||||||
|
if err != nil {
|
||||||
|
return nil, nil, err
|
||||||
|
}
|
||||||
|
f := os.NewFile(uintptr(fd), name)
|
||||||
|
if resourcePrivateEntry(f) != nil {
|
||||||
|
f.Close()
|
||||||
|
return nil, nil, errResourceState
|
||||||
|
}
|
||||||
|
b, err := io.ReadAll(io.LimitReader(f, 65537))
|
||||||
|
if err != nil || len(b) > 65536 || s.entryMatches(name, f) != nil {
|
||||||
|
f.Close()
|
||||||
|
return nil, nil, errResourceState
|
||||||
|
}
|
||||||
|
return f, b, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *resourceMonitorStore) trustedRead(name string, held *os.File) ([]byte, error) {
|
||||||
|
f, b, err := s.openEntry(name)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer f.Close()
|
||||||
|
if held == nil || s.entryMatches(name, held) != nil {
|
||||||
|
return nil, errResourceState
|
||||||
|
}
|
||||||
|
readInfo, err := f.Stat()
|
||||||
|
heldInfo, heldErr := held.Stat()
|
||||||
|
if err != nil || heldErr != nil || !os.SameFile(readInfo, heldInfo) {
|
||||||
|
return nil, errResourceState
|
||||||
|
}
|
||||||
|
return b, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Both content and held inode identities must still match the trusted snapshot.
|
||||||
|
// In particular, an identical signed replacement is not silently adopted.
|
||||||
|
func (s *resourceMonitorStore) snapshotGuard() error {
|
||||||
|
key, err := s.trustedRead(resourceKeyName, s.keyFile)
|
||||||
|
if err != nil || !hmac.Equal(key, s.key) {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
if s.existed {
|
||||||
|
b, err := s.trustedRead(resourceStateName, s.stateFile)
|
||||||
|
if err != nil || !bytes.Equal(b, s.previous) {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
if _, err := s.decode(b); err != nil {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
f, _, err := s.openEntry(resourceStateName)
|
||||||
|
if f != nil {
|
||||||
|
f.Close()
|
||||||
|
}
|
||||||
|
if !errors.Is(err, os.ErrNotExist) {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return s.guard()
|
||||||
|
}
|
||||||
|
func openResourceMonitorStore(path string, create bool) (*resourceMonitorStore, error) {
|
||||||
|
if managedDirectory(path) != nil {
|
||||||
|
return nil, errResourceState
|
||||||
|
}
|
||||||
|
fd, err := syscall.Open(path, syscall.O_RDONLY|syscall.O_DIRECTORY|syscall.O_NOFOLLOW|syscall.O_CLOEXEC, 0)
|
||||||
|
if err != nil {
|
||||||
|
return nil, errResourceState
|
||||||
|
}
|
||||||
|
s := &resourceMonitorStore{path: path, dir: os.NewFile(uintptr(fd), "resource state directory")}
|
||||||
|
fail := func() (*resourceMonitorStore, error) { s.close(); return nil, errResourceState }
|
||||||
|
stateFile, state, stateErr := s.openEntry(resourceStateName)
|
||||||
|
s.stateFile = stateFile
|
||||||
|
if stateErr != nil && !errors.Is(stateErr, os.ErrNotExist) {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
keyFile, key, err := s.openEntry(resourceKeyName)
|
||||||
|
s.keyFile = keyFile
|
||||||
|
if errors.Is(err, os.ErrNotExist) && errors.Is(stateErr, os.ErrNotExist) && create {
|
||||||
|
key = make([]byte, 32)
|
||||||
|
if _, err = rand.Read(key); err != nil {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
keyFD, err := syscall.Openat(fd, resourceKeyName, syscall.O_RDWR|syscall.O_CREAT|syscall.O_EXCL|syscall.O_NOFOLLOW|syscall.O_CLOEXEC, 0600)
|
||||||
|
if err != nil {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
f := os.NewFile(uintptr(keyFD), "resource key")
|
||||||
|
s.keyFile = f
|
||||||
|
_, writeErr := f.Write(key)
|
||||||
|
syncErr := f.Sync()
|
||||||
|
if errors.Join(writeErr, syncErr, s.dir.Sync()) != nil {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
} else if err != nil {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
if len(key) != 32 {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
s.key = append([]byte(nil), key...)
|
||||||
|
s.previous, s.existed = state, stateErr == nil
|
||||||
|
if s.existed {
|
||||||
|
if _, err := s.decode(state); err != nil {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if s.snapshotGuard() != nil {
|
||||||
|
return fail()
|
||||||
|
}
|
||||||
|
return s, nil
|
||||||
|
}
|
||||||
|
func (s *resourceMonitorStore) decode(b []byte) (resourceMonitorState, error) {
|
||||||
|
var v resourceMonitorState
|
||||||
|
d := json.NewDecoder(bytes.NewReader(b))
|
||||||
|
if uniqueResourceJSON(d, 0) != nil {
|
||||||
|
return v, errResourceState
|
||||||
|
}
|
||||||
|
if _, err := d.Token(); err != io.EOF {
|
||||||
|
return v, errResourceState
|
||||||
|
}
|
||||||
|
d = json.NewDecoder(bytes.NewReader(b))
|
||||||
|
d.DisallowUnknownFields()
|
||||||
|
if d.Decode(&v) != nil || v.Version != 1 || v.Updated < 0 || len(v.Rows) > 64 || !hmac.Equal([]byte(v.MAC), []byte(resourceStateMAC(s.key, v))) {
|
||||||
|
return v, errResourceState
|
||||||
|
}
|
||||||
|
seen := map[string]bool{}
|
||||||
|
services := map[string]bool{}
|
||||||
|
for _, r := range v.Rows {
|
||||||
|
id := r.Service + "/" + r.Resource
|
||||||
|
fp, err := hex.DecodeString(r.Fingerprint)
|
||||||
|
if !resourceID.MatchString(r.Service) || !resourceID.MatchString(r.Resource) || seen[id] || err != nil || len(fp) != 32 || r.LastCompleted < 0 || r.NextDue < 0 {
|
||||||
|
return v, errResourceState
|
||||||
|
}
|
||||||
|
seen[id] = true
|
||||||
|
services[r.Service] = true
|
||||||
|
if len(services) > 16 || r.LastCompleted > v.Updated || r.LastCompleted > 253402300799999 || (r.Health == "UNKNOWN" && r.NextDue != 0) || (r.Health != "UNKNOWN" && (r.NextDue < r.LastCompleted || r.NextDue-r.LastCompleted > 86400000)) {
|
||||||
|
return v, errResourceState
|
||||||
|
}
|
||||||
|
if r.Health != "UNKNOWN" && r.Health != "HEALTHY" && r.Health != "DEGRADED" {
|
||||||
|
return v, errResourceState
|
||||||
|
}
|
||||||
|
switch r.Outcome {
|
||||||
|
case "unknown", "pass", "dns_failed", "destination_denied", "transfer_failed", "expectation_failed", "timeout", "cancelled":
|
||||||
|
default:
|
||||||
|
return v, errResourceState
|
||||||
|
}
|
||||||
|
if (r.Health == "UNKNOWN" && (r.Outcome != "unknown" || r.LastCompleted != 0 || r.Failures != 0)) || (r.Health == "HEALTHY" && (r.Outcome != "pass" || r.Failures != 0)) || (r.Health == "DEGRADED" && (r.Outcome == "unknown" || r.Outcome == "pass" || r.Failures == 0)) {
|
||||||
|
return v, errResourceState
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return v, nil
|
||||||
|
}
|
||||||
|
func (s *resourceMonitorStore) load() (resourceMonitorState, error) {
|
||||||
|
if s.guard() != nil {
|
||||||
|
return resourceMonitorState{}, errResourceState
|
||||||
|
}
|
||||||
|
key, err := s.trustedRead(resourceKeyName, s.keyFile)
|
||||||
|
if err != nil || !hmac.Equal(key, s.key) {
|
||||||
|
return resourceMonitorState{}, errResourceState
|
||||||
|
}
|
||||||
|
b, err := s.trustedRead(resourceStateName, s.stateFile)
|
||||||
|
if errors.Is(err, os.ErrNotExist) && !s.existed {
|
||||||
|
return resourceMonitorState{Version: 1, Rows: []resourceMonitorRow{}}, nil
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return resourceMonitorState{}, errResourceState
|
||||||
|
}
|
||||||
|
v, err := s.decode(b)
|
||||||
|
if err != nil {
|
||||||
|
return v, err
|
||||||
|
}
|
||||||
|
if s.fault != nil {
|
||||||
|
s.fault("loaded", "")
|
||||||
|
}
|
||||||
|
// Authenticated operator repair on the original held inode can be reloaded;
|
||||||
|
// replaced files remain preserved/BLOCKED until deliberately repaired.
|
||||||
|
key, err = s.trustedRead(resourceKeyName, s.keyFile)
|
||||||
|
if err != nil || !hmac.Equal(key, s.key) || s.entryMatches(resourceStateName, s.stateFile) != nil {
|
||||||
|
return v, errResourceState
|
||||||
|
}
|
||||||
|
current, err := s.trustedRead(resourceStateName, s.stateFile)
|
||||||
|
if err != nil || !bytes.Equal(current, b) {
|
||||||
|
return v, errResourceState
|
||||||
|
}
|
||||||
|
s.previous = b
|
||||||
|
return v, nil
|
||||||
|
}
|
||||||
|
func (s *resourceMonitorStore) save(v resourceMonitorState) error {
|
||||||
|
if s.snapshotGuard() != nil {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
v.MAC = resourceStateMAC(s.key, v)
|
||||||
|
b, err := json.Marshal(v)
|
||||||
|
if err != nil {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
if _, err := s.decode(b); err != nil {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
// Keep the staged inode open from creation through publication; reopening a
|
||||||
|
// path after rename must never adopt a foreign replacement as our own inode.
|
||||||
|
f, err := os.CreateTemp(s.entryPath(""), ".resource-monitor-stage-")
|
||||||
|
if err != nil {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
stage := f.Name()
|
||||||
|
stageGuard := func() bool {
|
||||||
|
if s.entryMatches(filepath.Base(stage), f) != nil {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
contents, err := io.ReadAll(io.NewSectionReader(f, 0, 65537))
|
||||||
|
// Exact bytes bind the entire validated update, including its HMAC;
|
||||||
|
// truncation, append and in-place edits must fail before publication.
|
||||||
|
return err == nil && bytes.Equal(contents, b) && s.entryMatches(filepath.Base(stage), f) == nil
|
||||||
|
}
|
||||||
|
adopted := false
|
||||||
|
defer func() {
|
||||||
|
// Delete only the known staged inode, never an operator replacement.
|
||||||
|
info, err := os.Lstat(stage)
|
||||||
|
held, heldErr := f.Stat()
|
||||||
|
if err == nil && heldErr == nil && os.SameFile(info, held) && resourcePrivateEntry(f) == nil {
|
||||||
|
contents, readErr := io.ReadAll(io.NewSectionReader(f, 0, 65537))
|
||||||
|
if readErr == nil && bytes.Equal(contents, b) {
|
||||||
|
os.Remove(stage)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !adopted {
|
||||||
|
f.Close()
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
if _, err := f.Write(b); err != nil {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
if f.Sync() != nil {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
if s.fault != nil {
|
||||||
|
s.fault("staged", stage)
|
||||||
|
}
|
||||||
|
if s.snapshotGuard() != nil || !stageGuard() {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
if s.fault != nil {
|
||||||
|
s.fault("publish", stage)
|
||||||
|
}
|
||||||
|
// Detect benign/operator replacements immediately before publication. An
|
||||||
|
// actively hostile same-UID actor racing between this final check and rename
|
||||||
|
// is outside the guarantee: POSIX rename has no inode compare-and-swap.
|
||||||
|
if s.existed {
|
||||||
|
if s.snapshotGuard() != nil || !stageGuard() {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
if os.Rename(stage, s.entryPath(resourceStateName)) != nil {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
// Exclusive initial publication cannot clobber a newly appeared entry.
|
||||||
|
key, err := s.trustedRead(resourceKeyName, s.keyFile)
|
||||||
|
if err != nil || !hmac.Equal(key, s.key) || s.guard() != nil || !stageGuard() {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
if os.Link(stage, s.entryPath(resourceStateName)) != nil {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
stageInfo, stageErr := os.Lstat(stage)
|
||||||
|
heldInfo, heldErr := f.Stat()
|
||||||
|
if stageErr != nil || heldErr != nil || !os.SameFile(stageInfo, heldInfo) {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
if os.Remove(stage) != nil {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if s.entryMatches(resourceStateName, f) != nil {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
published, err := s.trustedRead(resourceStateName, f)
|
||||||
|
if err != nil || !bytes.Equal(published, b) {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
old := s.stateFile
|
||||||
|
s.stateFile, s.previous, s.existed, adopted = f, b, true, true
|
||||||
|
var closeErr error
|
||||||
|
if old != nil {
|
||||||
|
closeErr = old.Close()
|
||||||
|
}
|
||||||
|
if errors.Join(closeErr, s.dir.Sync()) != nil {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,541 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"io"
|
||||||
|
"net"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"reflect"
|
||||||
|
"strings"
|
||||||
|
"syscall"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func resourceMonitorFixture(t *testing.T) (resourceMonitorSource, []byte) {
|
||||||
|
t.Helper()
|
||||||
|
key := bytes.Repeat([]byte{7}, 32)
|
||||||
|
path := resourceCheckFile(t, resourceCheckExample)
|
||||||
|
s, err := readResourceMonitorSource(path, resourceCheckLookup, key)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return s, key
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceMonitorSchedule(t *testing.T) {
|
||||||
|
source, key := resourceMonitorFixture(t)
|
||||||
|
a, b, cooldown := 1000, 4000, 200
|
||||||
|
source.config.Services[0].Resources[0].IntervalMS = &a
|
||||||
|
source.config.Services[0].Resources[0].CooldownMS = &cooldown
|
||||||
|
source.config.Services[1].Resources[0].IntervalMS = &b
|
||||||
|
state := resourceMonitorState{Version: 1}
|
||||||
|
now := time.UnixMilli(10000)
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
var visits []string
|
||||||
|
var elapsed []int64
|
||||||
|
d := resourceMonitorDeps{key: key, now: func() time.Time { return now }, source: func() (resourceMonitorSource, error) { return source, nil }, guard: func() error { return nil }, load: func() (resourceMonitorState, error) { return state, nil }, save: func(v resourceMonitorState) error { state = v; return nil }, report: func(bool) {}, wait: func(_ context.Context, d time.Duration) error { now = now.Add(d); return nil }}
|
||||||
|
d.probe = func(_ context.Context, service string, _ resourceCheckSpec, _ resourceMonitorSource, guard func() error) (string, error) {
|
||||||
|
if err := guard(); err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
visits = append(visits, service)
|
||||||
|
elapsed = append(elapsed, now.UnixMilli()-10000)
|
||||||
|
if len(visits) == 8 {
|
||||||
|
cancel()
|
||||||
|
}
|
||||||
|
if len(visits) == 1 {
|
||||||
|
return "timeout", nil
|
||||||
|
}
|
||||||
|
return "pass", nil
|
||||||
|
}
|
||||||
|
if err := runResourceMonitor(ctx, d); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if !reflect.DeepEqual(visits, []string{"first", "second", "first", "first", "first", "first", "second", "first"}) || !reflect.DeepEqual(elapsed, []int64{0, 0, 200, 1200, 2200, 3200, 4000, 4200}) {
|
||||||
|
t.Fatalf("unfair/due schedule: %v %v", visits, elapsed)
|
||||||
|
}
|
||||||
|
if state.Rows[0].Health != "HEALTHY" || state.Rows[0].Failures != 0 || state.Rows[1].NextDue != 18000 {
|
||||||
|
t.Fatal(state)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceMonitorBlockedRepairAndPersistenceFailure(t *testing.T) {
|
||||||
|
for _, kind := range []string{"source", "ownership", "storage", "save"} {
|
||||||
|
t.Run(kind, func(t *testing.T) {
|
||||||
|
source, key := resourceMonitorFixture(t)
|
||||||
|
state := resourceMonitorState{Version: 1, MAC: "initial"}
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
tick, probes := 0, 0
|
||||||
|
d := resourceMonitorDeps{key: key, now: func() time.Time { return time.UnixMilli(10000 + int64(tick)*1000) }, report: func(bool) {}, wait: func(_ context.Context, _ time.Duration) error {
|
||||||
|
tick++
|
||||||
|
if tick == 4 {
|
||||||
|
cancel()
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}, source: func() (resourceMonitorSource, error) {
|
||||||
|
if kind == "source" && tick < 2 {
|
||||||
|
return resourceMonitorSource{}, errResourceConfig
|
||||||
|
}
|
||||||
|
return source, nil
|
||||||
|
}, guard: func() error {
|
||||||
|
if kind == "ownership" && tick < 2 {
|
||||||
|
return errResourceConfig
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}, load: func() (resourceMonitorState, error) {
|
||||||
|
if kind == "storage" && tick < 2 {
|
||||||
|
return state, errResourceState
|
||||||
|
}
|
||||||
|
return state, nil
|
||||||
|
}, save: func(v resourceMonitorState) error {
|
||||||
|
if kind == "save" {
|
||||||
|
return errResourceState
|
||||||
|
}
|
||||||
|
state = v
|
||||||
|
return nil
|
||||||
|
}}
|
||||||
|
d.probe = func(_ context.Context, _ string, _ resourceCheckSpec, _ resourceMonitorSource, g func() error) (string, error) {
|
||||||
|
if err := g(); err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
probes++
|
||||||
|
return "pass", nil
|
||||||
|
}
|
||||||
|
if err := runResourceMonitor(ctx, d); (err != nil) != (kind == "save") {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if kind == "save" {
|
||||||
|
if probes != 1 {
|
||||||
|
t.Fatalf("probed after persistence failure: %d", probes)
|
||||||
|
}
|
||||||
|
} else if probes != 2 {
|
||||||
|
t.Fatalf("did not resume safely: %d", probes)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceMonitorConfigReconcilePrivacy(t *testing.T) {
|
||||||
|
source, key := resourceMonitorFixture(t)
|
||||||
|
rows := reconcileResourceRows(key, source, nil)
|
||||||
|
rows[0].Health, rows[0].Outcome, rows[0].LastCompleted = "HEALTHY", "pass", 10000
|
||||||
|
rows[1].Health, rows[1].Outcome, rows[1].LastCompleted = "HEALTHY", "pass", 10000
|
||||||
|
old := rows[1].Fingerprint
|
||||||
|
source.config.Services[0].Resources[0].URL = "https://example.com/changed-secret"
|
||||||
|
changed := reconcileResourceRows(key, source, rows)
|
||||||
|
if changed[0].Health != "UNKNOWN" || changed[1].Health != "HEALTHY" || changed[1].Fingerprint != old {
|
||||||
|
t.Fatal(changed)
|
||||||
|
}
|
||||||
|
source.config.Services = source.config.Services[1:]
|
||||||
|
if got := reconcileResourceRows(key, source, changed); len(got) != 1 || got[0].Service != "second" {
|
||||||
|
t.Fatal(got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceMonitorStore(t *testing.T) {
|
||||||
|
source, key := resourceMonitorFixture(t)
|
||||||
|
dir := t.TempDir()
|
||||||
|
if err := os.Chmod(dir, 0700); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
s, err := openResourceMonitorStore(dir, true)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
key = s.key
|
||||||
|
state, err := s.load()
|
||||||
|
if err != nil || len(state.Rows) != 0 {
|
||||||
|
t.Fatal(state, err)
|
||||||
|
}
|
||||||
|
state.Updated = 10000
|
||||||
|
state.Rows = reconcileResourceRows(key, source, nil)
|
||||||
|
if err := s.save(state); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
s.close()
|
||||||
|
s, err = openResourceMonitorStore(dir, false)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer s.close()
|
||||||
|
loaded, err := s.load()
|
||||||
|
if err != nil || !reflect.DeepEqual(loaded.Rows, state.Rows) {
|
||||||
|
t.Fatal(loaded, err)
|
||||||
|
}
|
||||||
|
data, err := os.ReadFile(filepath.Join(dir, resourceStateName))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
for _, secret := range []string{"example.com", "private-token", "secret=value", "example.org", "Authorization"} {
|
||||||
|
if bytes.Contains(data, []byte(secret)) {
|
||||||
|
t.Fatal("secret serialized")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, name := range []string{resourceStateName, resourceKeyName} {
|
||||||
|
info, err := os.Stat(filepath.Join(dir, name))
|
||||||
|
if err != nil || info.Mode().Perm() != 0600 {
|
||||||
|
t.Fatal(info, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// Corruption is preserved; restoring a valid signed version resumes loading.
|
||||||
|
bad := []byte(`{"version":99}`)
|
||||||
|
if err := os.WriteFile(filepath.Join(dir, resourceStateName), bad, 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := s.load(); err == nil {
|
||||||
|
t.Fatal("corrupt state trusted")
|
||||||
|
}
|
||||||
|
if err := s.save(state); err == nil {
|
||||||
|
t.Fatal("corrupt state overwritten")
|
||||||
|
}
|
||||||
|
got, _ := os.ReadFile(filepath.Join(dir, resourceStateName))
|
||||||
|
if !bytes.Equal(got, bad) {
|
||||||
|
t.Fatal("evidence changed")
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(filepath.Join(dir, resourceStateName), data, 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := s.load(); err != nil {
|
||||||
|
t.Fatal("valid repair refused", err)
|
||||||
|
}
|
||||||
|
if err := os.Remove(filepath.Join(dir, resourceKeyName)); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := openResourceMonitorStore(dir, true); err == nil {
|
||||||
|
t.Fatal("missing key silently regenerated")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceMonitorUnsafeStorePreserved(t *testing.T) {
|
||||||
|
for _, kind := range []string{"symlink", "hardlink", "permissions", "wrong-owner", "directory-replaced"} {
|
||||||
|
t.Run(kind, func(t *testing.T) {
|
||||||
|
dir := filepath.Join(t.TempDir(), "state")
|
||||||
|
if err := os.Mkdir(dir, 0700); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
s, err := openResourceMonitorStore(dir, true)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer s.close()
|
||||||
|
path := filepath.Join(dir, resourceKeyName)
|
||||||
|
original, err := os.ReadFile(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
switch kind {
|
||||||
|
case "symlink":
|
||||||
|
if err := os.Rename(path, path+"-original"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.Symlink(path+"-original", path); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
case "hardlink":
|
||||||
|
if err := os.Link(path, path+"-alias"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
case "permissions":
|
||||||
|
if err := os.Chmod(path, 0644); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
case "wrong-owner":
|
||||||
|
if os.Geteuid() != 0 {
|
||||||
|
t.Skip("requires chown capability")
|
||||||
|
}
|
||||||
|
if err := os.Chown(path, 12345, -1); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
case "directory-replaced":
|
||||||
|
if err := os.Rename(dir, dir+"-old"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.Mkdir(dir, 0700); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
path = filepath.Join(dir+"-old", resourceKeyName)
|
||||||
|
}
|
||||||
|
if _, err := s.load(); err == nil {
|
||||||
|
t.Fatal("unsafe storage accepted")
|
||||||
|
}
|
||||||
|
if err := s.save(resourceMonitorState{Version: 1}); err == nil {
|
||||||
|
t.Fatal("unsafe storage overwritten")
|
||||||
|
}
|
||||||
|
data, err := os.ReadFile(path)
|
||||||
|
if err != nil || !bytes.Equal(data, original) {
|
||||||
|
t.Fatal("key changed", err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceMonitorReloadBeforeDNSAndTransfer(t *testing.T) {
|
||||||
|
for _, phase := range []string{"before-dns", "after-dns", "replace-after-dns", "env-after-dns", "owner-after-dns"} {
|
||||||
|
t.Run(phase, func(t *testing.T) {
|
||||||
|
key := bytes.Repeat([]byte{1}, 32)
|
||||||
|
path := resourceCheckFile(t, resourceCheckExample)
|
||||||
|
envChanged, ownerLost := false, false
|
||||||
|
lookup := func(k string) (string, bool) {
|
||||||
|
if envChanged && k == "CHECK_TOKEN" {
|
||||||
|
return "changed secret", true
|
||||||
|
}
|
||||||
|
return resourceCheckLookup(k)
|
||||||
|
}
|
||||||
|
source, err := readResourceMonitorSource(path, lookup, key)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
transfers, dnsCalls := 0, 0
|
||||||
|
guard := func() error {
|
||||||
|
if ownerLost {
|
||||||
|
return errResourceConfig
|
||||||
|
}
|
||||||
|
fresh, err := readResourceMonitorSource(path, lookup, key)
|
||||||
|
if err != nil || fresh.fingerprint != source.fingerprint {
|
||||||
|
return errResourceConfig
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
mutate := func() {
|
||||||
|
switch phase {
|
||||||
|
case "env-after-dns":
|
||||||
|
envChanged = true
|
||||||
|
case "owner-after-dns":
|
||||||
|
ownerLost = true
|
||||||
|
case "replace-after-dns":
|
||||||
|
data := strings.Replace(resourceCheckExample, "secret=value", "secret=rotated", 1)
|
||||||
|
if err := os.WriteFile(path+"-next", []byte(data), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := os.Rename(path+"-next", path); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
if err := os.WriteFile(path, []byte(`{"version":1}`), 0600); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if phase == "before-dns" {
|
||||||
|
mutate()
|
||||||
|
}
|
||||||
|
check := resourceCheckDeps{resolve: func(context.Context, string, string) ([]net.IP, error) {
|
||||||
|
dnsCalls++
|
||||||
|
mutate()
|
||||||
|
return []net.IP{net.ParseIP("93.184.216.34")}, nil
|
||||||
|
}, runner: func(context.Context, []string, io.Reader, io.Writer, io.Writer) error { transfers++; return nil }}
|
||||||
|
_, err = probeResourceMonitor(context.Background(), "first", source.config.Services[0].Resources[0], source, guard, check)
|
||||||
|
if err == nil || transfers != 0 || (phase == "before-dns" && dnsCalls != 0) {
|
||||||
|
t.Fatal("unsafe probe", transfers, dnsCalls, err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceMonitorBothTLSAndCancellationJoin(t *testing.T) {
|
||||||
|
source, _ := resourceMonitorFixture(t)
|
||||||
|
spec := source.config.Services[1].Resources[0]
|
||||||
|
check := resourceCheckDeps{resolve: func(context.Context, string, string) ([]net.IP, error) {
|
||||||
|
return []net.IP{net.ParseIP("93.184.216.34")}, nil
|
||||||
|
}}
|
||||||
|
calls := 0
|
||||||
|
check.runner = func(_ context.Context, args []string, _ io.Reader, out, diag io.Writer) error {
|
||||||
|
calls++
|
||||||
|
protocol := "tls12"
|
||||||
|
mode := ""
|
||||||
|
if strings.Contains(strings.Join(args, " "), "--tlsv1.3") {
|
||||||
|
protocol = "tls13"
|
||||||
|
mode = "verify"
|
||||||
|
}
|
||||||
|
configuredWire(spec.URL, spec.Method, protocol, mode, out, diag)
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
outcome, err := probeResourceMonitor(context.Background(), "second", spec, source, func() error { return nil }, check)
|
||||||
|
if err != nil || outcome == "pass" || calls != 2 {
|
||||||
|
t.Fatal(outcome, calls, err)
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
joined := false
|
||||||
|
check.runner = func(ctx context.Context, _ []string, _ io.Reader, _ io.Writer, _ io.Writer) error {
|
||||||
|
cancel()
|
||||||
|
<-ctx.Done()
|
||||||
|
joined = true
|
||||||
|
return ctx.Err()
|
||||||
|
}
|
||||||
|
outcome, err = probeResourceMonitor(ctx, "second", spec, source, func() error { return nil }, check)
|
||||||
|
if err != nil || outcome != "cancelled" || !joined {
|
||||||
|
t.Fatal("did not join cancellation", outcome, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceMonitorSchemaScheduling(t *testing.T) {
|
||||||
|
for _, field := range []string{"interval_ms", "cooldown_ms"} {
|
||||||
|
for _, value := range []string{"0", "-1", "99", "86400001", "null", "\"1000\""} {
|
||||||
|
data := strings.Replace(resourceCheckExample, `"timeout_ms":1000`, `"timeout_ms":1000,"`+field+`":`+value, 1)
|
||||||
|
if _, err := parseResourceChecks([]byte(data), resourceCheckLookup); err == nil {
|
||||||
|
t.Fatal(field, value)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceMonitorSingletonAndGenericDispatch(t *testing.T) {
|
||||||
|
d, env, _ := managedCLIFixture(t)
|
||||||
|
lock, err := acquireManagedMonitorSingleton(d)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer lock.close()
|
||||||
|
if _, err := acquireManagedMonitorSingleton(d); !errors.Is(err, syscall.EWOULDBLOCK) {
|
||||||
|
t.Fatal("singleton conflict ignored", err)
|
||||||
|
}
|
||||||
|
// Generic dispatch precedes invalid YouTube/adaptive settings. A cancelled
|
||||||
|
// generic startup needs no runtime/network, database or valid source JSON.
|
||||||
|
env["SOCKS_PORT"] = "1080"
|
||||||
|
env["ZAPRET_AUTOTUNE_SOCKS_PROXY"] = "socks5h://127.0.0.1:1080"
|
||||||
|
env["ZAPRET_AUTOTUNE_RESOURCE_CONFIG"] = "/missing-invalid-config"
|
||||||
|
env["ZAPRET_AUTOTUNE_VIDEO_ID"] = "bad"
|
||||||
|
env["ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE"] = "invalid"
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
cancel()
|
||||||
|
var out, diag bytes.Buffer
|
||||||
|
code := managedMonitorCLIContext(ctx, nil, &out, &diag, d, func(context.Context, managedMonitorDeps) error { t.Fatal("YouTube supervisor invoked"); return nil })
|
||||||
|
if code != 0 || !strings.Contains(out.String(), "generic resource monitor") {
|
||||||
|
t.Fatal(code, out.String(), diag.String())
|
||||||
|
}
|
||||||
|
delete(env, "ZAPRET_AUTOTUNE_RESOURCE_CONFIG")
|
||||||
|
if code := managedMonitorCLIContext(ctx, nil, &out, &diag, d, nil); code != 64 {
|
||||||
|
t.Fatal("legacy invalid config no longer rejected", code)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceMonitorInvalidInitialConfigAliveAndTERM(t *testing.T) {
|
||||||
|
d, env, _ := managedCLIFixture(t)
|
||||||
|
env["SOCKS_PORT"] = "1080"
|
||||||
|
env["ZAPRET_AUTOTUNE_SOCKS_PROXY"] = "socks5h://127.0.0.1:1080"
|
||||||
|
d.verifySOCKS = func(int) error { return nil }
|
||||||
|
initManagedCLI(t, d)
|
||||||
|
path := resourceCheckFile(t, `{"invalid":true}`)
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
started := make(chan struct{}, 1)
|
||||||
|
check := resourceCheckDeps{guard: func() error {
|
||||||
|
select {
|
||||||
|
case started <- struct{}{}:
|
||||||
|
default:
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}, resolve: func(context.Context, string, string) ([]net.IP, error) {
|
||||||
|
t.Error("DNS with invalid source")
|
||||||
|
return nil, nil
|
||||||
|
}, runner: func(context.Context, []string, io.Reader, io.Writer, io.Writer) error {
|
||||||
|
t.Error("transfer with invalid source")
|
||||||
|
return nil
|
||||||
|
}}
|
||||||
|
var out, diag bytes.Buffer
|
||||||
|
done := make(chan int, 1)
|
||||||
|
go func() { done <- resourceMonitorCLIWithCheck(ctx, path, &out, &diag, d, check) }()
|
||||||
|
// Source error short-circuits the runtime guard, so wait for authenticated
|
||||||
|
// BLOCKED persistence rather than relying on a diagnostic log race.
|
||||||
|
deadline := time.Now().Add(2 * time.Second)
|
||||||
|
for time.Now().Before(deadline) {
|
||||||
|
s, err := openResourceMonitorStore(d.directory, false)
|
||||||
|
if err == nil {
|
||||||
|
state, err := s.load()
|
||||||
|
s.close()
|
||||||
|
if err == nil && state.Blocked {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
time.Sleep(10 * time.Millisecond)
|
||||||
|
}
|
||||||
|
select {
|
||||||
|
case code := <-done:
|
||||||
|
t.Fatalf("invalid source terminated controller: %d", code)
|
||||||
|
default:
|
||||||
|
}
|
||||||
|
cancel()
|
||||||
|
select {
|
||||||
|
case code := <-done:
|
||||||
|
if code != 0 {
|
||||||
|
t.Fatal(code, diag.String())
|
||||||
|
}
|
||||||
|
case <-time.After(2 * time.Second):
|
||||||
|
t.Fatal("TERM did not join controller")
|
||||||
|
}
|
||||||
|
if !strings.Contains(diag.String(), "BLOCKED") {
|
||||||
|
t.Fatal("no blocked status", diag.String())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceMonitorConcurrentEnvironmentReload(t *testing.T) {
|
||||||
|
path := resourceCheckFile(t, resourceCheckExample)
|
||||||
|
t.Setenv("CHECK_URL", "https://example.org/private")
|
||||||
|
t.Setenv("CHECK_TOKEN", "initial-secret")
|
||||||
|
t.Setenv("ZAPRET_AUTOTUNE_SOCKS_PROXY", "socks5h://127.0.0.1:1080")
|
||||||
|
key := bytes.Repeat([]byte{3}, 32)
|
||||||
|
baseline, err := readResourceMonitorSource(path, os.LookupEnv, key)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
done := make(chan struct{})
|
||||||
|
go func() {
|
||||||
|
defer close(done)
|
||||||
|
for i := 0; i < 100; i++ {
|
||||||
|
os.Setenv("CHECK_TOKEN", "rotated-secret")
|
||||||
|
os.Setenv("CHECK_TOKEN", "initial-secret")
|
||||||
|
}
|
||||||
|
}()
|
||||||
|
for i := 0; i < 100; i++ {
|
||||||
|
s, err := readResourceMonitorSource(path, os.LookupEnv, key)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if s.fingerprint == "" {
|
||||||
|
t.Fatal("empty fingerprint")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
<-done
|
||||||
|
if err := os.Setenv("CHECK_TOKEN", "rotated-secret"); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
fresh, err := readResourceMonitorSource(path, os.LookupEnv, key)
|
||||||
|
if err != nil || fresh.fingerprint == baseline.fingerprint {
|
||||||
|
t.Fatal("secret rotation not bound", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResourceMonitorTargetConfigBoundary(t *testing.T) {
|
||||||
|
_, env, _ := managedCLIFixture(t)
|
||||||
|
for k, v := range env {
|
||||||
|
t.Setenv(k, v)
|
||||||
|
}
|
||||||
|
t.Setenv("SOCKS_PORT", "1080")
|
||||||
|
t.Setenv("ZAPRET_AUTOTUNE_SOCKS_PROXY", "socks5h://127.0.0.1:1080")
|
||||||
|
t.Setenv("ZAPRET_AUTOTUNE_RESOURCE_CONFIG", resourceCheckFile(t, `{"invalid":true}`))
|
||||||
|
t.Setenv("ZAPRET_AUTOTUNE_VIDEO_ID", "invalid")
|
||||||
|
t.Setenv("ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE", "invalid")
|
||||||
|
if code := run([]string{"target-config-check"}, io.Discard, io.Discard); code != 0 {
|
||||||
|
t.Fatal("bad resource JSON/YouTube blocked safe generic startup", code)
|
||||||
|
}
|
||||||
|
t.Setenv("ZAPRET_AUTOTUNE_SOCKS_PROXY", "socks5h://foreign.example:1080")
|
||||||
|
if code := run([]string{"target-config-check"}, io.Discard, io.Discard); code != 64 {
|
||||||
|
t.Fatal("bad proxy allowed", code)
|
||||||
|
}
|
||||||
|
t.Setenv("ZAPRET_AUTOTUNE_SOCKS_PROXY", "socks5h://127.0.0.1:1080")
|
||||||
|
t.Setenv("ZAPRET_AUTOTUNE_DNS_MODE", "invalid")
|
||||||
|
if code := run([]string{"target-config-check"}, io.Discard, io.Discard); code != 64 {
|
||||||
|
t.Fatal("bad DNS allowed", code)
|
||||||
|
}
|
||||||
|
t.Setenv("ZAPRET_AUTOTUNE_DNS_MODE", "doh")
|
||||||
|
t.Setenv("ZAPRET_AUTOTUNE_TARGET_ROLE", "foreign")
|
||||||
|
if code := run([]string{"target-config-check"}, io.Discard, io.Discard); code != 64 {
|
||||||
|
t.Fatal("bad target role allowed", code)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,135 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"os/exec"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestResourcePeerTLSCollectorBoundaries(t *testing.T) {
|
||||||
|
for _, code := range []int{7, 35, 51, 58, 60, 65} {
|
||||||
|
t.Run(fmt.Sprint(code), func(t *testing.T) {
|
||||||
|
negative := exec.Command("sh", "-c", fmt.Sprintf("exit %d", code)).Run()
|
||||||
|
for _, mixed := range []error{nil, context.Canceled, errors.New("unknown infrastructure"), &managedRecoveryFatal{Err: errors.New("guard failed")}} {
|
||||||
|
runner := func(_ context.Context, args []string, _ io.Reader, _, _ io.Writer) error {
|
||||||
|
if strings.Contains(strings.Join(args, " "), "--insecure") {
|
||||||
|
t.Fatal("certificate bypass")
|
||||||
|
}
|
||||||
|
return errors.Join(negative, mixed)
|
||||||
|
}
|
||||||
|
// Use curl's actual exit type for the unmixed production path.
|
||||||
|
if mixed == nil {
|
||||||
|
runner = func(context.Context, []string, io.Reader, io.Writer, io.Writer) error { return negative }
|
||||||
|
}
|
||||||
|
m, err := collectAssetMeasurement(context.Background(), "socks5://127.0.0.1:1080", "https://yt3.ggpht.com/fresh?private=secret", "yt3.ggpht.com", "tls12", "", runner)
|
||||||
|
if err == nil || len(m.Transfers) != 0 || strings.Contains(err.Error(), "secret") {
|
||||||
|
t.Fatal("accepted/leaked rejection", err)
|
||||||
|
}
|
||||||
|
peer := code == 35 || code == 51 || code == 60
|
||||||
|
var fatal *managedRecoveryFatal
|
||||||
|
if errors.As(err, &fatal) != (!peer || mixed != nil) {
|
||||||
|
t.Fatalf("code=%d mixed=%v fatal=%v", code, mixed, err)
|
||||||
|
}
|
||||||
|
if managedAssetTransferNonpass(err) != (peer && mixed == nil) || managedAssetHealthNonpass(err) != (peer && mixed == nil) {
|
||||||
|
t.Fatal("unsafe nonpass classification", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestManagedAvatarPeerTLSMonitorNoQualifiedCooldown(t *testing.T) {
|
||||||
|
for _, tc := range []struct {
|
||||||
|
code int
|
||||||
|
stage string
|
||||||
|
}{{35, "verify"}, {60, "verify"}, {35, "apply"}, {60, "apply"}} {
|
||||||
|
t.Run(fmt.Sprintf("%d/%s", tc.code, tc.stage), func(t *testing.T) {
|
||||||
|
d, front, _ := recoveryFixture(t, true)
|
||||||
|
r, events, _ := assetRuntimeFixture(t)
|
||||||
|
good := r.curl
|
||||||
|
negative := exec.Command("sh", "-c", fmt.Sprintf("exit %d", tc.code)).Run()
|
||||||
|
reject := func(ctx context.Context, args []string, in io.Reader, out, diag io.Writer) error {
|
||||||
|
if strings.Contains(strings.Join(args, " "), "--insecure") {
|
||||||
|
t.Fatal("certificate bypass")
|
||||||
|
}
|
||||||
|
return negative
|
||||||
|
}
|
||||||
|
if tc.stage == "verify" {
|
||||||
|
r.curl = reject
|
||||||
|
}
|
||||||
|
thumbnail := unitAssetCandidate(t, d.db, "i.ytimg.com", "--lua-desync=fake", true)
|
||||||
|
ggc := managedCandidateFixture(t, d.db, "googlevideo", "fresh.googlevideo.com", "--lua-desync=fake", true)
|
||||||
|
ids := []int64{front, thumbnail, ggc}
|
||||||
|
target, _, _ := managedCLIFixture(t)
|
||||||
|
initManagedCLI(t, target)
|
||||||
|
d.transaction.Dir = target.directory
|
||||||
|
if err := d.transaction.Apply(d.db, ids); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
before, err := d.transaction.read("config")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
d.service = true
|
||||||
|
d.frontHealth = func(context.Context) (string, error) { return "OK", nil }
|
||||||
|
d.resources = func(context.Context) (managedServiceInventory, error) {
|
||||||
|
return managedServiceInventory{images: map[string]string{"i.ytimg.com": "https://i.ytimg.com/fresh", "yt3.ggpht.com": "https://yt3.ggpht.com/fresh"}, proxy: r.proxy, runner: reject}, nil
|
||||||
|
}
|
||||||
|
d.verifyAssets = r.verify
|
||||||
|
d.health = managedServiceHealth(r.proxy, d.frontHealth, d.resources, func(ctx context.Context, args []string, in io.Reader, out, diag io.Writer) error {
|
||||||
|
config, _ := io.ReadAll(in)
|
||||||
|
if strings.Contains(string(config), "yt3.ggpht.com") {
|
||||||
|
return negative
|
||||||
|
}
|
||||||
|
return good(ctx, args, strings.NewReader(string(config)), out, diag)
|
||||||
|
})
|
||||||
|
monitor := monitorRuntimeFixture(t)
|
||||||
|
monitor.db, monitor.target, monitor.threshold = d.db, target, 1
|
||||||
|
if err := wireManagedRecovery(&monitor, d, func(error) {}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
monitor.reportRecoveryError = func(err error) {
|
||||||
|
if !errors.Is(err, errNoQualifiedStrategy) {
|
||||||
|
t.Fatal("not genuine candidate exhaustion", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
waits := 0
|
||||||
|
monitor.wait = func(ctx context.Context, _ time.Duration) error {
|
||||||
|
state, err := readMonitorState(d.db, monitor.serviceID)
|
||||||
|
if err != nil || state.RecoveryRunning || state.CooldownUntil.Before(monitor.now().Add(2*time.Hour)) {
|
||||||
|
t.Fatal("absence pause not durable", state, err)
|
||||||
|
}
|
||||||
|
waits++
|
||||||
|
if waits == 2 {
|
||||||
|
cancel()
|
||||||
|
return ctx.Err()
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if err := runManagedMonitor(ctx, monitor); !errors.Is(err, context.Canceled) {
|
||||||
|
t.Fatal("peer rejection killed monitor", err)
|
||||||
|
}
|
||||||
|
joins := 0
|
||||||
|
for _, event := range *events {
|
||||||
|
if event == "join" {
|
||||||
|
joins++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if joins != len(managedAssetSeeds) {
|
||||||
|
t.Fatal("candidate worker not joined", joins, *events)
|
||||||
|
}
|
||||||
|
assertActiveCandidates(t, d.transaction, d.db, ids)
|
||||||
|
after, err := d.transaction.read("config")
|
||||||
|
if err != nil || string(before) != string(after) {
|
||||||
|
t.Fatal("peer-negative apply changed committed config", err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -7,6 +7,7 @@ import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
func TestTargetConfigCheck(t *testing.T) {
|
func TestTargetConfigCheck(t *testing.T) {
|
||||||
|
t.Setenv("ZAPRET_AUTOTUNE_DNS_MODE", "native")
|
||||||
t.Setenv("ZAPRET_AUTOTUNE_YOUTUBE_SERVICE", "0")
|
t.Setenv("ZAPRET_AUTOTUNE_YOUTUBE_SERVICE", "0")
|
||||||
t.Setenv("ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE", "cold")
|
t.Setenv("ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE", "cold")
|
||||||
t.Setenv("ZAPRET_AUTOTUNE_VIDEO_ID", "dQw4w9WgXcQ")
|
t.Setenv("ZAPRET_AUTOTUNE_VIDEO_ID", "dQw4w9WgXcQ")
|
||||||
@@ -23,6 +24,21 @@ func TestTargetConfigCheck(t *testing.T) {
|
|||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
t.Run("DoH ignores deprecated native DNS", func(t *testing.T) {
|
||||||
|
t.Setenv("ZAPRET_AUTOTUNE_DNS_MODE", "doh")
|
||||||
|
t.Setenv("ZAPRET_AUTOTUNE_DOH_URL", defaultDoHURL)
|
||||||
|
t.Setenv("ZAPRET_AUTOTUNE_DOH_BOOTSTRAP_IPS", "8.8.8.8")
|
||||||
|
t.Setenv("ZAPRET_AUTOTUNE_DOH_TIMEOUT_SEC", "5")
|
||||||
|
t.Setenv("SS_DNS_SERVER", "ignored invalid legacy setting")
|
||||||
|
var out, errOut bytes.Buffer
|
||||||
|
if code := run([]string{"target-config-check"}, &out, &errOut); code != 0 || out.Len() != 0 || errOut.Len() != 0 {
|
||||||
|
t.Fatal(code, out.String(), errOut.String())
|
||||||
|
}
|
||||||
|
t.Setenv("ZAPRET_AUTOTUNE_DOH_BOOTSTRAP_IPS", "secret-bootstrap-host")
|
||||||
|
if code := run([]string{"target-config-check"}, &out, &errOut); code != 64 || strings.Contains(errOut.String(), "secret-bootstrap-host") {
|
||||||
|
t.Fatal(code, errOut.String())
|
||||||
|
}
|
||||||
|
})
|
||||||
t.Setenv("SS_DNS_SERVER", "")
|
t.Setenv("SS_DNS_SERVER", "")
|
||||||
for key, value := range map[string]string{
|
for key, value := range map[string]string{
|
||||||
"ZAPRET_AUTOTUNE_YOUTUBE_SERVICE": "true",
|
"ZAPRET_AUTOTUNE_YOUTUBE_SERVICE": "true",
|
||||||
|
|||||||
@@ -3,7 +3,15 @@ ZAPRET_AUTOTUNE_HOST_SOCKS_PORT=11080
|
|||||||
ZAPRET_AUTOTUNE_YOUTUBE_SERVICE=0
|
ZAPRET_AUTOTUNE_YOUTUBE_SERVICE=0
|
||||||
ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE=cold
|
ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE=cold
|
||||||
ZAPRET_AUTOTUNE_VIDEO_ID=dQw4w9WgXcQ
|
ZAPRET_AUTOTUNE_VIDEO_ID=dQw4w9WgXcQ
|
||||||
# Optional JSON array: 1..5 unique 11-character VideoIDs (not signed URLs).
|
# Missing/empty selects automatic YouTube inventory: primary + two public samples.
|
||||||
|
# Nonempty JSON overrides it: 1..5 unique 11-character VideoIDs (not signed URLs).
|
||||||
# ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS='["dQw4w9WgXcQ","aqz-KE-bpKQ","jNQXAC9IVRw"]'
|
# ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS='["dQw4w9WgXcQ","aqz-KE-bpKQ","jNQXAC9IVRw"]'
|
||||||
# Optional target ss-server native DNS; empty retains the base resolver behavior.
|
# Encrypted target DNS for both SOCKS hostname traffic and managed checks.
|
||||||
|
ZAPRET_AUTOTUNE_DNS_MODE=doh
|
||||||
|
ZAPRET_AUTOTUNE_DOH_URL=https://dns.google/dns-query
|
||||||
|
# Empty selects Google bootstrap only for the exact default URL above.
|
||||||
|
# Custom provider: change URL AND supply its literal bootstrap IPs (comma-separated).
|
||||||
|
ZAPRET_AUTOTUNE_DOH_BOOTSTRAP_IPS=
|
||||||
|
ZAPRET_AUTOTUNE_DOH_TIMEOUT_SEC=5
|
||||||
|
# Explicit native mode only; ignored in doh mode. Never an automatic fallback.
|
||||||
SS_DNS_SERVER=
|
SS_DNS_SERVER=
|
||||||
+224
-20
@@ -5,8 +5,133 @@ reconfigure, or stop the existing `zapret2-ss-autotune` container. The target
|
|||||||
starts from a disabled managed baseline; `managed monitor` bootstraps the
|
starts from a disabled managed baseline; `managed monitor` bootstraps the
|
||||||
frontend/API strategy, then discovers and applies an exact-host GGC strategy.
|
frontend/API strategy, then discovers and applies an exact-host GGC strategy.
|
||||||
|
|
||||||
|
## Recorded DoH acceptance (2026-10-13): live DNS and staged outage verified
|
||||||
|
|
||||||
|
Supplied verified reports, not remote evidence reread by this docs update:
|
||||||
|
source `544dda7c85965fc3200de3af17fca9ef9999b2ce`, managed image
|
||||||
|
`sha256:b3085b7ceca4a629c28bd58f23bf97f02a34c2a38df08b57756e40594ba2d427`,
|
||||||
|
container `4ad54175a54c`, owned loopback SOCKS11082. This source includes DoH
|
||||||
|
`367d956`, TCP timeout-reply fix `721cad4` and public-health `socks5h` fix
|
||||||
|
`544dda7`. Later docs commits are not the tested image source.
|
||||||
|
|
||||||
|
- LIVE 303-second snapshot: unchanged root and DNS22/SS59,60/monitor253 PIDs,
|
||||||
|
children PPID1; owned readiness ACK and parent FD8/9 anchors positive.
|
||||||
|
UDP/TCP IDs/questions correct, rcode0, answers2/8; SOCKS `0500`, WWW
|
||||||
|
HTTP200/SSLVerifyResult0, configured `health youtube` exit0. Wire evidence:
|
||||||
|
41 marked direct DoH connections, zero external port53. Scope `suffix`,
|
||||||
|
generation9, IDs `[1,9]`, hash preserved versus staging are measured snapshots,
|
||||||
|
not universal state; autonomous updates occurred since an earlier generation8.
|
||||||
|
- Staging blocked both literal bootstrap IPs for 61 seconds. UDP/TCP returned
|
||||||
|
SERVFAIL/rcode2 at about 5000ms; root, DNS, monitor and SS PIDs stayed the same,
|
||||||
|
external port53 stayed zero. Removing the block restored DNS immediately;
|
||||||
|
WWW HTTP200 passed after 46 seconds without restart. Graceful stop took
|
||||||
|
1 second, exit0, no survivors. This tests a DNS dependency outage, not all
|
||||||
|
conceivable failures or nonconfiguration guards. Temporary nft fault rules
|
||||||
|
were staging-only and removed; runtime firewall remains normal generated
|
||||||
|
native policy, not a new fault-injection policy.
|
||||||
|
- Existing shared GGC qualification is separate proof: no new 18-transfer
|
||||||
|
qualification under DoH. Fresh TLS1.3 media returned HTTP206/65536 bytes/MP4
|
||||||
|
`ftyp`/SSL0; TLS1.2 TIMEOUT is not qualified. Avatar and thumbnail qualification
|
||||||
|
remain independent outstanding gates. **No full-service READY claim.**
|
||||||
|
- Staging cleanup removed the test containers, leaving no new running tests;
|
||||||
|
old user SS1080 and user-stopped proxy were unchanged. State, credentials and
|
||||||
|
private env were preserved. No source/runtime changes or probes in this update.
|
||||||
|
|
||||||
|
Private reports: `/home/agent/zapret2-autotune-rc-doh-721cad4/live-upgrade-private/`,
|
||||||
|
then latest `/home/agent/zapret2-autotune-rc-doh-544dda7/live-upgrade-private/`.
|
||||||
|
Historical acceptance below remains scoped to its own source/image and DNS mode.
|
||||||
|
Generic service integration and maximum strategy collection remain future work,
|
||||||
|
not a new framework.
|
||||||
|
|
||||||
|
## Autonomous three-CDN field acceptance (2026-10-13): media qualified, NOT READY
|
||||||
|
|
||||||
|
Supplied field snapshot: source `3fc8563995e890310580ba8f78850b92fea42136`,
|
||||||
|
managed `Dockerfile.target` image
|
||||||
|
`sha256:a902e4efd949a7ffce4414fd99ab4a815a643b40cf9b9e30e2bd2db7b452817a`,
|
||||||
|
container `4c6b34…`, project `rc0398e6f…`, loopback SOCKS11082.
|
||||||
|
Natural 08:07–08:09 recovery after the existing cooldown discovered actual hosts
|
||||||
|
`rr16---sn-n8v7znsz.googlevideo.com`, `rr18---sn-n8v7zns6.googlevideo.com`,
|
||||||
|
and `rr5---sn-n8v7kn76.googlevideo.com`, without manual domains.
|
||||||
|
Each passed post-warmup 3/3 on both TLS protocols: all18 HTTP206, Range
|
||||||
|
`0..65535`, 65536 bytes, MP4 `ftyp`, valid SSL verification. Separate warm
|
||||||
|
attempts/failures were 2/0, 3/1, 3/1; no cold evidence is claimed.
|
||||||
|
|
||||||
|
Fresh version-1 `owned-nfq-media-range-v1` Adaptive proof checked at
|
||||||
|
`2026-10-13T08:08:28.752Z` measured three roots/prefixes and retained endpoint,
|
||||||
|
source-ID and native-owner Lua bindings. Native recovery automatically promoted
|
||||||
|
candidate10, generation9→10, IDs `[1,3,9]`→`[1,9,10]`, experimental scope→`suffix`;
|
||||||
|
frontend1, thumbnail9 and common `googlevideo.com` hostlist stayed intact.
|
||||||
|
This is representative composite two-strategy/per-host-learning qualification,
|
||||||
|
not a maximum catalogue or qualification of every suffix host. Native request
|
||||||
|
statistics were not persisted, so manual request count is unknown.
|
||||||
|
|
||||||
|
Reported 08:21:48 snapshot: running container, live monitor, whole-service health
|
||||||
|
false / DEGRADED. Probe failures405 at 08:20:37 do not independently diagnose an
|
||||||
|
avatar cause this phase. Fresh orchestrator owned-SOCKS greeting returned `0500`;
|
||||||
|
native `socks5h` respects SS DNS `8.8.8.8`. The passive SSH watch vanished after
|
||||||
|
about ten minutes; no completed 18-minute watch is claimed. No cooldown reset.
|
||||||
|
Supplied private proof: `/home/agent/zapret2-autotune-rc-3fc8563/field-private/proof/final.safe.json`,
|
||||||
|
not independently reread here. This docs update performs no deployment/probes;
|
||||||
|
later docs commits are not the tested image source.
|
||||||
|
|
||||||
|
Service-mode source now discovers representative inventory automatically and can
|
||||||
|
promote an existing experiment only through the unchanged version-1 owned native
|
||||||
|
proof factory: **three actual media hosts**, at least two roots and two prefixes,
|
||||||
|
post-warmup 3/3 on both TLS protocols. Discovery is bounded to three rounds,
|
||||||
|
15 requests maximum and three minutes; signed MP4 URLs need twenty minutes left.
|
||||||
|
VideoIDs are not CDN evidence. No manual domain list or single-CDN relaxation is
|
||||||
|
introduced. Missing/empty inventory defaults and explicit overrides are below.
|
||||||
|
|
||||||
|
Apply requires qualified frontend and fresh deployed checks of all three media
|
||||||
|
resources, not avatar readiness. Whole-service health remains DEGRADED until avatar
|
||||||
|
checks pass. Insufficient inventory preserves the committed experiment and uses
|
||||||
|
configured cooldown; true strategy exhaustion retains its minimum two-hour pause.
|
||||||
|
Suffix authorization remains extrapolative, not proof of every Googlevideo host.
|
||||||
|
|
||||||
|
Representative autonomous media promotion is field-qualified above; actual
|
||||||
|
avatar/whole-service acceptance remains open. Historical results below are not
|
||||||
|
the new factory proof. Maximum strategy collection/search diversity and PID1 global-failure
|
||||||
|
stay-alive are unfinished future increments, as is arbitrary declarative service
|
||||||
|
configuration. See [MANAGED_TARGET.md](../../MANAGED_TARGET.md) and
|
||||||
|
[GOOGLEVIDEO_DISCOVERY.md](../../GOOGLEVIDEO_DISCOVERY.md).
|
||||||
|
|
||||||
## Recorded Googlevideo suffix experiment (2026-10-12)
|
## Recorded Googlevideo suffix experiment (2026-10-12)
|
||||||
|
|
||||||
|
### Final field follow-up: DEGRADED / NOT READY
|
||||||
|
|
||||||
|
Supplied results for source `fd25cb89398300bb9849db4b5511af75bf86eccf`, correct
|
||||||
|
managed image `sha256:73e955e7fdc6efaef88c62ebe4f7afeab71c29d29ba3c88b2a2b7547bb647da5`,
|
||||||
|
container `a4e845a6d7…`, official managed PID 1 and SOCKS11082 supersede the
|
||||||
|
earlier pending media/fallback labels below. The `37823…` prototype is not active.
|
||||||
|
Watch `20:15:07.686Z` → `20:30:54.826Z` on 2026-10-12 completed over 15 minutes:
|
||||||
|
restart count0, monitor alive; a fresh post-report check returned SOCKS `0500`,
|
||||||
|
WWW HTTP200/SSLVerifyResult0. Native `socks5h` probes respect SS `-d 8.8.8.8`.
|
||||||
|
|
||||||
|
Post-warmup TLS1.2 and TLS1.3 each passed 3/3 on
|
||||||
|
`rr4---sn-axq7sn7z.googlevideo.com`, `rr5---sn-n8v7kn76.googlevideo.com`, and
|
||||||
|
`rr18---sn-axq7sn7z.googlevideo.com`: **18 actual HTTP206, 65536-byte MP4 `ftyp`
|
||||||
|
transfers**, valid Range `0..65535`. Initial warmup is separate: TLS1.2 three
|
||||||
|
timeouts, TLS1.3 three passes. No cold/all-host claim; the old rr5 `sn-ixh`
|
||||||
|
counterexample was not retested. Thumbnail JPEG decoding passed 3/3 per TLS
|
||||||
|
(2888 bytes, SSLVerifyResult0). Watch metadata found an allowed `yt3.ggpht.com`
|
||||||
|
avatar URL, but actual both-TLS avatar transfers returned curl28, zero samples.
|
||||||
|
There is no saved avatar evidence or qualified avatar profile.
|
||||||
|
|
||||||
|
Recovery now continued through validated peer-TLS/payload nonpasses with current
|
||||||
|
job-bound transaction/store/ownership guards: genuine no-qualified-strategy
|
||||||
|
cooldown advanced naturally while the monitor stayed alive, without a reset.
|
||||||
|
Snapshot generation9/IDs[1,3,9]/hash `0e34…`/experimental scope were unchanged;
|
||||||
|
candidate3 has no qualification/evidence. This is degraded recovery acceptance,
|
||||||
|
not successful avatar repair, durable family qualification or promotion.
|
||||||
|
Actual avatar transfer/strategy qualification and durable representative
|
||||||
|
multi-host family/mode qualification remain unfinished; automatic family
|
||||||
|
qualification is not implemented. Other targets on 1080/11080/11081 and Xray
|
||||||
|
were unchanged. No remote operations are performed by this docs update.
|
||||||
|
Private sanitized proof: `/home/agent/zapret2-autotune-rc-fd25cb8/proof/final.safe.json`.
|
||||||
|
Exact results and limits: [PROGRESS.md](../../PROGRESS.md).
|
||||||
|
|
||||||
|
### Historical initial experiment
|
||||||
|
|
||||||
Source RC `eade051c2bc335b71b1616d697326ca4c8019d70`, immutable image
|
Source RC `eade051c2bc335b71b1616d697326ca4c8019d70`, immutable image
|
||||||
`sha256:fdb60593e29d7c337a413a47aaddfbcaaeabdeb877042c58f506d4abdc98a13b`,
|
`sha256:fdb60593e29d7c337a413a47aaddfbcaaeabdeb877042c58f506d4abdc98a13b`,
|
||||||
isolated SOCKS `127.0.0.1:11082`: recorded journal generation3, active [1,2,3],
|
isolated SOCKS `127.0.0.1:11082`: recorded journal generation3, active [1,2,3],
|
||||||
@@ -30,9 +155,10 @@ replace this approved profile. Remaining acceptance is representative signed
|
|||||||
media coverage (at least three hosts across two roots/prefixes, post-warmup 3/3
|
media coverage (at least three hosts across two roots/prefixes, post-warmup 3/3
|
||||||
on both TLS protocols plus media), avatar qualification, actual restart/recreation
|
on both TLS protocols plus media), avatar qualification, actual restart/recreation
|
||||||
recovery of this suffix RC, and autonomous experimental recovery/autoqualification
|
recovery of this suffix RC, and autonomous experimental recovery/autoqualification
|
||||||
policy with accurate covered-versus-qualified status. See
|
policy with accurate covered-versus-qualified status. These were that initial
|
||||||
[PROGRESS.md](../../PROGRESS.md) for the five remaining gates. The older clean-run
|
RC's pending gates; see the final follow-up above for later field results and
|
||||||
PASS below does not close them.
|
[PROGRESS.md](../../PROGRESS.md) for current remaining work. The older clean-run
|
||||||
|
PASS below does not close family/avatar qualification.
|
||||||
|
|
||||||
## Provisioning
|
## Provisioning
|
||||||
|
|
||||||
@@ -67,6 +193,22 @@ in the private `.env` and pull that immutable image before starting the target.
|
|||||||
Use a managed target image, not the generic image. Do not deploy production
|
Use a managed target image, not the generic image. Do not deploy production
|
||||||
with a tag or local default.
|
with a tag or local default.
|
||||||
|
|
||||||
|
Build from the repository root with an explicit Dockerfile and assert the
|
||||||
|
image contract **before promotion**, even if its tag says managed-target:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
IMAGE=zapret-autotune-managed-target:local
|
||||||
|
docker build --pull -f Dockerfile.target -t "$IMAGE" .
|
||||||
|
test "$(docker image inspect --format '{{json .Config.Entrypoint}}' "$IMAGE")" = '["/bin/sh","/opt/managed-target-entrypoint.sh"]'
|
||||||
|
docker run --rm --network none --entrypoint /bin/sh "$IMAGE" -ec 'test "$(cat /etc/zapret-autotune/target-id)" = zapret-autotune-managed-v1; test -f /opt/managed-target-entrypoint.sh'
|
||||||
|
```
|
||||||
|
|
||||||
|
Root `Dockerfile` is the proposal/prototype image, not the managed PID 1 service;
|
||||||
|
omitting `-f Dockerfile.target` previously produced a mistagged prototype.
|
||||||
|
These positive entrypoint/marker assertions do not replace live acceptance.
|
||||||
|
Promote the same immutable image as described in
|
||||||
|
[MANAGED_TARGET.md](../../MANAGED_TARGET.md#build-once-promote-the-same-image).
|
||||||
|
|
||||||
Offline static guard (from repository root):
|
Offline static guard (from repository root):
|
||||||
`python3 deploy/managed-target/check_static.py`.
|
`python3 deploy/managed-target/check_static.py`.
|
||||||
Offline real pre-launch validation and private Compose forwarding fixtures:
|
Offline real pre-launch validation and private Compose forwarding fixtures:
|
||||||
@@ -81,9 +223,11 @@ Isolated, accelerated shell supervision fixtures (no network):
|
|||||||
Compose forwards `ZAPRET_AUTOTUNE_YOUTUBE_SERVICE` (`0` or `1`, default `0`),
|
Compose forwards `ZAPRET_AUTOTUNE_YOUTUBE_SERVICE` (`0` or `1`, default `0`),
|
||||||
`ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE` (`cold` or `adaptive`, default `cold`),
|
`ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE` (`cold` or `adaptive`, default `cold`),
|
||||||
`ZAPRET_AUTOTUNE_VIDEO_ID` (default `dQw4w9WgXcQ`) and
|
`ZAPRET_AUTOTUNE_VIDEO_ID` (default `dQw4w9WgXcQ`) and
|
||||||
`ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS` (JSON array of 1..5 unique VideoIDs;
|
`ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS` (optional JSON array of 1..5 unique VideoIDs).
|
||||||
default only the primary ID). Explicit empty/invalid settings are rejected,
|
Missing/empty inventory selects automatic service-mode samples: primary plus
|
||||||
not silently replaced. Adaptive requires service `1`. See `.env.example`.
|
`aqz-KE-bpKQ` and `jNQXAC9IVRw`, deduplicated. A nonempty explicit inventory
|
||||||
|
overrides those defaults, including an intentional single video. Invalid nonempty
|
||||||
|
settings are rejected. Adaptive requires service `1`. See `.env.example`.
|
||||||
Service `0` preserves legacy Player API/exact-host behavior.
|
Service `0` preserves legacy Player API/exact-host behavior.
|
||||||
|
|
||||||
Adaptive qualification is **post-warmup**, distinct from cold qualification;
|
Adaptive qualification is **post-warmup**, distinct from cold qualification;
|
||||||
@@ -96,18 +240,72 @@ asset checks cover sampled thumbnail/avatar routes, not the whole thumbnail pool
|
|||||||
The current service/adaptive hardware acceptance gate is **pending**; the old
|
The current service/adaptive hardware acceptance gate is **pending**; the old
|
||||||
clean-run PASS below does not qualify these new modes.
|
clean-run PASS below does not qualify these new modes.
|
||||||
|
|
||||||
Optional `SS_DNS_SERVER` is an unbracketed literal IPv4/IPv6 address only (for
|
**Managed DNS defaults to DoH**, independently of service/YouTube mode. PID 1
|
||||||
example `8.8.8.8` or `2001:4860:4860::8888`); empty preserves existing resolver
|
starts the stdlib forwarder inside the existing Go binary, with no dependency
|
||||||
behavior. No domain, port, zone, control characters or option strings are
|
or new container, on container loopback `127.0.0.1:5353` (UDP and
|
||||||
accepted. Validation runs before service launch and reports opaque `configerror`
|
TCP), validates its private PID readiness acknowledgement, then starts ss-server
|
||||||
without echoing input. DNS applies only to this target's native `ss-server`.
|
with `-d 127.0.0.1:5353`. No DNS port is published. Both client hostname traffic
|
||||||
The image build checks installed `ss-server -h` for native `-d <addr>` support;
|
and managed checks share this encrypted resolver path; it works for arbitrary
|
||||||
the validated nonsecret address is passed as a separate `-d` argv value, not
|
services, not only YouTube. HTTPS uses normal certificate verification, direct
|
||||||
eval or an assumed JSON key. Passwords still go only to private JSON configs.
|
literal-IP bootstrap and socket mark `0x50000000` excluding live/probe NFQUEUE rules;
|
||||||
The reported `8.8.8.8` diagnostic corrected a tampered avatar response, but does
|
there is no system-DNS, proxy or plaintext fallback, or recursive SOCKS resolution.
|
||||||
not fix/prove all thumbnail routes. Native DNS correction is **not full asset
|
The DoH resolver sees queries directly: they are not tunneled through SS.
|
||||||
availability proof**. No DoH bridge, permanent Google IP pins or global host DNS
|
Only the internal loopback5353 helper exchange is plaintext UDP/TCP. Endpoint errors
|
||||||
changes are supplied. PID 1 remains the sole startup/recovery owner.
|
return SERVFAIL while the forwarder stays alive, rather than switching DNS modes.
|
||||||
|
|
||||||
|
The private `.env` controls `ZAPRET_AUTOTUNE_DNS_MODE=doh|native`,
|
||||||
|
`ZAPRET_AUTOTUNE_DOH_URL` (default `https://dns.google/dns-query`),
|
||||||
|
`ZAPRET_AUTOTUNE_DOH_BOOTSTRAP_IPS` (missing/empty selects `8.8.8.8,8.8.4.4`
|
||||||
|
**only for the exact default Google URL**; otherwise explicit 1–4 literal
|
||||||
|
IPs, comma-separated without spaces), and `ZAPRET_AUTOTUNE_DOH_TIMEOUT_SEC`
|
||||||
|
(default `5`, integer 1–30). Changing provider requires changing **both** URL
|
||||||
|
and bootstrap IPs. Compose/example leave bootstrap empty, so changing only the
|
||||||
|
URL cannot silently reuse Google bootstrap. URL must use HTTPS/443 with a path, without userinfo, query
|
||||||
|
or fragment. A path can contain a private provider token: do not print it.
|
||||||
|
Validation reports opaque errors before any services start. The subprocess
|
||||||
|
gets provider configuration through env, never endpoint argv/logging.
|
||||||
|
|
||||||
|
Default public settings in the private `.env`:
|
||||||
|
|
||||||
|
```dotenv
|
||||||
|
ZAPRET_AUTOTUNE_DNS_MODE=doh
|
||||||
|
ZAPRET_AUTOTUNE_DOH_URL=https://dns.google/dns-query
|
||||||
|
ZAPRET_AUTOTUNE_DOH_BOOTSTRAP_IPS=
|
||||||
|
ZAPRET_AUTOTUNE_DOH_TIMEOUT_SEC=5
|
||||||
|
```
|
||||||
|
|
||||||
|
LIVE configuration is
|
||||||
|
`/home/agent/zapret-autotune-cleanops-3fc8563-1791882112/.env` (mode0600).
|
||||||
|
These four key names/defaults are public; credentials, custom provider paths,
|
||||||
|
queries and signed resource URLs are not. URL/bootstrap are never child argv.
|
||||||
|
Changing DNS requires official Compose recreation using that private env under
|
||||||
|
the controlled PID1 lifecycle, not a detached helper or ad hoc child restart.
|
||||||
|
The forwarder bounds queries to 4096 bytes, replies to 65535 bytes, HTTP headers
|
||||||
|
to 8192 bytes and concurrent workers to 16; it adds no DNS cache. No claim is
|
||||||
|
made about caching inside the existing SS resolver.
|
||||||
|
|
||||||
|
Only explicit `ZAPRET_AUTOTUNE_DNS_MODE=native` restores the old resolver
|
||||||
|
behavior, with no forwarder child. In that mode optional `SS_DNS_SERVER` accepts
|
||||||
|
only an unbracketed literal IPv4/IPv6 address; empty retains the base resolver.
|
||||||
|
It is ignored in DoH mode and is **not** a fallback. Native DNS can be plaintext.
|
||||||
|
The pinned ss-server's native `-d <addr>` is used, not an assumed JSON key;
|
||||||
|
passwords still go only to private JSON configs. No extra capability is added;
|
||||||
|
Compose sets container-local `ip_unprivileged_port_start=0` for port 5353.
|
||||||
|
Managed probes use `socks5h://127.0.0.1:1080`: Player, metadata, image,
|
||||||
|
media, recovery and warmup hostnames go through the owned SS listener rather than
|
||||||
|
curl's local DNS. Older private `socks5` overrides are validated and canonicalized
|
||||||
|
internally; credentials and unowned endpoints remain forbidden. Public legacy
|
||||||
|
discovery and root checker local `--resolve` pinning are unchanged.
|
||||||
|
This corrects a DNS path bypass, not a proven cause of earlier peer certificate
|
||||||
|
failures, and does not fix/prove all thumbnail routes. It is **not full asset
|
||||||
|
availability proof**. No permanent service IP pins or global host DNS changes
|
||||||
|
are supplied. Clients must request remote DNS (browser SOCKS5 remote DNS or
|
||||||
|
`curl --socks5-hostname 127.0.0.1:11082` when that is the chosen host port).
|
||||||
|
Client-local resolution and IP-literal requests bypass this hostname DNS path;
|
||||||
|
the bridge does not encrypt unrelated container processes' DNS. IP-transmitting
|
||||||
|
SOCKS clients with local resolution are not covered: use remote DNS/`socks5h`.
|
||||||
|
Supplied LIVE and staged-outage DNS acceptance is recorded above; earlier
|
||||||
|
clean-run PASS is not DoH evidence. PID 1 remains the sole startup/recovery owner.
|
||||||
|
|
||||||
Compose has no automatic restart. PID 1 restarts the entire owned runtime only
|
Compose has no automatic restart. PID 1 restarts the entire owned runtime only
|
||||||
for technical failures, with three starts per rolling six hours (including initial
|
for technical failures, with three starts per rolling six hours (including initial
|
||||||
@@ -158,7 +356,7 @@ use host or shared/container PID namespaces. It does not grant inspection of
|
|||||||
broader host processes. The full service/adaptive hardware gate remains
|
broader host processes. The full service/adaptive hardware gate remains
|
||||||
**pending**, not PASS; this ownership fix alone does not qualify it.
|
**pending**, not PASS; this ownership fix alone does not qualify it.
|
||||||
|
|
||||||
Startup order: ss-server/ss-local → listener 1080 → guarded managed init on
|
Startup order: owned DNS readiness (default DoH) → ss-server/ss-local → listener 1080 → guarded managed init on
|
||||||
fresh state → `managed reconcile` on every startup → target-local `zapret2 start`
|
fresh state → `managed reconcile` on every startup → target-local `zapret2 start`
|
||||||
→ supervised managed monitor. Reconciliation uses the fixed private target DB
|
→ supervised managed monitor. Reconciliation uses the fixed private target DB
|
||||||
and restores an interrupted prepared transaction's qualified last-good bytes
|
and restores an interrupted prepared transaction's qualified last-good bytes
|
||||||
@@ -172,7 +370,13 @@ Any startup command failure/timeout exits 78, never the retry code 75. These
|
|||||||
bounds do not shorten the monitor's 90-minute bootstrap/recovery scan budget.
|
bounds do not shorten the monitor's 90-minute bootstrap/recovery scan budget.
|
||||||
|
|
||||||
SIGTERM/SIGINT cancels and waits for the monitor before stopping target-local
|
SIGTERM/SIGINT cancels and waits for the monitor before stopping target-local
|
||||||
zapret and Shadowsocks children. Nonretry monitor fatal exits stop the
|
zapret and Shadowsocks children, then terminates/joins DNS within five seconds
|
||||||
|
and removes only its anchored, validated readiness inode. PID 1 retains private
|
||||||
|
directory/file descriptors through child death, including SIGKILL. Preexisting,
|
||||||
|
replaced or symlink readiness evidence is preserved and fails closed rather
|
||||||
|
than being deleted for a restart. DNS remains available throughout rollback and
|
||||||
|
SS shutdown. A DNS child crash participates in the existing technical retry
|
||||||
|
budget; upstream outages do not kill that child. Nonretry monitor fatal exits stop the
|
||||||
entrypoint; technical exits and Shadowsocks failures restart from private configs
|
entrypoint; technical exits and Shadowsocks failures restart from private configs
|
||||||
only after safe cleanup. Legacy monitor health is YouTube Player API status
|
only after safe cleanup. Legacy monitor health is YouTube Player API status
|
||||||
through the owned SOCKS, not video playback/SABR; legacy GGC candidates are scoped
|
through the owned SOCKS, not video playback/SABR; legacy GGC candidates are scoped
|
||||||
|
|||||||
@@ -27,12 +27,12 @@ with tempfile.TemporaryDirectory(prefix='target-config-') as tmp:
|
|||||||
ZAPRET_AUTOTUNE_YOUTUBE_SERVICE='0', ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE='cold',
|
ZAPRET_AUTOTUNE_YOUTUBE_SERVICE='0', ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE='cold',
|
||||||
ZAPRET_AUTOTUNE_VIDEO_ID='dQw4w9WgXcQ',
|
ZAPRET_AUTOTUNE_VIDEO_ID='dQw4w9WgXcQ',
|
||||||
ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS='["dQw4w9WgXcQ"]',
|
ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS='["dQw4w9WgXcQ"]',
|
||||||
ZAPRET_AUTOTUNE_SOCKS_PROXY='socks5://127.0.0.1:1080')
|
ZAPRET_AUTOTUNE_SOCKS_PROXY='socks5h://127.0.0.1:1080')
|
||||||
for dns, valid in (('', True), ('8.8.8.8', True), ('2001:4860:4860::8888', True),
|
for dns, valid in (('', True), ('8.8.8.8', True), ('2001:4860:4860::8888', True),
|
||||||
('example.org', False), ('8.8.8.8\n', False), ('-d 8.8.8.8', False)):
|
('example.org', False), ('8.8.8.8\n', False), ('-d 8.8.8.8', False)):
|
||||||
for path in private.iterdir():
|
for path in private.iterdir():
|
||||||
path.unlink()
|
path.unlink()
|
||||||
result = subprocess.run(['sh', str(entry)], env=dict(base, SS_DNS_SERVER=dns),
|
result = subprocess.run(['sh', str(entry)], env=dict(base, ZAPRET_AUTOTUNE_DNS_MODE='native', SS_DNS_SERVER=dns),
|
||||||
capture_output=True, timeout=5)
|
capture_output=True, timeout=5)
|
||||||
assert result.returncode == (0 if valid else 64), result.stderr
|
assert result.returncode == (0 if valid else 64), result.stderr
|
||||||
assert not result.stdout and base['SS_PASSWORD'].encode() not in result.stderr
|
assert not result.stdout and base['SS_PASSWORD'].encode() not in result.stderr
|
||||||
@@ -43,11 +43,41 @@ with tempfile.TemporaryDirectory(prefix='target-config-') as tmp:
|
|||||||
path = private / (name + '.json')
|
path = private / (name + '.json')
|
||||||
assert path.stat().st_mode & 0o777 == 0o600
|
assert path.stat().st_mode & 0o777 == 0o600
|
||||||
assert json.loads(path.read_text())['password'] == base['SS_PASSWORD']
|
assert json.loads(path.read_text())['password'] == base['SS_PASSWORD']
|
||||||
|
for overrides, valid in (({}, True), ({'SS_DNS_SERVER': 'ignored.example'}, True),
|
||||||
|
({'ZAPRET_AUTOTUNE_DOH_BOOTSTRAP_IPS': ''}, True),
|
||||||
|
({'ZAPRET_AUTOTUNE_DNS_MODE': ''}, False),
|
||||||
|
({'ZAPRET_AUTOTUNE_DOH_URL': 'http://secret.invalid/token'}, False),
|
||||||
|
({'ZAPRET_AUTOTUNE_DOH_URL': 'https://secret.invalid/token'}, False),
|
||||||
|
({'ZAPRET_AUTOTUNE_DOH_URL': 'https://secret.invalid/token',
|
||||||
|
'ZAPRET_AUTOTUNE_DOH_BOOTSTRAP_IPS': ''}, False),
|
||||||
|
({'ZAPRET_AUTOTUNE_DOH_URL': 'https://secret.invalid/token',
|
||||||
|
'ZAPRET_AUTOTUNE_DOH_BOOTSTRAP_IPS': '192.0.2.1'}, True),
|
||||||
|
({'ZAPRET_AUTOTUNE_DOH_TIMEOUT_SEC': '31'}, False)):
|
||||||
|
for path in private.iterdir():
|
||||||
|
path.unlink()
|
||||||
|
result = subprocess.run(['sh', str(entry)], env=dict(base, **overrides), capture_output=True, timeout=5)
|
||||||
|
assert result.returncode == (0 if valid else 64), result.stderr
|
||||||
|
assert b'secret.invalid' not in result.stderr and b'token' not in result.stderr
|
||||||
|
assert base['SS_PASSWORD'].encode() not in result.stdout + result.stderr
|
||||||
|
if not valid:
|
||||||
|
assert not list(private.iterdir())
|
||||||
# Render privately, never print the config/password; config does not deploy.
|
# Render privately, never print the config/password; config does not deploy.
|
||||||
compose_env = {k: v for k, v in base.items() if not k.startswith('ZAPRET_AUTOTUNE_')}
|
compose_env = {k: v for k, v in base.items() if not k.startswith('ZAPRET_AUTOTUNE_')}
|
||||||
|
factory_cases = []
|
||||||
for overrides, service, mode, ids in (
|
for overrides, service, mode, ids in (
|
||||||
({}, '0', 'cold', ['dQw4w9WgXcQ']),
|
({}, '0', 'cold', ['dQw4w9WgXcQ']),
|
||||||
({'ZAPRET_AUTOTUNE_VIDEO_ID': 'aqz-KE-bpKQ'}, '0', 'cold', ['aqz-KE-bpKQ']),
|
({'ZAPRET_AUTOTUNE_VIDEO_ID': 'aqz-KE-bpKQ'}, '0', 'cold', ['aqz-KE-bpKQ']),
|
||||||
|
({'ZAPRET_AUTOTUNE_YOUTUBE_SERVICE': '1', 'ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE': 'adaptive'},
|
||||||
|
'1', 'adaptive', ['dQw4w9WgXcQ', 'aqz-KE-bpKQ', 'jNQXAC9IVRw']),
|
||||||
|
({'ZAPRET_AUTOTUNE_YOUTUBE_SERVICE': '1', 'ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE': 'adaptive',
|
||||||
|
'ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS': ''},
|
||||||
|
'1', 'adaptive', ['dQw4w9WgXcQ', 'aqz-KE-bpKQ', 'jNQXAC9IVRw']),
|
||||||
|
({'ZAPRET_AUTOTUNE_YOUTUBE_SERVICE': '1', 'ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE': 'adaptive',
|
||||||
|
'ZAPRET_AUTOTUNE_VIDEO_ID': 'aqz-KE-bpKQ'},
|
||||||
|
'1', 'adaptive', ['aqz-KE-bpKQ', 'jNQXAC9IVRw']),
|
||||||
|
({'ZAPRET_AUTOTUNE_YOUTUBE_SERVICE': '1', 'ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE': 'adaptive',
|
||||||
|
'ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS': '["aaaaaaaaaaa"]'},
|
||||||
|
'1', 'adaptive', ['aaaaaaaaaaa']),
|
||||||
({'ZAPRET_AUTOTUNE_YOUTUBE_SERVICE': '1', 'ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE': 'adaptive',
|
({'ZAPRET_AUTOTUNE_YOUTUBE_SERVICE': '1', 'ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE': 'adaptive',
|
||||||
'ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS': '["dQw4w9WgXcQ","aqz-KE-bpKQ","jNQXAC9IVRw"]',
|
'ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS': '["dQw4w9WgXcQ","aqz-KE-bpKQ","jNQXAC9IVRw"]',
|
||||||
'SS_DNS_SERVER': '8.8.8.8'}, '1', 'adaptive', ['dQw4w9WgXcQ', 'aqz-KE-bpKQ', 'jNQXAC9IVRw']),
|
'SS_DNS_SERVER': '8.8.8.8'}, '1', 'adaptive', ['dQw4w9WgXcQ', 'aqz-KE-bpKQ', 'jNQXAC9IVRw']),
|
||||||
@@ -60,6 +90,42 @@ with tempfile.TemporaryDirectory(prefix='target-config-') as tmp:
|
|||||||
settings = json.loads(result.stdout)['services']['target']['environment']
|
settings = json.loads(result.stdout)['services']['target']['environment']
|
||||||
assert settings['ZAPRET_AUTOTUNE_YOUTUBE_SERVICE'] == service
|
assert settings['ZAPRET_AUTOTUNE_YOUTUBE_SERVICE'] == service
|
||||||
assert settings['ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE'] == mode
|
assert settings['ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE'] == mode
|
||||||
assert json.loads(settings['ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS']) == ids
|
assert settings['ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS'] == overrides.get('ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS', '')
|
||||||
assert settings['SS_DNS_SERVER'] == overrides.get('SS_DNS_SERVER', '')
|
assert settings['SS_DNS_SERVER'] == overrides.get('SS_DNS_SERVER', '')
|
||||||
|
assert settings['ZAPRET_AUTOTUNE_DNS_MODE'] == 'doh'
|
||||||
|
assert settings['ZAPRET_AUTOTUNE_DOH_URL'] == 'https://dns.google/dns-query'
|
||||||
|
assert settings['ZAPRET_AUTOTUNE_DOH_BOOTSTRAP_IPS'] == ''
|
||||||
|
assert settings['ZAPRET_AUTOTUNE_DOH_TIMEOUT_SEC'] == '5'
|
||||||
|
assert settings['ZAPRET_AUTOTUNE_SOCKS_PROXY'] == 'socks5h://127.0.0.1:1080'
|
||||||
|
factory_cases.append({'environment': {k: v for k, v in settings.items()
|
||||||
|
if k.startswith('ZAPRET_AUTOTUNE_')}, 'ids': ids})
|
||||||
|
# Validate the actual rendered DNS environment, not an invented equivalent.
|
||||||
|
# Compose must not inject Google bootstrap into a custom-provider request.
|
||||||
|
for service in ('0', '1'):
|
||||||
|
for provider in ({}, {'ZAPRET_AUTOTUNE_DOH_URL': 'https://secret.invalid/token'}):
|
||||||
|
for bootstrap in ({}, {'ZAPRET_AUTOTUNE_DOH_BOOTSTRAP_IPS': ''},
|
||||||
|
{'ZAPRET_AUTOTUNE_DOH_BOOTSTRAP_IPS': '192.0.2.1'}):
|
||||||
|
overrides = dict(provider, **bootstrap, ZAPRET_AUTOTUNE_YOUTUBE_SERVICE=service)
|
||||||
|
result = subprocess.run(['docker', 'compose', '--env-file', '/dev/null', '-f',
|
||||||
|
str(ROOT / 'deploy/managed-target/compose.yaml'),
|
||||||
|
'config', '--format', 'json'], env=dict(compose_env, **overrides),
|
||||||
|
capture_output=True, timeout=10)
|
||||||
|
assert result.returncode == 0, 'Compose config failed (output suppressed)'
|
||||||
|
settings = json.loads(result.stdout)['services']['target']['environment']
|
||||||
|
assert settings['ZAPRET_AUTOTUNE_DOH_BOOTSTRAP_IPS'] == bootstrap.get('ZAPRET_AUTOTUNE_DOH_BOOTSTRAP_IPS', '')
|
||||||
|
for path in private.iterdir():
|
||||||
|
path.unlink()
|
||||||
|
validated = subprocess.run(['sh', str(entry)], env=dict(env, **settings), capture_output=True, timeout=5)
|
||||||
|
valid = not provider or bool(bootstrap.get('ZAPRET_AUTOTUNE_DOH_BOOTSTRAP_IPS'))
|
||||||
|
assert validated.returncode == (0 if valid else 64), 'Rendered DNS config validation failed'
|
||||||
|
assert b'secret.invalid' not in validated.stderr and b'token' not in validated.stderr
|
||||||
|
assert base['SS_PASSWORD'].encode() not in validated.stdout + validated.stderr
|
||||||
|
if not valid:
|
||||||
|
assert not list(private.iterdir()), 'Invalid DNS config created service configs'
|
||||||
|
# Only sanitized fixture settings cross into Go; never persist the password.
|
||||||
|
fixture = root / 'compose-factory.json'
|
||||||
|
fixture.write_text(json.dumps(factory_cases))
|
||||||
|
subprocess.run(['go', 'test', './cmd/zapret-autotune', '-run', '^TestManagedGooglevideoComposeForwarding$',
|
||||||
|
'-count=1'], cwd=ROOT, env=dict(env, AUTOTUNE_COMPOSE_CONFIG_FIXTURE=str(fixture)),
|
||||||
|
check=True, timeout=60)
|
||||||
print('target config fixtures passed')
|
print('target config fixtures passed')
|
||||||
@@ -15,6 +15,7 @@ assert " image: ${ZAPRET_AUTOTUNE_IMAGE:-zapret-autotune-managed-target:local
|
|||||||
assert " cap_drop: [ALL]\n" in compose
|
assert " cap_drop: [ALL]\n" in compose
|
||||||
assert " cap_add: [NET_ADMIN, NET_RAW, SETUID, SETGID, KILL, SYS_PTRACE]\n" in compose
|
assert " cap_add: [NET_ADMIN, NET_RAW, SETUID, SETGID, KILL, SYS_PTRACE]\n" in compose
|
||||||
assert "pid:" not in compose # Keep inspection confined to the private PID namespace.
|
assert "pid:" not in compose # Keep inspection confined to the private PID namespace.
|
||||||
|
assert 'ZAPRET_AUTOTUNE_SOCKS_PROXY: socks5h://127.0.0.1:1080' in compose
|
||||||
assert compose.count(" - type: bind\n") == 1
|
assert compose.count(" - type: bind\n") == 1
|
||||||
assert " source: ./target-state\n" in compose
|
assert " source: ./target-state\n" in compose
|
||||||
assert " target: /var/lib/zapret-autotune/target\n" in compose
|
assert " target: /var/lib/zapret-autotune/target\n" in compose
|
||||||
@@ -22,14 +23,28 @@ assert " create_host_path: false\n" in compose
|
|||||||
assert "SS_PASSWORD: ${SS_PASSWORD:?Set SS_PASSWORD in the deployment environment}" in compose
|
assert "SS_PASSWORD: ${SS_PASSWORD:?Set SS_PASSWORD in the deployment environment}" in compose
|
||||||
for setting in (
|
for setting in (
|
||||||
'SS_DNS_SERVER: ${SS_DNS_SERVER-}',
|
'SS_DNS_SERVER: ${SS_DNS_SERVER-}',
|
||||||
|
'ZAPRET_AUTOTUNE_DNS_MODE: ${ZAPRET_AUTOTUNE_DNS_MODE-doh}',
|
||||||
|
'ZAPRET_AUTOTUNE_DOH_URL: ${ZAPRET_AUTOTUNE_DOH_URL-https://dns.google/dns-query}',
|
||||||
|
'ZAPRET_AUTOTUNE_DOH_BOOTSTRAP_IPS: ${ZAPRET_AUTOTUNE_DOH_BOOTSTRAP_IPS-}',
|
||||||
|
'ZAPRET_AUTOTUNE_DOH_TIMEOUT_SEC: ${ZAPRET_AUTOTUNE_DOH_TIMEOUT_SEC-5}',
|
||||||
'ZAPRET_AUTOTUNE_YOUTUBE_SERVICE: ${ZAPRET_AUTOTUNE_YOUTUBE_SERVICE-0}',
|
'ZAPRET_AUTOTUNE_YOUTUBE_SERVICE: ${ZAPRET_AUTOTUNE_YOUTUBE_SERVICE-0}',
|
||||||
'ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE: ${ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE-cold}',
|
'ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE: ${ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE-cold}',
|
||||||
'ZAPRET_AUTOTUNE_VIDEO_ID: ${ZAPRET_AUTOTUNE_VIDEO_ID-dQw4w9WgXcQ}',
|
'ZAPRET_AUTOTUNE_VIDEO_ID: ${ZAPRET_AUTOTUNE_VIDEO_ID-dQw4w9WgXcQ}',
|
||||||
"ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS: '${ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS-[\"${ZAPRET_AUTOTUNE_VIDEO_ID-dQw4w9WgXcQ}\"]}'",
|
"ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS: '${ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS:-}'",
|
||||||
):
|
):
|
||||||
assert setting in compose, setting
|
assert setting in compose, setting
|
||||||
assert entrypoint.index('target-config-check || exit 64') < entrypoint.index('cat > "$config_dir/server.json"')
|
assert entrypoint.index('target-config-check || exit 64') < entrypoint.index('cat > "$config_dir/server.json"')
|
||||||
assert 'set -- "$@" -d "$SS_DNS_SERVER"' in entrypoint
|
assert 'set -- "$@" -d "$dns_address"' in entrypoint
|
||||||
|
assert entrypoint.index('start_dns || return $?') < entrypoint.index('set -- ss-server')
|
||||||
|
assert 'dns-forwarder readiness-check "$dns_pid"' in entrypoint
|
||||||
|
assert 'dns-forwarder readiness-cleanup "$dns_pid"' in entrypoint
|
||||||
|
assert 'exec 8<"$config_dir"' in entrypoint
|
||||||
|
assert 'exec 9<"$config_dir/dns-ready"' in entrypoint
|
||||||
|
assert 'rm -f "$config_dir/dns-ready"' not in entrypoint
|
||||||
|
assert 'while dns_alive && kill -0 "$server_pid"' in entrypoint
|
||||||
|
assert entrypoint.index('bounded_wait "$pid" 5') < entrypoint.index('bounded_wait "$dns_pid" 5')
|
||||||
|
assert '5353' not in compose
|
||||||
|
assert 'net.ipv4.ip_unprivileged_port_start: "0"' in compose
|
||||||
assert 'eval ' not in entrypoint
|
assert 'eval ' not in entrypoint
|
||||||
assert "secrets:" not in compose
|
assert "secrets:" not in compose
|
||||||
assert "ss_password=${SS_PASSWORD-}" in entrypoint
|
assert "ss_password=${SS_PASSWORD-}" in entrypoint
|
||||||
@@ -44,21 +59,24 @@ assert " stop_grace_period: 240s\n" in compose
|
|||||||
assert 'bounded_wait "$monitor_pid" 45' in entrypoint
|
assert 'bounded_wait "$monitor_pid" 45' in entrypoint
|
||||||
assert '/bin/sh /opt/managed-target-supervision.sh --watchdog' in entrypoint
|
assert '/bin/sh /opt/managed-target-supervision.sh --watchdog' in entrypoint
|
||||||
assert 'setsid /bin/sh /opt/managed-target-supervision.sh --monitor' in entrypoint
|
assert 'setsid /bin/sh /opt/managed-target-supervision.sh --monitor' in entrypoint
|
||||||
assert entrypoint.index('if ! cleanup_session; then') < entrypoint.index('if timeout -k 5 45 /usr/local/bin/zapret-autotune managed reconcile; then')
|
assert entrypoint.index('if ! cleanup_session; then') < entrypoint.index('if cleanup_command 45 /usr/local/bin/zapret-autotune managed reconcile; then')
|
||||||
assert 'COPY managed-target-supervision.sh /opt/managed-target-supervision.sh' in (directory.parents[1] / 'Dockerfile.target').read_text()
|
assert 'COPY managed-target-supervision.sh /opt/managed-target-supervision.sh' in (directory.parents[1] / 'Dockerfile.target').read_text()
|
||||||
assert '!managed-target-supervision.sh' in (directory.parents[1] / '.dockerignore').read_text()
|
assert '!managed-target-supervision.sh' in (directory.parents[1] / '.dockerignore').read_text()
|
||||||
assert '"$count" -lt 3' in entrypoint
|
assert '"$count" -ge 3' in entrypoint
|
||||||
|
assert 'BLOCKED: generic target restart budget; checks paused' in entrypoint
|
||||||
assert '"$((now - started))" -lt 21600' in entrypoint
|
assert '"$((now - started))" -lt 21600' in entrypoint
|
||||||
assert "backoff=15" in entrypoint
|
assert "backoff=15" in entrypoint
|
||||||
assert "systemctl" not in entrypoint and "docker " not in entrypoint
|
assert "systemctl" not in entrypoint and "docker " not in entrypoint
|
||||||
assert "timeout -k 5 30 /opt/zapret2/init.d/sysv/zapret2 stop" in entrypoint
|
assert "cleanup_command 30 /opt/zapret2/init.d/sysv/zapret2 stop" in entrypoint
|
||||||
|
assert 'generic_cleanup_deadline=$(($(date +%s) + 80))' in entrypoint
|
||||||
|
assert 'timeout -k 1 "$cleanup_command_cap" "$@"' in entrypoint
|
||||||
for command, seconds in (("/usr/local/bin/zapret-autotune managed init", 45),
|
for command, seconds in (("/usr/local/bin/zapret-autotune managed init", 45),
|
||||||
("/usr/local/bin/zapret-autotune managed reconcile", 45),
|
("/usr/local/bin/zapret-autotune managed reconcile", 45),
|
||||||
("/opt/zapret2/init.d/sysv/zapret2 start", 30)):
|
("/opt/zapret2/init.d/sysv/zapret2 start", 30)):
|
||||||
assert f"timeout -k 5 {seconds} {command} || exit 78" in entrypoint
|
assert f"timeout -k 5 {seconds} {command} || return 78" in entrypoint
|
||||||
assert entrypoint.index("managed reconcile || exit 78") < entrypoint.index("zapret2 start || exit 78")
|
assert entrypoint.index("managed reconcile || return 78") < entrypoint.index("zapret2 start || return 78")
|
||||||
experimental = 'managed apply --googlevideo-experiment native-circular --googlevideo-scope experimental_googlevideo_suffix --googlevideo-request startup || exit 78'
|
experimental = 'managed apply --googlevideo-experiment native-circular --googlevideo-scope experimental_googlevideo_suffix --googlevideo-request startup || exit 78'
|
||||||
assert entrypoint.index('zapret2 start || exit 78') < entrypoint.index(experimental)
|
assert entrypoint.index('zapret2 start || return 78') < entrypoint.index(experimental)
|
||||||
assert entrypoint.index(experimental) < entrypoint.index('setsid /bin/sh /opt/managed-target-supervision.sh --monitor')
|
assert entrypoint.index(experimental) < entrypoint.index('setsid /bin/sh /opt/managed-target-supervision.sh --monitor')
|
||||||
assert entrypoint.index('managed experimental-request-check || exit 78') < entrypoint.index('managed init || exit 78')
|
assert entrypoint.index('managed experimental-request-check || return 78') < entrypoint.index('managed init || return 78')
|
||||||
assert 'rm "${ZAPRET_CONFIG%/*}/.autotune-experimental-suffix-request"' not in entrypoint
|
assert 'rm "${ZAPRET_CONFIG%/*}/.autotune-experimental-suffix-request"' not in entrypoint
|
||||||
@@ -4,6 +4,7 @@ Only a temporary COPY of the entrypoint is redirected to fake executables;
|
|||||||
production has no test switches, real network probes, or accelerated timers.
|
production has no test switches, real network probes, or accelerated timers.
|
||||||
"""
|
"""
|
||||||
import ctypes
|
import ctypes
|
||||||
|
import atexit
|
||||||
import os
|
import os
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
import shutil
|
import shutil
|
||||||
@@ -13,6 +14,15 @@ import time
|
|||||||
|
|
||||||
|
|
||||||
ROOT = Path(__file__).resolve().parents[2]
|
ROOT = Path(__file__).resolve().parents[2]
|
||||||
|
# Real readiness CLI checks root ownership. Use an isolated user namespace for
|
||||||
|
# offline fixtures when invoked as an ordinary user; no host-root requirement.
|
||||||
|
if os.geteuid() != 0:
|
||||||
|
os.execvp('unshare', ['unshare', '-Ur', 'python3', str(Path(__file__).resolve())])
|
||||||
|
CLI_TMP = tempfile.TemporaryDirectory(prefix='dns-readiness-cli-')
|
||||||
|
atexit.register(CLI_TMP.cleanup)
|
||||||
|
DNS_CLI = Path(CLI_TMP.name) / 'autotune'
|
||||||
|
subprocess.run(['go', 'build', '-o', str(DNS_CLI), './cmd/zapret-autotune'],
|
||||||
|
cwd=ROOT, env=dict(os.environ, GOPROXY='off', GOTOOLCHAIN='local'), check=True)
|
||||||
FAKE = r'''#!/usr/bin/env python3
|
FAKE = r'''#!/usr/bin/env python3
|
||||||
import ctypes, os, pathlib, signal, subprocess, sys, time
|
import ctypes, os, pathlib, signal, subprocess, sys, time
|
||||||
root = pathlib.Path(os.environ['FIXTURE'])
|
root = pathlib.Path(os.environ['FIXTURE'])
|
||||||
@@ -27,7 +37,11 @@ def increment(key):
|
|||||||
path.write_text(str(n))
|
path.write_text(str(n))
|
||||||
return n
|
return n
|
||||||
def linger(code=0):
|
def linger(code=0):
|
||||||
signal.signal(signal.SIGTERM, lambda *_: sys.exit(code))
|
def stop(*_):
|
||||||
|
if os.environ.get('ZAPRET_AUTOTUNE_DNS_MODE', 'doh') == 'doh' and scenario not in ('dns-death', 'dns-killed', 'dns-replace', 'dns-replace-symlink', 'dns-prestart'):
|
||||||
|
os.kill(int((root / 'dns-pid').read_text()), 0)
|
||||||
|
sys.exit(code)
|
||||||
|
signal.signal(signal.SIGTERM, stop)
|
||||||
while True:
|
while True:
|
||||||
time.sleep(.01)
|
time.sleep(.01)
|
||||||
def assert_workers_gone():
|
def assert_workers_gone():
|
||||||
@@ -53,10 +67,19 @@ if name == 'awk':
|
|||||||
sys.exit(0)
|
sys.exit(0)
|
||||||
if name == 'date':
|
if name == 'date':
|
||||||
n = increment('clock')
|
n = increment('clock')
|
||||||
print(100000 + (n * 21601 if scenario == 'window' else n))
|
print(100000 + (n * 21601 if scenario == 'window' else n + (21601 if (root / 'budget-pause').exists() else 0)))
|
||||||
sys.exit(0)
|
sys.exit(0)
|
||||||
if name == 'sleep':
|
if name == 'sleep':
|
||||||
seconds = int(sys.argv[1])
|
seconds = int(sys.argv[1])
|
||||||
|
if scenario.startswith('generic-budget-') and seconds != 60:
|
||||||
|
os.execv(os.environ['REAL_SLEEP'], [os.environ['REAL_SLEEP'], *sys.argv[1:]])
|
||||||
|
if seconds == 60 and scenario in ('generic-ss', 'generic-unsafe', 'generic-budget-monitor', 'generic-budget-reconcile', 'generic-budget-ack'):
|
||||||
|
log('generic-pause')
|
||||||
|
(root / 'budget-pause').touch()
|
||||||
|
if scenario != 'generic-ss':
|
||||||
|
os.kill(os.getppid(), signal.SIGTERM)
|
||||||
|
time.sleep(.01)
|
||||||
|
sys.exit(0)
|
||||||
if seconds in (15, 30):
|
if seconds in (15, 30):
|
||||||
log('cooldown ' + str(seconds))
|
log('cooldown ' + str(seconds))
|
||||||
if scenario == 'cancel':
|
if scenario == 'cancel':
|
||||||
@@ -74,13 +97,21 @@ if name in ('ss-server', 'ss-local'):
|
|||||||
f.write(str(os.getpid()) + '\n')
|
f.write(str(os.getpid()) + '\n')
|
||||||
assert 'SS_PASSWORD' not in os.environ
|
assert 'SS_PASSWORD' not in os.environ
|
||||||
assert sys.argv[1] == '-c'
|
assert sys.argv[1] == '-c'
|
||||||
dns = os.environ.get('SS_DNS_SERVER', '') if name == 'ss-server' else ''
|
dns = ('127.0.0.1:5353' if os.environ.get('ZAPRET_AUTOTUNE_DNS_MODE', 'doh') == 'doh'
|
||||||
|
else os.environ.get('SS_DNS_SERVER', '')) if name == 'ss-server' else ''
|
||||||
assert sys.argv[3:] == (['-d', dns] if dns else [])
|
assert sys.argv[3:] == (['-d', dns] if dns else [])
|
||||||
|
if dns == '127.0.0.1:5353':
|
||||||
|
assert (root / 'private' / 'dns-ready').read_text().strip() == (root / 'dns-pid').read_text()
|
||||||
|
os.kill(int((root / 'dns-pid').read_text()), 0)
|
||||||
assert (pathlib.Path(sys.argv[2]).stat().st_mode & 0o777) == 0o600
|
assert (pathlib.Path(sys.argv[2]).stat().st_mode & 0o777) == 0o600
|
||||||
log(name)
|
log(name)
|
||||||
if name == 'ss-local' and scenario in ('ss', 'mixed', 'joinfail') and increment('ss_runs') == 1:
|
if scenario.startswith('generic-budget-'):
|
||||||
|
signal.signal(signal.SIGTERM, signal.SIG_IGN)
|
||||||
|
while True:
|
||||||
|
time.sleep(.01)
|
||||||
|
if name == 'ss-local' and scenario in ('ss', 'mixed', 'joinfail', 'generic-ss') and increment('ss_runs') <= (3 if scenario == 'generic-ss' else 1):
|
||||||
# Let startup/monitor begin before simulating a child crash.
|
# Let startup/monitor begin before simulating a child crash.
|
||||||
while not (root / 'monitor_runs').exists():
|
while not (root / 'monitor_runs').exists() or (scenario == 'generic-ss' and int((root / 'monitor_runs').read_text()) < int((root / 'ss_runs').read_text())):
|
||||||
time.sleep(.01)
|
time.sleep(.01)
|
||||||
time.sleep(.03)
|
time.sleep(.03)
|
||||||
sys.exit(137)
|
sys.exit(137)
|
||||||
@@ -96,6 +127,73 @@ if name == 'zapret2':
|
|||||||
assert name == 'autotune'
|
assert name == 'autotune'
|
||||||
if sys.argv[1:] == ['target-config-check']:
|
if sys.argv[1:] == ['target-config-check']:
|
||||||
sys.exit(0)
|
sys.exit(0)
|
||||||
|
if sys.argv[1] == 'dns-forwarder':
|
||||||
|
if sys.argv[2] in ('readiness-check', 'readiness-cleanup'):
|
||||||
|
# exec preserves PID1's actual inherited anchors; no simulated unlink.
|
||||||
|
os.execv(os.environ['DNS_CLI'], [os.environ['DNS_CLI'], *sys.argv[1:]])
|
||||||
|
mode = os.environ.get('ZAPRET_AUTOTUNE_DNS_MODE', 'doh')
|
||||||
|
if sys.argv[2] == 'mode':
|
||||||
|
print(mode)
|
||||||
|
sys.exit(0)
|
||||||
|
if sys.argv[2] == 'address':
|
||||||
|
print('127.0.0.1:5353' if mode == 'doh' else os.environ.get('SS_DNS_SERVER', ''))
|
||||||
|
sys.exit(0)
|
||||||
|
assert sys.argv[2:] == ['serve'] and mode == 'doh'
|
||||||
|
assert 'SS_PASSWORD' not in os.environ
|
||||||
|
with (root / 'children').open('a') as f:
|
||||||
|
f.write(str(os.getpid()) + '\n')
|
||||||
|
(root / 'dns-pid').write_text(str(os.getpid()))
|
||||||
|
log('dns-start')
|
||||||
|
if scenario == 'dns-early':
|
||||||
|
sys.exit(1)
|
||||||
|
ready = root / 'private' / 'dns-ready'
|
||||||
|
owned = None
|
||||||
|
if scenario == 'dns-preack-foreign':
|
||||||
|
ready.write_text('foreign evidence\n')
|
||||||
|
sys.exit(1)
|
||||||
|
if scenario not in ('dns-noack', 'dns-term', 'dns-int'):
|
||||||
|
with ready.open('x') as f:
|
||||||
|
os.chmod(ready, 0o600)
|
||||||
|
f.write(str(os.getpid() + (1 if scenario == 'dns-wrongack' else 0)) + '\n')
|
||||||
|
owned = ready.stat()
|
||||||
|
if scenario == 'dns-permissions':
|
||||||
|
ready.chmod(0o644)
|
||||||
|
if scenario == 'dns-symlink':
|
||||||
|
payload = ready.with_name('dns-payload')
|
||||||
|
ready.rename(payload)
|
||||||
|
ready.symlink_to(payload)
|
||||||
|
log('dns-ready')
|
||||||
|
def stop_dns(*_):
|
||||||
|
assert not (root / 'prepared').exists(), 'DNS stopped before restoration'
|
||||||
|
# SS children have been joined, not merely signalled.
|
||||||
|
for pid in (root / 'children').read_text().splitlines():
|
||||||
|
if int(pid) != os.getpid():
|
||||||
|
assert not pathlib.Path('/proc/' + pid).exists(), ('DNS stopped before child join', pid)
|
||||||
|
log('dns-stop')
|
||||||
|
if ready.exists() and not ready.is_symlink() and owned and os.path.samestat(owned, ready.stat()):
|
||||||
|
ready.unlink()
|
||||||
|
sys.exit(0)
|
||||||
|
signal.signal(signal.SIGTERM, stop_dns)
|
||||||
|
if scenario.startswith('generic-budget-'):
|
||||||
|
signal.signal(signal.SIGTERM, signal.SIG_IGN)
|
||||||
|
if scenario in ('dns-term', 'dns-int'):
|
||||||
|
os.kill(os.getppid(), signal.SIGTERM if scenario == 'dns-term' else signal.SIGINT)
|
||||||
|
if scenario in ('dns-death', 'dns-killed', 'dns-replace', 'dns-replace-symlink') and increment('dns_runs') == 1:
|
||||||
|
while not (root / 'monitor_runs').exists():
|
||||||
|
time.sleep(.01)
|
||||||
|
if scenario.startswith('dns-replace'):
|
||||||
|
# Retain old inode so a replacement cannot reuse its number.
|
||||||
|
ready.rename(ready.with_name('retired-ready'))
|
||||||
|
payload = ready.with_name('foreign-ready')
|
||||||
|
payload.write_text('foreign evidence\n')
|
||||||
|
payload.chmod(0o600)
|
||||||
|
if scenario == 'dns-replace-symlink':
|
||||||
|
ready.symlink_to(payload)
|
||||||
|
else:
|
||||||
|
payload.rename(ready)
|
||||||
|
os.kill(os.getpid(), signal.SIGKILL)
|
||||||
|
while True:
|
||||||
|
time.sleep(.01)
|
||||||
command = sys.argv[2]
|
command = sys.argv[2]
|
||||||
if command == 'init':
|
if command == 'init':
|
||||||
log('init')
|
log('init')
|
||||||
@@ -103,19 +201,45 @@ if command == 'init':
|
|||||||
elif command == 'reconcile':
|
elif command == 'reconcile':
|
||||||
n = increment('reconciles')
|
n = increment('reconciles')
|
||||||
log('reconcile')
|
log('reconcile')
|
||||||
|
if scenario.startswith('generic-budget-') and n == 2:
|
||||||
|
with (root / 'children').open('a') as f:
|
||||||
|
f.write(str(os.getpid()) + '\n')
|
||||||
|
signal.signal(signal.SIGTERM, signal.SIG_IGN)
|
||||||
|
while True:
|
||||||
|
time.sleep(.01)
|
||||||
if scenario == 'orphans':
|
if scenario == 'orphans':
|
||||||
assert_workers_gone()
|
assert_workers_gone()
|
||||||
if scenario == 'prestartfail':
|
if scenario in ('prestartfail', 'generic-unsafe'):
|
||||||
sys.exit(78)
|
sys.exit(78)
|
||||||
|
if scenario == 'dns-prestart':
|
||||||
|
os.kill(int((root / 'dns-pid').read_text()), signal.SIGKILL)
|
||||||
|
time.sleep(.03)
|
||||||
if scenario in ('unknown', 'unknown75') and n == 2:
|
if scenario in ('unknown', 'unknown75') and n == 2:
|
||||||
sys.exit(78)
|
sys.exit(78)
|
||||||
(root / 'prepared').unlink(missing_ok=True)
|
(root / 'prepared').unlink(missing_ok=True)
|
||||||
elif command == 'monitor':
|
elif command == 'monitor':
|
||||||
|
if scenario.startswith('generic-'):
|
||||||
|
assert os.environ['ZAPRET_AUTOTUNE_RESOURCE_CONFIG'] == '/operator/resources.json'
|
||||||
with (root / 'children').open('a') as f:
|
with (root / 'children').open('a') as f:
|
||||||
f.write(str(os.getpid()) + '\n')
|
f.write(str(os.getpid()) + '\n')
|
||||||
n = increment('monitor_runs')
|
n = increment('monitor_runs')
|
||||||
log('monitor ' + str(n))
|
log('monitor ' + str(n))
|
||||||
assert os.getsid(0) == os.getpid(), 'monitor not in its own verified session'
|
assert os.getsid(0) == os.getpid(), 'monitor not in its own verified session'
|
||||||
|
if scenario.startswith('generic-budget-'):
|
||||||
|
time.sleep(.03)
|
||||||
|
os.kill(os.getppid(), signal.SIGTERM)
|
||||||
|
if scenario == 'generic-budget-monitor':
|
||||||
|
signal.signal(signal.SIGTERM, signal.SIG_IGN)
|
||||||
|
while True:
|
||||||
|
time.sleep(.01)
|
||||||
|
linger()
|
||||||
|
if scenario == 'generic-ss' and n <= 3:
|
||||||
|
linger()
|
||||||
|
if scenario in ('dns-death', 'dns-killed', 'dns-replace', 'dns-replace-symlink') and n == 1:
|
||||||
|
linger()
|
||||||
|
if scenario == 'dns-outage':
|
||||||
|
# Upstream SERVFAIL is modelled by a live DNS child, not child failure.
|
||||||
|
assert (root / 'dns-pid').exists()
|
||||||
if n == 1 and scenario == 'orphans':
|
if n == 1 and scenario == 'orphans':
|
||||||
subprocess.Popen([str(root / 'bin' / 'blockcheck')], preexec_fn=os.setpgrp)
|
subprocess.Popen([str(root / 'bin' / 'blockcheck')], preexec_fn=os.setpgrp)
|
||||||
while not (root / 'workers').exists() or len((root / 'workers').read_text().splitlines()) != 2:
|
while not (root / 'workers').exists() or len((root / 'workers').read_text().splitlines()) != 2:
|
||||||
@@ -144,7 +268,7 @@ elif command == 'monitor':
|
|||||||
'''
|
'''
|
||||||
|
|
||||||
|
|
||||||
def run(scenario, expected, dns=''):
|
def run(scenario, expected, dns='', mode='native'):
|
||||||
with tempfile.TemporaryDirectory(prefix='supervision-') as tmp:
|
with tempfile.TemporaryDirectory(prefix='supervision-') as tmp:
|
||||||
root = Path(tmp)
|
root = Path(tmp)
|
||||||
binary = root / 'bin'
|
binary = root / 'bin'
|
||||||
@@ -154,23 +278,49 @@ def run(scenario, expected, dns=''):
|
|||||||
fake.chmod(0o755)
|
fake.chmod(0o755)
|
||||||
for name in ('awk', 'date', 'sleep', 'timeout', 'ss-server', 'ss-local', 'autotune', 'zapret2', 'blockcheck', 'nfq'):
|
for name in ('awk', 'date', 'sleep', 'timeout', 'ss-server', 'ss-local', 'autotune', 'zapret2', 'blockcheck', 'nfq'):
|
||||||
(binary / name).symlink_to(fake)
|
(binary / name).symlink_to(fake)
|
||||||
|
realtime = scenario.startswith('generic-budget-')
|
||||||
|
if realtime:
|
||||||
|
for name in ('date', 'timeout'):
|
||||||
|
(binary / name).unlink()
|
||||||
configs = root / 'private'
|
configs = root / 'private'
|
||||||
configs.mkdir()
|
configs.mkdir()
|
||||||
|
if scenario.startswith('dns-stale'):
|
||||||
|
if scenario == 'dns-stale-symlink':
|
||||||
|
(configs / 'dns-ready').symlink_to(configs / 'missing')
|
||||||
|
else:
|
||||||
|
(configs / 'dns-ready').write_text('foreign evidence\n')
|
||||||
text = (ROOT / 'managed-target-entrypoint.sh').read_text()
|
text = (ROOT / 'managed-target-entrypoint.sh').read_text()
|
||||||
helper = ROOT / 'managed-target-supervision.sh'
|
helper = ROOT / 'managed-target-supervision.sh'
|
||||||
if scenario == 'ackfail':
|
if scenario == 'ackfail':
|
||||||
broken_helper = root / 'broken-helper.sh'
|
broken_helper = root / 'broken-helper.sh'
|
||||||
broken_helper.write_text(helper.read_text().replace('sleep "$seconds" & sleeper=$!', 'exit 78'))
|
broken_helper.write_text(helper.read_text().replace('sleep "$seconds" & sleeper=$!', 'exit 78'))
|
||||||
helper = broken_helper
|
helper = broken_helper
|
||||||
|
if scenario == 'generic-budget-ack':
|
||||||
|
delayed_helper = root / 'delayed-helper.sh'
|
||||||
|
delayed_helper.write_text(helper.read_text().replace(': > "$config_dir/join-ready"',
|
||||||
|
f'{shutil.which("sleep")} 300\n : > "$config_dir/join-ready"'))
|
||||||
|
helper = delayed_helper
|
||||||
text = text.replace('/opt/managed-target-supervision.sh', str(helper))
|
text = text.replace('/opt/managed-target-supervision.sh', str(helper))
|
||||||
text = text.replace('config_dir=/run/zapret-autotune-ss', f'config_dir={configs}')
|
text = text.replace('config_dir=/run/zapret-autotune-ss', f'config_dir={configs}')
|
||||||
text = text.replace('/usr/local/bin/zapret-autotune', str(binary / 'autotune'))
|
text = text.replace('/usr/local/bin/zapret-autotune', str(binary / 'autotune'))
|
||||||
text = text.replace('/opt/zapret2/init.d/sysv/zapret2', str(binary / 'zapret2'))
|
text = text.replace('/opt/zapret2/init.d/sysv/zapret2', str(binary / 'zapret2'))
|
||||||
|
if realtime:
|
||||||
|
# Private fixture scaling only: production has no configurable grace.
|
||||||
|
text = text.replace('$(date +%s) + 80', '$(date +%s) + 12')
|
||||||
entry = root / 'entry.sh'
|
entry = root / 'entry.sh'
|
||||||
entry.write_text(text)
|
entry.write_text(text)
|
||||||
env = dict(os.environ, PATH=f'{binary}:{os.environ["PATH"]}', FIXTURE=tmp,
|
env = dict(os.environ, PATH=f'{binary}:{os.environ["PATH"]}', FIXTURE=tmp,
|
||||||
SCENARIO=scenario, SS_PASSWORD='ab' * 32, SS_DNS_SERVER=dns,
|
SCENARIO=scenario, SS_PASSWORD='ab' * 32, SS_DNS_SERVER=dns,
|
||||||
ZAPRET_CONFIG=str(root / 'config'))
|
ZAPRET_CONFIG=str(root / 'config'), DNS_CLI=str(DNS_CLI),
|
||||||
|
REAL_SLEEP=shutil.which('sleep'))
|
||||||
|
if scenario.startswith('generic-'):
|
||||||
|
env['ZAPRET_AUTOTUNE_RESOURCE_CONFIG'] = '/operator/resources.json'
|
||||||
|
else:
|
||||||
|
env.pop('ZAPRET_AUTOTUNE_RESOURCE_CONFIG', None)
|
||||||
|
if mode is None:
|
||||||
|
env.pop('ZAPRET_AUTOTUNE_DNS_MODE', None)
|
||||||
|
else:
|
||||||
|
env['ZAPRET_AUTOTUNE_DNS_MODE'] = mode
|
||||||
peer = None
|
peer = None
|
||||||
if scenario == 'orphans':
|
if scenario == 'orphans':
|
||||||
# Local fixture is not PID 1. Adopt orphan workers here so the test
|
# Local fixture is not PID 1. Adopt orphan workers here so the test
|
||||||
@@ -179,17 +329,20 @@ def run(scenario, expected, dns=''):
|
|||||||
peer = subprocess.Popen([shutil.which('sleep'), '300'], start_new_session=True)
|
peer = subprocess.Popen([shutil.which('sleep'), '300'], start_new_session=True)
|
||||||
(root / 'peer').write_text(str(peer.pid))
|
(root / 'peer').write_text(str(peer.pid))
|
||||||
try:
|
try:
|
||||||
result = subprocess.run(['sh', str(entry)], env=env, capture_output=True, timeout=15)
|
started = time.monotonic()
|
||||||
|
result = subprocess.run(['sh', str(entry)], env=env, capture_output=True, timeout=18 if realtime else 15)
|
||||||
|
if realtime:
|
||||||
|
assert time.monotonic() - started < 16, (scenario, 'aggregate cleanup exceeded grace')
|
||||||
if peer:
|
if peer:
|
||||||
assert peer.poll() is None, 'unowned helper was killed'
|
assert peer.poll() is None, 'unowned helper was killed'
|
||||||
finally:
|
finally:
|
||||||
if peer:
|
if peer:
|
||||||
peer.terminate()
|
peer.terminate()
|
||||||
peer.wait(timeout=2)
|
peer.wait(timeout=2)
|
||||||
events = (root / 'events').read_text().splitlines()
|
events = (root / 'events').read_text().splitlines() if (root / 'events').exists() else []
|
||||||
assert result.returncode == expected, (scenario, result.returncode, result.stderr, events)
|
assert result.returncode == expected, (scenario, result.returncode, result.stderr, events)
|
||||||
assert b'ab' * 32 not in result.stdout + result.stderr
|
assert b'ab' * 32 not in result.stdout + result.stderr
|
||||||
for pid in (root / 'children').read_text().splitlines():
|
for pid in ((root / 'children').read_text().splitlines() if (root / 'children').exists() else []):
|
||||||
assert not Path(f'/proc/{pid}').exists(), (scenario, 'child not joined', pid)
|
assert not Path(f'/proc/{pid}').exists(), (scenario, 'child not joined', pid)
|
||||||
if scenario == 'orphans':
|
if scenario == 'orphans':
|
||||||
for pid in (root / 'workers').read_text().splitlines():
|
for pid in (root / 'workers').read_text().splitlines():
|
||||||
@@ -205,6 +358,19 @@ def run(scenario, expected, dns=''):
|
|||||||
for i, event in enumerate(events):
|
for i, event in enumerate(events):
|
||||||
if event in ('start', 'stop'):
|
if event in ('start', 'stop'):
|
||||||
assert events[i - 1] == 'reconcile', (scenario, events)
|
assert events[i - 1] == 'reconcile', (scenario, events)
|
||||||
|
foreign = scenario in ('dns-symlink', 'dns-preack-foreign', 'dns-replace', 'dns-replace-symlink', 'dns-stale', 'dns-stale-symlink')
|
||||||
|
if foreign:
|
||||||
|
ready = configs / 'dns-ready'
|
||||||
|
assert ready.exists() or ready.is_symlink(), (scenario, 'foreign readiness deleted')
|
||||||
|
if scenario != 'dns-stale-symlink' and scenario != 'dns-symlink':
|
||||||
|
assert ready.read_text() == 'foreign evidence\n', (scenario, 'foreign bytes changed')
|
||||||
|
elif realtime and (configs / 'dns-ready').exists():
|
||||||
|
# Exhaustion cannot authorize unlinking without the owned Go helper.
|
||||||
|
assert (configs / 'dns-ready').read_text().strip() == (root / 'dns-pid').read_text()
|
||||||
|
else:
|
||||||
|
assert not (configs / 'dns-ready').exists(), (scenario, 'stale DNS readiness')
|
||||||
|
if mode == 'native':
|
||||||
|
assert 'dns-start' not in events
|
||||||
return events
|
return events
|
||||||
|
|
||||||
|
|
||||||
@@ -229,10 +395,43 @@ events = run('cancel', 0)
|
|||||||
assert events.count('start') == 1
|
assert events.count('start') == 1
|
||||||
events = run('prestartfail', 78)
|
events = run('prestartfail', 78)
|
||||||
assert 'start' not in events and 'stop' not in events
|
assert 'start' not in events and 'stop' not in events
|
||||||
|
events = run('generic-ss', 0, mode='doh')
|
||||||
|
assert events.count('start') == 4 and events.count('stop') == 4, events
|
||||||
|
assert events.count('generic-pause') == 1, events
|
||||||
|
assert events.count('dns-start') == events.count('dns-stop') == 4, events
|
||||||
|
events = run('generic-unsafe', 0)
|
||||||
|
assert 'start' not in events and events.count('generic-pause') == 1, events
|
||||||
|
for scenario in ('generic-budget-monitor', 'generic-budget-reconcile', 'generic-budget-ack'):
|
||||||
|
events = run(scenario, 78, mode='doh')
|
||||||
|
assert events.count('start') == 1 and 'stop' not in events, (scenario, events)
|
||||||
for dns in ('8.8.8.8', '2001:4860:4860::8888'):
|
for dns in ('8.8.8.8', '2001:4860:4860::8888'):
|
||||||
events = run('fatal64', 64, dns)
|
events = run('fatal64', 64, dns)
|
||||||
assert events.count('ss-server') == 1 and events.count('ss-local') == 1
|
assert events.count('ss-server') == 1 and events.count('ss-local') == 1
|
||||||
|
|
||||||
|
for scenario in ('dns-outage', 'technical', 'ss', 'cancel', 'dns-death', 'dns-killed'):
|
||||||
|
events = run(scenario, 0, dns='8.8.8.8', mode=None)
|
||||||
|
assert events.index('dns-ready') < events.index('ss-server')
|
||||||
|
assert events.index('dns-ready') < events.index('ss-local')
|
||||||
|
if scenario not in ('dns-death', 'dns-killed'):
|
||||||
|
assert events.count('dns-start') == events.count('dns-stop')
|
||||||
|
assert events.index('stop') < events.index('dns-stop')
|
||||||
|
if scenario == 'dns-outage':
|
||||||
|
assert events.count('start') == 1 and 'cooldown 15' not in events
|
||||||
|
for scenario, code in (('dns-early', 78), ('dns-noack', 78), ('dns-wrongack', 78),
|
||||||
|
('dns-permissions', 78), ('dns-symlink', 78), ('dns-prestart', 78)):
|
||||||
|
events = run(scenario, code, mode='doh')
|
||||||
|
assert not any(e.startswith('monitor ') for e in events)
|
||||||
|
if scenario != 'dns-prestart':
|
||||||
|
assert 'ss-server' not in events and 'ss-local' not in events
|
||||||
|
for scenario in ('dns-stale', 'dns-stale-symlink', 'dns-preack-foreign', 'dns-replace', 'dns-replace-symlink'):
|
||||||
|
events = run(scenario, 78, mode='doh')
|
||||||
|
assert events.count('dns-start') <= 1 and 'cooldown 15' not in events
|
||||||
|
if scenario.startswith('dns-stale'):
|
||||||
|
assert not events
|
||||||
|
for scenario in ('dns-term', 'dns-int'):
|
||||||
|
events = run(scenario, 0, mode='doh')
|
||||||
|
assert events == ['dns-start', 'dns-stop'], events
|
||||||
|
|
||||||
|
|
||||||
def check_watchdog_traps(shell):
|
def check_watchdog_traps(shell):
|
||||||
"""Same bounded_wait from normal flow, plain EXIT, and TERM -> EXIT traps."""
|
"""Same bounded_wait from normal flow, plain EXIT, and TERM -> EXIT traps."""
|
||||||
@@ -250,6 +449,7 @@ def check_watchdog_traps(shell):
|
|||||||
mode=$1
|
mode=$1
|
||||||
shift
|
shift
|
||||||
config_dir={directory}
|
config_dir={directory}
|
||||||
|
generic_mode=0
|
||||||
. {ROOT / 'managed-target-supervision.sh'}
|
. {ROOT / 'managed-target-supervision.sh'}
|
||||||
{bounded_wait}
|
{bounded_wait}
|
||||||
cleanup() {{
|
cleanup() {{
|
||||||
|
|||||||
@@ -13,16 +13,21 @@ services:
|
|||||||
environment:
|
environment:
|
||||||
SS_PASSWORD: ${SS_PASSWORD:?Set SS_PASSWORD in the deployment environment}
|
SS_PASSWORD: ${SS_PASSWORD:?Set SS_PASSWORD in the deployment environment}
|
||||||
SS_DNS_SERVER: ${SS_DNS_SERVER-}
|
SS_DNS_SERVER: ${SS_DNS_SERVER-}
|
||||||
|
ZAPRET_AUTOTUNE_DNS_MODE: ${ZAPRET_AUTOTUNE_DNS_MODE-doh}
|
||||||
|
ZAPRET_AUTOTUNE_DOH_URL: ${ZAPRET_AUTOTUNE_DOH_URL-https://dns.google/dns-query}
|
||||||
|
ZAPRET_AUTOTUNE_DOH_BOOTSTRAP_IPS: ${ZAPRET_AUTOTUNE_DOH_BOOTSTRAP_IPS-}
|
||||||
|
ZAPRET_AUTOTUNE_DOH_TIMEOUT_SEC: ${ZAPRET_AUTOTUNE_DOH_TIMEOUT_SEC-5}
|
||||||
ZAPRET_AUTOTUNE_YOUTUBE_SERVICE: ${ZAPRET_AUTOTUNE_YOUTUBE_SERVICE-0}
|
ZAPRET_AUTOTUNE_YOUTUBE_SERVICE: ${ZAPRET_AUTOTUNE_YOUTUBE_SERVICE-0}
|
||||||
|
ZAPRET_AUTOTUNE_RESOURCE_CONFIG: ${ZAPRET_AUTOTUNE_RESOURCE_CONFIG-}
|
||||||
ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE: ${ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE-cold}
|
ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE: ${ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE-cold}
|
||||||
ZAPRET_AUTOTUNE_VIDEO_ID: ${ZAPRET_AUTOTUNE_VIDEO_ID-dQw4w9WgXcQ}
|
ZAPRET_AUTOTUNE_VIDEO_ID: ${ZAPRET_AUTOTUNE_VIDEO_ID-dQw4w9WgXcQ}
|
||||||
ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS: '${ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS-["${ZAPRET_AUTOTUNE_VIDEO_ID-dQw4w9WgXcQ}"]}'
|
ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS: '${ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS:-}'
|
||||||
SOCKS_PORT: "1080"
|
SOCKS_PORT: "1080"
|
||||||
ZAPRET_AUTOTUNE_TARGET_ROLE: managed-test
|
ZAPRET_AUTOTUNE_TARGET_ROLE: managed-test
|
||||||
ZAPRET_AUTOTUNE_TARGET_ID: zapret-autotune-managed-v1
|
ZAPRET_AUTOTUNE_TARGET_ID: zapret-autotune-managed-v1
|
||||||
ZAPRET_AUTOTUNE_TARGET_DIR: /var/lib/zapret-autotune/target
|
ZAPRET_AUTOTUNE_TARGET_DIR: /var/lib/zapret-autotune/target
|
||||||
ZAPRET_CONFIG: /var/lib/zapret-autotune/target/config
|
ZAPRET_CONFIG: /var/lib/zapret-autotune/target/config
|
||||||
ZAPRET_AUTOTUNE_SOCKS_PROXY: socks5://127.0.0.1:1080
|
ZAPRET_AUTOTUNE_SOCKS_PROXY: socks5h://127.0.0.1:1080
|
||||||
ZAPRET_AUTOTUNE_DB: /var/lib/zapret-autotune/target/candidates.db
|
ZAPRET_AUTOTUNE_DB: /var/lib/zapret-autotune/target/candidates.db
|
||||||
ports:
|
ports:
|
||||||
- "127.0.0.1:${ZAPRET_AUTOTUNE_HOST_SOCKS_PORT:-11080}:1080/tcp"
|
- "127.0.0.1:${ZAPRET_AUTOTUNE_HOST_SOCKS_PORT:-11080}:1080/tcp"
|
||||||
@@ -38,6 +43,8 @@ services:
|
|||||||
networks: [target]
|
networks: [target]
|
||||||
restart: "no"
|
restart: "no"
|
||||||
stop_grace_period: 240s
|
stop_grace_period: 240s
|
||||||
|
sysctls:
|
||||||
|
net.ipv4.ip_unprivileged_port_start: "0"
|
||||||
networks:
|
networks:
|
||||||
target:
|
target:
|
||||||
driver: bridge
|
driver: bridge
|
||||||
+191
-22
@@ -14,6 +14,11 @@ esac
|
|||||||
[ "${#ss_password}" -ge 32 ] || { echo 'error: target secret must be at least 32 hex characters' >&2; exit 64; }
|
[ "${#ss_password}" -ge 32 ] || { echo 'error: target secret must be at least 32 hex characters' >&2; exit 64; }
|
||||||
# Validate before creating configs or launching any service. Errors never echo input.
|
# Validate before creating configs or launching any service. Errors never echo input.
|
||||||
/usr/local/bin/zapret-autotune target-config-check || exit 64
|
/usr/local/bin/zapret-autotune target-config-check || exit 64
|
||||||
|
dns_mode=$(/usr/local/bin/zapret-autotune dns-forwarder mode) || exit 64
|
||||||
|
dns_address=$(/usr/local/bin/zapret-autotune dns-forwarder address) || exit 64
|
||||||
|
generic_mode=0
|
||||||
|
[ -z "${ZAPRET_AUTOTUNE_RESOURCE_CONFIG-}" ] || generic_mode=1
|
||||||
|
case "$dns_mode" in doh|native) ;; *) exit 64 ;; esac
|
||||||
config_dir=/run/zapret-autotune-ss
|
config_dir=/run/zapret-autotune-ss
|
||||||
awk '$2 == "/run/zapret-autotune-ss" && $3 == "tmpfs" { found=1 } END { exit !found }' /proc/mounts || {
|
awk '$2 == "/run/zapret-autotune-ss" && $3 == "tmpfs" { found=1 } END { exit !found }' /proc/mounts || {
|
||||||
echo 'error: target Shadowsocks config directory must be a dedicated tmpfs' >&2; exit 78;
|
echo 'error: target Shadowsocks config directory must be a dedicated tmpfs' >&2; exit 78;
|
||||||
@@ -29,17 +34,55 @@ chmod 0600 "$config_dir/server.json" "$config_dir/local.json"
|
|||||||
unset ss_password
|
unset ss_password
|
||||||
|
|
||||||
server_pid=
|
server_pid=
|
||||||
|
dns_pid=
|
||||||
|
dns_anchor_dir=0
|
||||||
|
dns_anchor_file=0
|
||||||
local_pid=
|
local_pid=
|
||||||
monitor_pid=
|
monitor_pid=
|
||||||
monitor_sid=
|
monitor_sid=
|
||||||
zapret_started=0
|
zapret_started=0
|
||||||
cooldown_pid=
|
cooldown_pid=
|
||||||
|
generic_cleanup_deadline=
|
||||||
|
generic_cleanup_failed=0
|
||||||
# Signal only descendants of an owned child, never the container/host namespace.
|
# Signal only descendants of an owned child, never the container/host namespace.
|
||||||
# The monitor gets TERM itself first so it can join scans and restore bytes;
|
# The monitor gets TERM itself first so it can join scans and restore bytes;
|
||||||
# descendant signalling is reserved for SS shutdown and forced watchdog cleanup.
|
# descendant signalling is reserved for SS shutdown and forced watchdog cleanup.
|
||||||
. /opt/managed-target-supervision.sh
|
. /opt/managed-target-supervision.sh
|
||||||
|
# One budget per cleanup cycle, including ACKs, kill escalation and helper joins.
|
||||||
|
# Ten seconds of Docker's 90s grace remain for coarse clocks and final reaping.
|
||||||
|
cleanup_remaining() {
|
||||||
|
cleanup_left=$((generic_cleanup_deadline - $(date +%s)))
|
||||||
|
[ "$cleanup_left" -ge 0 ] || cleanup_left=0
|
||||||
|
}
|
||||||
|
cleanup_command() {
|
||||||
|
if [ "$generic_mode" -eq 0 ]; then
|
||||||
|
timeout -k 5 "$@"
|
||||||
|
return $?
|
||||||
|
fi
|
||||||
|
cleanup_command_cap=$1
|
||||||
|
shift
|
||||||
|
cleanup_remaining
|
||||||
|
# timeout's KILL grace is part of, not additional to, the shared budget.
|
||||||
|
[ "$cleanup_left" -gt 1 ] || return 78
|
||||||
|
cleanup_command_left=$((cleanup_left - 1))
|
||||||
|
[ "$cleanup_command_cap" -le "$cleanup_command_left" ] || cleanup_command_cap=$cleanup_command_left
|
||||||
|
timeout -k 1 "$cleanup_command_cap" "$@"
|
||||||
|
}
|
||||||
|
cleanup_reap() {
|
||||||
|
reap_pid=$1
|
||||||
|
reap_deadline=$2
|
||||||
|
[ "$reap_deadline" -le "$((generic_cleanup_deadline + 2))" ] || reap_deadline=$((generic_cleanup_deadline + 2))
|
||||||
|
child_joined=0
|
||||||
|
while proc_identity "$reap_pid" && [ "$proc_state" != Z ]; do
|
||||||
|
[ "$(date +%s)" -lt "$reap_deadline" ] || return 0
|
||||||
|
pause 1
|
||||||
|
done
|
||||||
|
waited_code=0
|
||||||
|
wait "$reap_pid" 2>/dev/null || waited_code=$?
|
||||||
|
child_joined=1
|
||||||
|
}
|
||||||
pause() {
|
pause() {
|
||||||
sleep "$1" </dev/null >/dev/null 2>&1 & cooldown_pid=$!
|
sleep "$1" 8<&- 9<&- </dev/null >/dev/null 2>&1 & cooldown_pid=$!
|
||||||
wait "$cooldown_pid"
|
wait "$cooldown_pid"
|
||||||
cooldown_pid=
|
cooldown_pid=
|
||||||
}
|
}
|
||||||
@@ -51,6 +94,9 @@ cleanup_session() {
|
|||||||
for session_signal in TERM KILL; do
|
for session_signal in TERM KILL; do
|
||||||
signal_session "$monitor_sid" "$session_signal" || return 78
|
signal_session "$monitor_sid" "$session_signal" || return 78
|
||||||
session_deadline=$(($(date +%s) + 3))
|
session_deadline=$(($(date +%s) + 3))
|
||||||
|
if [ "$generic_mode" -eq 1 ] && [ "$session_deadline" -gt "$generic_cleanup_deadline" ]; then
|
||||||
|
session_deadline=$generic_cleanup_deadline
|
||||||
|
fi
|
||||||
while :; do
|
while :; do
|
||||||
signal_session "$monitor_sid" CHECK || return 78
|
signal_session "$monitor_sid" CHECK || return 78
|
||||||
if [ "$session_live" -eq 0 ]; then
|
if [ "$session_live" -eq 0 ]; then
|
||||||
@@ -79,14 +125,30 @@ capture_monitor_session() {
|
|||||||
bounded_wait() {
|
bounded_wait() {
|
||||||
child=$1
|
child=$1
|
||||||
seconds=$2
|
seconds=$2
|
||||||
|
if [ "$generic_mode" -eq 1 ]; then
|
||||||
|
cleanup_remaining
|
||||||
|
# Reserve ACK setup (5s) and post-KILL reaping (2s) before granting wait.
|
||||||
|
if [ "$cleanup_left" -le 7 ]; then
|
||||||
|
signal_tree "$child" KILL
|
||||||
|
cleanup_reap "$child" "$(($(date +%s) + 2))"
|
||||||
|
waited_code=${waited_code-137}
|
||||||
|
join_timed_out=1
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
cleanup_wait_left=$((cleanup_left - 7))
|
||||||
|
[ "$seconds" -le "$cleanup_wait_left" ] || seconds=$cleanup_wait_left
|
||||||
|
fi
|
||||||
rm -f "$config_dir/join-timeout" "$config_dir/join-ready"
|
rm -f "$config_dir/join-timeout" "$config_dir/join-ready"
|
||||||
# Fresh interpreter: a forked shell can inherit a deferred EXIT/TERM trap
|
# Fresh interpreter: a forked shell can inherit a deferred EXIT/TERM trap
|
||||||
# context and never run its cancellation handler during PID 1 shutdown.
|
# context and never run its cancellation handler during PID 1 shutdown.
|
||||||
/bin/sh /opt/managed-target-supervision.sh --watchdog "$child" "$seconds" "$config_dir" </dev/null >/dev/null 2>&1 &
|
/bin/sh /opt/managed-target-supervision.sh --watchdog "$child" "$seconds" "$config_dir" 8<&- 9<&- </dev/null >/dev/null 2>&1 &
|
||||||
watchdog=$!
|
watchdog=$!
|
||||||
# A fast-exiting child must not cancel the watchdog before its trap exists.
|
# A fast-exiting child must not cancel the watchdog before its trap exists.
|
||||||
# Fail closed if exec/setup fails before acknowledgement; never wait forever.
|
# Fail closed if exec/setup fails before acknowledgement; never wait forever.
|
||||||
ack_deadline=$(($(date +%s) + 5))
|
ack_deadline=$(($(date +%s) + 5))
|
||||||
|
if [ "$generic_mode" -eq 1 ] && [ "$ack_deadline" -gt "$generic_cleanup_deadline" ]; then
|
||||||
|
ack_deadline=$generic_cleanup_deadline
|
||||||
|
fi
|
||||||
while [ ! -e "$config_dir/join-ready" ]; do
|
while [ ! -e "$config_dir/join-ready" ]; do
|
||||||
if ! kill -0 "$watchdog" 2>/dev/null || [ "$(date +%s)" -ge "$ack_deadline" ]; then
|
if ! kill -0 "$watchdog" 2>/dev/null || [ "$(date +%s)" -ge "$ack_deadline" ]; then
|
||||||
: > "$config_dir/join-timeout"
|
: > "$config_dir/join-timeout"
|
||||||
@@ -96,13 +158,39 @@ bounded_wait() {
|
|||||||
fi
|
fi
|
||||||
done
|
done
|
||||||
waited_code=0
|
waited_code=0
|
||||||
|
if [ "$generic_mode" -eq 1 ]; then
|
||||||
|
cleanup_join_deadline=$(($(date +%s) + seconds))
|
||||||
|
[ "$cleanup_join_deadline" -le "$generic_cleanup_deadline" ] || cleanup_join_deadline=$generic_cleanup_deadline
|
||||||
|
cleanup_reap "$child" "$cleanup_join_deadline"
|
||||||
|
if [ "$child_joined" -eq 0 ]; then
|
||||||
|
: > "$config_dir/join-timeout"
|
||||||
|
signal_tree "$child" KILL
|
||||||
|
cleanup_reap "$child" "$(($(date +%s) + 2))"
|
||||||
|
fi
|
||||||
|
joined_service=$child_joined
|
||||||
|
else
|
||||||
wait "$child" 2>/dev/null || waited_code=$?
|
wait "$child" 2>/dev/null || waited_code=$?
|
||||||
|
fi
|
||||||
kill -TERM "$watchdog" 2>/dev/null || true
|
kill -TERM "$watchdog" 2>/dev/null || true
|
||||||
|
if [ "$generic_mode" -eq 1 ]; then
|
||||||
|
service_waited_code=$waited_code
|
||||||
|
cleanup_reap "$watchdog" "$(($(date +%s) + 2))"
|
||||||
|
if [ "$child_joined" -eq 0 ]; then
|
||||||
|
signal_tree "$watchdog" KILL
|
||||||
|
cleanup_reap "$watchdog" "$(($(date +%s) + 2))"
|
||||||
|
fi
|
||||||
|
child_joined=$joined_service
|
||||||
|
waited_code=$service_waited_code
|
||||||
|
else
|
||||||
wait "$watchdog" 2>/dev/null || true
|
wait "$watchdog" 2>/dev/null || true
|
||||||
|
fi
|
||||||
join_timed_out=0
|
join_timed_out=0
|
||||||
[ ! -e "$config_dir/join-timeout" ] || join_timed_out=1
|
[ ! -e "$config_dir/join-timeout" ] || join_timed_out=1
|
||||||
}
|
}
|
||||||
cleanup_cycle() {
|
cleanup_cycle() {
|
||||||
|
if [ "$generic_mode" -eq 1 ] && [ -z "$generic_cleanup_deadline" ]; then
|
||||||
|
generic_cleanup_deadline=$(($(date +%s) + 80))
|
||||||
|
fi
|
||||||
cleanup_code=0
|
cleanup_code=0
|
||||||
# Let the monitor cancel/join scans and restore committed bytes BEFORE stop.
|
# Let the monitor cancel/join scans and restore committed bytes BEFORE stop.
|
||||||
if [ -n "$monitor_pid" ]; then
|
if [ -n "$monitor_pid" ]; then
|
||||||
@@ -127,8 +215,8 @@ cleanup_cycle() {
|
|||||||
# Stop also sources config: clear trust before reconciliation, never
|
# Stop also sources config: clear trust before reconciliation, never
|
||||||
# source unknown bytes left by a crashed monitor, even for shutdown.
|
# source unknown bytes left by a crashed monitor, even for shutdown.
|
||||||
zapret_started=0
|
zapret_started=0
|
||||||
if timeout -k 5 45 /usr/local/bin/zapret-autotune managed reconcile; then
|
if cleanup_command 45 /usr/local/bin/zapret-autotune managed reconcile; then
|
||||||
timeout -k 5 30 /opt/zapret2/init.d/sysv/zapret2 stop || cleanup_code=78
|
cleanup_command 30 /opt/zapret2/init.d/sysv/zapret2 stop || cleanup_code=78
|
||||||
else
|
else
|
||||||
cleanup_code=78
|
cleanup_code=78
|
||||||
fi
|
fi
|
||||||
@@ -136,9 +224,11 @@ cleanup_cycle() {
|
|||||||
for pid in "$local_pid" "$server_pid"; do
|
for pid in "$local_pid" "$server_pid"; do
|
||||||
[ -z "$pid" ] || signal_tree "$pid" TERM
|
[ -z "$pid" ] || signal_tree "$pid" TERM
|
||||||
done
|
done
|
||||||
|
ss_joined=1
|
||||||
for pid in "$local_pid" "$server_pid"; do
|
for pid in "$local_pid" "$server_pid"; do
|
||||||
if [ -n "$pid" ]; then
|
if [ -n "$pid" ]; then
|
||||||
bounded_wait "$pid" 5
|
bounded_wait "$pid" 5
|
||||||
|
if [ "$generic_mode" -eq 1 ] && [ "$child_joined" -eq 0 ]; then ss_joined=0; fi
|
||||||
# Its original failure triggered supervision; only forced shutdown
|
# Its original failure triggered supervision; only forced shutdown
|
||||||
# is a cleanup failure, not the already-dead Shadowsocks status.
|
# is a cleanup failure, not the already-dead Shadowsocks status.
|
||||||
[ "$join_timed_out" -eq 0 ] || { echo 'error: Shadowsocks join timeout' >&2; cleanup_code=78; }
|
[ "$join_timed_out" -eq 0 ] || { echo 'error: Shadowsocks join timeout' >&2; cleanup_code=78; }
|
||||||
@@ -146,6 +236,28 @@ cleanup_cycle() {
|
|||||||
done
|
done
|
||||||
server_pid=
|
server_pid=
|
||||||
local_pid=
|
local_pid=
|
||||||
|
# DNS remains available through monitor rollback, zapret stop and SS joins.
|
||||||
|
if [ -n "$dns_pid" ] && [ "$ss_joined" -eq 1 ]; then
|
||||||
|
kill -TERM "$dns_pid" 2>/dev/null || true
|
||||||
|
bounded_wait "$dns_pid" 5
|
||||||
|
[ "$join_timed_out" -eq 0 ] || { echo 'error: DNS join timeout' >&2; cleanup_code=78; }
|
||||||
|
if [ "$generic_mode" -eq 1 ] && [ "$child_joined" -eq 0 ]; then
|
||||||
|
cleanup_code=78
|
||||||
|
elif [ "$generic_mode" -eq 1 ]; then
|
||||||
|
cleanup_command 5 /usr/local/bin/zapret-autotune dns-forwarder readiness-cleanup "$dns_pid" || cleanup_code=78
|
||||||
|
else
|
||||||
|
/usr/local/bin/zapret-autotune dns-forwarder readiness-cleanup "$dns_pid" || cleanup_code=78
|
||||||
|
fi
|
||||||
|
if [ "$generic_mode" -eq 0 ] || [ "$child_joined" -eq 1 ]; then dns_pid=; fi
|
||||||
|
fi
|
||||||
|
if [ -z "$dns_pid" ]; then
|
||||||
|
if [ "$dns_anchor_file" -eq 1 ]; then exec 9<&-; dns_anchor_file=0; fi
|
||||||
|
if [ "$dns_anchor_dir" -eq 1 ]; then exec 8<&-; dns_anchor_dir=0; fi
|
||||||
|
fi
|
||||||
|
if [ "$generic_mode" -eq 1 ]; then
|
||||||
|
[ "$cleanup_code" -eq 0 ] || generic_cleanup_failed=1
|
||||||
|
generic_cleanup_deadline=
|
||||||
|
fi
|
||||||
return "$cleanup_code"
|
return "$cleanup_code"
|
||||||
}
|
}
|
||||||
cleanup() {
|
cleanup() {
|
||||||
@@ -158,66 +270,103 @@ cleanup() {
|
|||||||
wait "$cooldown_pid" 2>/dev/null || true
|
wait "$cooldown_pid" 2>/dev/null || true
|
||||||
fi
|
fi
|
||||||
cleanup_cycle || final_code=78
|
cleanup_cycle || final_code=78
|
||||||
|
[ "$generic_cleanup_failed" -eq 0 ] || final_code=78
|
||||||
exit "$final_code"
|
exit "$final_code"
|
||||||
}
|
}
|
||||||
trap cleanup EXIT
|
trap cleanup EXIT
|
||||||
trap 'exit 0' INT TERM
|
trap 'exit 0' INT TERM
|
||||||
|
|
||||||
# Never invoke /opt/entrypoint.sh: zapret must start AFTER reconciliation.
|
# Never invoke /opt/entrypoint.sh: zapret must start AFTER reconciliation.
|
||||||
|
dns_alive() {
|
||||||
|
[ "$dns_mode" = native ] || { [ -n "$dns_pid" ] && kill -0 "$dns_pid" 2>/dev/null; }
|
||||||
|
}
|
||||||
|
start_dns() {
|
||||||
|
[ "$dns_mode" = doh ] || return 0
|
||||||
|
exec 9<&-
|
||||||
|
exec 8<"$config_dir"
|
||||||
|
dns_anchor_dir=1
|
||||||
|
# Unknown readiness is evidence, not debris. Never delete it to start.
|
||||||
|
if [ -e "$config_dir/dns-ready" ] || [ -L "$config_dir/dns-ready" ]; then
|
||||||
|
echo 'error: target DNS readiness already exists' >&2
|
||||||
|
return 78
|
||||||
|
fi
|
||||||
|
/usr/local/bin/zapret-autotune dns-forwarder serve &
|
||||||
|
dns_pid=$!
|
||||||
|
dns_deadline=$(($(date +%s) + 5))
|
||||||
|
while :; do
|
||||||
|
dns_alive || { echo 'error: target DNS child exited' >&2; return 75; }
|
||||||
|
if [ -s "$config_dir/dns-ready" ]; then
|
||||||
|
[ -f "$config_dir/dns-ready" ] && [ ! -L "$config_dir/dns-ready" ] &&
|
||||||
|
[ "$(stat -c '%u:%a' "$config_dir/dns-ready")" = "$(id -u):600" ] || return 78
|
||||||
|
# Retain inode/directory anchors in PID 1 even after child SIGKILL.
|
||||||
|
exec 9<"$config_dir/dns-ready"
|
||||||
|
dns_anchor_file=1
|
||||||
|
/usr/local/bin/zapret-autotune dns-forwarder readiness-check "$dns_pid" || return 78
|
||||||
|
dns_alive || return 75
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
[ "$(date +%s)" -lt "$dns_deadline" ] || { echo 'error: target DNS readiness timeout' >&2; return 75; }
|
||||||
|
pause 1
|
||||||
|
done
|
||||||
|
}
|
||||||
run_cycle() {
|
run_cycle() {
|
||||||
|
start_dns || return $?
|
||||||
set -- ss-server -c "$config_dir/server.json"
|
set -- ss-server -c "$config_dir/server.json"
|
||||||
# DNS is not a secret. Native -d avoids assuming support for a JSON DNS key.
|
# DNS is not a secret. Native -d avoids assuming support for a JSON DNS key.
|
||||||
[ -z "${SS_DNS_SERVER-}" ] || set -- "$@" -d "$SS_DNS_SERVER"
|
[ -z "$dns_address" ] || set -- "$@" -d "$dns_address"
|
||||||
"$@" &
|
"$@" 8<&- 9<&- &
|
||||||
server_pid=$!
|
server_pid=$!
|
||||||
ss-local -c "$config_dir/local.json" &
|
ss-local -c "$config_dir/local.json" 8<&- 9<&- &
|
||||||
local_pid=$!
|
local_pid=$!
|
||||||
|
|
||||||
# Bounded readiness, no network probe. Managed commands verify socket ownership.
|
# Bounded readiness, no network probe. Managed commands verify socket ownership.
|
||||||
attempt=0
|
attempt=0
|
||||||
until awk '$2 == "00000000:0438" && $4 == "0A" { found=1 } END { exit !found }' /proc/net/tcp; do
|
until awk '$2 == "00000000:0438" && $4 == "0A" { found=1 } END { exit !found }' /proc/net/tcp; do
|
||||||
{ kill -0 "$server_pid" && kill -0 "$local_pid"; } || return 75
|
{ dns_alive && kill -0 "$server_pid" && kill -0 "$local_pid"; } || return 75
|
||||||
attempt=$((attempt + 1))
|
attempt=$((attempt + 1))
|
||||||
[ "$attempt" -lt 30 ] || { echo 'error: target SOCKS listener timeout' >&2; return 75; }
|
[ "$attempt" -lt 30 ] || { echo 'error: target SOCKS listener timeout' >&2; return 75; }
|
||||||
pause 1
|
pause 1
|
||||||
done
|
done
|
||||||
|
dns_alive || return 75
|
||||||
# Reject unsafe operator requests before init/reconcile can mutate owned state.
|
# Reject unsafe operator requests before init/reconcile can mutate owned state.
|
||||||
timeout -k 5 45 /usr/local/bin/zapret-autotune managed experimental-request-check || exit 78
|
if [ "$generic_mode" -eq 0 ]; then
|
||||||
|
timeout -k 5 45 /usr/local/bin/zapret-autotune managed experimental-request-check || return 78
|
||||||
|
fi
|
||||||
if [ ! -e "$ZAPRET_CONFIG" ]; then
|
if [ ! -e "$ZAPRET_CONFIG" ]; then
|
||||||
timeout -k 5 45 /usr/local/bin/zapret-autotune managed init || exit 78
|
timeout -k 5 45 /usr/local/bin/zapret-autotune managed init || return 78
|
||||||
fi
|
fi
|
||||||
# Never activate pending or unknown bytes. Any reconciliation failure is fatal,
|
# Never activate pending or unknown bytes. Any reconciliation failure is fatal,
|
||||||
# not a transient (75) restart request. Reload restores known bytes first.
|
# not a transient (75) restart request. Reload restores known bytes first.
|
||||||
timeout -k 5 45 /usr/local/bin/zapret-autotune managed reconcile || exit 78
|
timeout -k 5 45 /usr/local/bin/zapret-autotune managed reconcile || return 78
|
||||||
zapret_started=1
|
zapret_started=1
|
||||||
timeout -k 5 30 /opt/zapret2/init.d/sysv/zapret2 start || exit 78
|
timeout -k 5 30 /opt/zapret2/init.d/sysv/zapret2 start || return 78
|
||||||
|
|
||||||
# Private operator request, consumed only after a successful guarded transaction.
|
# Private operator request, consumed only after a successful guarded transaction.
|
||||||
# Never suppress qualification gates globally or race the owned monitor.
|
# Never suppress qualification gates globally or race the owned monitor.
|
||||||
if [ -e "${ZAPRET_CONFIG%/*}/.autotune-experimental-suffix-request" ] || [ -L "${ZAPRET_CONFIG%/*}/.autotune-experimental-suffix-request" ]; then
|
if [ "$generic_mode" -eq 0 ] && { [ -e "${ZAPRET_CONFIG%/*}/.autotune-experimental-suffix-request" ] || [ -L "${ZAPRET_CONFIG%/*}/.autotune-experimental-suffix-request" ]; }; then
|
||||||
timeout -k 5 120 /usr/local/bin/zapret-autotune managed apply --googlevideo-experiment native-circular --googlevideo-scope experimental_googlevideo_suffix --googlevideo-request startup || exit 78
|
timeout -k 5 120 /usr/local/bin/zapret-autotune managed apply --googlevideo-experiment native-circular --googlevideo-scope experimental_googlevideo_suffix --googlevideo-request startup || exit 78
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# A replacement SS child can fail during reconciliation/activation. Do not
|
# A replacement SS child can fail during reconciliation/activation. Do not
|
||||||
# fork a monitor we already need to cancel: an early TERM can race interpreter
|
# fork a monitor we already need to cancel: an early TERM can race interpreter
|
||||||
# startup before its cancellation handler exists. Clean this cycle directly.
|
# startup before its cancellation handler exists. Clean this cycle directly.
|
||||||
{ kill -0 "$server_pid" 2>/dev/null && kill -0 "$local_pid" 2>/dev/null; } || return 75
|
{ dns_alive && kill -0 "$server_pid" 2>/dev/null && kill -0 "$local_pid" 2>/dev/null; } || return 75
|
||||||
rm -f "$config_dir/monitor-ready"
|
rm -f "$config_dir/monitor-ready"
|
||||||
setsid /bin/sh /opt/managed-target-supervision.sh --monitor "$config_dir" /usr/local/bin/zapret-autotune managed monitor &
|
setsid /bin/sh /opt/managed-target-supervision.sh --monitor "$config_dir" /usr/local/bin/zapret-autotune managed monitor 8<&- 9<&- &
|
||||||
monitor_pid=$!
|
monitor_pid=$!
|
||||||
monitor_ack_deadline=$(($(date +%s) + 5))
|
monitor_ack_deadline=$(($(date +%s) + 5))
|
||||||
while [ ! -e "$config_dir/monitor-ready" ]; do
|
while [ ! -e "$config_dir/monitor-ready" ]; do
|
||||||
if ! kill -0 "$monitor_pid" 2>/dev/null || [ "$(date +%s)" -ge "$monitor_ack_deadline" ]; then
|
if ! kill -0 "$monitor_pid" 2>/dev/null || [ "$(date +%s)" -ge "$monitor_ack_deadline" ]; then
|
||||||
signal_tree "$monitor_pid" KILL
|
signal_tree "$monitor_pid" KILL
|
||||||
exit 78
|
return 78
|
||||||
fi
|
fi
|
||||||
done
|
done
|
||||||
capture_monitor_session || exit 78
|
capture_monitor_session || return 78
|
||||||
if proc_identity "$monitor_pid" && [ "$proc_sid" != "$monitor_sid" ]; then
|
if proc_identity "$monitor_pid" && [ "$proc_sid" != "$monitor_sid" ]; then
|
||||||
monitor_sid=
|
monitor_sid=
|
||||||
exit 78
|
return 78
|
||||||
fi
|
fi
|
||||||
while kill -0 "$server_pid" && kill -0 "$local_pid" && kill -0 "$monitor_pid"; do
|
while dns_alive && kill -0 "$server_pid" && kill -0 "$local_pid" && kill -0 "$monitor_pid"; do
|
||||||
pause 1
|
pause 1
|
||||||
done
|
done
|
||||||
if ! kill -0 "$monitor_pid" 2>/dev/null; then
|
if ! kill -0 "$monitor_pid" 2>/dev/null; then
|
||||||
@@ -250,11 +399,29 @@ while :; do
|
|||||||
count=$((count + 1))
|
count=$((count + 1))
|
||||||
fi
|
fi
|
||||||
done
|
done
|
||||||
[ "$count" -lt 3 ] || { echo 'error: target restart budget exhausted' >&2; exit 78; }
|
if [ "$count" -ge 3 ]; then
|
||||||
|
if [ "$generic_mode" -eq 1 ]; then
|
||||||
|
echo 'BLOCKED: generic target restart budget; checks paused' >&2
|
||||||
|
pause 60
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
echo 'error: target restart budget exhausted' >&2; exit 78
|
||||||
|
fi
|
||||||
starts="$recent $now"
|
starts="$recent $now"
|
||||||
code=0
|
code=0
|
||||||
run_cycle || code=$?
|
run_cycle || code=$?
|
||||||
cleanup_cycle || code=78
|
if ! cleanup_cycle; then
|
||||||
|
if [ "$generic_mode" -eq 1 ]; then
|
||||||
|
echo 'BLOCKED: generic target unsafe cleanup; operator repair/restart required' >&2
|
||||||
|
while :; do pause 60; done
|
||||||
|
fi
|
||||||
|
code=78
|
||||||
|
fi
|
||||||
|
if [ "$generic_mode" -eq 1 ] && [ "$code" -eq 78 ]; then
|
||||||
|
echo 'BLOCKED: generic target trust/startup failure; checks paused' >&2
|
||||||
|
pause 60
|
||||||
|
continue
|
||||||
|
fi
|
||||||
[ "$code" -eq 75 ] || exit "$code"
|
[ "$code" -eq 75 ] || exit "$code"
|
||||||
# A long-running monitor may outlive the window; recheck at failure time.
|
# A long-running monitor may outlive the window; recheck at failure time.
|
||||||
now=$(date +%s)
|
now=$(date +%s)
|
||||||
@@ -262,7 +429,9 @@ while :; do
|
|||||||
for started in $starts; do
|
for started in $starts; do
|
||||||
[ "$((now - started))" -ge 21600 ] || retry_count=$((retry_count + 1))
|
[ "$((now - started))" -ge 21600 ] || retry_count=$((retry_count + 1))
|
||||||
done
|
done
|
||||||
[ "$retry_count" -lt 3 ] || { echo 'error: target restart budget exhausted' >&2; exit 78; }
|
if [ "$retry_count" -ge 3 ] && [ "$generic_mode" -eq 0 ]; then
|
||||||
|
echo 'error: target restart budget exhausted' >&2; exit 78
|
||||||
|
fi
|
||||||
pause "$backoff"
|
pause "$backoff"
|
||||||
[ "$backoff" -ge 60 ] || backoff=$((backoff * 2))
|
[ "$backoff" -ge 60 ] || backoff=$((backoff * 2))
|
||||||
done
|
done
|
||||||
@@ -0,0 +1,27 @@
|
|||||||
|
{
|
||||||
|
"version": 1,
|
||||||
|
"services": [
|
||||||
|
{
|
||||||
|
"id": "example",
|
||||||
|
"resources": [
|
||||||
|
{
|
||||||
|
"id": "homepage",
|
||||||
|
"group": "web",
|
||||||
|
"kind": "http",
|
||||||
|
"url": "https://example.com/",
|
||||||
|
"method": "GET",
|
||||||
|
"tls": "both",
|
||||||
|
"timeout_ms": 10000,
|
||||||
|
"interval_ms": 300000,
|
||||||
|
"cooldown_ms": 30000,
|
||||||
|
"expect": {
|
||||||
|
"statuses": [200],
|
||||||
|
"content_type": "text/html",
|
||||||
|
"min_bytes": 1,
|
||||||
|
"max_bytes": 1048576
|
||||||
|
}
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
Reference in new issue
Block a user