Compare commits

..
21 Commits
Author SHA1 Message Date
ZeusinaandGPT-6 Bot 88564b340a fix: include adapter packages in container builds
Co-authored-by: GPT-6 <gpt-6@openai.com>
2026-10-15 18:25:03 +03:00
ZeusinaandGPT-6 Bot e270810f33 refactor: execute adapter plans through shared core
Execute the private YouTube qualification plan through core-owned scheduling while retaining worker cleanup, proof sealing and legacy policy boundaries. Preserve unsafe collector failures alongside concurrent cancellation or deadlines.

Send bounded Player JSON through stdin instead of argv and update all shared runner callers. Generic controller cutover and remote runtime acceptance remain pending.

Co-authored-by: GPT-6 <gpt-6@openai.com>
2026-10-15 17:25:12 +03:00
ZeusinaandGPT-6 Bot f4853173b3 feat: extract versioned YouTube service adapter
Wire pure Player parsing and strict qualification planning through a compiled-in kind/version registry. Fail closed before discovery requests when the required module is absent or mismatched.

Require sealed module-bound v2 evidence for new writes while preserving fresh trusted v1 apply eligibility and historical rollback. Keep legacy SOCKS5h routing, separate suffix/asset policy and public configuration unchanged; generic controller cutover and field acceptance remain pending.

Co-authored-by: GPT-6 <gpt-6@openai.com>
2026-10-15 16:23:54 +03:00
ZeusinaandGPT-6 Bot d324b4137b docs: record scoped managed v2 verification
Record bounded typed qualification, apply, monitor fault/repair and dated independent cleanup; retain body/live-FD and untested v1/YouTube acceptance limits. Keep adapter registry and version-bound module migration explicitly future work.

Co-authored-by: GPT-6 <gpt-6@openai.com>
2026-10-15 15:33:26 +03:00
ZeusinaandGPT-6 Bot cd05e75189 feat: add typed generic image and MP4 checks
Preserve strict v1 semantics and fingerprints; bind v2 typed expectations to a new proof namespace. Reuse bounded shared collection across checks, monitoring, qualification and apply health. Document prefix-only MP4 and deferred public GIF/plugin/runtime boundaries.

Co-authored-by: GPT-6 <gpt-6@openai.com>
2026-10-15 15:07:54 +03:00
ZeusinaandGPT-6 Bot 9ab9fd73a9 docs: record shared-core parity and rotation fault checks
Record scoped real-curl parity separately from older generic rotation fault evidence and pending managed runtime acceptance. Preserve strict YouTube qualification, refusal attribution limits and the next typed generic image/media capability.

Co-authored-by: GPT-6 <gpt-6@openai.com>
2026-10-15 12:17:00 +03:00
ZeusinaandGPT-6 Bot 45d28e3733 refactor: share strict resource validation through generic core
Use private typed generic, image and MP4 response contracts in the existing bounded HTTPS execution pipeline. Preserve generic mismatch measurements, strict legacy DTOs and producer behavior; reject overflowing Content-Range integers before normalization.

Add injected-wire adapter parity and direct-core regressions. Local targeted EUID-0/race, full Go and config fixtures passed; native acceptance remains pending. The unrelated long Python supervision timeout remains unresolved. No public schema, suffix-learning or controller migration.

Co-authored-by: GPT-6 <gpt-6@openai.com>
2026-10-15 12:04:28 +03:00
ZeusinaandGPT-6 Bot 2149a4abc8 docs: record bounded verified-profile rotation observations
Separate positive A/B/replay/A field observations from missing full native and rotation fault acceptance. Retain evidence provenance, cleanup limits and the next old-policy rollback/prepared-repair gate.

Co-authored-by: GPT-6 <gpt-6@openai.com>
2026-10-15 11:09:49 +03:00
ZeusinaandGPT-6 Bot 8d43dd9363 feat: rotate generic profiles across verified alternatives
Select bounded canonical strategy allowlists at explicit PID1 startup within the same immutable authenticated registry, matrix and source. Reconstruct stored policies independently for historical replay, reconciliation and rollback; preserve freshness for new policies even when rendered bytes match. Retain legacy refusal and one-shot startup guards.

Local fixtures and independent local review passed; live native rotation acceptance remains pending. No uninterrupted rotation, new qualification or proof-store overwrite.

Co-authored-by: GPT-6 <gpt-6@openai.com>
2026-10-15 10:18:14 +03:00
ZeusinaandGPT-6 Bot 5d0be0288f docs: record guarded generic apply fault acceptance
Record independently verified bounded replay, rollback, prepared repair and cancellation gates; retain conservative TERM78, per-PID1 authorization and coverage limits.

Co-authored-by: GPT-6 <gpt-6@openai.com>
2026-10-15 08:21:57 +03:00
ZeusinaandGPT-6 Bot 643d0c6220 fix: prevent automatic retries of failed startup apply
Latch explicit resource apply once per PID 1 lifetime. Fail closed before launch and permanently pause after authoritative cleanup on failed or incomplete apply; retain reconciliation on successful monitor and service restarts.

Cover deterministic 180-second retry windows, launch/ACK/child/session/service failures, cancellation, successful restarts and fresh-PID1 flags. Document RAM-only reset limits and repair-only startup; no remote health acceptance is claimed.

Co-authored-by: GPT-6 <gpt-6@openai.com>
2026-10-15 08:05:27 +03:00
ZeusinaandGPT-6 Bot 36aace02da fix: publish controller startup acknowledgements atomically
Publish a complete PID acknowledgement by private temporary file and same-directory rename. Preserve unknown acknowledgement artifacts and remove only retained owned stale state. Keep malformed acknowledgement failures closed.

Cover deterministic publication barriers, shared apply/monitor/qualify launch, foreign artifacts, owned replacement, and private-directory checks. This does not establish hostile root or same-UID writer no-clobber guarantees; the private mode-700 directory is the trust boundary.

Co-authored-by: GPT-6 <gpt-6@openai.com>
2026-10-14 23:00:06 +03:00
ZeusinaandGPT-6 Bot ade3b89137 chore: add private-safe controller startup diagnostics
Add fixed startup branch labels and numeric apply exit status without relaxing startup guards or exposing private values. Cover shared launch/ACK and wrapper branches plus failed apply cleanup. Diagnostics only; the ACK publication race remains unchanged.

Co-authored-by: GPT-6 <gpt-6@openai.com>
2026-10-14 22:37:48 +03:00
ZeusinaandGPT-6 Bot de453ed2c8 feat: add guarded generic startup apply
Co-authored-by: GPT-6 <gpt-6@openai.com>
2026-10-14 22:02:00 +03:00
ZeusinaandGPT-6 3743cdbd3e docs: record bounded qualification and planner acceptance
Co-authored-by: GPT-6 <noreply@openai.com>
2026-10-14 19:23:40 +03:00
ZeusinaandGPT-6 Bot 5cdc0cd767 fix: diagnose qualification worker scan failures
Report fixed procfs scan reasons, typed errno and numeric worker identities without exposing paths or errors. Preserve scan metadata through attestation and restore normalization while retaining fail-closed decisions and cleanup classification.

Co-authored-by: GPT-6 <gpt-6@openai.com>
2026-10-14 15:40:48 +03:00
ZeusinaandGPT-6 Bot 83921350fd fix: diagnose opaque qualification predicates
Add fixed attestation, factory guard and restoration reasons without changing fail-closed predicates or cleanup order. Retain nested reasons under joined ownership vetoes and distinguish prior cleanup veto from a new restoration failure.

Co-authored-by: GPT-6 <gpt-6@openai.com>
2026-10-14 15:13:32 +03:00
ZeusinaandGPT-6 Bot 63cdc7d490 fix: add privacy-safe qualification failure diagnostics
Report fixed operation and collector reason enums with measurement ordinals, keeping peer-negative classification and cleanup/ACK vetoes unchanged. Preserve original errors without emitting their values.

Co-authored-by: GPT-6 <gpt-6@openai.com>
2026-10-14 14:53:08 +03:00
ZeusinaandGPT-6 Bot 610916ce66 fix: reject concealed qualification alias traversal
Validate the raw pinned package alias before path normalization so symlink/.. traversal cannot attest one executable and launch another. Preserve package ownership, held-file, hash, inode and Lua checks.

Co-authored-by: GPT-6 <gpt-6@openai.com>
2026-10-14 14:19:33 +03:00
ZeusinaandGPT-6 Bot a1a76bff76 fix: attest trusted packaged qualification engine alias
Resolve only the fixed nfq2 package alias into trusted binaries parents, hold the no-follow engine descriptor and verify inode/hash against the running worker before nft activation. Keep Lua/import guards and completed-proof publication unchanged.

Co-authored-by: GPT-6 <gpt-6@openai.com>
2026-10-14 13:57:58 +03:00
ZeusinaandGPT-6 Bot 573da3f797 feat: add candidate-bound domain qualification and planning
Co-authored-by: GPT-6 <gpt-6@openai.com>
2026-10-14 12:21:52 +03:00
74 changed files with 9274 additions and 258 deletions

No files matched your search

+2
View File
@@ -3,6 +3,8 @@
!go.sum
!cmd/
!cmd/**
!internal/
!internal/**
!Dockerfile
!autotune-entrypoint.sh
!Dockerfile.target
+1
View File
@@ -4,6 +4,7 @@ ENV CGO_ENABLED=0 GOTOOLCHAIN=local
COPY go.mod go.sum ./
RUN go mod download
COPY cmd/ ./cmd/
COPY internal/ ./internal/
RUN --mount=type=cache,target=/root/.cache/go-build go build -trimpath -o /out/zapret-autotune ./cmd/zapret-autotune
FROM vernette/ss-zapret2@sha256:ba158180f27cd71a291ee521c291490374fd3fcf2f81eb1434ddfb19d35de45b
+1
View File
@@ -4,6 +4,7 @@ ENV CGO_ENABLED=0 GOTOOLCHAIN=local
COPY go.mod go.sum ./
RUN go mod download
COPY cmd/ ./cmd/
COPY internal/ ./internal/
RUN --mount=type=cache,target=/root/.cache/go-build go build -trimpath -o /out/zapret-autotune ./cmd/zapret-autotune
FROM vernette/ss-zapret2@sha256:ba158180f27cd71a291ee521c291490374fd3fcf2f81eb1434ddfb19d35de45b
+96 -2
View File
@@ -1,6 +1,6 @@
# Periodic configured resource checks
`managed monitor --resource-config FILE` runs resource-config v1 periodically.
`managed monitor --resource-config FILE` runs resource-config v1 or v2 periodically.
PID 1 selects the same mode when `ZAPRET_AUTOTUNE_RESOURCE_CONFIG` is nonempty;
unset preserves the existing YouTube monitor. The modes share one singleton
lock. Supply an absolute container-visible config path (for example mount a
@@ -19,9 +19,69 @@ not playback health, generic recovery or whole-production READY: HEALTHY is
base passed native full/race checks, independent security review and isolated
remote fault acceptance for `4efc066` plus ownership-status fix `5c85f3f`.
See [PROGRESS.md](PROGRESS.md) for supplied evidence and its narrow scope.
Version1 is bounded to 16 services / 64 resources with stable configured IDs;
Both versions are bounded to 16 services / 64 resources with stable configured IDs;
valid configuration/state does not imply an automatic strategy assignment.
V1 remains HTTP GET/HEAD only, with explicit `kind: http`. See
`resource-check-v2.example.json` for v2: omitted kind defaults to `http`; all
expectations explicitly declare statuses/content_type/min_bytes/max_bytes.
`image` is GET-only, [200], PNG/JPEG with full bounded decode (5 MiB maximum,
16 MiPixels); public GIF is deferred pending an aggregate frame/pixel budget,
without changing legacy GIF adapters. `mp4_range` is GET-only, [206], audio/mp4
or video/mp4, 16–65536 bytes, checked Content-Range and ftyp. Range stays fixed
at 0-65535; a smaller max_bytes bounds the actual response, not the requested end.
Operator Range/Accept-Encoding headers remain forbidden in both versions.
Timeouts (100–30000 ms) and declared byte limits apply to every kind. Typed
wire-valid semantic failures degrade checks; malformed/unsafe execution cannot
qualify a candidate. V2 version/kind/expectations invalidate old HMAC rows/proofs;
historic v1 fingerprints remain unchanged. MP4 prefix health is not playback or
YouTube proof: three discovered CDNs, >=1024-byte producer samples, warmup,
suffix authorization and adaptive policy still belong to the legacy adapter.
The private compiled-in YouTube module and core `executeAdapterPlan[T]` bridge
now drive the existing owned qualifier's ordered warmup and steady collection.
They do not add public module configuration or change this monitor's schemas,
controller, suffix authorization or legacy SOCKS5h routing boundary. Generic
controller cutover still needs dynamic-inventory fingerprints and warmup-bearing
generic proof binding; images/avatars and full YouTube READY remain unresolved.
Local source/closure review and injected-wire checks are
recorded alongside scoped managed v2 field observations below; full acceptance
is not implied.
### Scoped managed v2 monitor observations (2026-10-15)
Independent closeout at **2026-10-15T12:28:52Z** records **SCOPED_PASS** for source
`cd05e75189f90c32e1346536c50f0973e7105cb3`, audit runtime unchanged. See
[RESOURCE_PLAN.md](RESOURCE_PLAN.md#managed-v2-scoped_pass-observations-2026-10-15)
for the root-private report/audit pointers, qualification/apply counts, dated
cleanup and full evidence limits. Official owned PID1 SS/DoH/public v2 paths
qualified PNG/JPEG and video/audio MP4 prefixes on both TLS versions (16 M3
cells / 48 TLS transfers); eight apply and eight initial monitor checks passed.
Five HMAC-authenticated snapshots, including initial state, show real host-fixture
PNG truncation and MP4 total overflow degrading only the affected row to
`DEGRADED / expectation_failed`, followed by all-HEALTHY repair. The SAME
controller PID 379/start 62777447 and journal/config/proof/key/input hashes were
retained, without restart or requalification. HEALTHY is still CHECKS_ONLY.
Both official stops exited 0; dated audit found all five owned containers,
owned network and 174 recorded process identities/namespaces gone, and three
foreign raw inspect snapshots unchanged. Historical Xray equality is only a
harness assertion without retained initial raw process inventory.
Typed semantics are supported by real production validation, authenticated
outcomes and fixture metadata, **not independent re-decoding of retained bodies**:
bodies were deliberately not retained for privacy. Live executable/held-FD
snapshots were also not retained; PID/start/parent/netns/native argv checks and
reviewed ACK enforcement are not a durable live-FD certificate. The initial
DoH-forwarder/worker observer misclassification was corrected in the harness,
retaining logs and the same qualified proof, not fixing a product failure.
No new public-v1 managed field run, negative M3/cancel/freshness/old-matrix remote
campaign, actual YouTube discovery/three-CDN/suffix/adaptive/warmup/avatar/playback
was run. Local v1 parity and unit rejection coverage do not fill those gaps.
The static test image did not replace the user's working SS service. Missing
body/live-FD evidence and untested cases remain limits before full acceptance
and YouTube adapter cutover; there is no full safety, migration or READY claim.
Invalid resource JSON initially or after reload is BLOCKED, with no DNS or
transfers until repaired. Bad static target role, proxy or DNS settings still
fail startup validation; resource JSON is deliberately not part of
@@ -70,3 +130,37 @@ within a 90-second Docker stop limit (the supplied Compose grants 240 seconds).
Bounded final reaps/worst-case cleanup were verified offline; measured remote
normal stops were 1.45s, 0.59s and 0.57s, not an 80-second runtime worst-case test.
These changes do not claim to fix legacy YouTube exit 78.
### Internal service-adapter boundary (local extraction, field acceptance pending)
Legacy YouTube execution now explicitly selects compiled-in `youtube/1` via
`cmd/zapret-autotune/service_adapters.go`. `internal/adapter/youtube` parses
bounded Player media JSON and supplies signed-MP4 freshness/diversity semantics
and the owned qualification plan. Unknown/absent kind or version fails closed,
without a generic HTTP fallback. The shared `internal/adapter` DTOs are internal
only: signed URLs/inventory and plan caps have `json:"-"` fields.
The existing controller still owns discovery requests/retries, transport,
origin/SNI/certificate/TLS checks, worker guards/cleanup and sealing after join.
Legacy probes retain SOCKS5h and the owned encrypted-resolver guard; they do not
run the generic all-A/AAAA approval/connect-to IP pinning path. Full request-policy
capabilities (including future metadata POST) and parity are prerequisites before
generic cutover, not guarantees supplied by this minimal extraction.
Public generic checks and their HMAC/state formats have not migrated to this
module interface. Suffix/experimental authorization is separate from measured
endpoints, and avatar/thumbnail health remains independent. A module plan is
neither proof nor permission to claim READY.
New adaptive proof v2 seals adapter kind/version/policy digest along with the
measured facts. Unknown identities, mismatched policy, missing attestation and
unsealed DTOs cannot publish adaptive evidence. Legacy v1 is explicitly the
immutable historical YouTube policy for journal reconstruction, not a request
to reinterpret old evidence using the latest module. Expired historical proofs
can reconstruct their owned config. Fresh trusted stored v1 retains its original
apply eligibility; expiry still blocks fresh apply. Only new producer writes
require sealed module-bound v2; public/unsealed DTOs cannot publish evidence.
Discovery/asset/health entrypoints preflight the required kind/version/policy
before requests. Missing/mismatched modules are fatal configuration failures,
not retryable inventory absence or a genuine-no-strategy cooldown.
No public module field, dynamic plugin ABI, controller cutover, same-UID sandbox,
remote YouTube acceptance, or fix for legacy exit 78 is claimed here.
+849
View File
@@ -0,0 +1,849 @@
# Resource compatibility planner and isolated qualifier
## First collector migration (shared-core parity verified; managed v2 scoped observations)
Private typed response contracts now select generic GET/HEAD, full decoded image,
or bounded MP4 range validation inside the existing shared HTTPS transfer core.
The redundant legacy forwarding wrapper and media boolean are removed; image and
media adapters retain their measurements and GooglevideoPass DTOs. Generic v1
status/MIME/byte expectations use the same validation entry, with a distinct
completed-mismatch result retaining the same measured facts. No public Range config, schema,
proof importer, registry or dependency was added.
Injected-wire tests exercise the real generic producer and legacy callers, both
TLS versions, normalized byte/hash/timing facts, private stdin/pinning, origin,
redirect, framing, bounds and strict range/ftyp/image decoder rejection. Direct
core tests ensure strict validation is not an unused legacy-only helper.
MP4 requests use Range `0-65535` (a 64 KiB upper bound, not an exact response
size requirement). The core accepts validated 16–65536-byte ranges; the existing
producer additionally requires at least 1024 measured bytes. Content-Range end
and total must parse without overflow, with total greater than end; rejected or
unknown totals cannot produce a media PASS DTO.
This is collector migration only: public schemas and the legacy YouTube producer
remain unchanged, including three discovered CDNs, adaptive A/E, warmup then 3/3
samples on each TLS version, explicit googlevideo.com suffix authorization, fresh
discovery, and independent qualification of image/avatar assets. The legacy path
is retained. Generic GET success is never full YouTube qualification.
Recorded local verification: archived `2149a4a` versus current source passed 86
parity subcases; 14 new integration cases, 11 focused EUID-0 serial/race tests and
full Go tests passed (full Go about 55s). Python static/config/startup checks passed;
the long supervision check timed out at 18s and again at 120s and is unresolved.
Do not call all tests green or infer a regression from that timeout: entrypoint
code was unchanged and later full Go entry tests passed. These are prior results,
not reruns in this docs-only update.
### Scoped remote shared-core parity (2026-10-15)
Privileged-read evidence: `/home/agent/core-runtime-parity-45d28e3-20261015/report.md`.
Exact-source baseline `2149a4a` and current
`45d28e37330069ddf0efa3120fec8a558d37ba7f` each passed 48 real curl/HTTPS/SOCKS5h
fixture subtests (24 cases × TLS1.2/1.3). Of the normalized rows, 46 matched and
two intentionally differed: overflowing Content-Range total, formerly saturated
by ParseInt, now rejects without a PASS DTO. Known negative classifications are
preserved; invalid/unknown/overflow totals are not proof.
Coverage includes GET/HEAD expectations and retained mismatch facts, bounds,
redirect/framing/trust rejection, real PNG/JPEG/GIF decode and rejection, strict
206 range/ftyp checks, and stdin/argv/result privacy sentinels. Real 65536-byte
and 1024-byte media responses passed the same Range request. Each version made
60 SOCKS5h connections/SNI handshakes and 58 authenticated HTTP requests, including
18 actual Range requests; two untrusted-certificate attempts stopped before HTTP.
Original hostname/SNI was retained; curl verified a fresh private fixture CA,
without TLS bypass. Timing/speed were positive but excluded from exact normalized
parity hashes; this is not a performance comparison or fatal-reason-tree parity.
The proxy maps only allowlisted fixture names to a loopback TLS listener: it is
not an SS proxy or DoH resolver. **NOT RUN:** normal public CLI under owned PID1
SS/DoH, managed ownership/guard checks and probe orchestration, NFQ/native behavior,
real YouTube discovery/three-CDN qualification, or candidate/apply/monitor flow.
No DoH validation or production qualification follows from helper tests. No
candidate/PASS/DB/JSON proof imports occurred. Helpers exited and joined listeners;
no owned containers/networks were created. Root-private artifacts, keys, manifests
and hashes remain retained. Selected foreign identity/running/start facts matched
before/after, not arbitrary foreign application-data hashes.
### Public typed checks (v2; scoped managed field verification)
`resource-check-v2.example.json` adds closed `kind` values `http`, `image` and
`mp4_range`; omitted v2 kind canonicalizes to `http`. V1 still requires explicit
`http` and retains its serialization/fingerprints. V2 requires explicit statuses,
content_type, min_bytes and max_bytes. HTTP retains GET/HEAD semantics; typed
checks require GET. Image requires exactly [200] and image/png or image/jpeg,
full decoding, positive dimensions at most 16 MiPixels and a configured body
ceiling at most 5 MiB. Public GIF/omitted image MIME are deliberately deferred
until a decoder with an aggregate frame/pixel budget exists; legacy GIF remains.
MP4 requires exactly [206], video/mp4 or audio/mp4, and 16 <= min <= max <= 65536.
The core always requests bytes 0-65535, even with a smaller max_bytes; actual
returned bytes must fit that smaller bound and the validated Content-Range/ftyp.
This proves a bounded MP4 prefix, **not playback or YouTube qualification**.
All kinds use configured 100–30000 ms timeouts and byte ceilings, private stdin,
strict framing/TLS/origin and all-answer DNS policy through the same public check,
periodic monitor, isolated M=3 qualifier and apply-health path. Wire-valid typed
semantic failures are complete expectation negatives; unsafe output, malformed
metrics/framing, cancellation and ownership failures cannot become benign proof.
V2 source and row HMACs bind version, normalized kind and expectations, so old
plain-HTTP evidence/state cannot promote typed checks. V1 historical journal
reconstruction is unchanged. No public proof import or module framework exists.
Local verification for this v2 slice: full Go tests (`-p 1 -parallel 1`), affected
resource race tests, vet, CGO-disabled build, four managed-target Python fixtures
and shell syntax passed. Current and archived `45d28e3` each passed the existing
86 legacy adapter/generic-mismatch parity subtests. Privileged tests ran as EUID 0
inside the current user namespace (UID 0 maps to host UID 100000), not host-initial
root. These are local injected-wire/fixture results, not remote network acceptance.
Next let the YouTube adapter supply qualified exact endpoints/dynamic inventory into
the generic registry, preserving fresh discovery and the three-CDN contract.
Keep measured endpoint proof separate from explicit suffix/adaptive-learning
authorization: both capabilities are prerequisites to complete managed migration.
No plugin framework, legacy deletion or full controller/journal/profile migration
is implied. Full runtime acceptance remains incomplete; avatars and legacy exit78 remain
unresolved. No production READY claim.
### Managed v2 SCOPED_PASS observations (2026-10-15)
Source `cd05e75189f90c32e1346536c50f0973e7105cb3`. Privileged reads of
`/home/agent/managed-v2-cd05e75-20261015/report.md` and
`field-private/independent-audit.safe.json` record independent closeout at
**2026-10-15T12:28:52Z**, verdict **SCOPED_PASS**, with
`runtime_state_changed_by_audit: false`. These are bounded positive observations,
not unequivocal whole-criterion acceptance, full safety or READY. The static test
binary/image is not a deployment to the user's working SS service. The audit
matched 137 archived product files and the pinned native NFQ engine (1.0.2).
- Official public v2 CLI under owned PID1 SS/DoH exercised PNG/JPEG full decode
and video/mp4 + audio/mp4 bounded prefixes, each over TLS1.2/1.3 with verified
fixture CA/SNI. Two candidates × four resources × both TLS versions produced
16 M=3 cells / 48 real qualification TLS transfers. Queue-200 worker argv and
PID/start matched; exact-payload registry/matrix HMACs, v2 source and semantic
row fingerprints were independently recomputed, not imported PASS evidence.
- Official apply reused that proof without requalification: queue 300, canonical
`multidisorder`, three exact fixture hosts, journal v4 / generation 1 / committed
generic. Eight actual typed apply checks and eight initial monitor checks passed.
Source fingerprint:
`cc5a2622133daed58df0b2c22254f6112a3d9da7fbd9391ce7d1c6f467761886`;
committed config SHA-256:
`27f9a8ffd56a4c19d4bd522dd9f5a5e8b49b18b0ae48d41431ef9b22975ea772`.
- Five authenticated monitor snapshots (including initial state) showed truncated
PNG and overflowing video MP4 Content-Range each degrading **only** its affected
row to `DEGRADED / expectation_failed`, then repair restoring all four HEALTHY.
Host fixture changes produced real responses; controller PID 379/start 62777447,
journal/config/proofs/key/input hashes stayed the same, without requalification.
Apply/monitor wire evidence totals 48 transfers (33 healthy-mode, 7 PNG-fault,
8 MP4-fault); fixture timing/size/status is not durable product DTO telemetry.
- Dated closeout independently rechecked both official target stops exited 0;
all five recorded owned containers, owned network and 174 recorded PID/start
identities/namespaces were absent. Three foreign raw Docker inspect snapshots
remained byte-identical: `f6c9d42858e1` running, `4ad54175a54c` and `8cc36c813d6f`
stopped. Historical Xray inventory equality is a harness assertion only: its
initial raw process inventory was not retained, so it is not independently proved.
**Evidence limits:** production strict validation, authenticated outcomes and actual
fixture metadata establish this scoped typed-response observation. Bodies were
intentionally not retained to avoid data exposure and were **not independently
re-decoded**. Actual PID/start/parent/netns/native argv were independently checked;
live executable/held-FD snapshots were **not retained**. ACK/snapshot guards are
reviewed code/log enforcement, not a durable live-FD certificate. These missing
body/live-FD certificates prevent an unqualified full acceptance claim.
The initial observer misclassified the intentionally live DoH forwarder as an
unjoined worker. Original logs/harness remain; the corrected continuation used
the SAME proof, with no new qualification. This is a harness assertion correction,
not an observed product bug.
**NOT RUN in this field campaign:** public v1 managed GET/HEAD (86 current/old
local parity cases passed, not a remote v1 run); negative M3 qualification,
cancellation, freshness/24h expiry or old-matrix rejection; actual YouTube
discovery/three-CDN, suffix authorization, adaptive learning, warmup, avatar or
playback. New kind/version cannot promote old proofs by local unit coverage,
not a fresh remote rejection campaign. MP4 proves a prefix, not playback or an
exact 64-KiB body; public GIF remains deferred and legacy GIF retained. Prior full
Go/race/Python results are recorded above, not rerun for this docs-only change.
**Next minimum slice (design, NOT implemented):** a real YouTube/core adapter
boundary wrapping existing discovery and strict qualification calls, not a full
framework. Core owns networking, limits, proof, persistence and apply; YouTube
owns fresh URLs, three-CDN warmup/sampling, per-host adaptive learning and explicit
suffix policy. Plan a minimal compiled-in Go registry keyed by kind + version,
fail closed for unknown contracts; bind module/policy version into source/proof
and preserve authenticated historical restore under its original contract. No
dynamic plugin/RPC API exists. Do not generalize public POST/discovery until an
actual adapter needs that request contract. Retain legacy functionality until
parity and the remaining acceptance gaps are closed before full adapter cutover.
`plan` remains **proposal-only** and never changes runtime or config.
Explicit generic startup apply/replay and the per-PID-1 failure latch have passed
bounded remote acceptance with independent privileged verification (see below).
This is not automatic recovery, full concurrency acceptance or production READY.
Explicit isolated qualification temporarily installs an owned NFQ worker and
queue rules, but never changes legacy SQL groups, renderer or apply journals.
Monitor HEALTHY, old SQL measurements and public check JSON are not proof.
Without a completed owned qualification batch, compatibility is UNKNOWN and
hosts are unassigned. No PASS/evidence import is exposed.
Commands require root and the existing `managed-test` target settings:
```
zapret-autotune strategy-import --strategies /private/strategies.json
zapret-autotune plan --config /private/resources.json
zapret-autotune qualify --config /private/resources.json --strategies /private/strategies.json --isolated
```
`strategy-import` requires an owned, regular, single-link 0600 file. Only
definitions are accepted; duplicate keys, unknown fields and evidence fields
are rejected. Example:
```json
{
"version": 1,
"strategies": [
{
"id": "split-one",
"args": "--payload=tls_client_hello --lua-desync=multisplit:pos=1",
"selectors": {"transport":"tcp","ports":"443","l7":"tls","tls":"both"}
}
]
}
```
The registry is one immutable batch (1–32 unique IDs). This conservative slice
accepts only TCP/443/TLS selectors and `tls12`, `tls13`, or `both` policy.
Argument whitespace is canonicalized; option/Lua stage order is preserved.
Existing payload/range validation is reused. Lua stages are restricted to
fake, multisplit, multidisorder, fakeddisorder, split and disorder; parameters
to pos, repeats, tcp_ts, tcp_flags_set, seqovl and blob=fake_default_tls.
File/inline Lua, queue/user/fwmark/filter options and arbitrary stages are
rejected. Acceptance of a definition is **not** runtime verification.
The private target directory holds `.resource-plan-key` (32 random bytes),
`.resource-plan-strategies.json` and, when an owned producer exists,
`.resource-plan-matrix.json`. JSON envelopes have `payload` and an HMAC-SHA256
`mac`. Each file is write-once, owned, 0600, no-follow, single-link and bounded
to 64 KiB. Held directory/file identities and exact snapshots are checked;
foreign entries, replacement inodes and modified bytes are never clobbered or
silently adopted. Interrupted writes remain blocked for explicit operator
repair. There is no automatic rotation/update/repair in this slice. The private
key and same-UID/root boundary must be protected: HMAC is not protection from an
actor who can read the key and replace the whole target state between processes.
Matrix v1 binds the registry digest and at most one completed run per candidate.
Each run binds ID, definition digest (canonical args **and selectors**), keyed
config fingerprint, start/completion milliseconds, resource/TLS outcomes and
keyed resource fingerprints. Its attestation records collector/runtime,
definition digest, composite executable/Lua input hash, installed rules hash,
actual worker-argv hash, queue and worker PID/start identity, plus
restored/joined completion. No URLs, headers, resolved IPs, bodies or raw errors
are persisted. A nonserialized producer seal protects the completed in-memory
snapshot against mutation before publication. Only the owned producer and
test-internal fixtures create seals; JSON DTOs cannot authorize publication.
Every resource/protocol cell records exactly three actual attempts; any nonpass
in those attempts makes the cell negative (never an average or partial PASS).
`plan` loads only authenticated private state and the existing resource-config
snapshot (including resolved environment secrets, DNS/proxy and private headers
in keyed fingerprints). Every resource on an exact host, across all services
and input group labels, must have matching complete results for every required
TLS protocol. Complete failed checks are negative; missing, expired (24 hours),
future, mismatched or partial checks are unknown. Changed config invalidates old
run evidence. Malformed/unauthenticated state yields blocked, unassigned output
and exit 78; valid missing evidence yields unknown output and exit 0.
Output contains `proposal_only`, `evidence` (authentication status, **not**
health), per-host `validated`/`negative`/`unknown` candidate alternatives,
`assignments` and `unassigned`. Compatible equal args/selectors merge coverage
across input labels. Greedy greatest-uncovered coverage, then lexical candidate
ID, produces deterministic disjoint exact-host assignments. It does not claim
global optimality or suffix coverage. Explicit legacy YouTube suffix policy is
unchanged.
## INTRUSIVE / ISOLATED startup only
Set **both** `ZAPRET_AUTOTUNE_QUALIFY_CONFIG` and
`ZAPRET_AUTOTUNE_STRATEGY_CONFIG` to container paths on a dedicated target.
Do not send working user traffic to this target's published SOCKS port. This is
not noninterrupting background search. Provision the private resource and 0600
strategy files separately; compose forwards paths but does not mount files for
you. Start with the managed disabled baseline, not an existing active profile.
Qualification v1 accepts lowercase DNS hostnames, not literal IP targets; the
ordinary resource-check command's broader literal-IP policy is unchanged.
PID 1 starts its owned SS+DoH services, initializes/reconciles known state, then
launches one finite `qualify ... --isolated` session **instead of** starting the
live zapret daemon or monitor. Qualifier checks root/target namespace, PID 1's
actual entrypoint/environment, direct-child executable/arguments/session ACK,
owned SS and DNS readiness anchors. A flag or docker-exec environment alone
does not authorize qualification. Missing either startup path fails closed.
Without `--isolated`, `qualify` remains UNKNOWN/exit 78 without runtime changes.
The shared monitor singleton excludes legacy and generic controllers. Any
existing NFQ worker/queue, active profile, unknown config/journal or existing
immutable matrix is refused. Known baseline config/journal/backup/identity
inodes and bytes remain held unchanged throughout the job. This first producer
does **not** pause/stop/restart existing active profiles; job-bound maintenance
rollback is deferred rather than pretending a path-only daemon stop is safe.
Each candidate uses the existing worker-before-nft/queue-ready lifecycle on
queue 200, fixed Lua imports and fwmark 0x10000000, plus native
`--filter-l7=tls`. The actual child executable, argv, parent, start identity,
queue socket ownership, executable/Lua hashes, native `--version` output 1.0.2
and installed nft rule bytes are attested before capture and checked again at
each collector guard. DoH's existing 0x50000000 socket mark bypass remains.
TLS policy is collector/eligibility policy, not a nonexistent native TLS-version
filter. All configured protocols are measured; incompatible selector policies
remain UNKNOWN in the proposal.
The existing GET/HEAD generic collector preserves TLS origin verification,
configured body/status/content-type expectations, private stdin and IP pinning.
Completed expectation failures and typed peer response/TLS/transport negatives
do not stop the remaining repeats/cells. Guard/source/unknown infrastructure or
batch cancellation veto the entire batch; no partial positive matrix is saved.
Config (including resolved environment/header secrets) and strategy-source
snapshots are checked after DNS/before transfer and before sealing/publication.
Workers are joined and temporary nft queues removed before verifying unchanged
disabled baseline/runtime and sealing. The batch has a fixed 15-minute deadline
with separate bounded cleanup; PID 1 keeps SS+DoH alive through cleanup.
The bounded 64 KiB matrix size (including its authenticated envelope and
worst-case field widths) is checked before any worker/nft operation; excessive
catalogue/resource combinations return UNKNOWN/exit 64 even within individual
count caps (32 strategies, 64 resources).
Successful batches, including complete negatives, return the assignment proposal.
Incomplete/unsafe jobs return UNKNOWN/blocked exit 78 without proof. PID 1 then
stays safe-idle with SS+DoH for inspection, for either success or blocked
completion: no qualification retry loop, duplicate controller or automatic
generic monitor start. Retests require future explicit immutable snapshot
rotation, not overwriting prior evidence.
Completed controller cleanup is acknowledged separately from qualification in
an exclusive, 0600 HMAC-authenticated `qualification-completed.json` on the
current boot's SS tmpfs. It binds the launched PID, exit outcome (0 or 78) and
verified cleanup; it is **not** strategy proof. PID 1 checks that ACK against its
private session launch and revalidates the disabled baseline, singleton,
SS/DoH ownership and absence of NFQ workers/temporary nft queues before shutting
down dependencies. Thus TERM after a clean blocked qualifier has exited but
before its parent reaps it can exit cleanly. Worker/nft/join/restoration failure,
missing/foreign/mutated ACK or unverified early failure still veto clean shutdown
with exit 78. Ordinary legacy/generic monitor exit 78 remains a safety veto.
No unsupported `--filter-mark` or blockcheck embedded/stream/progress/deadline
flags are used. Background noninterrupting search, active-profile maintenance
and immutable snapshot rotation for repeated runs remain deferred.
## Explicit generic transactional startup apply (local implementation)
On the target with its already completed private qualification batch, unset both
qualification startup paths and set:
```
ZAPRET_AUTOTUNE_RESOURCE_CONFIG=/private/resources.json
ZAPRET_AUTOTUNE_RESOURCE_APPLY=1
ZAPRET_AUTOTUNE_STOP_GRACE_PERIOD=90s
```
Compose forwards the apply flag and configurable stop grace (legacy default
remains 240s). Generic PID 1 cleanup has one shared 80-second budget. No external
manager, new proof importer, SQL frontend surrogate or parallel journal exists.
PID 1 starts owned SS/DoH, reconciles known bytes, launches the acknowledged
`managed resource-apply --resource-config FILE` child before the checks-only
monitor, and starts the native daemon only with verified committed config.
This is at most one apply launch per PID 1 lifetime. A RAM latch is marked failed
before launch and succeeds only after child wait=0 and owned-session cleanup.
Launch/ACK failure, nonzero/signal exit, SS/DoH loss, or incomplete cleanup leaves
it failed: after authoritative cleanup PID 1 remains cancellably BLOCKED, without
restarting SS/DoH, native zapret, apply or monitoring, even across former 60s retry
windows. Cleanup still stops SS/DoH (DNS last); BLOCKED does not keep them alive.
Successful apply is skipped on later monitor/SS restarts, but every runtime cycle
still reconciles committed refs before native start. Qualification and legacy
retry behavior are unchanged.
**Ceiling:** this is not a durable manual-intent latch. A fresh PID 1 (including
an external Docker restart policy) resets authorization and, with flag=1, permits
one new attempt. Operators must disable unattended external restart/recreation
for this workflow. Set `ZAPRET_AUTOTUNE_RESOURCE_APPLY=0` for explicit repair-only
startup: prepared state is reconciled, never promoted or reapplied. Identical
committed requests still resume by authenticated refs without a new transaction.
The command refuses standalone/docker-exec callers: actual PID 1 environment,
entrypoint, namespace, direct-child executable/argv and held launch ACK are
required, plus the existing controller singleton and no pre-existing NFQ worker.
First apply supports **disabled baseline → generic**. Explicit PID 1 startup
selection also permits generic → generic rotation among already verified
alternatives in the **same immutable authenticated registry/matrix/source**.
Active legacy profiles, changed source/catalogue/proofs and uninterrupted live
rotation remain unsupported. Every configured resource must require `both` TLS protocols and
every assigned exact host must have complete, candidate-bound passing proof for
all its resources across all input labels. New apply requires completion within
24 hours, never in the future. Negative/unknown/unassigned hosts cannot authorize
apply. Registry definitions may still describe single-TLS proposals, but native
apply rejects them instead of widening unsupported TLS-version selectors.
Canonical ordered args/selectors define profiles separated by `--new`; each
contains `--filter-tcp=443 --filter-l7=tls --hostlist-domains=^a,^b`. Exact-host
semantics use the pinned native source
`https://raw.githubusercontent.com/bol-van/zapret2/4df94497869e18ed4fdf1272141427b2fbfc6604/nfq2/hostlist.c`:
inline `AppendHostlistItem` uses `addpool`, leading `^` sets
`HOSTLIST_POOL_FLAG_STRICT_MATCH`, and `SearchHostList` rejects suffix fallback
when that flag is set and `!bHostFull`. Existing YouTube suffix policy is unchanged.
The packaged executable/Lua hash must match private attestation; held native,
Lua, source, marker and proof identities/bytes are guarded through the transaction.
No active queue version probe or mark/filter change is made (DoH500/probe100/live400).
The **existing** journal gains v4 independently tagged committed/pending refs
(`baseline`, `legacy`, `generic`). Generic refs exclude legacy IDs/scopes and bind
exact authenticated registry/matrix envelope hashes, source fingerprint,
internally derived canonical assignment digest and complete config hash.
Optional scalar `strategy_ids` retains the canonical selection policy on each
generic ref; missing/empty means the original default greedy policy. No policy
is permitted on baseline/legacy refs. This remains v4: older strict-decoding
engines reject the new field, so downgrade is blocked, not silently reinterpreted.
Unknown versions/tags, mixed refs and unresolved historical reconstruction block
without adopting or erasing foreign state; v1–v3 legacy behavior remains.
Staging/backup/prepared/replace/fixed restart/health/commit reuse the same engine.
All required resources/TLS are checked through the existing synchronous owned
SS/DoH collector (all-answer SSRF validation, CONNECT pin, SNI/cert validation,
private stdin). No Player API health or public check JSON authorizes commit.
Committed identical requests are idempotent: historical proof expiry alone does
not trigger reload, reapply or rejection on restart. Authenticated integrity,
source and native drift still block. Prepared recovery **never** promotes pending
bytes, including after successful health: it restores prior bytes and reloads.
Generic rollback additionally checks all resources before finalizing the journal.
**Strict first-slice ceiling:** a DPI-broken disabled baseline can legitimately
fail these checks; restored bytes then remain `prepared`/BLOCKED until operator
repair. Physical restoration is not falsely reported as healthy rollback.
Legacy rollback has no new health requirement. Cancellation joins synchronous
checks before rollback; PID 1 bounds the owned session and retains prepared state
on unsafe/unfinished cleanup. `ActiveCandidates` refuses generic deployments;
managed/resource status validates generic refs without inventing legacy groups.
### Explicit startup selection (implemented; bounded positive field observations)
Set `ZAPRET_AUTOTUNE_RESOURCE_STRATEGY_IDS=s1,s2` with
`ZAPRET_AUTOTUNE_RESOURCE_CONFIG` and `ZAPRET_AUTOTUNE_RESOURCE_APPLY=1` on a
fresh container PID 1. Empty retains default greedy selection. The bounded list
accepts at most 32 unique registry IDs, canonicalized in lexical order; malformed,
duplicate, unknown or incomplete-coverage requests cannot authorize a transaction.
Eligibility is filtered **before** equal-args/selectors coverage merging; excluded
candidates cannot lend proof to a selected profile. Compatibility evidence and
all alternatives remain visible; the policy is not proof or a new qualification.
No registry/matrix files are written by rotation. Exact hosts, both TLS versions,
all required cells/M=3, native/source integrity and the 24-hour freshness gate
remain required for every new selection. Identical committed selection replays
without reload even after expiry. Distinct redundant allowlists are distinct
policies and may create a transaction even if rendered bytes coincide.
PID 1 forwards the setting in environment only; the apply child authenticates it
against PID 1's original environment, not CLI arguments. Syntax is validated
even with APPLY=0; nonempty policy requires generic resource config. With APPLY=0
selection is ignored for reconciliation/monitoring (registry membership is not
resolved): committed and pending refs reconstruct independently from their own
stored policies and historical completion horizon, never the requested setting.
Prepared recovery restores the old generic bytes **and policy**, never promotes
the pending side; failed rollback health retains prepared state for repair.
Startup still allows just one attempt per PID 1; external restart policies reset
that authorization. There is no monitor autotuning, new qualification, proof
import, snapshot overwrite or uninterrupted maintenance in this slice.
Local root fixture tests cover the production snapshot factory and shared engine;
independent local review and deterministic 180-second retry-window fixtures passed.
The earlier bounded baseline gates below cover guarded apply/replay, strict rollback,
prepared repair and the per-PID-1 failure latch; they do not establish rotation fault
acceptance or authorize live rollout.
### Verified-profile rotation observations — source `8d43dd9`
**Positive field observations only; partial native acceptance, not full rotation
safety acceptance or READY.** Saved-artifact privileged audits observed default A
generation 1 → selected multisplit B generation 2 → identical B replay generation 2
→ A return generation 3. No repeat positive sequence is required for the next gate.
- A config SHA-256 was `27f9a8ffd56a4c19d4bd522dd9f5a5e8b49b18b0ae48d41431ef9b22975ea772`;
B was `51faacf53e76fa0ce95fead1f02da3846c1706ecb9edc8ddaba095c089ea2c82`.
Replay retained transaction `09dfd03e9741abdf8e6b9f3bd44b2118`; A return used
`49a29d560f086a11f9de770913777e07` and restored the exact original A bytes.
Historical prepared snapshots held actual old-A backup before B and actual old-B
backup before A. Sampled old-committed → prepared → new-committed transitions
demonstrate these observations, not crash safety or failed-rotation rollback.
- Fourteen saved registry/matrix snapshots passed exact-payload HMAC and byte
invariance checks; source fingerprint was unchanged. The two original runs,
16 M=3 cells and 48 prior both-TLS qualification transfers were reused, about
12 hours old at startup, **not requalified**. Native A/B argument hashes differed;
A return matched original A, with native queue 300 socket-inode/PID linkage
observed. Early A-return process inventory is missing; full phase-path native
argv acceptance is not established.
- Six authenticated monitor snapshots each showed four HEALTHY resources and zero
failures. Saved fixture records showed 64 successful GETs, TLS 32/32, matching
SNI/Host and expected-header presence; A return contributed 32, TLS 16/16.
This proves fixture compatibility, **not causal DPI benefit** or continuously live
change. Original A-return official stop exited 0; fixture stop exited 2, not an
observed product failure in this run.
- Evidence base: `/home/agent/rotation-8d43dd9/field-private/` (root-private).
`report.safe.json` covers only generations 1/2/2, not final completion. Generation
3 is supported separately by `a-return-timeline.private.json`, retained state and
`final-passive-evidence/{before,stopped}/` snapshots. The initial harness ended
with KeyboardInterrupt; two continuation-construction bugs and observer filename
alias corrections are disclosed in `harness-path-alias.safe.json`. Corrections
affected host observer artifacts only, not proof inputs.
- Final saved cleanup verification is
`final-passive-evidence/final-cleanup-verification.safe.json`, timestamp
`1792051407108`: both exact owned stage containers and the owned network absent,
eight recorded PID paths absent, both known network namespaces without PIDs,
and three foreign full-inspect hashes unchanged. Owned containers/network were
explicitly removed with orchestrator approval; private state/evidence/keys were
retained. Foreign SS remained running and the two older containers unchanged.
Xray was not mutated; no fresh Xray inventory or exhaustive process audit is claimed.
- The first independent forensic audit passed the bounded 14-HMAC positive evidence,
but the automated grader rejected missing independently refreshed dated evidence
and exact native argv phase paths. Do not upgrade this to full native/safety
acceptance. Unknown-ID refusal and old-profile fault/repair observations were
subsequently recorded below, with narrower attribution than a branch-specific
refusal proof. Crash safety remains untested. Natural 24-hour expiry remains
unit-only, without a 24-hour wait.
- Prior local verification recorded 17 EUID-0 fixtures in one race/parallel pass
with 57 subrecords; namespace UID 0 mapped to host UID 100000, not host-initial
root. Full producer Go/race and four Python checks passed serially after parallel
resource-contention failures; this does not establish a contention-free baseline.
These are prior checks, not tests rerun for this documentation-only update.
### Subsequent old-generic rotation faults (bounded saved-artifact acceptance)
Actual remote source was `8d43dd9`, **not** shared-core `45d28e3`; these separate
results do not establish end-to-end acceptance of a combined release. Privileged
reads verified `/home/agent/rotation-fault-8d43dd9/field-private/ACCEPTANCE.safe.md`
and `standaloneprivilegedaudit.safe.json`, dated `1792054235151`
(2026-10-15 08:50:35.151 UTC), verdict
`PASS_BOUNDED_SAVED_ARTIFACTS_AND_CURRENT_CLEANUP`.
- All 13 snapshots passed exact-serialized-payload HMAC/immutable proof checks.
Old A is generic with empty/default policy, not baseline; B selects multisplit.
Historical old/new refs reconstruct independently.
- One-shot: eight B apply checks (one HTTP500), eight successful A rollback checks,
A committed/pending clear, no automatic reapply through 80s; official stop0.
- Persistent: 24 checks across apply/rollback/cleanup reconciliation, three
HTTP500; A bytes physically restored with PREPARED old-A/pending-B retained,
no reapply through 80s; stop78 retained the unsafe-cleanup latch. Healthy fixture
plus only APPLY=1→0 (requested B still set) gave eight reconcile + eight monitor
successes, old A committed/pending clear and four HEALTHY resources; stop0.
- Total: 56 authenticated requests, TLS28/28, four controlled HTTP500. At the
dated audit all 150 recorded owned PID/start identities were gone, recorded netns
had no processes, five owned containers and network were removed, and three
foreign raw-inspect hashes were unchanged. Root-private evidence was retained;
sampled identities are not an exhaustive all-helper inventory.
- Valid-syntax `unknown-safe-id` reached the actual CLI apply entry, which exited
78 and latched BLOCKED, with zero requests. Before/stopped config, journal,
backup and proof bytes were unchanged; no staging was sampled. This is an
**observed refusal with an unknown-ID request**, not uniquely attributed to the
unknown-ID branch: the generic diagnostic cannot exclude another trust error.
Source validation order supports an inference, not a specific logged reason
or complete pre-mutation proof. Official stop0 is separate from CLI exit78.
Startup remains one-shot per PID1, without automatic apply retry. These fixture
faults do not establish crash/cancellation safety, continuously live rotation,
causal DPI benefit or READY. Next migration work is the minimal typed generic
image/media capability above; owned managed v2 scoped observations are recorded
above, with full runtime acceptance still incomplete.
New-batch/versioned stores, uninterrupted rotation, strategy breadth and Prometheus
are separate future work, not prerequisites to documenting this collector slice.
## Bounded generic apply/replay and failure-latch acceptance
**DONE within the stated bounds**, not full generic autotuning or service READY.
Fault acceptance used source `643d0c6`; the earlier healthy committed replay used
`36aace0` after the atomic launch-ACK correction. Independent privileged audit
returned `PASS_BOUNDED_GENERIC_APPLY_FAULTS` for the observed A/B/C safety gates.
Root-private evidence references (no private payloads reproduced here):
- `/home/agent/generic-latch-643d0c6/field-private/field-report.md`
- `/home/agent/generic-latch-643d0c6/field-private/independent-audit.safe.json`
- `/home/agent/ack-recreate-36aace0/field-private/field-report.md`
### Observed acceptance
- Real apply health reused authentic, still-fresh qualification: 16 M=3 cells,
48 prior TLS transfers, two strategies and four resources. No new qualification,
timestamp rewrite, fake ACK or imported proof was used. Pinned native 1.0.2/Lua
stayed unchanged. The earlier healthy start/recreate each made eight TLS checks
with four authenticated HEALTHY resources; v4 committed generation 1, refs,
config and proof bytes remained unchanged. Both official stops exited 0.
- **A — one HTTP 500:** eight apply plus eight rollback requests, one 500 total.
Rollback committed prior baseline generation 1 and cleared pending. Request
count, journal and config stayed frozen at 45.096/90.084/135.124 seconds;
one apply only, no automatic retry. PID 1 stayed alive/BLOCKED with SS/DNS and
other owned runtime stopped. Official stop exited 0.
- **B — persistent HTTP 500:** 24 requests and three 500s: eight apply, eight
rollback and eight cleanup-reconcile checks, not a second apply. Prior baseline
bytes were physically restored, but failed health retained PREPARED committed
baseline/pending generic refs. State stayed unchanged for 80.097 seconds with
no reapply; official stop exited 78. After repairing only fixture behavior and
changing startup APPLY=1 to APPLY=0, the same state/source/environment completed
eight reconcile and eight monitor checks. Baseline became committed, pending
cleared, and generic was never promoted; official stop exited 0.
- **C — TERM during a real slow request:** one canceled request plus eight passing
rollback checks. The canceled curl was joined before the first rollback request;
baseline generation 1 committed and pending cleared, with no false generic
commit. Actual PID 1 exit was **78**, with `monitor cleanup status 78`, not 0.
Apply health cancellation is collapsed into `errPlan`/CLI 78; outside the
qualification-specific completion path, the supervisor accepts only 0/143 and
retains a sticky failure even after verified rollback. This is conservative
**outcome conflation**, not an observed rollback-health failure or cleanup
defect. Do not blanket-allow 78 or manufacture a completion ACK. A normal-0
path would require authenticated completion plus fresh guards; that optional
future refinement is not a blocker for this bounded P0 safety slice.
### Bounds and remaining work
- A/B/repair/C totalled 65 actual HTTPS requests. All 171 observed owned PID/start
identities disappeared, owned netns were empty, and owned targets/network were
removed. Foreign SS/container full-inspect hashes stayed unchanged. Passive
100ms inventories do not cover every shorter-lived helper; no exhaustive argv
or packet-level privacy audit is claimed. Same-root HMAC-key mutation is outside
the trust boundary.
- The latch is one RAM authorization per PID 1, **not durable manual intent**.
Fresh startup/external restart policy reauthorizes ENV=1; keep unattended
external restart/recreation disabled. ENV=0 repairs/reconciles only. Historical
committed replay/repair allows proof expiry; a new apply still requires fresh,
complete source-bound qualification within 24 hours. Natural remote 24-hour
expiry remains unit-covered only; remote qualifier singleton conflict is still
untested.
- Use an execution-free remote harness: stage inputs before startup; no target
`exec`, `cp` or `nsenter` during active guards. Observe host metadata/private
files and Docker logs/inspect; official stop is the cancellation operation.
Earlier scan EACCES nonrecurrence does not establish its cause.
- No production READY, causal DPI benefit, active-profile maintenance, plan or
snapshot rotation, automatic recovery or monitor autotuning is accepted here.
Next roadmap work is the missing rotation fault/repair gate above, then dynamic
profile/plan rotation and compatible selectors,
generic resource tuning and bounded strategy-DB collections; bounded-ID
Prometheus remains a separate planned function. No new framework, probes or
native/live upgrade was introduced. Legacy YouTube exit 78 and unfinished
images/avatars remain NOT READY.
## Bounded remote acceptance — source `5cdc0cd`
The isolated producer/registry/matrix/planner slice is **bounded-qualified**.
Independent privileged evidence audits passed the observed positive, fault and
active-state refusal gates. **Full remote concurrency acceptance is PARTIAL**;
this is not generic apply, active-profile maintenance or service READY.
Root-private proof references (retained reports, not logs copied into this doc):
- `/home/agent/zapret2-autotune-matrix-5cdc0cd/execution-free-private/execution-free-report.md`
- `/home/agent/zapret2-autotune-matrix-5cdc0cd/fault-final-private/qualification-fault-report.md`
- `/home/agent/zapret2-autotune-matrix-5cdc0cd/refusal-gap-private/remaining-refusal-reports.md`
### Observed positive and fault gates
- The execution-free private batch made 60 actual transfers: two candidates,
five resources, both TLS protocols, three attempts per cell. Its 20 aggregated
cells contain 16 PASS and four `expectation_failed`: mandatory resource d's
HTTP 500 fails for both candidates/protocols, leaving d unassigned. Exact
hosts a/b/c receive the greedy multidisorder proposal with authentic multisplit
alternatives. Input groups remain metadata, not assignment boundaries;
there is no generic suffix coverage or automatic apply.
- Independent verification authenticated the exact stored payload HMAC, not a
reserialized JSON equivalent. Distinct native arguments/worker-argv hashes,
executable/Lua/rules evidence, queue 200 and NFQ 1.0.2 attestations bind the
two joined/restored runs. Disabled baseline bytes stayed unchanged; sampled
workers, queue and temporary NFT rules were gone. Official PID 1 shutdown
exited 0 in 1.31s; foreign inspect hashes stayed unchanged.
- Mid-batch cancellation and effective semantic config drift each stopped
after two requests, published no matrix and preserved the native baseline.
Official PID 1 exits were 0 in 2.46s and 0.44s respectively; the deliberately
changed config source was retained, not silently rolled back.
- Unsafe Lua/queue/fwmark/user/file definitions and evidence/schema imports
were rejected without transfers. Oversized capacity was rejected before
worker launch; unauthorized isolated startup was refused. Corrupt MAC blocked
planning while preserving payload bytes; changed effective config produced
unknown/unassigned output without changing the original authenticated matrix.
Immutable catalogue replacement and completed-matrix retest were refused
without transfers or overwriting original state.
- A genuine archived v3 committed journal with two candidates passed official
`reconcile`/`ActiveCandidates` validation before qualification refused active
profile state. Profile/config, journal, last-good and target identity remained
byte-identical; zero transfers/workers and no matrix. Actual PID 1 exit 78 was
an expected early preflight refusal, **not** an exit-0 cleanup ACK. This did
not activate or maintain a running active-profile NFQ daemon; that remains
unsupported. Whole-DB byte preservation is not claimed.
### Explicit limits and testing lesson
- Remote qualifier singleton conflict is **NOT RUN**. The earlier planted-lock
fixture failed initialization before that branch and does not count. The
shared-lock unit second-acquire `EWOULDBLOCK` check passed, but is not remote
qualifier concurrency acceptance. No deterministic official lifecycle seam
exists for this bounded test; no test framework or racy injection was added.
- Natural 24-hour remote expiry is **NOT RUN**. The existing unit check rejects
a structurally valid 25-hour-old proof; this is not observed remote expiry.
- Privacy sentinel absence was checked in tested logs/state and sampled fault/
refusal argv. Positive active argv and exhaustive DNS packet-level privacy
were not proved. Configured/observed encrypted DNS is not exhaustive proof
against transient fallback.
- Compatibility on this baseline is not causal DPI-desync benefit. Historical
worker-scan EACCES origin remains unknown; nonrecurrence in the execution-free
batch does not identify its cause. Earlier harness failures remain recorded
in the referenced reports, not erased or relabeled as product acceptance.
- Stage all inputs before target startup. From startup through qualification
completion, prohibit target `exec`, `cp` and `nsenter` (even passive exec starts
a process). Observe host `/proc` metadata and Docker logs/inspect only; inspect
runtime and run `plan` after finalization/cleanup. The explicit official stop
used for cancellation is the documented exception, not an observer exec.
- Owned test containers/network were removed and private evidence retained.
Foreign SS1080 stayed running and legacy11082 stayed stopped with exit 78.
No live upgrade or push occurred; unresolved legacy exit 78, YouTube/image
readiness, transactional generic apply/profile-journal migration, immutable
rotation and noninterrupting search remain outside this producer acceptance;
the later bounded apply/replay gates are recorded separately above.
## Internal YouTube adapter slice (local; field acceptance pending)
`internal/adapter` is a small closed compiled-in contract (`Binding`, ephemeral
`Resource`/`Plan`, `Module`); `internal/adapter/youtube` implements `youtube/1`.
The explicit kind/version registry in `service_adapters.go` has no init hooks,
dynamic loading, public configuration, HTTP fallback or module privileges.
Player media parsing (`playerMediaResource`) and owned native qualification
(`verifyGooglevideo`) actually call it. Discovery retry/rotation/cancellation and
representative selection remain in the legacy controller; signed MP4 freshness
and the host/TLS qualification plan are pure service semantics in the module.
The next private execution bridge is now wired: `verifyGooglevideo` runs the
actual owned-worker collector through `executeAdapterPlan[T]`. Core independently
pins the known module binding and exact three-host/TLS1.2+1.3, 10-attempt/120-second
warmup, three steady samples/protocol and 1024-byte policy before worker startup
and probes. Invalid, duplicate, overflowing or weakened plans fail closed. The
executor preserves host order, alternating warmups, discarded learning successes,
strict measured steady facts and negative/fatal classification. It has no worker,
goroutine, persistence or seal capability; existing core range/ftyp/TLS validation
and post-worker-join proof validation/sealing remain the authority.
This is a **private execution bridge**, not public adapter configuration or a new
generic service/matrix proof. Dynamic-inventory fingerprints and warmup-bearing
generic proof binding are still required before generic-controller cutover.
The inherited legacy SOCKS5h boundary below is unchanged, not an all-DNS-answer
pinning guarantee. Three-CDN qualification does not itself grant suffix scope;
images/avatars and runtime acceptance remain separate and not READY.
Player JSON now travels through bounded private stdin (`curl --json @-`), not
process argv; the shared runner passes no stdin for metadata GETs. Video IDs are
validated before execution, POST bytes and headers remain unchanged, and opaque
public errors retain typed cancellation/unsafe-output causes. This is a private
Player transport change, not generic public POST/discovery configuration.
Bridge-local checks: an unchanged fixture run against archived `f485317` and the
wired producer has normalized proof SHA256
`fb380324e320a78f1ceb011c59b82b9de4ff4f31f5f032f469a181292987e3b2`.
Only warmup duration, attestation time and private seals are excluded; exact
probe order and cleanup/join are checked. Baseline and focused serial/race logs:
`/tmp/opencode/adapter-executor-{baseline,focused}.log`. These are injected local
checks, not network/native acceptance or a full-suite rerun.
Cancellation review follow-up: both warmup and steady probe boundaries now use
`managedRecoveryContextError`, preserving concurrent unsafe I/O/output overflow
alongside cancellation/deadline causes. Production collector regressions check
fatal/non-restartable classification, full cleanup/join and no attestation/seal;
plain cancellation remains timeout-safe. A pre-fix Go-overlay negative control
fails all eight mixed-failure cases; the fixed producer retains the hash above.
Actual `sudo` EUID-0 serial full Go checks passed (1806 test/subtest pass events),
as did 227 affected serial/race events, separate module race tests, vet and a
CGO-disabled build. Logs: `/tmp/opencode/adapter-executor-root-full.json`,
`adapter-executor-context-root-race.json`, `adapter-executor-context-focused.log`
and `adapter-executor-context-negative-control.log` in that same directory.
The reviewer-reported earlier `SavedFirstAndFailClosed/nonpass` freshness failure
did not recur in ten focused repeats or this full run; no freshness guards or
fixtures were changed, and its earlier cause is not established as a baseline
flake. Python checks were not rerun in this follow-up. These are local tests,
not remote runtime acceptance, a commit/deployment or a READY claim.
Core origin/TLS validation, shared image/media transfer code, bounded worker
lifecycle and post-cleanup/join sealing remain authoritative. Legacy media and
metadata routing retains SOCKS5h with the owned encrypted-resolver guard and
exact origin/SNI/certificate/redirect checks. It does **not** use public generic
`runResourceProtocol`'s all-A/AAAA approval and connect-to IP pinning. That inherited
ceiling is unchanged, not a pinning guarantee: full request-policy capabilities
(including future metadata POST) and parity are prerequisites to generic cutover.
No new DNS/IP routing policy is introduced by this extraction. Both TLS
versions must learn separately, then each has three measured MP4 samples per
representative host, minimum 1024 bytes (not an exact 64 KiB requirement).
Measured endpoints do not authorize a suffix: existing exact/suffix/experimental
policy and independent thumbnail/avatar qualification are unchanged. The module
cannot manufacture evidence, request wildcard apply or claim YouTube READY.
New adaptive proof **v2** includes adapter kind/version/immutable policy SHA256
inside the producer seal. Public JSON cannot recreate that seal. Missing or
mismatched binding/version/digest fails closed; v1 with v2 fields is rejected.
Historical proof **v1** explicitly uses the pinned legacy YouTube v1 contract,
including historical freshness-independent journal reconstruction. Its config
bytes/hash and journal refs remain stable. Fresh stored trusted v1 proofs retain
their original apply/qualification eligibility, including unchanged attestation
freshness and media gates. Expired v1 can reconstruct but cannot authorize fresh
apply. Only **new production/persistence** requires sealed, module-bound v2;
neither sealed-new-v1 nor public unsealed v1/v2 DTOs can publish new evidence.
Cold v1 evidence, SQL schema 7, native arguments,
public GET/HEAD v1/v2, state keys and HMAC formats are unchanged.
This slice is not full generic-controller migration, native-engine runtime
acceptance, a security sandbox (including against same-UID actors), or a fix for
unresolved avatars/legacy exit 78. No legacy path is deleted. Local tests and
archived baseline parity are recorded separately; no remote validation or commit
is part of this extraction review.
Discovery preflight resolves the required module identity and policy digest
before any Player/WWW/avatar/front/media requests, including injected callbacks,
legacy proposal discovery and managed discovery hooks. Missing/mismatched modules
produce a fixed private sentinel wrapped as a fatal configuration failure, not
retryable unavailable inventory, a transfer-negative or genuine no-strategy pause.
The compiled production registry remains fixed; synchronized test replacement
does not add public runtime module configuration or loading.
### Local verification for this extraction (2026-10-15)
- Fresh serial `go test -json -p 1 ./... -parallel 1 -count=1 -timeout 600s`:
399 top-level / 1737 including subtest PASS events, zero failures.
- Affected serial race run (`-race -p 1 -parallel 1 -count=1`) covers adapters,
Player/discovery, owned qualification/proof gates, historical journal parity,
independent assets and generic resource core: 61 top-level / 307 including
subtest PASS events, zero failures. Full regex and logs are retained at
`/tmp/opencode/youtube-adapter-fixes/{full,race}.json`.
- Archived `d324b4137b2e95f1c5f4d6c8240470e3f8749f0c` and current builds each
passed 275 affected test/subtest events. The portable legacy parser/journal
corpus ran **unmodified** on both: file SHA256
`7b8eea9c9bbcdd8f0fadde3f78596d41e9d032a0a4f7b2ca90d1223d106584d5`.
Both reconstructed historical config SHA256
`172c7377388334e4baa2dc1009270478dc271a7b2f00d28cfada631f44a35364`,
retaining journal bytes/refs after reconcile and ActiveCandidates validation.
An additional unchanged portable v1 corpus passed 2 top-level / 8 including
subtest events on **each** build: fresh stored v1 shortlist/apply eligibility,
expired fresh-apply refusal, and staged/backup/prepared/replaced/reloaded/healthy
crash rollback with expired historical proofs. Backup/config bytes and hashes
are stable; pre-prepare journal bytes are unchanged and post-prepare generation/
reload/ref transitions follow the original contract. Corpus SHA256:
`6df53b12af5d80d164ea3201a614a5fd5723561dc7b2d6bc2d2aa37f845f053d`.
New negative tests separately reject even internally sealed NEW v1 writes and
public unsealed DTOs; missing/changed module identity/policy makes zero Player,
WWW/avatar/media/front/worker calls and does not become a two-hour absence.
- All four `python3 deploy/managed-target/check_{static,config,startup_diagnostics,supervision}.py`
checks passed serially; supervision was bounded with `timeout 600s`, no retry.
`go vet ./...`, `CGO_ENABLED=0 go build`, `sh -n` on all three entrypoint/
supervision scripts, and `git diff --check` passed. Build SHA256:
`8327faf86098145bb41306b3de895cdd622aeccdcda90f8ec73e19a759c8f98a`.
- Ownership-sensitive final runs used `sudo -n` EUID 0 **inside the existing
user namespace**, mapping UID 0 to parent/host UID 100000, not initial-host
root. The initial extra single-UID `unshare -Ur` run failed ownership fixtures
(`chown: invalid argument` / `invalid private qualification data`); a stale
test assertion also expected newly produced proof v1 and was updated to v2.
Those prior extraction failures are retained in
`/tmp/opencode/youtube-adapter-review/go-full.json`, not reported as passes. Vet's
initial unkeyed shared-DTO literals were fixed before final successful checks.
Reproduction commands (run serially; root is only for offline ownership fixtures):
```sh
pattern='Test(YoutubeLegacy|ManagedGooglevideo|ManagedAdaptive|ExperimentalPromotion|Googlevideo|Resource(Core|Discovery|Contract)|Player|Discover|ManagedAsset|Asset)'
sudo -n env GOPROXY=off GOTOOLCHAIN=local GOMODCACHE="$(go env GOMODCACHE)" "$(command -v go)" test -json -p 1 ./... -parallel 1 -count=1 -timeout 600s
sudo -n env GOPROXY=off GOTOOLCHAIN=local GOMODCACHE="$(go env GOMODCACHE)" "$(command -v go)" test -race -json -p 1 ./internal/adapter/... ./cmd/zapret-autotune -parallel 1 -run "${pattern}|Test(ServiceAdapter|LegacyYoutubeV1|QualificationPlan|SignedSample|DiscoverResponse)" -count=1 -timeout 600s
# Baseline/current parity: the same $pattern, without -race, on both trees.
# Baseline tree: git archive d324b4137b2e95f1c5f4d6c8240470e3f8749f0c;
# add identical youtube_legacy_parity_test.go and youtube_legacy_apply_parity_test.go.
# The additional portable v1 corpus runs separately with -run '^TestLegacyYoutubeV1'.
```
+125
View File
@@ -0,0 +1,125 @@
package main
import (
"context"
"errors"
"time"
"zapret-autotune/internal/adapter"
"zapret-autotune/internal/adapter/youtube"
)
var errAdapterPlan = errors.New("invalid private adapter execution plan")
// This is core policy, not module-supplied caps. Adding a binding requires an
// audited execution policy here; even a correctly identified module is untrusted.
func validateAdapterPlan(binding adapter.Binding, plan adapter.Plan, resources map[string]mediaResource) error {
if binding != (youtube.Module{}).Binding() || len(plan.Hosts) != 3 ||
plan.Protocols != [2]string{"tls12", "tls13"} || plan.Samples != 3 ||
plan.MinBytes != 1024 || plan.WarmupLimit != 10 || plan.WarmupTimeout != 120*time.Second {
return errAdapterPlan
}
if (youtube.Module{}).ValidateHosts(plan.Hosts) != nil {
return errAdapterPlan
}
for _, host := range plan.Hosts {
resource, ok := resources[host]
if !ok || !validDomain(host) || resource.host != host {
return errAdapterPlan
}
if _, err := privateResourceURL(resource.url, host); err != nil {
return errAdapterPlan
}
if !youtube.SignedSampleFresh(youtube.Resource{URL: resource.url, Host: host, Kind: resource.kind}, time.Now(), 120*time.Second) {
return errAdapterPlan
}
}
return nil
}
// Typed measurements stay private and ephemeral. Only core callbacks collect
// and classify them; modules cannot supply callbacks, attestations or seals.
type adapterMeasurement[T any] struct {
value T
bytes int64
}
type adapterHostMeasurements[T any] struct {
host string
warmupAttempts, warmupFailures int
warmupLimit int
warmupMS float64
samples []T
}
// executeAdapterPlan owns scheduling only. Its caller owns the worker, transport
// validation and post-join proof validation/sealing. No goroutines or persistence.
func executeAdapterPlan[T any](ctx context.Context, binding adapter.Binding, plan adapter.Plan, resources map[string]mediaResource,
probe func(context.Context, mediaResource, string) (adapterMeasurement[T], error), nonpass func(error) bool,
) ([]adapterHostMeasurements[T], error) {
if err := validateAdapterPlan(binding, plan, resources); err != nil {
return nil, err
}
if err := ctx.Err(); err != nil {
return nil, err
}
hosts := make([]adapterHostMeasurements[T], len(plan.Hosts))
for i, host := range plan.Hosts {
hosts[i].host = host
}
for i := range hosts {
h := &hosts[i]
h.warmupLimit = plan.WarmupLimit
start := time.Now()
warmCtx, cancel := context.WithTimeout(ctx, plan.WarmupTimeout)
learned := map[string]bool{}
for h.warmupAttempts < h.warmupLimit && len(learned) < len(plan.Protocols) && warmCtx.Err() == nil {
protocol := plan.Protocols[h.warmupAttempts%len(plan.Protocols)]
h.warmupAttempts++
measurement, err := probe(warmCtx, resources[h.host], protocol)
err = managedRecoveryContextError(warmCtx, err)
if err != nil {
h.warmupFailures++
if !nonpass(err) && (warmCtx.Err() == nil || !managedRecoveryTimeoutSafe(err)) {
cancel()
return hosts, err
}
} else if measurement.bytes < plan.MinBytes {
h.warmupFailures++
} else {
learned[protocol] = true
}
}
h.warmupMS = float64(time.Since(start).Microseconds()) / 1000
cancel()
if err := ctx.Err(); err != nil {
return hosts, err
}
if len(learned) != len(plan.Protocols) || h.warmupMS > float64(plan.WarmupTimeout/time.Millisecond) {
return hosts, errCandidateNonpass
}
for _, protocol := range plan.Protocols {
for n := 0; n < plan.Samples; n++ {
if err := ctx.Err(); err != nil {
return hosts, err
}
measurement, err := probe(ctx, resources[h.host], protocol)
err = managedRecoveryContextError(ctx, err)
if ctx.Err() != nil {
return hosts, err
}
if err != nil {
if nonpass(err) {
return hosts, errCandidateNonpass
}
return hosts, err
}
if measurement.bytes < plan.MinBytes {
return hosts, errCandidateNonpass
}
h.samples = append(h.samples, measurement.value)
}
}
}
return hosts, nil
}
@@ -0,0 +1,148 @@
package main
import (
"context"
"crypto/sha256"
"encoding/json"
"errors"
"fmt"
"io"
"reflect"
"strings"
"testing"
"time"
)
// Also runnable unchanged against archived f485317. Only wall-clock duration,
// attestation time and process-private seals are excluded from the golden hash.
func TestAdapterExecutorLegacyParity(t *testing.T) {
r, c, resources, events := googlevideoRuntimeFixture(t)
runner := r.curl
var order []string
r.curl = func(ctx context.Context, args []string, stdin io.Reader, out, diagnostic io.Writer) error {
protocol := "tls12"
if strings.Contains(strings.Join(args, " "), "--tlsv1.3") {
protocol = "tls13"
}
order = append(order, protocol)
if len(order) <= 2 {
return context.DeadlineExceeded
}
return runner(ctx, args, stdin, out, diagnostic)
}
p, err := r.verifyGooglevideo(context.Background(), c, resources)
if err != nil || p.producerSeal == nil {
t.Fatal(err, p)
}
if !strings.HasSuffix(strings.Join(*events, ","), "delete,join") {
t.Fatal(*events)
}
expected := []string{"tls12", "tls13"}
for range c.Endpoints {
expected = append(expected, "tls12", "tls13", "tls12", "tls12", "tls12", "tls13", "tls13", "tls13")
}
if !reflect.DeepEqual(order, expected) {
t.Fatal(order, expected)
}
for i := range p.Hosts {
p.Hosts[i].WarmupMS = 0
}
p.Attestation.CheckedAt = time.Unix(1700000000, 0).UTC()
raw, err := json.Marshal(p)
if err != nil {
t.Fatal(err)
}
hash := fmt.Sprintf("%x", sha256.Sum256(raw))
t.Log("normalized proof sha256:", hash)
const baseline = "fb380324e320a78f1ceb011c59b82b9de4ff4f31f5f032f469a181292987e3b2"
if hash != baseline {
t.Fatalf("proof parity changed: %s\n%s", hash, raw)
}
}
func TestAdapterExecutorProductionCancellationJoinsWithoutSeal(t *testing.T) {
for _, at := range []int{1, 3} {
t.Run(fmt.Sprint(at), func(t *testing.T) {
r, c, resources, events := googlevideoRuntimeFixture(t)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
runner, calls := r.curl, 0
r.curl = func(probeCtx context.Context, args []string, stdin io.Reader, out, diagnostic io.Writer) error {
calls++
if calls == at {
cancel()
return ctx.Err()
}
return runner(probeCtx, args, stdin, out, diagnostic)
}
p, err := r.verifyGooglevideo(ctx, c, resources)
if !errors.Is(err, context.Canceled) || calls != at || p.Attestation != nil || p.producerSeal != nil ||
!strings.HasSuffix(strings.Join(*events, ","), "delete,join") {
t.Fatal(p, err, calls, *events)
}
if !managedRecoveryTimeoutSafe(err) {
t.Fatal("plain cancellation became unsafe", err)
}
})
}
}
func TestAdapterExecutorProductionCancellationPreservesUnsafeTransfer(t *testing.T) {
unsafeIO := errors.New("unsafe collector IO")
for _, at := range []int{1, 3} { // first warmup or first steady after both TLS versions learn
for _, deadline := range []bool{false, true} {
for _, overflow := range []bool{false, true} {
t.Run(fmt.Sprintf("probe%d/deadline%t/overflow%t", at, deadline, overflow), func(t *testing.T) {
r, c, resources, events := googlevideoRuntimeFixture(t)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
wantContext := context.Canceled
if deadline {
var stop context.CancelFunc
ctx, stop = context.WithTimeout(ctx, time.Second)
defer stop()
wantContext = context.DeadlineExceeded
}
runner, calls := r.curl, 0
r.curl = func(probeCtx context.Context, args []string, stdin io.Reader, out, diagnostic io.Writer) error {
calls++
if calls != at {
return runner(probeCtx, args, stdin, out, diagnostic)
}
if deadline {
<-ctx.Done()
} else {
cancel()
}
if overflow {
// Exercise the real bounded collector writer, not a forged fatal DTO.
_, writeErr := out.Write(make([]byte, 256<<10))
return errors.Join(writeErr, ctx.Err())
}
return errors.Join(unsafeIO, ctx.Err())
}
p, err := r.verifyGooglevideo(ctx, c, resources)
var fatal *managedRecoveryFatal
if !errors.Is(err, wantContext) || !errors.As(err, &fatal) || managedRecoveryTimeoutSafe(err) || managedResourceTransferNonpass(err, false, false) {
t.Fatalf("lost mixed safety/context error: %v", err)
}
if overflow {
var collected *resourceError
if !errors.As(err, &collected) || collected.reason != reasonOutputBound {
t.Fatal("lost collector overflow", err)
}
} else if !errors.Is(err, unsafeIO) {
t.Fatal("lost unsafe IO cause", err)
}
if calls != at || p.Attestation != nil || p.producerSeal != nil ||
!strings.HasSuffix(strings.Join(*events, ","), "delete,join") {
t.Fatal(p, err, calls, *events)
}
if at == 3 && (p.Hosts[0].WarmupAttempts != 2 || len(p.Hosts[0].Passes) != 0) {
t.Fatal("learning leaked into steady proof", p.Hosts)
}
})
}
}
}
}
@@ -0,0 +1,248 @@
package main
import (
"context"
"errors"
"math"
"reflect"
"testing"
"time"
"zapret-autotune/internal/adapter"
"zapret-autotune/internal/adapter/youtube"
)
func adapterExecutorFixture(t *testing.T) (adapter.Binding, adapter.Plan, map[string]mediaResource) {
t.Helper()
hosts := []string{"rr1---sn-alpha.googlevideo.com", "rr2---sn-beta.googlevideo.com", "rr5---sn-gamma.googlevideo.com"}
resources := map[string]mediaResource{}
var samples []youtube.Resource
for _, host := range hosts {
url := "https://" + host + "/videoplayback?expire=4102444800&sig=private"
resources[host] = mediaResource{host: host, url: url, kind: "video/mp4"}
samples = append(samples, youtube.Resource{Host: host, URL: url, Kind: "video/mp4"})
}
m := youtube.Module{}
p, err := m.PlanQualification(hosts, samples, time.Now())
if err != nil {
t.Fatal(err)
}
return m.Binding(), p, resources
}
func TestAdapterExecutorRejectsUntrustedPlanBeforeProbe(t *testing.T) {
mutations := map[string]func(*adapter.Binding, *adapter.Plan, map[string]mediaResource){
"binding-kind": func(b *adapter.Binding, _ *adapter.Plan, _ map[string]mediaResource) { b.Kind = "other" },
"binding-version": func(b *adapter.Binding, _ *adapter.Plan, _ map[string]mediaResource) { b.Version++ },
"binding-policy": func(b *adapter.Binding, _ *adapter.Plan, _ map[string]mediaResource) { b.PolicySHA256 = "" },
"missing-host": func(_ *adapter.Binding, p *adapter.Plan, _ map[string]mediaResource) { p.Hosts = p.Hosts[:2] },
"extra-host": func(_ *adapter.Binding, p *adapter.Plan, _ map[string]mediaResource) {
p.Hosts = append(p.Hosts, "rr9---sn-delta.googlevideo.com")
},
"duplicate-host": func(_ *adapter.Binding, p *adapter.Plan, _ map[string]mediaResource) { p.Hosts[1] = p.Hosts[0] },
"host-url": func(_ *adapter.Binding, p *adapter.Plan, _ map[string]mediaResource) {
p.Hosts[0] = "https://example.com"
},
"host-case": func(_ *adapter.Binding, p *adapter.Plan, _ map[string]mediaResource) {
p.Hosts[0] = "RR1---sn-alpha.googlevideo.com"
},
"host-suffix": func(_ *adapter.Binding, p *adapter.Plan, _ map[string]mediaResource) {
p.Hosts[0] = "rr1---sn-alpha.example.com"
},
"missing-resource": func(_ *adapter.Binding, p *adapter.Plan, r map[string]mediaResource) { delete(r, p.Hosts[0]) },
"resource-host": func(_ *adapter.Binding, p *adapter.Plan, r map[string]mediaResource) {
v := r[p.Hosts[0]]
v.host = p.Hosts[1]
r[p.Hosts[0]] = v
},
"resource-url": func(_ *adapter.Binding, p *adapter.Plan, r map[string]mediaResource) {
v := r[p.Hosts[0]]
v.url = "https://evil.example/"
r[p.Hosts[0]] = v
},
"missing-protocol": func(_ *adapter.Binding, p *adapter.Plan, _ map[string]mediaResource) { p.Protocols[1] = "" },
"duplicate-protocol": func(_ *adapter.Binding, p *adapter.Plan, _ map[string]mediaResource) { p.Protocols[1] = p.Protocols[0] },
"unknown-protocol": func(_ *adapter.Binding, p *adapter.Plan, _ map[string]mediaResource) { p.Protocols[1] = "http" },
"protocol-order": func(_ *adapter.Binding, p *adapter.Plan, _ map[string]mediaResource) {
p.Protocols = [2]string{"tls13", "tls12"}
},
}
for _, n := range []int{math.MinInt, -1, 0, 1, 4, math.MaxInt} {
value := n
mutations["samples-"+time.Duration(n).String()] = func(_ *adapter.Binding, p *adapter.Plan, _ map[string]mediaResource) { p.Samples = value }
}
for _, n := range []int{-1, 0, 1, 11, math.MaxInt} {
value := n
mutations["warmup-"+time.Duration(n).String()] = func(_ *adapter.Binding, p *adapter.Plan, _ map[string]mediaResource) { p.WarmupLimit = value }
}
for _, n := range []int64{math.MinInt64, -1, 0, 1, 1025, math.MaxInt64} {
value := n
mutations["bytes-"+time.Duration(n).String()] = func(_ *adapter.Binding, p *adapter.Plan, _ map[string]mediaResource) { p.MinBytes = value }
}
for _, n := range []time.Duration{-1, 0, time.Second, 121 * time.Second, time.Duration(math.MaxInt64)} {
value := n
mutations["timeout-"+n.String()] = func(_ *adapter.Binding, p *adapter.Plan, _ map[string]mediaResource) { p.WarmupTimeout = value }
}
for name, mutate := range mutations {
t.Run(name, func(t *testing.T) {
b, p, resources := adapterExecutorFixture(t)
mutate(&b, &p, resources)
_, err := executeAdapterPlan(context.Background(), b, p, resources,
func(context.Context, mediaResource, string) (adapterMeasurement[int], error) {
t.Fatal("invalid plan probed")
return adapterMeasurement[int]{}, nil
},
func(error) bool { t.Fatal("invalid plan classified"); return false })
if !errors.Is(err, errAdapterPlan) {
t.Fatal(err)
}
})
}
}
func TestAdapterExecutorOrderedWarmupAndTypedSamples(t *testing.T) {
b, p, resources := adapterExecutorFixture(t)
var order []string
results, err := executeAdapterPlan(context.Background(), b, p, resources,
func(ctx context.Context, r mediaResource, protocol string) (adapterMeasurement[int], error) {
if _, ok := ctx.Deadline(); len(order)%8 < 2 && !ok {
t.Fatal("warmup lacks deadline")
}
order = append(order, r.host+"/"+protocol)
return adapterMeasurement[int]{value: len(order), bytes: 1024}, nil
}, func(error) bool { return false })
if err != nil {
t.Fatal(err)
}
var expected []string
for i, host := range p.Hosts {
for _, protocol := range []string{"tls12", "tls13", "tls12", "tls12", "tls12", "tls13", "tls13", "tls13"} {
expected = append(expected, host+"/"+protocol)
}
h := results[i]
base := i * 8
if h.host != host || h.warmupAttempts != 2 || h.warmupFailures != 0 || h.warmupLimit != 10 || !reflect.DeepEqual(h.samples, []int{base + 3, base + 4, base + 5, base + 6, base + 7, base + 8}) {
t.Fatal(h)
}
}
if !reflect.DeepEqual(order, expected) {
t.Fatal(order, expected)
}
}
func TestAdapterExecutorFailureAndCancellation(t *testing.T) {
fatal := errors.New("infrastructure failure")
negative := errors.New("known negative")
for _, mode := range []string{"warm-negative", "warm-small", "warm-fatal", "warm-mixed", "steady-negative", "steady-small", "steady-fatal", "cancel-before", "cancel-warm", "deadline-warm", "cancel-steady"} {
t.Run(mode, func(t *testing.T) {
b, p, resources := adapterExecutorFixture(t)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
want := errCandidateNonpass
wantCalls := 10
if mode == "cancel-before" {
cancel()
want = context.Canceled
wantCalls = 0
}
if mode == "warm-fatal" {
want = fatal
wantCalls = 1
}
if mode == "warm-mixed" {
want = fatal
wantCalls = 1
}
if mode == "steady-negative" || mode == "steady-small" {
wantCalls = 3
}
if mode == "steady-fatal" {
want = fatal
wantCalls = 3
}
if mode == "cancel-warm" {
want = context.Canceled
wantCalls = 1
}
if mode == "cancel-steady" {
want = context.Canceled
wantCalls = 3
}
if mode == "deadline-warm" {
var stop context.CancelFunc
ctx, stop = context.WithTimeout(ctx, time.Second)
defer stop()
want = context.DeadlineExceeded
wantCalls = 1
}
calls := 0
hosts, err := executeAdapterPlan(ctx, b, p, resources,
func(probeCtx context.Context, _ mediaResource, _ string) (adapterMeasurement[int], error) {
calls++
m := adapterMeasurement[int]{value: calls, bytes: 1024}
switch mode {
case "warm-negative":
return m, negative
case "warm-small":
m.bytes = 1023
case "warm-fatal":
return m, fatal
case "warm-mixed":
return m, errors.Join(negative, fatal)
case "steady-negative":
if calls == 3 {
return m, negative
}
case "steady-small":
if calls == 3 {
m.bytes = 0
}
case "steady-fatal":
if calls == 3 {
return m, fatal
}
case "cancel-warm":
cancel()
return m, ctx.Err()
case "deadline-warm":
<-probeCtx.Done()
return m, probeCtx.Err()
case "cancel-steady":
if calls == 3 {
cancel()
return m, ctx.Err()
}
}
return m, nil
}, func(err error) bool { return err == negative })
if !errors.Is(err, want) || calls != wantCalls {
t.Fatal(err, calls, want, wantCalls)
}
for _, host := range hosts {
if len(host.samples) != 0 {
t.Fatal("phantom steady sample", host)
}
}
if calls == 10 && (hosts[0].warmupAttempts != 10 || hosts[0].warmupFailures != 10) {
t.Fatal(hosts)
}
})
}
}
type weakenedExecutorAdapter struct{ youtube.Module }
func (m weakenedExecutorAdapter) PlanQualification(hosts []string, resources []adapter.Resource, now time.Time) (adapter.Plan, error) {
p, err := m.Module.PlanQualification(hosts, resources, now)
p.Samples = 1
return p, err
}
func TestAdapterExecutorProductionPreflightRejectsWeakenedModule(t *testing.T) {
replaceServiceAdapters(t, serviceAdapterRegistry{{youtube.Kind, youtube.Version}: weakenedExecutorAdapter{}})
r, c, resources, events := googlevideoRuntimeFixture(t)
p, err := r.verifyGooglevideo(context.Background(), c, resources)
if !errors.Is(err, errAdapterPlan) || len(*events) != 0 || p.Attestation != nil || p.producerSeal != nil {
t.Fatal("module weakened production execution policy", p, err, *events)
}
}
+3 -1
View File
@@ -271,7 +271,9 @@ func TestAssetCLIApplyPartialFailureAndStatus(t *testing.T) {
t.Fatalf("requested status: %s %s", out, diagnostic)
}
if fail {
d.player = func(context.Context, []string, io.Writer, io.Writer) error { return errors.New("unit health failure") }
d.player = func(context.Context, []string, io.Reader, io.Writer, io.Writer) error {
return errors.New("unit health failure")
}
}
code, _, diagnostic := callManaged(t, d, "apply", "--asset-candidate", fmt.Sprintf("%d,%d", a, b))
if (code != 0) != fail {
+26 -3
View File
@@ -9,6 +9,8 @@ import (
"math"
"strings"
"time"
"zapret-autotune/internal/adapter/youtube"
)
// Additive only: old proposals are not independently qualified candidates.
@@ -351,6 +353,7 @@ type GooglevideoHostProof struct {
}
type GooglevideoQualificationProof struct {
Adapter *youtube.Binding `json:"adapter,omitempty"`
Version int `json:"version"`
Mode QualificationMode `json:"mode"`
SourceKind string `json:"source_kind"`
@@ -371,7 +374,25 @@ func (p GooglevideoQualificationProof) validate(c storedCandidate) error {
}
func (p GooglevideoQualificationProof) validateStored(c storedCandidate) error {
if p.Version != 1 || p.Mode != c.Mode || p.Args != c.Args || len(p.Hosts) != len(c.Endpoints) || len(p.Hosts) < 3 {
// Missing binding is explicitly the immutable legacy YouTube v1 policy,
// never the current registry default. Retain historical reconstruction.
if p.Version == 1 {
if p.Adapter != nil {
return fmt.Errorf("legacy proof cannot carry adapter binding")
}
} else {
if p.Mode != QualificationAdaptive || p.Adapter == nil {
return fmt.Errorf("version-bound adaptive proof required")
}
module, err := lookupServiceAdapter(p.Adapter.Kind, p.Adapter.Version)
if err != nil {
return err
}
if *p.Adapter != module.Binding() {
return fmt.Errorf("adapter policy mismatch")
}
}
if (p.Version != 1 && p.Version != 2) || p.Mode != c.Mode || p.Args != c.Args || len(p.Hosts) != len(c.Endpoints) || len(p.Hosts) < 3 {
return fmt.Errorf("invalid representative proof definition")
}
if (p.Mode == QualificationCold && p.SourceKind != "media-range") || (p.Mode == QualificationAdaptive && p.SourceKind != NativeCircularPostWarmup) || (p.Mode != QualificationCold && p.Mode != QualificationAdaptive) {
@@ -380,7 +401,7 @@ func (p GooglevideoQualificationProof) validateStored(c storedCandidate) error {
if err := (&candidate{Args: p.Args}).validateArgs(); err != nil {
return err
}
if err := validateGooglevideoHosts(c.Endpoints); err != nil {
if err := validateLegacyYoutubeV1Hosts(c.Endpoints); err != nil {
return err
}
if p.Mode == QualificationCold && p.Attestation != nil {
@@ -494,7 +515,9 @@ func validateAdaptiveGooglevideoEvidence(db *sql.DB, c storedCandidate, fresh bo
// One transactional insert, no cold candidate_evidence summaries. A private
// producer seal is required as well as valid DTO facts and stored definition.
func saveAdaptiveGooglevideoEvidence(tx *sql.Tx, id int64, proof GooglevideoQualificationProof) error {
if proof.producerSeal == nil || *proof.producerSeal != googlevideoProofSeal(proof) {
// Existing trusted v1 rows retain their original read/apply contract. Only
// this new-write boundary requires the version-bound producer protocol.
if proof.Version != 2 || proof.producerSeal == nil || *proof.producerSeal != googlevideoProofSeal(proof) {
return fmt.Errorf("completed owned collector attestation required")
}
var c storedCandidate
+11 -3
View File
@@ -24,7 +24,7 @@ const playerResponseLimit = 1 << 20
var videoIDPattern = regexp.MustCompile(`^[A-Za-z0-9_-]{11}$`)
var discoveryDNSLabel = regexp.MustCompile(`^[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?$`)
type playerRunner func(context.Context, []string, io.Writer, io.Writer) error
type playerRunner func(context.Context, []string, io.Reader, io.Writer, io.Writer) error
// Keep request diagnostics private while retaining typed causes for supervision.
type playerRequestError struct{ cause error }
@@ -32,8 +32,9 @@ type playerRequestError struct{ cause error }
func (e *playerRequestError) Error() string { return "player request failed or exceeded output limit" }
func (e *playerRequestError) Unwrap() error { return e.cause }
func runPlayerCurl(ctx context.Context, args []string, stdout, stderr io.Writer) error {
func runPlayerCurl(ctx context.Context, args []string, stdin io.Reader, stdout, stderr io.Writer) error {
cmd := exec.CommandContext(ctx, "curl", args...)
cmd.Stdin = stdin
cmd.Stdout, cmd.Stderr = stdout, stderr
return cmd.Run()
}
@@ -148,6 +149,12 @@ func discoverYoutube(args []string, stdout, stderr io.Writer, runner playerRunne
}
func requestPlayer(videoID, proxy string, runner playerRunner) ([]byte, error) {
if _, err := requiredYoutubeAdapter(); err != nil {
return nil, err
}
if !videoIDPattern.MatchString(videoID) {
return nil, errors.New("invalid player video ID")
}
payload, _ := json.Marshal(map[string]interface{}{
"videoId": videoID,
"context": map[string]interface{}{"client": map[string]interface{}{
@@ -161,8 +168,9 @@ func requestPlayer(videoID, proxy string, runner playerRunner) ([]byte, error) {
output := &playerBoundedWriter{dst: &body, limit: playerResponseLimit}
diagnostics := &playerBoundedWriter{dst: io.Discard, limit: 64 << 10}
// Disable curlrc and NO_PROXY bypasses: the explicit proxy must be used.
// Keep the bounded JSON body off argv; curl retains its JSON POST headers.
err := runner(ctx, []string{"--disable", "--silent", "--show-error", "--fail", "--max-time", "30",
"--noproxy", "", "--proxy", proxy, "--user-agent", playerUserAgent, "--json", string(payload), playerURL}, output, diagnostics)
"--noproxy", "", "--proxy", proxy, "--user-agent", playerUserAgent, "--json", "@-", playerURL}, bytes.NewReader(payload), output, diagnostics)
if err != nil || output.exceeded || diagnostics.exceeded {
if output.exceeded || diagnostics.exceeded {
// An output failure remains unsafe even if the child was also canceled.
+6 -6
View File
@@ -20,13 +20,13 @@ func discoveryArgs() []string {
func TestDiscoverYoutubeSuccess(t *testing.T) {
var stdout, stderr bytes.Buffer
called := false
runner := func(ctx context.Context, args []string, out, diagnostics io.Writer) error {
runner := func(ctx context.Context, args []string, stdin io.Reader, out, diagnostics io.Writer) error {
called = true
if _, ok := ctx.Deadline(); !ok {
t.Error("request must have a deadline")
}
prefix := []string{"--disable", "--silent", "--show-error", "--fail", "--max-time", "30", "--noproxy", "", "--proxy", "socks5://localhost:1080", "--user-agent", playerUserAgent, "--json"}
if len(args) != len(prefix)+2 || !reflect.DeepEqual(args[:len(prefix)], prefix) || args[len(args)-1] != playerURL {
if len(args) != len(prefix)+2 || !reflect.DeepEqual(args[:len(prefix)], prefix) || args[len(prefix)] != "@-" || args[len(args)-1] != playerURL {
t.Fatalf("unexpected curl arguments: %q", args)
}
var payload struct {
@@ -42,7 +42,7 @@ func TestDiscoverYoutubeSuccess(t *testing.T) {
} `json:"client"`
} `json:"context"`
}
if err := json.Unmarshal([]byte(args[len(prefix)]), &payload); err != nil {
if err := json.NewDecoder(stdin).Decode(&payload); err != nil {
t.Fatal(err)
}
client := payload.Context.Client
@@ -79,7 +79,7 @@ func TestDiscoverYoutubeInvalidArgs(t *testing.T) {
}
for _, args := range tests {
var stdout, stderr bytes.Buffer
runner := func(context.Context, []string, io.Writer, io.Writer) error {
runner := func(context.Context, []string, io.Reader, io.Writer, io.Writer) error {
t.Fatal("invalid arguments must not invoke curl")
return nil
}
@@ -114,7 +114,7 @@ func TestPlayerHostnameFailsClosed(t *testing.T) {
}
for _, fixture := range fixtures {
var stdout, stderr bytes.Buffer
runner := func(_ context.Context, _ []string, out, diagnostics io.Writer) error {
runner := func(_ context.Context, _ []string, _ io.Reader, out, diagnostics io.Writer) error {
io.WriteString(diagnostics, "SECRET")
_, err := io.WriteString(out, fixture)
return err
@@ -136,7 +136,7 @@ func TestDiscoverYoutubeRequestFailures(t *testing.T) {
for _, mode := range []string{"runner", "stdout limit", "stderr limit", "write result"} {
t.Run(mode, func(t *testing.T) {
var stdout, stderr bytes.Buffer
runner := func(_ context.Context, _ []string, out, diagnostics io.Writer) error {
runner := func(_ context.Context, _ []string, _ io.Reader, out, diagnostics io.Writer) error {
switch mode {
case "runner":
io.WriteString(out, discoveryFixture)
+4
View File
@@ -44,6 +44,10 @@ func googlevideoProposal(flags []string, stdout, stderr io.Writer, discover play
fmt.Fprintln(stderr, "error: tune googlevideo", err)
return 2
}
if _, err := requiredYoutubeAdapter(); err != nil {
fmt.Fprintln(stderr, "error: tune googlevideo", err)
return 1
}
started := time.Now()
db, err := openStore(databasePath())
if err != nil {
+8 -4
View File
@@ -17,16 +17,20 @@ var cacheTestFlags = []string{"--video-id", "dQw4w9WgXcQ", "--proxy", "socks5://
func cachePlayer(t *testing.T, order *[]string) playerRunner {
t.Helper()
return func(_ context.Context, args []string, out, diagnostics io.Writer) error {
return func(_ context.Context, args []string, stdin io.Reader, out, diagnostics io.Writer) error {
*order = append(*order, "discover")
invocation := strings.Join(args, " ")
for _, want := range []string{"--proxy socks5://localhost:1080", "21.02.35", "dQw4w9WgXcQ", playerURL} {
for _, want := range []string{"--proxy socks5://localhost:1080", "21.02.35", "--json @-", playerURL} {
if !strings.Contains(invocation, want) {
t.Fatalf("discovery missing %q: %s", want, invocation)
}
}
payload, err := io.ReadAll(stdin)
if err != nil || !strings.Contains(string(payload), `"videoId":"dQw4w9WgXcQ"`) || strings.Contains(invocation, "dQw4w9WgXcQ") {
t.Fatal("player payload must be supplied only on stdin")
}
fmt.Fprint(diagnostics, "https://secret.invalid/signed?token=SECRET")
_, err := fmt.Fprintf(out, `{"playabilityStatus":{"status":"OK"},"streamingData":{"serverAbrStreamingUrl":"https://%s/videoplayback?token=SECRET"}}`, cacheTestHost)
_, err = fmt.Fprintf(out, `{"playabilityStatus":{"status":"OK"},"streamingData":{"serverAbrStreamingUrl":"https://%s/videoplayback?token=SECRET"}}`, cacheTestHost)
return err
}
}
@@ -209,7 +213,7 @@ func TestTuneGooglevideoFailures(t *testing.T) {
var order []string
player := cachePlayer(t, &order)
if mode == "discovery" {
player = func(context.Context, []string, io.Writer, io.Writer) error {
player = func(context.Context, []string, io.Reader, io.Writer, io.Writer) error {
order = append(order, "discover")
return fmt.Errorf("https://signed.invalid/?SECRET")
}
+5 -5
View File
@@ -125,20 +125,20 @@ func TestRunHealthYoutube(t *testing.T) {
t.Setenv("NO_PROXY", "*")
var stdout, stderr bytes.Buffer
calls := 0
runner := func(ctx context.Context, args []string, out, diagnostics io.Writer) error {
runner := func(ctx context.Context, args []string, stdin io.Reader, out, diagnostics io.Writer) error {
calls++
deadline, ok := ctx.Deadline()
if !ok || time.Until(deadline) <= 0 || time.Until(deadline) > 35*time.Second {
t.Error("request must have bounded deadline")
}
prefix := []string{"--disable", "--silent", "--show-error", "--fail", "--max-time", "30", "--noproxy", "", "--proxy", proxy, "--user-agent", playerUserAgent, "--json"}
if len(args) != len(prefix)+2 || !reflect.DeepEqual(args[:len(prefix)], prefix) || args[len(args)-1] != playerURL {
if len(args) != len(prefix)+2 || !reflect.DeepEqual(args[:len(prefix)], prefix) || args[len(prefix)] != "@-" || args[len(args)-1] != playerURL {
t.Fatalf("unexpected proxy/API arguments: %q", args)
}
var payload struct {
VideoID string `json:"videoId"`
}
if err := json.Unmarshal([]byte(args[len(prefix)]), &payload); err != nil || payload.VideoID != "abcdefghijk" {
if err := json.NewDecoder(stdin).Decode(&payload); err != nil || payload.VideoID != "abcdefghijk" {
t.Fatalf("unexpected video payload: %+v %v", payload, err)
}
io.WriteString(diagnostics, "SECRET")
@@ -158,7 +158,7 @@ func TestHealthYoutubeFailures(t *testing.T) {
t.Run(mode, func(t *testing.T) {
var stdout, stderr bytes.Buffer
calls := 0
runner := func(_ context.Context, _ []string, out, diagnostics io.Writer) error {
runner := func(_ context.Context, _ []string, _ io.Reader, out, diagnostics io.Writer) error {
calls++
io.WriteString(diagnostics, "SECRET")
switch mode {
@@ -192,7 +192,7 @@ func TestHealthYoutubeFailures(t *testing.T) {
func TestHealthDispatchAndConfigFailWithoutRequest(t *testing.T) {
setHealthEnvironment(t)
runner := func(context.Context, []string, io.Writer, io.Writer) error {
runner := func(context.Context, []string, io.Reader, io.Writer, io.Writer) error {
t.Fatal("invalid command/config must not request or fall back")
return nil
}
+20
View File
@@ -9,6 +9,12 @@ import (
const usage = `Usage: zapret-autotune <command>
Commands:
strategy-import --strategies FILE
Import immutable definitions only into the private target registry
plan --config FILE
Authenticated domain compatibility proposal (never apply)
qualify --config FILE --strategies FILE --isolated
Intrusive finite batch; only PID 1 isolated startup on disabled baseline
dns-forwarder mode|address|serve
Managed DNS configuration or loopback DoH forwarder
dns-forwarder readiness-check|readiness-cleanup <expected-dns-pid>
@@ -45,6 +51,12 @@ func run(args []string, stdout, stderr io.Writer) int {
}
func runWithPlayer(args []string, stdout, stderr io.Writer, player playerRunner) int {
if len(args) > 0 && args[0] == "qualification-completion-check" {
return qualificationCompletionCLI(args[1:], managedDependencies(player))
}
if len(args) > 0 && (args[0] == "plan" || args[0] == "strategy-import" || args[0] == "qualify") {
return resourcePlanCLI(args, stdout, stderr, managedDependencies(player))
}
if len(args) == 3 && args[0] == "check" && args[1] == "--config" {
return configuredResourceCheck(args[2], stdout, stderr)
}
@@ -53,6 +65,14 @@ func runWithPlayer(args []string, stdout, stderr io.Writer, player playerRunner)
}
if len(args) == 1 && args[0] == "target-config-check" {
path, _ := os.LookupEnv("ZAPRET_AUTOTUNE_RESOURCE_CONFIG")
policy, _ := os.LookupEnv(resourceStrategyIDsEnv)
if _, err := parseResourceStrategyIDs(policy); err != nil || policy != "" && path == "" {
fmt.Fprintln(stderr, "configerror: invalid generic strategy selection")
return 64
}
if path == "" {
path, _ = os.LookupEnv("ZAPRET_AUTOTUNE_QUALIFY_CONFIG")
}
if path != "" {
port, err := managedTargetSettings(os.LookupEnv)
if err != nil || port != 1080 {
+12 -11
View File
@@ -234,12 +234,12 @@ func (r managedAssetRuntime) withOwnedWorker(ctx context.Context, endpoints []st
}
occupied, e := r.queue(0)
if e != nil || occupied {
return &managedRecoveryFatal{Err: errors.Join(e, errors.New("asset queue unavailable"))}
return &managedRecoveryFatal{Err: qualificationOperation(errors.Join(e, errors.New("asset queue unavailable")), stageWorkerQueue)}
}
args := managedProbeWorkerArgs(endpoints, strategyArgs)
worker, e := r.start(args)
if e != nil {
return &managedRecoveryFatal{Err: e}
return &managedRecoveryFatal{Err: qualificationOperation(e, stageWorkerStart)}
}
joined, table := false, false
defer func() {
@@ -249,28 +249,29 @@ func (r managedAssetRuntime) withOwnedWorker(ctx context.Context, endpoints []st
// Remove queues before killing the consumer.
if table {
_, cleanupErr = r.nft(cleanup, "delete table inet "+managedAssetTable+"\n")
cleanupErr = qualificationOperation(cleanupErr, stageNFTDelete)
}
if !joined {
select {
case e := <-worker.done:
joined = true
cleanupErr = errors.Join(cleanupErr, e, errors.New("asset worker exited unexpectedly"))
cleanupErr = errors.Join(cleanupErr, qualificationOperation(errors.Join(e, errors.New("asset worker exited unexpectedly")), stageStopJoin))
default:
}
}
if !joined {
cleanupErr = errors.Join(cleanupErr, worker.stop())
cleanupErr = errors.Join(cleanupErr, qualificationOperation(worker.stop(), stageStopJoin))
select {
case e := <-worker.done:
joined = true
if e != nil {
var exit *exec.ExitError
if !errors.As(e, &exit) || !exit.ProcessState.Sys().(syscall.WaitStatus).Signaled() || exit.ProcessState.Sys().(syscall.WaitStatus).Signal() != syscall.SIGKILL {
cleanupErr = errors.Join(cleanupErr, e)
cleanupErr = errors.Join(cleanupErr, qualificationOperation(e, stageStopJoin))
}
}
case <-cleanup.Done():
cleanupErr = errors.Join(cleanupErr, errors.New("asset worker join timeout"))
cleanupErr = errors.Join(cleanupErr, qualificationOperation(errors.New("asset worker join timeout"), stageStopJoin))
}
}
if cleanupErr != nil {
@@ -285,12 +286,12 @@ func (r managedAssetRuntime) withOwnedWorker(ctx context.Context, endpoints []st
select {
case e := <-worker.done:
joined = true
return &managedRecoveryFatal{Err: errors.Join(e, errors.New("asset worker exited before READY"))}
return &managedRecoveryFatal{Err: qualificationOperation(errors.Join(e, errors.New("asset worker exited before READY")), stageWorkerExit)}
default:
}
ready, e := r.queue(worker.pid)
if e != nil {
return &managedRecoveryFatal{Err: e}
return &managedRecoveryFatal{Err: qualificationOperation(e, stageWorkerQueue)}
}
if ready {
break
@@ -300,7 +301,7 @@ func (r managedAssetRuntime) withOwnedWorker(ctx context.Context, endpoints []st
if ctx.Err() != nil {
return ctx.Err()
}
return &managedRecoveryFatal{Err: errors.New("asset queue READY timeout")}
return &managedRecoveryFatal{Err: qualificationOperation(errors.New("asset queue READY timeout"), stageWorkerQueue)}
case <-tick.C:
}
}
@@ -311,7 +312,7 @@ func (r managedAssetRuntime) withOwnedWorker(ctx context.Context, endpoints []st
table = true
// Recheck ownership after installing the queue and before the first capture.
if ready, e := r.queue(worker.pid); e != nil || !ready {
return &managedRecoveryFatal{Err: errors.Join(e, errors.New("owned queue lost before capture"))}
return &managedRecoveryFatal{Err: qualificationOperation(errors.Join(e, errors.New("owned queue lost before capture")), stageWorkerQueue)}
}
probeCtx, stopProbe := context.WithCancel(ctx)
watchDone := make(chan struct{})
@@ -320,7 +321,7 @@ func (r managedAssetRuntime) withOwnedWorker(ctx context.Context, endpoints []st
defer close(watchDone)
select {
case e := <-worker.done:
exited <- errors.Join(e, errors.New("asset worker exited during probe"))
exited <- qualificationOperation(errors.Join(e, errors.New("asset worker exited during probe")), stageWorkerExit)
stopProbe()
case <-probeCtx.Done():
}
+10 -1
View File
@@ -71,6 +71,9 @@ type managedServiceInventory struct {
func managedServiceResources(cfg healthConfig, player func(context.Context) ([]byte, error), page playerRunner) func(context.Context) (managedServiceInventory, error) {
return func(ctx context.Context) (managedServiceInventory, error) {
i := managedServiceInventory{images: make(map[string]string), proxy: cfg.SOCKSProxy, runner: runResourceCurl}
if _, err := requiredYoutubeAdapter(); err != nil {
return i, err
}
if err := ctx.Err(); err != nil {
return i, managedRecoveryContextError(ctx, err)
}
@@ -105,6 +108,9 @@ func managedServiceResources(cfg healthConfig, player func(context.Context) ([]b
}
i.media, _, err = playerMediaResource(body)
if err != nil {
if errors.Is(err, errServiceAdapterUnavailable) {
return i, err
}
i.resourceErr = &managedResourceFailure{"media", "unknown", err}
}
for attempt := 0; attempt < 2; attempt++ {
@@ -122,7 +128,7 @@ func managedServiceResources(cfg healthConfig, player func(context.Context) ([]b
var output bytes.Buffer
out := &playerBoundedWriter{dst: &output, limit: 5 << 20}
// Fixed trusted WWW origin; no redirects, credentials or diagnostics.
err = page(bounded, []string{"--disable", "--silent", "--show-error", "--fail", "--noproxy", "", "--proxy", proxy, "--proto", "=https", "--max-time", "30", "--max-filesize", "5242880", origin}, out, io.Discard)
err = page(bounded, []string{"--disable", "--silent", "--show-error", "--fail", "--noproxy", "", "--proxy", proxy, "--proto", "=https", "--max-time", "30", "--max-filesize", "5242880", origin}, nil, out, io.Discard)
requestErr := bounded.Err()
cancel()
if ctx.Err() != nil {
@@ -152,6 +158,9 @@ func managedServiceResources(cfg healthConfig, player func(context.Context) ([]b
func managedServiceHealth(proxy string, front func(context.Context) (string, error), inventory func(context.Context) (managedServiceInventory, error), runner subprocessRunner) func(context.Context) (string, error) {
return func(ctx context.Context) (string, error) {
if _, err := requiredYoutubeAdapter(); err != nil {
return "DEGRADED", err
}
status, err := front(ctx)
if err != nil || status != "OK" {
return "DEGRADED", &managedResourceFailure{"front", managedFrontendHost, err}
@@ -66,7 +66,7 @@ func TestManagedAvatarInventoryFallback(t *testing.T) {
calls := 0
resources := managedServiceResources(cfg, func(context.Context) ([]byte, error) {
return []byte(`{"playabilityStatus":{"status":"OK"},"videoDetails":{"thumbnail":{"thumbnails":[{"url":"https://i.ytimg.com/fresh"}]}},"streamingData":{"formats":[{"url":"https://fresh.googlevideo.com/private","mimeType":"video/mp4"}]}}`), nil
}, func(ctx context.Context, args []string, out, diagnostic io.Writer) error {
}, func(ctx context.Context, args []string, _ io.Reader, out, diagnostic io.Writer) error {
calls++
if _, ok := ctx.Deadline(); !ok || diagnostic != io.Discard {
t.Fatal("unbounded metadata or diagnostics")
@@ -148,7 +148,7 @@ func TestManagedAvatarInventoryCancellationNoFallback(t *testing.T) {
cancel()
}
return []byte(`{"playabilityStatus":{"status":"OK"}}`), nil
}, func(ctx context.Context, args []string, out, _ io.Writer) error {
}, func(ctx context.Context, args []string, _ io.Reader, out, _ io.Writer) error {
pages++
assertManagedDNSArgs(t, args, "socks5h://127.0.0.1:11080")
if args[len(args)-1] != "https://www.youtube.com/" {
+60
View File
@@ -271,6 +271,21 @@ func managedReport(m *managedTransaction, out io.Writer, openDB func() (*sql.DB,
if managedHash(b) != j.Hash && (j.State != "prepared" || managedHash(b) != j.Next) {
return errManagedDrift
}
if j.Version == 4 {
if activeOnly {
return fmt.Errorf("v4 deployment has no legacy active IDs")
}
if err := m.validateConfigRef(nil, j.Ref, j.Candidates, j.Hash, j.Scope); err != nil {
return err
}
if j.State == "prepared" {
if err := m.validateConfigRef(nil, j.PendingRef, j.PendingCandidates, j.Next, j.PendingScope); err != nil {
return err
}
}
_, err := fmt.Fprintf(out, "managed-test generation=%d hash=%s state=%s kind=%s\nLive resource health: NOT CHECKED; configured resources only, not production READY.\n", j.Generation, j.Hash, j.State, j.Ref.Kind)
return err
}
groups := map[string]bool{}
operatorCoverage := map[string]bool{}
assetHosts := map[string]bool{}
@@ -420,6 +435,9 @@ func managedTargetSettings(lookup func(string) (string, bool)) (int, error) {
}
func managedCLI(args []string, stdout, stderr io.Writer, d managedDeps) int {
if len(args) == 3 && args[0] == "resource-apply" && args[1] == "--resource-config" {
return resourceApplyCLI(args[2], stdout, stderr, d)
}
if len(args) == 1 && args[0] == "resource-status" {
return resourceStatusCLI(stdout, d)
}
@@ -449,7 +467,22 @@ func managedCLI(args []string, stdout, stderr io.Writer, d managedDeps) int {
if command == "reconcile" {
err = managedPrestartReconcile(m, d)
} else if command == "status" || command == "active-ids" {
j, loadErr := m.load()
if loadErr != nil {
err = loadErr
} else if j.Version == 4 {
config, _ := d.lookup("ZAPRET_AUTOTUNE_RESOURCE_CONFIG")
s, e := openResourceApplySnapshot(config, d)
if e != nil {
err = errResourcePlan
} else {
m.resource = s
err = managedReport(m, stdout, d.openDB, command == "active-ids")
s.close()
}
} else {
err = managedReport(m, stdout, d.openDB, command == "active-ids", len(args) == 2 && args[1] == "--assets")
}
} else if command == "experiment" {
startup := false
for i := 1; i < len(args); i += 2 {
@@ -504,6 +537,27 @@ func managedCLI(args []string, stdout, stderr io.Writer, d managedDeps) int {
// Startup may activate only a known baseline or DB-qualified rendered config.
// Reconcile restores bytes before its fixed target-local restart hook runs.
func managedPrestartReconcile(m *managedTransaction, d managedDeps) (err error) {
j, loadErr := m.load()
if loadErr != nil {
return loadErr
}
if j.Version == 4 {
config, _ := d.lookup("ZAPRET_AUTOTUNE_RESOURCE_CONFIG")
s, e := openResourceApplySnapshot(config, d)
if e != nil {
return errResourcePlan
}
defer s.close()
m.resource = s
defer func() { m.resource = nil }()
checks := productionResourceCheckDeps()
m.RollbackHealth = func() (string, error) {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
return resourceApplyHealth(ctx, s, checks)
}
return m.Reconcile()
}
path, err := managedMonitorDatabase(d)
if err != nil {
return err
@@ -543,6 +597,9 @@ func managedPrestartReconcile(m *managedTransaction, d managedDeps) (err error)
var errManagedScopeUnresolved = errors.New("managed deployment policy unresolved; explicit reconcile or fresh managed state required")
func managedActivate(m *managedTransaction, d managedDeps, ids []int64, expected map[int64]string, scopes ...GooglevideoScope) error {
if _, err := requiredYoutubeAdapter(); err != nil {
return err
}
info, err := os.Stat(d.script)
if err != nil || !info.Mode().IsRegular() || info.Mode().Perm()&0111 == 0 {
return fmt.Errorf("target zapret daemon init executable unavailable")
@@ -552,6 +609,9 @@ func managedActivate(m *managedTransaction, d managedDeps, ids []int64, expected
return err
}
m.Health = func() (string, error) {
if _, err := requiredYoutubeAdapter(); err != nil {
return "", err
}
portText, _ := d.lookup("SOCKS_PORT")
port, err := strconv.Atoi(portText)
if err != nil {
+2 -2
View File
@@ -46,7 +46,7 @@ func managedCLIFixture(t *testing.T) (managedDeps, map[string]string, *int) {
}
return nil
},
player: func(_ context.Context, args []string, stdout, _ io.Writer) error {
player: func(_ context.Context, args []string, _ io.Reader, stdout, _ io.Writer) error {
if !strings.Contains(strings.Join(args, "|"), "--noproxy||--proxy|socks5h://127.0.0.1:11080|") {
t.Fatalf("unsafe health argv: %q", args)
}
@@ -212,7 +212,7 @@ func TestManagedCLIApplyAndRollback(t *testing.T) {
}
switch mode {
case "health":
d.player = func(_ context.Context, _ []string, stdout, _ io.Writer) error {
d.player = func(_ context.Context, _ []string, _ io.Reader, stdout, _ io.Writer) error {
_, err := io.WriteString(stdout, `{"playabilityStatus":{"status":"ERROR","secret":"do-not-print"}}`)
return err
}
+1 -1
View File
@@ -48,7 +48,7 @@ func TestManagedDNSProductionPlayerAndGuard(t *testing.T) {
return nil
}
requests := 0
target.player = func(_ context.Context, args []string, out, _ io.Writer) error {
target.player = func(_ context.Context, args []string, _ io.Reader, out, _ io.Writer) error {
assertManagedDNSArgs(t, args, "socks5h://127.0.0.1:11080")
if args[len(args)-1] != playerURL {
t.Fatal("Player hostname changed", args)
@@ -86,6 +86,9 @@ func managedExperimentalRequest(dir string, consume bool, apply func() error) er
// Runs before the PID-1-owned monitor starts; never races a recovery worker.
// A real Player-discovered identity is saved WITHOUT cold/adaptive evidence.
func managedExperimentalSuffix(m *managedTransaction, d managedDeps) error {
if _, err := requiredYoutubeAdapter(); err != nil {
return err
}
singleton, err := acquireManagedMonitorSingleton(d)
if err != nil {
return err
@@ -43,7 +43,7 @@ func TestExperimentalProductionProbeCollectorClassification(t *testing.T) {
if strings.HasPrefix(mode, "metadata-") {
d.resources = managedServiceResources(healthConfig{SOCKSProxy: "socks5://127.0.0.1:11080", VideoID: "dQw4w9WgXcQ"}, func(context.Context) ([]byte, error) {
return []byte(fmt.Sprintf(`{"playabilityStatus":{"status":"OK"},"streamingData":{"formats":[{"url":%q,"mimeType":"video/mp4"}]}}`, experimentalResourceFixture().url)), nil
}, func(_ context.Context, args []string, out, diagnostic io.Writer) error {
}, func(_ context.Context, args []string, _ io.Reader, out, diagnostic io.Writer) error {
origin := args[len(args)-1]
if (origin != "https://www.youtube.com/" && !(mode == "metadata-malformed" && origin == "https://www.youtube.com/watch?v=dQw4w9WgXcQ")) || diagnostic != io.Discard {
t.Fatal("metadata origin/diagnostics changed")
@@ -132,7 +132,7 @@ func TestExperimentalSuffixCLIExplicit(t *testing.T) {
func TestExperimentalSuffixCLIRejectsBypassBeforeWork(t *testing.T) {
d, _, _ := managedCLIFixture(t)
d.lookup = func(string) (string, bool) { t.Fatal("bypass reached runtime work"); return "", false }
d.player = func(context.Context, []string, io.Writer, io.Writer) error {
d.player = func(context.Context, []string, io.Reader, io.Writer, io.Writer) error {
t.Fatal("bypass requested Player")
return nil
}
@@ -280,7 +280,7 @@ func TestExperimentalSuffixCLIRejectsInheritedPolicy(t *testing.T) {
if err := m.Apply(db, []int64{front, asset, id}, GooglevideoScopeExperimental); err != nil {
t.Fatal(err)
}
d.player = func(context.Context, []string, io.Writer, io.Writer) error {
d.player = func(context.Context, []string, io.Reader, io.Writer, io.Writer) error {
t.Fatal("inherited bypass requested Player")
return nil
}
@@ -319,7 +319,7 @@ func TestExperimentalSuffixCLIInterleavedPolicyChange(t *testing.T) {
}
return db, nil
}
d.player = func(context.Context, []string, io.Writer, io.Writer) error {
d.player = func(context.Context, []string, io.Reader, io.Writer, io.Writer) error {
t.Fatal("racing CLI requested Player")
return nil
}
@@ -179,7 +179,7 @@ func TestExperimentalPromotionRequiresRepresentativeOwnedProof(t *testing.T) {
}
assertActiveCandidates(t, d.transaction, d.db, []int64{front, thumb, avatar, id})
p, err := readGooglevideoProof(d.db, id)
if err != nil || p.Version != 1 || len(p.Hosts) != 3 {
if err != nil || p.Version != 2 || p.Adapter == nil || len(p.Hosts) != 3 {
t.Fatal(p, err)
}
c, err := readStrategyCandidate(d.db, experiment)
@@ -5,9 +5,10 @@ import (
"errors"
"fmt"
"sort"
"strconv"
"strings"
"time"
"zapret-autotune/internal/adapter/youtube"
)
// Ordinary unresolved state, not a target/ownership guard failure.
@@ -28,7 +29,7 @@ const managedGooglevideoDiscoveryTimeout = 3 * time.Minute
// Three hosts each need up to 120s warmup + six 35s transfers. Leave margin
// for daemon lifecycle as well; short-lived URLs are refetched, never rewritten.
const managedGooglevideoSampleTTL = 20 * time.Minute
const managedGooglevideoSampleTTL = youtube.InventoryTTL
func freshGooglevideoSample(r mediaResource) bool {
// The same exact GGC hostname syntax as the representative gate; reject bad
@@ -37,13 +38,10 @@ func freshGooglevideoSample(r mediaResource) bool {
if !ok || !strings.HasSuffix(r.host, ".googlevideo.com") || !validDomain(r.host) || r.host != strings.ToLower(r.host) || root == "" || len(prefix) < 3 || !strings.HasPrefix(prefix, "rr") || strings.Trim(prefix[2:], "0123456789") != "" {
return false
}
u, err := privateResourceURL(r.url, r.host)
if err != nil || u.Path != "/videoplayback" || (r.kind != "video/mp4" && r.kind != "audio/mp4") {
if _, err := privateResourceURL(r.url, r.host); err != nil {
return false
}
q := u.Query()
expires, err := strconv.ParseInt(q.Get("expire"), 10, 64)
return err == nil && expires > time.Now().Add(managedGooglevideoSampleTTL).Unix() && (q.Get("sig") != "" || q.Get("signature") != "" || q.Get("lsig") != "")
return youtube.SignedSampleFresh(youtube.Resource{URL: r.url, Host: r.host, Kind: r.kind}, time.Now(), managedGooglevideoSampleTTL)
}
// Choose exactly three actual hosts passing the unchanged representative gate.
@@ -70,6 +68,9 @@ func representativeGooglevideoSamples(byHost map[string]mediaResource) []mediaRe
func managedGooglevideoSamples(ids []string, request func(context.Context, string) ([]byte, error)) func(context.Context) ([]mediaResource, error) {
ids = append([]string(nil), ids...)
return func(ctx context.Context) ([]mediaResource, error) {
if _, err := requiredYoutubeAdapter(); err != nil {
return nil, err
}
if len(ids) == 0 || len(ids) > 5 || request == nil {
return nil, &managedRecoveryFatal{Err: errors.New("invalid sample collector configuration")}
}
@@ -108,6 +109,9 @@ func managedGooglevideoSamples(ids []string, request func(context.Context, strin
}
r, _, err := playerMediaResource(body)
if err != nil {
if errors.Is(err, errServiceAdapterUnavailable) {
return nil, err
}
stats.Unavailable++
continue
} // Missing/SABR is unknown, never fabricated.
@@ -9,9 +9,10 @@ import (
"os"
"os/exec"
"reflect"
"strconv"
"strings"
"time"
"zapret-autotune/internal/adapter/youtube"
)
// ponytail: one audited composite only. New native variants need an explicit
@@ -93,84 +94,55 @@ func newManagedGooglevideoVerifier(target managedDeps) func(context.Context, sto
}
func (r managedAssetRuntime) verifyGooglevideo(ctx context.Context, c storedCandidate, resources []mediaResource) (GooglevideoQualificationProof, error) {
p := GooglevideoQualificationProof{Version: 1, Mode: QualificationAdaptive, SourceKind: NativeCircularPostWarmup, Args: c.Args}
p := GooglevideoQualificationProof{Version: 2, Mode: QualificationAdaptive, SourceKind: NativeCircularPostWarmup, Args: c.Args}
if c.Group != "googlevideo" || c.Mode != QualificationAdaptive {
return p, errors.New("adaptive googlevideo definition required")
}
if err := validateNativeGooglevideoArgs(c.Args); err != nil {
return p, err
}
byHost := map[string]mediaResource{}
for _, resource := range resources {
u, err := privateResourceURL(resource.url, resource.host)
if err != nil || u.Path != "/videoplayback" || (resource.kind != "video/mp4" && resource.kind != "audio/mp4") {
return p, errGooglevideoInsufficientSamples
module, err := requiredYoutubeAdapter()
if err != nil {
return p, err
}
q := u.Query()
expires, err := strconv.ParseInt(q.Get("expire"), 10, 64)
if err != nil || expires <= time.Now().Unix()+120 || (q.Get("sig") == "" && q.Get("signature") == "" && q.Get("lsig") == "") {
binding := module.Binding()
p.Adapter = &binding
byHost := map[string]mediaResource{}
var samples []youtube.Resource
for _, resource := range resources {
if _, err := privateResourceURL(resource.url, resource.host); err != nil {
return p, errGooglevideoInsufficientSamples
}
byHost[resource.host] = resource
samples = append(samples, youtube.Resource{URL: resource.url, Host: resource.host, Kind: resource.kind})
}
// Validate representative host diversity using the shared gate before capture.
for _, host := range c.Endpoints {
if _, ok := byHost[host]; !ok {
if validateGooglevideoHosts(c.Endpoints) != nil {
return p, errGooglevideoInsufficientSamples
}
plan, err := module.PlanQualification(c.Endpoints, samples, time.Now())
if err != nil {
return p, errGooglevideoInsufficientSamples
}
if !reflect.DeepEqual(plan.Hosts, c.Endpoints) {
return p, errAdapterPlan
}
if err := validateAdapterPlan(binding, plan, byHost); err != nil {
return p, err
}
for _, host := range plan.Hosts {
p.Hosts = append(p.Hosts, GooglevideoHostProof{Host: host})
}
if err := validateGooglevideoHosts(c.Endpoints); err != nil {
return p, errGooglevideoInsufficientSamples
}
err := r.withOwnedWorker(ctx, c.Endpoints, c.Args, func(probeCtx context.Context) error {
for i := range p.Hosts {
h := &p.Hosts[i]
h.WarmupLimit = 10
start := time.Now()
warmCtx, cancel := context.WithTimeout(probeCtx, 120*time.Second)
learned := map[string]bool{}
for h.WarmupAttempts < h.WarmupLimit && len(learned) < 2 && warmCtx.Err() == nil {
protocol := []string{"tls12", "tls13"}[h.WarmupAttempts%2]
h.WarmupAttempts++
_, pass, err := collectMediaTransfer(warmCtx, r.proxy, byHost[h.Host], protocol, r.curl)
if err != nil {
h.WarmupFailures++
if !managedGooglevideoTransferNonpass(err) && warmCtx.Err() == nil {
cancel()
return err
}
} else if pass.Bytes < 1024 {
h.WarmupFailures++
} else {
learned[protocol] = true
}
}
h.WarmupMS = float64(time.Since(start).Microseconds()) / 1000
cancel()
if probeCtx.Err() != nil {
return probeCtx.Err()
}
if len(learned) != 2 || h.WarmupMS > 120000 {
return errCandidateNonpass
}
for _, protocol := range []string{"tls12", "tls13"} {
for n := 0; n < 3; n++ {
_, pass, err := collectMediaTransfer(probeCtx, r.proxy, byHost[h.Host], protocol, r.curl)
if err != nil {
if managedGooglevideoTransferNonpass(err) {
return errCandidateNonpass
err = r.withOwnedWorker(ctx, c.Endpoints, c.Args, func(probeCtx context.Context) error {
hosts, err := executeAdapterPlan(probeCtx, binding, plan, byHost,
func(ctx context.Context, resource mediaResource, protocol string) (adapterMeasurement[GooglevideoPass], error) {
_, pass, err := collectMediaTransfer(ctx, r.proxy, resource, protocol, r.curl)
return adapterMeasurement[GooglevideoPass]{value: pass, bytes: pass.Bytes}, err
}, managedGooglevideoTransferNonpass)
for i, h := range hosts {
p.Hosts[i] = GooglevideoHostProof{Host: h.host, WarmupLimit: h.warmupLimit,
WarmupAttempts: h.warmupAttempts, WarmupFailures: h.warmupFailures, WarmupMS: h.warmupMS, Passes: h.samples}
}
return err
}
if pass.Bytes < 1024 {
return errCandidateNonpass
}
h.Passes = append(h.Passes, pass)
}
}
}
return nil
})
if err != nil {
return p, err
@@ -239,16 +211,28 @@ func managedResourceTransferNonpass(err error, metadata, image bool) bool {
}
func validateGooglevideoHosts(hosts []string) error {
seen, roots, prefixes := map[string]bool{}, map[string]bool{}, map[string]bool{}
for _, host := range hosts {
prefix, root, ok := strings.Cut(strings.TrimSuffix(host, ".googlevideo.com"), "---sn-")
if !ok || !strings.HasSuffix(host, ".googlevideo.com") || !validDomain(host) || host != strings.ToLower(host) || seen[host] || root == "" || len(prefix) < 3 || !strings.HasPrefix(prefix, "rr") || strings.Trim(prefix[2:], "0123456789") != "" {
if !validDomain(host) {
return errGooglevideoInsufficientSamples
}
seen[host], roots[strings.Split(root, "-")[0]], prefixes[prefix] = true, true, true
}
if len(hosts) < 3 || len(roots) < 2 || len(prefixes) < 2 {
module, err := requiredYoutubeAdapter()
if err != nil {
return err
}
if module.ValidateHosts(hosts) != nil {
return errGooglevideoInsufficientSamples
}
return nil
}
// Historical v1 reconstruction must not depend on a future registry default.
// This pinned contract remains available even when the execution module is absent.
func validateLegacyYoutubeV1Hosts(hosts []string) error {
for _, host := range hosts {
if !validDomain(host) {
return errGooglevideoInsufficientSamples
}
}
return (youtube.Module{}).ValidateHosts(hosts)
}
@@ -352,7 +352,7 @@ func TestManagedMonitorProductionAdapterBinding(t *testing.T) {
recovery, _, _ := recoveryFixture(t, false)
target, _, _ := managedCLIFixture(t)
target.directory = recovery.transaction.Dir
target.player = func(ctx context.Context, args []string, out, _ io.Writer) error {
target.player = func(ctx context.Context, args []string, _ io.Reader, out, _ io.Writer) error {
if ctx.Err() != nil {
t.Fatal(ctx.Err())
}
@@ -580,7 +580,7 @@ func TestManagedMonitorCLIBindingAndCancellation(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
requests := 0
d.player = func(ctx context.Context, args []string, out, _ io.Writer) error {
d.player = func(ctx context.Context, args []string, _ io.Reader, out, _ io.Writer) error {
requests++
if !strings.Contains(strings.Join(args, "|"), "--noproxy||--proxy|socks5h://127.0.0.1:11080|") {
t.Fatal(args)
+8 -2
View File
@@ -47,6 +47,9 @@ type managedRecoveryDeps struct {
// The injected command runner receives only fixed script/argv from orchestration.
func managedRecoveryDependencies(target managedDeps, db *sql.DB, scan cacheScan, check cacheCheck, run func(context.Context, string, ...string) error) (managedRecoveryDeps, error) {
d := managedRecoveryDeps{db: db, scan: scan, check: check, run: run}
if _, err := requiredYoutubeAdapter(); err != nil {
return d, err
}
if target.lookup == nil || target.player == nil || db == nil || scan == nil || check == nil || run == nil {
return d, fmt.Errorf("complete target-local recovery adapter dependencies required")
}
@@ -59,6 +62,9 @@ func managedRecoveryDependencies(target managedDeps, db *sql.DB, scan cacheScan,
}
d.transaction = &managedTransaction{Dir: target.directory, Name: "config"}
requestVideo := func(ctx context.Context, videoID string) ([]byte, error) {
if _, err := requiredYoutubeAdapter(); err != nil {
return nil, err
}
if err := ctx.Err(); err != nil {
return nil, err
}
@@ -67,8 +73,8 @@ func managedRecoveryDependencies(target managedDeps, db *sql.DB, scan cacheScan,
}
bounded, cancel := context.WithTimeout(ctx, 35*time.Second)
defer cancel()
body, err := requestPlayer(videoID, cfg.SOCKSProxy, func(_ context.Context, args []string, out, diagnostics io.Writer) error {
return target.player(bounded, args, out, diagnostics)
body, err := requestPlayer(videoID, cfg.SOCKSProxy, func(_ context.Context, args []string, stdin io.Reader, out, diagnostics io.Writer) error {
return target.player(bounded, args, stdin, out, diagnostics)
})
if bounded.Err() != nil {
return nil, managedRecoveryContextError(ctx, errors.Join(err, bounded.Err()))
+1 -1
View File
@@ -146,7 +146,7 @@ func TestManagedServiceFreshPrivateInventory(t *testing.T) {
cfg := healthConfig{SOCKSProxy: "socks5://127.0.0.1:11080"}
resources := managedServiceResources(cfg, func(context.Context) ([]byte, error) {
return []byte(`{"playabilityStatus":{"status":"OK"},"videoDetails":{"thumbnail":{"thumbnails":[{"url":"https://i.ytimg.com/fresh?private=thumbnail"}]}},"streamingData":{}}`), nil
}, func(_ context.Context, args []string, out, _ io.Writer) error {
}, func(_ context.Context, args []string, _ io.Reader, out, _ io.Writer) error {
if args[len(args)-1] != "https://www.youtube.com/" || strings.Contains(strings.Join(args, " "), "private=") {
t.Fatal(args)
}
+91 -10
View File
@@ -40,17 +40,22 @@ type managedJournal struct {
PendingCandidates []int64 `json:"pending_candidate_ids,omitempty"`
Scope GooglevideoScope `json:"committed_googlevideo_scope,omitempty"`
PendingScope GooglevideoScope `json:"pending_googlevideo_scope,omitempty"`
Ref *managedConfigRef `json:"committed_ref,omitempty"`
PendingRef *managedConfigRef `json:"pending_ref,omitempty"`
}
// Hooks take no commands or arguments. The target adapter binds Reload
// exclusively to /opt/zapret2/init.d/sysv/zapret2 restart and Health to
// health youtube over its configured loopback SOCKS, returning exactly "OK".
// exclusively to /opt/zapret2/init.d/sysv/zapret2 restart. Health is the
// adapter's owned protocol check, returning exactly "OK" (legacy YouTube or
// every required generic resource/TLS pair); optional rollback health is strict.
// Reload switches target-local firewall and daemons for apply and rollback.
// This engine does not manage SOCKS, health scheduling or DB active state.
type managedTransaction struct {
Dir, Name string
Reload func() error
Health func() (string, error)
RollbackHealth func() (string, error)
resource *resourceApplySnapshot
// Ordinary candidate-ID CLI calls cannot inherit an operator experiment.
// Evaluated against the effective policy only while holding the journal lock.
forbidExperimentalApply bool
@@ -186,7 +191,7 @@ func (m *managedTransaction) load() (managedJournal, error) {
if err != nil {
return j, err
}
if err := json.Unmarshal(b, &j); err != nil {
if err := strictPlanJSON(b, &j); err != nil {
return j, err
}
validHash := func(s string) bool {
@@ -208,9 +213,12 @@ func (m *managedTransaction) load() (managedJournal, error) {
}
return true
}
if (j.Version != 1 && j.Version != 2 && j.Version != 3) || !j.Scope.valid() || !j.PendingScope.valid() || (j.Version < 3 && (j.Scope != "" || j.PendingScope != "")) || (j.Version == 3 && (j.Scope == "" || (j.State == "prepared" && j.PendingScope == ""))) || j.ID == "" || !validHash(j.Hash) || (j.State != "committed" && j.State != "prepared") || (j.State == "prepared" && !validHash(j.Next)) || (j.State == "committed" && (j.Next != "" || len(j.PendingCandidates) != 0 || j.PendingScope != "")) || !validIDs(j.Candidates) || !validIDs(j.PendingCandidates) || (j.Version == 1 && (len(j.Candidates) != 0 || len(j.PendingCandidates) != 0)) {
if (j.Version != 1 && j.Version != 2 && j.Version != 3 && j.Version != 4) || !j.Scope.valid() || !j.PendingScope.valid() || (j.Version < 3 && (j.Scope != "" || j.PendingScope != "")) || (j.Version == 3 && (j.Scope == "" || (j.State == "prepared" && j.PendingScope == ""))) || j.ID == "" || !validHash(j.Hash) || (j.State != "committed" && j.State != "prepared") || (j.State == "prepared" && !validHash(j.Next)) || (j.State == "committed" && (j.Next != "" || len(j.PendingCandidates) != 0 || j.PendingScope != "" || j.PendingRef != nil)) || !validIDs(j.Candidates) || !validIDs(j.PendingCandidates) || (j.Version == 1 && (len(j.Candidates) != 0 || len(j.PendingCandidates) != 0)) {
return j, fmt.Errorf("invalid managed journal")
}
if j.Version < 4 && (j.Ref != nil || j.PendingRef != nil) || j.Version == 4 && (!validManagedConfigRef(j.Ref, j.Candidates, j.Scope, j.Hash) || j.State == "prepared" && !validManagedConfigRef(j.PendingRef, j.PendingCandidates, j.PendingScope, j.Next)) {
return j, fmt.Errorf("invalid managed journal references")
}
return j, nil
}
@@ -234,7 +242,12 @@ func (m *managedTransaction) Initialize() error {
func (m *managedTransaction) checkpoint(point string) error {
if m.fault != nil {
return m.fault(point)
if err := m.fault(point); err != nil {
return err
}
}
if m.resource != nil {
return m.resource.guard()
}
return nil
}
@@ -265,6 +278,9 @@ func (m *managedTransaction) ActiveCandidates(db *sql.DB) ([]int64, error) {
if j.State != "committed" {
return fmt.Errorf("incomplete transaction: reconcile first")
}
if j.Version == 4 && j.Ref.Kind == "generic" {
return fmt.Errorf("generic deployment has no legacy candidate IDs")
}
b, err := m.read(m.Name)
if err != nil {
return err
@@ -299,6 +315,10 @@ func committedManagedJournal(j managedJournal) managedJournal {
if j.Version == 3 {
j.Scope, j.PendingScope = j.PendingScope, ""
}
if j.Version == 4 {
j.Scope, j.PendingScope = j.PendingScope, ""
j.Ref, j.PendingRef = j.PendingRef, nil
}
return j
}
@@ -342,6 +362,10 @@ func (m *managedTransaction) applyRenderedDB(b []byte, db *sql.DB, ids ...int64)
}
func (m *managedTransaction) applyRenderedPolicy(b []byte, db *sql.DB, requested *GooglevideoScope, ids ...int64) error {
return m.applyPolicy(b, db, requested, nil, ids...)
}
func (m *managedTransaction) applyPolicy(b []byte, db *sql.DB, requested *GooglevideoScope, generic *managedConfigRef, ids ...int64) error {
return m.locked(func() error {
j, err := m.load()
if err != nil {
@@ -350,9 +374,32 @@ func (m *managedTransaction) applyRenderedPolicy(b []byte, db *sql.DB, requested
if j.State != "committed" {
return fmt.Errorf("incomplete transaction: reconcile first")
}
if err := validateManagedRefs(db, j.Candidates, j.Hash, j.Scope); err != nil {
if err := m.validateConfigRef(db, j.Ref, j.Candidates, j.Hash, j.Scope); err != nil {
return err
}
if generic == nil && j.Version == 4 {
return fmt.Errorf("generic journal cannot authorize legacy apply")
}
if generic != nil {
if m.resource == nil || *generic != m.resource.ref || m.resource.guard() != nil || m.resource.validate(generic, managedHash(b)) != nil {
return errResourcePlan
}
if j.Version == 4 && j.Ref.Kind == "generic" {
if reflect.DeepEqual(j.Ref, generic) {
current, e := m.read(m.Name)
if e != nil || managedHash(current) != j.Hash {
return errManagedDrift
}
return nil
}
}
if !(j.Version == 4 && j.Ref.Kind == "generic") && (len(j.Candidates) != 0 || j.Hash != managedHash([]byte(managedBaseline))) {
return fmt.Errorf("generic apply requires disabled baseline")
}
if m.resource.fresh() != nil {
return errResourcePlan
}
}
scope := j.Scope
if scope == "" {
scope = GooglevideoScopeExact
@@ -360,6 +407,9 @@ func (m *managedTransaction) applyRenderedPolicy(b []byte, db *sql.DB, requested
if requested != nil {
scope = *requested
}
if generic != nil {
scope = ""
}
if m.forbidExperimentalApply && scope == GooglevideoScopeExperimental {
return fmt.Errorf("experimental CLI apply requires explicit guarded operator path")
}
@@ -391,7 +441,7 @@ func (m *managedTransaction) applyRenderedPolicy(b []byte, db *sql.DB, requested
return fmt.Errorf("fresh qualified candidate state required")
}
}
if err := validateManagedRefs(db, ids, managedHash(b), scope); err != nil {
if err := m.validateConfigRef(db, generic, ids, managedHash(b), scope); err != nil {
return err
}
prior, err := m.read(m.Name)
@@ -433,7 +483,14 @@ func (m *managedTransaction) applyRenderedPolicy(b []byte, db *sql.DB, requested
return err
}
j.Generation++
if requested != nil || j.Version == 3 {
if generic != nil {
if j.Version != 4 {
j.Ref = &managedConfigRef{Kind: "baseline"}
}
j.Version = 4
j.Scope, j.PendingScope = "", ""
j.PendingRef = generic
} else if requested != nil || j.Version == 3 {
j.Version = 3
if j.Scope == "" {
j.Scope = GooglevideoScopeExact
@@ -469,9 +526,14 @@ func (m *managedTransaction) applyRenderedPolicy(b []byte, db *sql.DB, requested
return fault
}
}
if err == nil {
if generic != nil {
err = m.resource.guard()
}
if err == nil {
err = m.Reload()
}
}
if err == nil {
if fault := m.checkpoint("reloaded"); fault != nil {
return fault
@@ -491,6 +553,15 @@ func (m *managedTransaction) applyRenderedPolicy(b []byte, db *sql.DB, requested
}
if err == nil {
committed := committedManagedJournal(j)
if generic != nil {
err = m.resource.guard()
}
if err != nil {
if rollback := m.restore(j); rollback != nil {
return &managedRollbackError{Cause: err, Rollback: rollback}
}
return err
}
current, err = m.read(m.Name)
if err == nil && managedHash(current) != j.Next {
return errManagedDrift
@@ -542,6 +613,15 @@ func (m *managedTransaction) restore(j managedJournal) error {
if err := m.Reload(); err != nil {
return err
}
if m.RollbackHealth != nil {
status, err := m.RollbackHealth()
if err != nil {
return err
}
if status != "OK" {
return &managedHealthRejection{Status: status}
}
}
if err := m.owns(j, false); err != nil {
return err
}
@@ -555,6 +635,7 @@ func (m *managedTransaction) restore(j managedJournal) error {
j.State, j.Next = "committed", ""
j.PendingCandidates = nil
j.PendingScope = ""
j.PendingRef = nil
return m.journal(j)
}
@@ -570,10 +651,10 @@ func (m *managedTransaction) Reconcile(databases ...*sql.DB) error {
if len(databases) != 0 {
db = databases[0]
}
if err := validateManagedRefs(db, j.Candidates, j.Hash, j.Scope); err != nil {
if err := m.validateConfigRef(db, j.Ref, j.Candidates, j.Hash, j.Scope); err != nil {
return err
}
if err := validateManagedRefs(db, j.PendingCandidates, j.Next, j.PendingScope); err != nil {
if err := m.validateConfigRef(db, j.PendingRef, j.PendingCandidates, j.Next, j.PendingScope); err != nil {
return err
}
if j.State == "prepared" {
@@ -34,7 +34,7 @@ func (w playerCancelWriter) Write(p []byte) (int, error) {
func canceledPlayerProcess(t *testing.T, cancel context.CancelFunc, extra error, overflow string) playerRunner {
t.Helper()
return func(ctx context.Context, _ []string, out, diagnostics io.Writer) error {
return func(ctx context.Context, _ []string, _ io.Reader, out, diagnostics io.Writer) error {
cmd := exec.CommandContext(ctx, os.Args[0], "-test.run=^TestPlayerCancellationChild$")
cmd.Env = append(os.Environ(), "AUTOTUNE_PLAYER_CANCEL_CHILD=1")
cmd.Stdout = playerCancelWriter{cancel: cancel}
@@ -52,8 +52,8 @@ func TestPlayerRequestCancellationCause(t *testing.T) {
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
runner := canceledPlayerProcess(t, cancel, nil, "")
_, err := requestPlayer("abcdefghijk", "socks5://127.0.0.1:11080", func(_ context.Context, args []string, out, diagnostics io.Writer) error {
return runner(ctx, args, out, diagnostics)
_, err := requestPlayer("abcdefghijk", "socks5://127.0.0.1:11080", func(_ context.Context, args []string, stdin io.Reader, out, diagnostics io.Writer) error {
return runner(ctx, args, stdin, out, diagnostics)
})
var child *exec.ExitError
if !errors.As(err, &child) || !managedRecoveryTimeoutSafe(err) {
@@ -67,12 +67,17 @@ func TestPlayerRequestCancellationCause(t *testing.T) {
func TestPlayerRequestPublicErrorsOpaque(t *testing.T) {
const secret = "https://user:password@example.com/video?signature=private"
cause := errors.New(secret)
runner := func(_ context.Context, _ []string, _, diagnostics io.Writer) error {
runner := func(_ context.Context, args []string, stdin io.Reader, _, diagnostics io.Writer) error {
payload, err := io.ReadAll(stdin)
if err != nil || !bytes.Contains(payload, []byte("abcdefghijk")) || strings.Contains(strings.Join(args, " "), "abcdefghijk") {
t.Fatal("private POST body missing from stdin or exposed in argv")
}
io.WriteString(diagnostics, secret)
return cause
diagnostics.Write(payload)
return errors.Join(cause, errors.New(string(payload)))
}
_, err := requestPlayer("abcdefghijk", "socks5://127.0.0.1:11080", runner)
if !errors.Is(err, cause) || strings.Contains(err.Error(), secret) || managedRecoveryTimeoutSafe(err) {
if !errors.Is(err, cause) || strings.Contains(err.Error(), secret) || strings.Contains(err.Error(), "abcdefghijk") || managedRecoveryTimeoutSafe(err) {
t.Fatal("opaque error lost its unsafe cause or exposed it")
}
t.Setenv("ZAPRET_AUTOTUNE_VIDEO_ID", "abcdefghijk")
@@ -84,7 +89,7 @@ func TestPlayerRequestPublicErrorsOpaque(t *testing.T) {
} else {
code = discoverYoutube([]string{"--video-id", "abcdefghijk", "--proxy", "socks5://127.0.0.1:11080"}, &out, &diagnostics, runner)
}
if code != 1 || out.Len() != 0 || strings.Contains(diagnostics.String(), secret) || !strings.Contains(diagnostics.String(), "player request failed") {
if code != 1 || out.Len() != 0 || strings.Contains(diagnostics.String(), secret) || strings.Contains(diagnostics.String(), "abcdefghijk") || !strings.Contains(diagnostics.String(), "player request failed") {
t.Fatalf("%s: code=%d diagnostics=%q", command, code, diagnostics.String())
}
}
@@ -141,7 +146,7 @@ func TestManagedMonitorSIGTERMChild(t *testing.T) {
env["ZAPRET_AUTOTUNE_DB"] = managedTargetDirectory + "/candidates.db"
ctx, cancel := signal.NotifyContext(context.Background(), syscall.SIGTERM)
defer cancel()
d.player = func(ctx context.Context, _ []string, _, diagnostics io.Writer) error {
d.player = func(ctx context.Context, _ []string, _ io.Reader, _, diagnostics io.Writer) error {
cmd := exec.CommandContext(ctx, os.Args[0], "-test.run=^TestPlayerCancellationChild$")
cmd.Env = append(os.Environ(), "AUTOTUNE_PLAYER_CANCEL_CHILD=1")
cmd.Stdout, cmd.Stderr = os.Stdout, diagnostics
+74
View File
@@ -0,0 +1,74 @@
package main
import (
"bytes"
"context"
"errors"
"io"
"os"
"path/filepath"
"strings"
"testing"
"time"
)
func playerCurlStub(t *testing.T, script string) string {
t.Helper()
dir := t.TempDir()
if err := os.WriteFile(filepath.Join(dir, "curl"), []byte("#!/bin/sh\n"+script), 0700); err != nil {
t.Fatal(err)
}
t.Setenv("PATH", dir+string(os.PathListSeparator)+os.Getenv("PATH"))
return dir
}
func TestPlayerPOSTPrivateStdin(t *testing.T) {
dir := playerCurlStub(t, `printf '%s\n' "$@" > "$PLAYER_CAPTURE.args"
cat > "$PLAYER_CAPTURE.body"
printf '%s' "$PLAYER_RESPONSE"
`)
t.Setenv("PLAYER_CAPTURE", filepath.Join(dir, "request"))
t.Setenv("PLAYER_RESPONSE", discoveryFixture)
body, err := requestPlayer("SECRET12345", "socks5://127.0.0.1:11080", runPlayerCurl)
if err != nil || string(body) != discoveryFixture {
t.Fatal("real runner did not preserve the response", err)
}
args, err := os.ReadFile(filepath.Join(dir, "request.args"))
if err != nil {
t.Fatal(err)
}
if strings.Contains(string(args), "SECRET12345") || !strings.Contains(string(args), "--json\n@-\n") {
t.Fatal("POST body must travel only on stdin")
}
payload, err := os.ReadFile(filepath.Join(dir, "request.body"))
const want = `{"context":{"client":{"androidSdkVersion":30,"clientName":"ANDROID","clientVersion":"21.02.35","osName":"Android","osVersion":"11","userAgent":"com.google.youtube/21.02.35(Linux;U;Android11)gzip"}},"videoId":"SECRET12345"}`
if err != nil || string(payload) != want {
t.Fatal("POST bytes changed", err)
}
}
func TestPlayerPOSTInputBound(t *testing.T) {
for _, id := range []string{"", strings.Repeat("S", playerResponseLimit+1), "SECRET\n1234"} {
_, err := requestPlayer(id, "socks5://127.0.0.1:11080", func(context.Context, []string, io.Reader, io.Writer, io.Writer) error {
t.Fatal("invalid input invoked curl")
return nil
})
if err == nil || strings.Contains(err.Error(), "SECRET") {
t.Fatal("invalid POST input must fail privately before execution")
}
}
}
func TestPlayerCurlStdinCancellation(t *testing.T) {
playerCurlStub(t, "cat > /dev/null\nprintf ready\nexec sleep 60\n")
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
var diagnostics bytes.Buffer
err := runPlayerCurl(ctx, []string{"--json", "@-"}, strings.NewReader(`{"videoId":"SECRET12345"}`), playerCancelWriter{cancel: cancel}, &diagnostics)
if err == nil || !errors.Is(ctx.Err(), context.Canceled) || !managedRecoveryTimeoutSafe(err) {
t.Fatal("real curl runner must join a canceled child", err)
}
if strings.Contains(diagnostics.String(), "SECRET12345") {
t.Fatal("stdin leaked into diagnostics")
}
}
@@ -0,0 +1,257 @@
package main
import (
"errors"
"fmt"
"io"
"syscall"
)
// These enums are closed: no caller-controlled strings enter diagnostics.
type qualificationStage uint8
const (
stageUnknown qualificationStage = iota
stagePreDNSGuard
stagePostDNSGuard
stageDNS
stageTransfer
stageAttestation
stageWorkerStart
stageWorkerQueue
stageWorkerExit
stageNFTDelete
stageStopJoin
stageRestore
stageFinalStoreGuard
stageACKWrite
)
type qualificationReason uint8
const (
reasonUnknown qualificationReason = iota
reasonExecution
reasonOutputBound
reasonMetrics
reasonOrigin
reasonTLSEvidence
reasonFraming
reasonBody
reasonMIME
reasonWorkerIdentity
reasonRules
reasonFiles
reasonDefinition
reasonRuntime
reasonCollector
reasonGuard
reasonWorkerScan
reasonEngine
reasonArgv
reasonParent
reasonStarttime
reasonQueue
reasonVersion
reasonRulesRead
reasonMode
reasonPlanStore
reasonContext
reasonQueuePresentOrUnreadable
reasonNFTRead
reasonNFTPresent
reasonBaseline
reasonPriorCleanupVeto
reasonWorkerScanRead
reasonWorkerScanReadlink
reasonWorkerScanForeign
)
func (s qualificationStage) String() string {
names := [...]string{"unknown", "pre_dns_guard", "post_dns_guard", "dns", "transfer", "attestation", "worker_start", "worker_queue", "worker_exit", "nft_delete", "stop_join", "restore", "final_store_guard", "ack_write"}
if int(s) >= len(names) {
return names[0]
}
return names[s]
}
func (r qualificationReason) String() string {
names := [...]string{"unknown", "execution", "output_bound", "metrics", "origin", "tls_evidence", "framing", "body", "mime", "worker_identity", "rules", "files", "definition", "runtime", "collector", "guard", "worker_scan", "engine", "argv", "parent", "starttime", "queue", "version", "rules_read", "mode", "plan_store", "context", "queue_present_or_unreadable", "nft_read", "nft_present", "baseline", "prior_cleanup_veto", "worker_scan_read", "worker_scan_readlink", "worker_scan_foreign"}
if int(r) >= len(names) {
return names[0]
}
return names[r]
}
type qualificationDiagnostic struct {
Err error
Stage qualificationStage
Reason qualificationReason
Candidate, Resource, Attempt int
TLS uint8
Errno qualificationErrno
ScanPID, OwnedPID int
}
type qualificationErrno uint8
const (
errnoNone qualificationErrno = iota
errnoEACCES
errnoEPERM
errnoESRCH
errnoENOENT
errnoOther
)
func (e qualificationErrno) String() string {
names := [...]string{"none", "eacces", "eperm", "esrch", "enoent", "other"}
if int(e) >= len(names) {
return "other"
}
return names[e]
}
func qualificationScanErrno(err error) qualificationErrno {
for _, item := range []struct {
err error
code qualificationErrno
}{{syscall.EACCES, errnoEACCES}, {syscall.EPERM, errnoEPERM}, {syscall.ESRCH, errnoESRCH}, {syscall.ENOENT, errnoENOENT}} {
if errors.Is(err, item.err) {
return item.code
}
}
if err == nil {
return errnoNone
}
return errnoOther
}
func workerScanReason(reason qualificationReason) bool {
return reason == reasonWorkerScanRead || reason == reasonWorkerScanReadlink || reason == reasonWorkerScanForeign
}
// Normalize classification without retaining private predicate errors.
func qualificationPlanPredicate(err error, fallback qualificationReason) error {
d := &qualificationDiagnostic{Err: errResourcePlan, Reason: qualificationErrorReason(err, fallback)}
copyQualificationScanMetadata(d, err)
return d
}
func copyQualificationScanMetadata(d *qualificationDiagnostic, err error) {
var source *qualificationDiagnostic
if workerScanReason(d.Reason) && errors.As(err, &source) {
d.Errno, d.ScanPID, d.OwnedPID = source.Errno, source.ScanPID, source.OwnedPID
}
}
func (e *qualificationDiagnostic) Error() string { return "qualification operation failed" }
func (e *qualificationDiagnostic) Unwrap() error { return e.Err }
func qualificationOperation(err error, stage qualificationStage) error {
if err == nil {
return nil
}
d := &qualificationDiagnostic{Err: err, Stage: stage, Reason: qualificationErrorReason(err, reasonUnknown)}
copyQualificationScanMetadata(d, err)
return d
}
func qualificationPredicate(err error, reason qualificationReason) error {
d := &qualificationDiagnostic{Err: err, Reason: reason}
copyQualificationScanMetadata(d, err)
return d
}
func qualificationErrorReason(err error, fallback qualificationReason) qualificationReason {
var diagnostic *qualificationDiagnostic
if errors.As(err, &diagnostic) && diagnostic.Reason != reasonUnknown {
return diagnostic.Reason
}
return fallback
}
func attestationMismatch(a, b qualificationAttestation) qualificationReason {
switch {
case a.PID != b.PID || a.StartTime != b.StartTime || a.WorkerArgsHash != b.WorkerArgsHash || a.Queue != b.Queue:
return reasonWorkerIdentity
case a.RulesHash != b.RulesHash:
return reasonRules
case a.LuaHash != b.LuaHash:
return reasonFiles
case a.Definition != b.Definition:
return reasonDefinition
case a.Runtime != b.Runtime:
return reasonRuntime
case a.Collector != b.Collector:
return reasonCollector
default:
return reasonUnknown
}
}
// Traverse only for reporting, never for classification. Bound even pathological
// error trees; output contains enums and ordinals, never Error() or type names.
func writeQualificationDiagnostics(w io.Writer, err error) {
budget := 32
var visit func(error, qualificationDiagnostic, bool)
visit = func(err error, inherited qualificationDiagnostic, cleanup bool) {
if err == nil || budget == 0 {
return
}
budget--
switch e := err.(type) {
case *qualificationDiagnostic:
if e.Candidate != 0 {
inherited.Candidate = e.Candidate
}
if e.Resource != 0 {
inherited.Resource = e.Resource
}
if e.Attempt != 0 {
inherited.Attempt = e.Attempt
}
if e.TLS != 0 {
inherited.TLS = e.TLS
}
if e.Stage != stageUnknown && (e.Stage != stageAttestation || inherited.Stage == stageUnknown) {
inherited.Stage = e.Stage
}
if e.Reason != reasonUnknown {
inherited.Reason = e.Reason
}
if workerScanReason(e.Reason) {
inherited.Errno, inherited.ScanPID, inherited.OwnedPID = e.Errno, e.ScanPID, e.OwnedPID
}
cleanup = cleanup || e.Stage == stageNFTDelete || e.Stage == stageStopJoin || e.Stage == stageRestore
visit(e.Err, inherited, cleanup)
return
case *resourceError:
inherited.Reason = e.reason
case interface{ Unwrap() []error }:
for _, child := range e.Unwrap() {
visit(child, inherited, cleanup)
}
return
}
if e, ok := err.(interface{ Unwrap() error }); ok {
visit(e.Unwrap(), inherited, cleanup)
return
}
role, tls := "primary", "none"
if cleanup {
role = "cleanup"
}
if inherited.TLS == 12 {
tls = "tls12"
} else if inherited.TLS == 13 {
tls = "tls13"
}
fmt.Fprintf(w, "qualification diagnostic: role=%s stage=%s reason=%s candidate=%d resource=%d attempt=%d tls=%s", role, inherited.Stage, inherited.Reason, inherited.Candidate, inherited.Resource, inherited.Attempt, tls)
if workerScanReason(inherited.Reason) {
fmt.Fprintf(w, " errno=%s scan_pid=%d owned_pid=%d", inherited.Errno, inherited.ScanPID, inherited.OwnedPID)
}
fmt.Fprintln(w)
}
visit(err, qualificationDiagnostic{}, false)
}
@@ -0,0 +1,239 @@
package main
import (
"bytes"
"context"
"errors"
"io"
"strings"
"testing"
)
func TestQualificationDiagnosticStagesAndPrivacy(t *testing.T) {
secret := errors.New("PRIVATE https://secret.example/path Authorization IP env")
for _, mode := range []string{"transfer", "next_guard", "post_guard", "attestation", "metrics", "mime", "cleanup"} {
t.Run(mode, func(t *testing.T) {
q, s, registry, _, calls := qualifierFixture(t)
original := q.checks.runner
q.checks.runner = func(ctx context.Context, args []string, in io.Reader, out, diag io.Writer) error {
if *calls == 3 && (mode == "transfer" || mode == "cleanup") {
return secret
}
var wire bytes.Buffer
err := original(ctx, args, in, &wire, diag)
if *calls == 4 {
if mode == "metrics" {
wire.WriteString("PRIVATE")
}
if mode == "mime" {
wire = *bytes.NewBufferString(strings.Replace(wire.String(), "text/plain", "text/plain; PRIVATE", 1))
}
}
_, _ = out.Write(wire.Bytes())
return err
}
guards := 0
q.checks.guard = func() error {
guards++
if (mode == "next_guard" && guards == 9) || (mode == "post_guard" && guards == 8) {
return secret
}
return nil
}
attest := q.attest
q.attest = func(ctx context.Context, pid int, args []string, d strategyDefinition) (qualificationAttestation, error) {
a, err := attest(ctx, pid, args, d)
if mode == "attestation" && *calls == 4 {
a.RulesHash = strings.Repeat("b", 64)
}
return a, err
}
if mode == "cleanup" {
q.restore = func(context.Context) error { return secret }
}
_, err := q.complete(context.Background(), s, registry)
if err == nil {
t.Fatal("failure accepted")
}
var diagnostic bytes.Buffer
writeQualificationDiagnostics(&diagnostic, err)
got := diagnostic.String()
if strings.Contains(got, "PRIVATE") || strings.Contains(got, "example") || strings.Contains(got, "SECRET") {
t.Fatal(got)
}
want := map[string]string{"transfer": "stage=transfer reason=execution candidate=1 resource=1 attempt=1 tls=tls13", "cleanup": "stage=transfer reason=execution candidate=1 resource=1 attempt=1 tls=tls13", "next_guard": "stage=pre_dns_guard reason=unknown candidate=1 resource=1 attempt=2 tls=tls13", "post_guard": "stage=post_dns_guard reason=unknown candidate=1 resource=1 attempt=1 tls=tls13", "attestation": "stage=pre_dns_guard reason=rules", "metrics": "stage=transfer reason=metrics", "mime": "stage=transfer reason=mime"}[mode]
if !strings.Contains(got, want) {
t.Fatalf("want %s, got %s", want, got)
}
if mode == "cleanup" {
if !strings.Contains(got, "role=cleanup stage=restore") || qualificationCleanupVerified(err, func(context.Context) error { t.Fatal("cleanup retry"); return nil }) {
t.Fatal(got)
}
}
if mode == "transfer" && !errors.Is(err, secret) {
t.Fatal("lost original error")
}
})
}
}
func TestQualificationDiagnosticsDoNotChangeNegatives(t *testing.T) {
for _, mode := range []string{"status500", "mime_mismatch"} {
t.Run(mode, func(t *testing.T) {
q, s, registry, _, calls := qualifierFixture(t)
if mode == "mime_mismatch" {
source, err := q.source()
if err != nil {
t.Fatal(err)
}
for i := range source.config.Services {
for j := range source.config.Services[i].Resources {
source.config.Services[i].Resources[j].Expect.ContentType = "image/png"
}
}
q.source = func() (resourceMonitorSource, error) { return source, nil }
} else {
original := q.checks.runner
q.checks.runner = func(ctx context.Context, args []string, in io.Reader, out, diag io.Writer) error {
var wire bytes.Buffer
err := original(ctx, args, in, &wire, diag)
_, _ = io.WriteString(out, strings.ReplaceAll(strings.ReplaceAll(wire.String(), "200 OK", "500 OK"), `"http_code":200`, `"http_code":500`))
return err
}
}
proof, err := q.complete(context.Background(), s, registry)
if err != nil || *calls != 12 {
t.Fatalf("%d %v", *calls, err)
}
for _, check := range proof.matrix.Runs[0].Checks {
if check.Outcome != "expectation_failed" || check.Attempts != 3 {
t.Fatal(check)
}
}
})
}
}
func TestQualificationDiagnosticVetoAndFinalStages(t *testing.T) {
secret := errors.New("PRIVATE sentinel")
for _, stage := range []qualificationStage{stageFinalStoreGuard, stageACKWrite, stageNFTDelete, stageStopJoin, stageWorkerQueue, stageWorkerExit} {
err := qualificationOperation(secret, stage)
var out bytes.Buffer
writeQualificationDiagnostics(&out, err)
if !strings.Contains(out.String(), "stage="+stage.String()) || strings.Contains(out.String(), "PRIVATE") || !errors.Is(err, secret) {
t.Fatal(out.String())
}
}
cancel := qualificationOperation(context.Canceled, stagePreDNSGuard)
if !managedCancellationOnly(cancel, context.Canceled) {
t.Fatal("pure cancellation changed")
}
for _, veto := range []error{secret, &managedRecoveryFatal{Err: context.Canceled}, &managedAssetCleanupFailure{Err: secret}} {
if managedCancellationOnly(errors.Join(cancel, veto), context.Canceled) {
t.Fatal("mixed veto lost")
}
}
if resourceRejectionOnly(qualificationOperation(&resourcePeerTLSRejection{error: secret}, stageTransfer)) {
t.Fatal("arbitrary rejection unwrapping added")
}
}
func TestQualificationFinalizeDiagnosticsAndACKGate(t *testing.T) {
secret := errors.New("PRIVATE sentinel")
for _, mode := range []string{"cleanup", "owned_cleanup", "restore", "guard", "ack", "success"} {
t.Run(mode, func(t *testing.T) {
var primary error
if mode == "cleanup" {
primary = &qualificationCleanupFailure{Err: secret}
}
if mode == "owned_cleanup" {
primary = errors.Join(context.Canceled, &managedRecoveryFatal{Err: &managedAssetCleanupFailure{Err: secret}})
}
restores, guards, acks := 0, 0, 0
err := qualificationFinalize(primary, func(context.Context) error {
restores++
if mode == "restore" {
return secret
}
return nil
}, func() error {
guards++
if mode == "guard" {
return secret
}
return nil
}, func() error {
acks++
if mode == "ack" {
return secret
}
return nil
})
if mode == "success" {
if err != nil || acks != 1 {
t.Fatal(err, acks)
}
return
}
var out bytes.Buffer
writeQualificationDiagnostics(&out, err)
stage := stageRestore
if mode == "guard" {
stage = stageFinalStoreGuard
}
if mode == "ack" {
stage = stageACKWrite
}
if !strings.Contains(out.String(), "stage="+stage.String()) || strings.Contains(out.String(), "PRIVATE") {
t.Fatal(out.String())
}
if mode != "ack" && acks != 0 {
t.Fatal("ACK passed failed gate")
}
if mode == "cleanup" || mode == "owned_cleanup" {
if restores != 0 || guards != 0 || acks != 0 || !strings.Contains(out.String(), "reason=prior_cleanup_veto") {
t.Fatal("cleanup failure retried or misdiagnosed", out.String())
}
}
})
}
}
func TestQualificationOpaqueAttestationReasonUnderPostDNSGuard(t *testing.T) {
secret := errors.New("PRIVATE attestation path env argv")
q, s, registry, _, transfers := qualifierFixture(t)
attest, calls := q.attest, 0
q.attest = func(ctx context.Context, pid int, args []string, d strategyDefinition) (qualificationAttestation, error) {
calls++
if calls == 3 { // Initial attestation, pre-DNS, then post-DNS.
return qualificationAttestation{}, qualificationPredicate(secret, reasonWorkerScan)
}
return attest(ctx, pid, args, d)
}
proof, err := q.complete(context.Background(), s, registry)
if !errors.Is(err, errResourcePlan) || !errors.Is(err, secret) || proof.seal != "" || *transfers != 0 {
t.Fatal("ownership veto or execution order changed", err, *transfers)
}
var out bytes.Buffer
writeQualificationDiagnostics(&out, err)
if strings.Contains(out.String(), "PRIVATE") || strings.Contains(out.String(), "reason=unknown") ||
strings.Count(out.String(), "stage=post_dns_guard reason=worker_scan candidate=1 resource=1 attempt=1 tls=tls12") != 2 {
t.Fatal(out.String())
}
}
func TestQualificationAttestationErrorRetainsOwnershipVeto(t *testing.T) {
q, s, registry, _, _ := qualifierFixture(t)
attest, calls := q.attest, 0
q.attest = func(ctx context.Context, pid int, args []string, d strategyDefinition) (qualificationAttestation, error) {
calls++
if calls == 2 {
return qualificationAttestation{}, context.Canceled
}
return attest(ctx, pid, args, d)
}
_, err := q.complete(context.Background(), s, registry)
if !errors.Is(err, context.Canceled) || !errors.Is(err, errResourcePlan) || managedCancellationOnly(err, context.Canceled) {
t.Fatal("attestation ownership veto changed", err)
}
}
@@ -0,0 +1,198 @@
package main
import (
"bytes"
"context"
"errors"
"go/ast"
"go/format"
"go/parser"
"go/token"
"reflect"
"strings"
"testing"
)
func TestQualificationRestorePredicatesAndOrder(t *testing.T) {
secret := errors.New("PRIVATE nft process baseline sentinel")
for _, reason := range []qualificationReason{reasonContext, reasonGuard, reasonMode, reasonPlanStore, reasonRuntime, reasonWorkerScan, reasonQueuePresentOrUnreadable, reasonNFTRead, reasonNFTPresent, reasonBaseline, reasonUnknown} {
t.Run(reason.String(), func(t *testing.T) {
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
if reason == reasonContext {
cancel()
}
var order []string
q := resourceQualifier{guard: func() error {
order = append(order, "guard")
if reason == reasonGuard {
return secret
}
if reason == reasonMode || reason == reasonPlanStore || reason == reasonRuntime {
return qualificationPredicate(secret, reason)
}
return nil
}}
q.runtime.nft = func(_ context.Context, input string) (string, error) {
order = append(order, "nft")
if input != "list tables\n" {
t.Fatal(input)
}
if reason == reasonNFTRead {
return "", secret
}
if reason == reasonNFTPresent {
return "table inet " + managedAssetTable, nil
}
return "", nil
}
err := q.restoreBaseline(ctx, func() error {
order = append(order, "worker_queue")
if reason == reasonWorkerScan || reason == reasonQueuePresentOrUnreadable {
return qualificationPredicate(secret, reason)
}
return nil
}, func() error {
order = append(order, "baseline")
if reason == reasonBaseline {
return errResourcePlan
}
return nil
})
want := []string{"guard", "worker_queue", "nft", "baseline"}
switch reason {
case reasonContext:
want = nil
case reasonGuard, reasonMode, reasonPlanStore, reasonRuntime:
want = want[:1]
case reasonWorkerScan, reasonQueuePresentOrUnreadable:
want = want[:2]
case reasonNFTRead, reasonNFTPresent:
want = want[:3]
}
if !reflect.DeepEqual(order, want) {
t.Fatal(order, want)
}
if reason == reasonUnknown {
if err != nil {
t.Fatal(err)
}
return
}
if !errors.Is(err, errResourcePlan) || errors.Is(err, secret) {
t.Fatal("original restore classification changed", err)
}
var out bytes.Buffer
writeQualificationDiagnostics(&out, qualificationOperation(err, stageRestore))
if !strings.Contains(out.String(), "role=cleanup stage=restore reason="+reason.String()) || strings.Contains(out.String(), "PRIVATE") {
t.Fatal(out.String())
}
})
}
}
// Check the actual production factory, not just injected diagnostic errors. This
// pins opaque predicate order/strictness without starting workers or changing nft.
func TestQualificationProductionPredicateMap(t *testing.T) {
fset := token.NewFileSet()
f, err := parser.ParseFile(fset, "resource_qualify.go", nil, 0)
if err != nil {
t.Fatal(err)
}
text := func(n ast.Node) string {
var b bytes.Buffer
if err := format.Node(&b, fset, n); err != nil {
t.Fatal(err)
}
return b.String()
}
var attest *ast.FuncLit
var restore, guard, noWorker, onlyWorker, scanWorkers string
ast.Inspect(f, func(n ast.Node) bool {
if fn, ok := n.(*ast.FuncDecl); ok {
switch fn.Name.Name {
case "qualificationNoWorker":
noWorker = text(fn.Body)
case "qualificationOnlyWorker":
onlyWorker = text(fn.Body)
case "qualificationScanWorkers":
scanWorkers = text(fn.Body)
}
}
a, ok := n.(*ast.AssignStmt)
if !ok || len(a.Lhs) != 1 || len(a.Rhs) != 1 {
return true
}
if text(a.Lhs[0]) == "q.attest" {
attest, _ = a.Rhs[0].(*ast.FuncLit)
}
if text(a.Lhs[0]) == "q.restore" {
restore = text(a.Rhs[0])
}
if text(a.Lhs[0]) == "q.guard" && guard == "" {
guard = text(a.Rhs[0])
}
return true
})
if attest == nil || !strings.Contains(restore, "q.restoreBaseline(ctx, qualificationNoWorker, func() error { return qualificationBaseline(d.directory) })") {
t.Fatal("production restore wiring changed", restore)
}
for _, check := range []string{
"if e := qualificationMode(config, strategies, d); e != nil {\n\t\treturn qualificationPredicate(errResourcePlan, qualificationErrorReason(e, reasonMode))",
"if e := s.guard(); e != nil {\n\t\treturn qualificationPredicate(errResourcePlan, qualificationErrorReason(e, reasonPlanStore))",
"if e := r.guard(); e != nil {\n\t\treturn qualificationPredicate(e, qualificationErrorReason(e, reasonRuntime))",
} {
if !strings.Contains(guard, check) {
t.Fatal("production guard predicate changed", guard)
}
}
if !strings.Contains(noWorker, "qualificationOnlyWorker(0)") || !strings.Contains(noWorker, "qualificationPredicate(e, qualificationErrorReason(e, reasonWorkerScan))") ||
!strings.Contains(noWorker, "managedAssetQueueReady(0)") || !strings.Contains(noWorker, "e != nil || occupied") || !strings.Contains(noWorker, "qualificationPredicate(errResourcePlan, reasonQueuePresentOrUnreadable)") {
t.Fatal("production worker/queue predicate changed", noWorker)
}
if !strings.Contains(onlyWorker, "qualificationScanWorkers(owned, os.ReadDir, os.Readlink)") ||
!strings.Contains(scanWorkers, "if errors.Is(e, os.ErrNotExist) {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\treturn &qualificationDiagnostic{Err: errResourcePlan, Reason: reasonWorkerScanReadlink") {
t.Fatal("unexpected procfs readlink errors must remain fail-closed", onlyWorker, scanWorkers)
}
var got []string
for _, stmt := range attest.Body.List {
branch, ok := stmt.(*ast.IfStmt)
if !ok {
continue
}
predicate := text(branch.Cond)
if branch.Init != nil {
predicate = text(branch.Init) + "; " + predicate
}
ret, ok := branch.Body.List[0].(*ast.ReturnStmt)
if !ok || len(ret.Results) != 2 {
t.Fatal("unexpected attestation branch", text(branch))
}
got = append(got, predicate+" => "+text(ret.Results[1]))
}
want := []string{
"e := q.guard(); e != nil => qualificationPredicate(errResourcePlan, qualificationErrorReason(e, reasonGuard))",
"e := qualificationOnlyWorker(pid); e != nil => qualificationPlanPredicate(e, reasonWorkerScan)",
`engine == nil || engine.validate() != nil || engine.running(fmt.Sprintf("/proc/%d/exe", pid)) != nil => qualificationPredicate(errResourcePlan, reasonEngine)`,
`e != nil || string(cmd) != strings.Join(append([]string{qualificationEngine}, args...), "\x00")+"\x00" => qualificationPredicate(errResourcePlan, reasonArgv)`,
"e != nil => qualificationPredicate(e, reasonParent)",
"end < 0 => qualificationPredicate(errResourcePlan, reasonParent)",
"len(tail) < 2 || tail[1] != strconv.Itoa(os.Getpid()) => qualificationPredicate(errResourcePlan, reasonParent)",
"e != nil => qualificationPredicate(e, reasonStarttime)",
"e != nil || !ready => qualificationPredicate(errResourcePlan, reasonQueue)",
"e != nil => qualificationPredicate(e, reasonFiles)",
"version.Run() != nil => qualificationPredicate(errResourcePlan, reasonVersion)",
"!valid => qualificationPredicate(errResourcePlan, reasonVersion)",
`e != nil || strings.TrimSpace(rules) == "" => qualificationPredicate(errResourcePlan, reasonRulesRead)`,
}
if !reflect.DeepEqual(got, want) {
t.Fatalf("production attestation predicates:\n%v\nwant:\n%v", got, want)
}
// The factory's strict mode check must precede any store, worker or nft access.
q := newResourceQualifier("", "", managedDeps{lookup: func(string) (string, bool) { return "", false }}, nil)
defer q.close()
_, err = q.attest(context.Background(), 0, nil, strategyDefinition{})
if !errors.Is(err, errResourcePlan) || qualificationErrorReason(err, reasonUnknown) != reasonMode {
t.Fatal(err)
}
}
@@ -0,0 +1,168 @@
package main
import (
"bytes"
"context"
"errors"
"fmt"
"os"
"path/filepath"
"strings"
"syscall"
"testing"
)
func TestQualificationWorkerScanBranches(t *testing.T) {
secret := errors.New("PRIVATE path argv propertyID sentinel")
dir := t.TempDir()
for _, name := range []string{"0-nonnumeric", "41"} {
if err := os.Mkdir(filepath.Join(dir, name), 0700); err != nil {
t.Fatal(err)
}
}
entries, err := os.ReadDir(dir)
if err != nil {
t.Fatal(err)
}
for _, tc := range []struct {
name string
err error
code qualificationErrno
}{
{"eacces", syscall.EACCES, errnoEACCES},
{"eperm", syscall.EPERM, errnoEPERM},
{"esrch", syscall.ESRCH, errnoESRCH},
{"enoent", syscall.ENOENT, errnoENOENT},
{"not_exist", os.ErrNotExist, errnoOther},
{"other", secret, errnoOther},
} {
for _, readDirFailure := range []bool{false, true} {
t.Run(fmt.Sprintf("%s/readdir=%t", tc.name, readDirFailure), func(t *testing.T) {
wrapped := &os.PathError{Op: "PRIVATE", Path: secret.Error(), Err: errors.Join(tc.err, secret)}
calls := 0
err := qualificationScanWorkers(42, func(path string) ([]os.DirEntry, error) {
if path != "/proc" {
t.Fatal(path)
}
if readDirFailure {
return nil, wrapped
}
return entries, nil
}, func(path string) (string, error) {
calls++
if path != "/proc/41/exe" {
t.Fatal("nonnumeric entry scanned", path)
}
return "", wrapped
})
if !readDirFailure && errors.Is(wrapped, os.ErrNotExist) {
if err != nil || calls != 1 {
t.Fatal("wrapped ENOENT must skip", err, calls)
}
return
}
var d *qualificationDiagnostic
if !errors.As(err, &d) || d.Errno != tc.code || d.OwnedPID != 42 {
t.Fatal("missing typed metadata", err)
}
if readDirFailure {
if d.Reason != reasonWorkerScanRead || d.ScanPID != 0 || calls != 0 || !errors.Is(err, wrapped) {
t.Fatal("readdir classification changed", d, calls)
}
} else if d.Reason != reasonWorkerScanReadlink || d.ScanPID != 41 || !errors.Is(err, errResourcePlan) || errors.Is(err, wrapped) {
t.Fatal("readlink classification changed", d)
}
assertWorkerScanOutput(t, err, stageUnknown, d.Reason, tc.code, d.ScanPID, 42)
})
}
}
for _, tc := range []struct {
name, exe string
owned int
deny bool
}{
{"owned", "/PRIVATE/nfqws2", 41, false},
{"foreign", "/PRIVATE/nfqws2", 42, true},
// A foreign --version invocation has the same exe: never inspect argv
// or exempt it from the exclusive-worker predicate.
{"foreign_version", "/PRIVATE/nfqws2", 0, true},
{"unrelated", "/PRIVATE/other", 42, false},
} {
t.Run(tc.name, func(t *testing.T) {
err := qualificationScanWorkers(tc.owned, func(string) ([]os.DirEntry, error) { return entries, nil }, func(string) (string, error) { return tc.exe, nil })
if !tc.deny {
if err != nil {
t.Fatal(err)
}
return
}
if !errors.Is(err, errResourcePlan) {
t.Fatal("foreign worker accepted", err)
}
assertWorkerScanOutput(t, err, stageUnknown, reasonWorkerScanForeign, errnoNone, 41, tc.owned)
})
}
}
func assertWorkerScanOutput(t *testing.T, err error, stage qualificationStage, reason qualificationReason, errno qualificationErrno, scan, owned int) {
t.Helper()
var out bytes.Buffer
writeQualificationDiagnostics(&out, err)
want := fmt.Sprintf(" errno=%s scan_pid=%d owned_pid=%d", errno, scan, owned)
if !strings.Contains(out.String(), "stage="+stage.String()+" reason="+reason.String()) || !strings.Contains(out.String(), want) || strings.Contains(out.String(), "PRIVATE") {
t.Fatal(out.String())
}
}
func TestQualificationWorkerScanNormalization(t *testing.T) {
secret := errors.New("PRIVATE path argv propertyID sentinel")
for _, reason := range []qualificationReason{reasonWorkerScanRead, reasonWorkerScanReadlink, reasonWorkerScanForeign} {
for _, errno := range []qualificationErrno{errnoNone, errnoEACCES, errnoEPERM, errnoESRCH, errnoENOENT, errnoOther} {
scan := &qualificationDiagnostic{Err: secret, Reason: reason, Errno: errno, ScanPID: 41, OwnedPID: 42}
attestation := qualificationPlanPredicate(qualificationOperation(scan, stageAttestation), reasonWorkerScan)
if !errors.Is(attestation, errResourcePlan) || errors.Is(attestation, secret) {
t.Fatal("attestation classification changed", attestation)
}
assertWorkerScanOutput(t, qualificationOperation(attestation, stagePostDNSGuard), stagePostDNSGuard, reason, errno, 41, 42)
q := resourceQualifier{guard: func() error { return nil }}
q.runtime.nft = func(context.Context, string) (string, error) {
t.Fatal("restore passed scan veto")
return "", nil
}
restore := q.restoreBaseline(context.Background(), func() error {
return qualificationPredicate(scan, qualificationErrorReason(scan, reasonWorkerScan))
}, func() error { t.Fatal("unsafe baseline restore"); return nil })
if !errors.Is(restore, errResourcePlan) || errors.Is(restore, secret) {
t.Fatal("restore classification changed", restore)
}
assertWorkerScanOutput(t, qualificationOperation(restore, stageRestore), stageRestore, reason, errno, 41, 42)
}
}
var out bytes.Buffer
writeQualificationDiagnostics(&out, qualificationPredicate(secret, reasonEngine))
if strings.Contains(out.String(), "errno=") || strings.Contains(out.String(), "_pid=") || strings.Contains(out.String(), "PRIVATE") {
t.Fatal("scan metadata on non-scan diagnostic", out.String())
}
}
func TestQualificationWorkerScanPostDNSVeto(t *testing.T) {
q, s, registry, _, transfers := qualifierFixture(t)
attest, calls := q.attest, 0
q.attest = func(ctx context.Context, pid int, args []string, d strategyDefinition) (qualificationAttestation, error) {
calls++
if calls == 3 {
scan := &qualificationDiagnostic{Err: errors.New("PRIVATE path argv propertyID sentinel"), Reason: reasonWorkerScanReadlink, Errno: errnoEPERM, ScanPID: 41, OwnedPID: pid}
return qualificationAttestation{}, qualificationPlanPredicate(scan, reasonWorkerScan)
}
return attest(ctx, pid, args, d)
}
proof, err := q.complete(context.Background(), s, registry)
if !errors.Is(err, errResourcePlan) || proof.seal != "" || *transfers != 0 {
t.Fatal("ownership veto or execution order changed", err, *transfers)
}
var out bytes.Buffer
writeQualificationDiagnostics(&out, err)
if strings.Count(out.String(), "stage=post_dns_guard reason=worker_scan_readlink candidate=1 resource=1 attempt=1 tls=tls12 errno=eperm scan_pid=41 owned_pid=") != 2 || strings.Contains(out.String(), "PRIVATE") {
t.Fatal(out.String())
}
}
+1 -1
View File
@@ -66,7 +66,7 @@ func TestRecoverGooglevideo(t *testing.T) {
var order, checked []string
player := cachePlayer(t, &order)
if mode == "discovery-error" {
player = func(context.Context, []string, io.Writer, io.Writer) error {
player = func(context.Context, []string, io.Reader, io.Writer, io.Writer) error {
order = append(order, "discover")
return fmt.Errorf("SECRET discovery failure")
}
+468
View File
@@ -0,0 +1,468 @@
package main
import (
"bytes"
"context"
"database/sql"
"fmt"
"io"
"net"
"os"
"os/signal"
"sort"
"strconv"
"strings"
"syscall"
"time"
)
// v4 tags each side independently; generic references never impersonate SQL IDs.
type managedConfigRef struct {
Kind string `json:"kind"`
Registry string `json:"registry_hash,omitempty"`
Matrix string `json:"matrix_hash,omitempty"`
Source string `json:"source_fingerprint,omitempty"`
Plan string `json:"plan_digest,omitempty"`
Config string `json:"config_hash,omitempty"`
Policy string `json:"strategy_ids,omitempty"`
}
const resourceStrategyIDsEnv = "ZAPRET_AUTOTUNE_RESOURCE_STRATEGY_IDS"
// Policy is a canonical bounded allowlist, never qualification evidence.
func parseResourceStrategyIDs(value string) (string, error) {
if value == "" {
return "", nil
}
if len(value) > 4096 {
return "", errResourcePlan
}
ids := strings.Split(value, ",")
if len(ids) > 32 {
return "", errResourcePlan
}
seen := map[string]bool{}
for _, id := range ids {
if !resourceID.MatchString(id) || seen[id] {
return "", errResourcePlan
}
seen[id] = true
}
sort.Strings(ids)
return strings.Join(ids, ","), nil
}
func validManagedConfigRef(r *managedConfigRef, ids []int64, scope GooglevideoScope, hash string) bool {
if r == nil {
return false
}
switch r.Kind {
case "generic":
policy, e := parseResourceStrategyIDs(r.Policy)
return e == nil && policy == r.Policy && len(ids) == 0 && scope == "" && hashOK(r.Registry) && hashOK(r.Matrix) && hashOK(r.Source) && hashOK(r.Plan) && r.Config == hash
case "baseline", "legacy":
return r.Policy == "" && r.Registry == "" && r.Matrix == "" && r.Source == "" && r.Plan == "" && r.Config == "" && (r.Kind == "legacy" && len(ids) > 0 || r.Kind == "baseline" && len(ids) == 0 && scope == "" && hash == managedHash([]byte(managedBaseline)))
}
return false
}
func (m *managedTransaction) validateConfigRef(db *sql.DB, ref *managedConfigRef, ids []int64, hash string, scope GooglevideoScope) error {
if ref == nil {
return validateManagedRefs(db, ids, hash, scope)
}
if !validManagedConfigRef(ref, ids, scope, hash) {
return errResourcePlan
}
if ref.Kind != "generic" {
return validateManagedRefs(db, ids, hash, scope)
}
if m.resource == nil {
return errResourcePlan
}
return m.resource.validate(ref, hash)
}
type resourceApplySnapshot struct {
store *resourcePlanStore
source resourceMonitorSource
registry strategyRegistry
matrix qualificationMatrix
ref managedConfigRef
rendered []byte
guard func() error
now func() time.Time
close func()
}
// Historical reconstruction uses the proof's original completion horizon. Only
// new apply checks wall-clock freshness; expiry cannot erase committed identity.
func resourceApplyPlan(source resourceMonitorSource, key []byte, registry strategyRegistry, matrix qualificationMatrix, now time.Time, policies ...string) (resourcePlan, []byte, error) {
policy := ""
if len(policies) != 0 {
policy = policies[0]
}
canonical, e := parseResourceStrategyIDs(policy)
if e != nil || canonical != policy {
return resourcePlan{}, nil, errResourcePlan
}
if policy != "" {
for _, id := range strings.Split(policy, ",") {
found := false
for _, d := range registry.Strategies {
found = found || d.ID == id
}
if !found {
return resourcePlan{}, nil, errResourcePlan
}
}
}
p := deriveResourcePlan(source, key, registry, matrix, now, policy)
if p.Evidence != "authenticated" || len(p.Unassigned) != 0 || len(p.Assignments) == 0 {
return p, nil, errResourcePlan
}
for _, s := range source.config.Services {
for _, r := range s.Resources {
if r.TLS != "both" {
return p, nil, errResourcePlan
}
}
}
profiles := []string{}
seen := map[string]bool{}
for _, a := range p.Assignments {
d, e := canonicalStrategy(strategyDefinition{ID: a.Strategy, Args: a.Args, Selectors: a.Selectors})
if e != nil || d.Selectors.TLS != "both" || len(a.Hosts) == 0 {
return p, nil, errResourcePlan
}
hosts := []string{}
for _, host := range a.Hosts {
if !validDomain(host) || host != strings.ToLower(host) || net.ParseIP(host) != nil || seen[host] {
return p, nil, errResourcePlan
}
seen[host] = true
hosts = append(hosts, "^"+host)
}
profiles = append(profiles, "--filter-tcp=443 --filter-l7=tls --hostlist-domains="+strings.Join(hosts, ",")+" "+d.Args)
}
return p, []byte("# Autotuner-owned generic config; runtime acceptance pending.\nNFQWS2_ENABLE=1\n" + managedFirewallConfig + "NFQWS2_OPT='" + strings.Join(profiles, " --new ") + "'\n"), nil
}
func (s *resourceApplySnapshot) fresh() error {
p, b, e := resourceApplyPlan(s.source, s.store.files.key, s.registry, s.matrix, s.now(), s.ref.Policy)
if e != nil || planDigest(p.Assignments) != s.ref.Plan || managedHash(b) != s.ref.Config {
return errResourcePlan
}
return s.guard()
}
// Each journal side reconstructs independently inside the same immutable boundary.
func (s *resourceApplySnapshot) validate(ref *managedConfigRef, hash string) error {
if ref == nil || ref.Kind != "generic" || ref.Registry != s.ref.Registry || ref.Matrix != s.ref.Matrix || ref.Source != s.ref.Source || hash != ref.Config {
return errResourcePlan
}
p, b, e := s.historicalPlan(ref.Policy)
if e != nil || planDigest(p.Assignments) != ref.Plan || managedHash(b) != ref.Config {
return errResourcePlan
}
return s.guard()
}
func (s *resourceApplySnapshot) historicalPlan(policy string) (resourcePlan, []byte, error) {
var horizon int64
for _, run := range s.matrix.Runs {
if run.Completed > horizon {
horizon = run.Completed
}
}
return resourceApplyPlan(s.source, s.store.files.key, s.registry, s.matrix, time.UnixMilli(horizon), policy)
}
func (s *resourceApplySnapshot) selectPolicy(policy string) error {
p, b, e := s.historicalPlan(policy)
if e != nil {
return e
}
s.ref.Policy, s.ref.Plan, s.ref.Config = policy, planDigest(p.Assignments), managedHash(b)
s.rendered = b
return nil
}
// No imported proposal or version probe. Keep private proof/source/native files
// held across staging, health and commit and bind every selected candidate.
func openResourceApplySnapshot(config string, d managedDeps) (*resourceApplySnapshot, error) {
return openResourceApplySnapshotAt(config, d, "/opt/zapret2")
}
func openResourceApplySnapshotAt(config string, d managedDeps, root string) (*resourceApplySnapshot, error) {
store, e := openResourcePlanStore(d.directory, false)
if e != nil {
return nil, errResourcePlan
}
s := &resourceApplySnapshot{store: store, now: time.Now}
var engine *qualificationBinary
var sourceFile *os.File
lua := map[string]*os.File{}
s.close = func() {
for _, f := range lua {
f.Close()
}
if sourceFile != nil {
sourceFile.Close()
}
if engine != nil {
engine.file.Close()
}
store.close()
}
fail := func() (*resourceApplySnapshot, error) { s.close(); return nil, errResourcePlan }
s.registry, e = store.registry()
if e != nil {
return fail()
}
s.matrix, e = store.matrix(s.registry)
if e != nil {
return fail()
}
marker, markerBytes, e := store.files.openEntry(managedMarkerName)
if e != nil {
return fail()
}
store.held[managedMarkerName], store.snapshots[managedMarkerName] = marker, markerBytes
fd, e := syscall.Open(config, syscall.O_RDONLY|syscall.O_NOFOLLOW|syscall.O_NONBLOCK|syscall.O_CLOEXEC, 0)
if e != nil {
return fail()
}
sourceFile = os.NewFile(uintptr(fd), "resource source")
if resourcePrivateEntry(sourceFile) != nil {
return fail()
}
sourceBytes, e := io.ReadAll(io.LimitReader(sourceFile, 65537))
if e != nil || len(sourceBytes) > 65536 {
return fail()
}
s.source, e = readResourceMonitorSource(config, d.lookup, store.files.key)
if e != nil {
return fail()
}
engine, e = openQualificationEngine(root)
if e != nil {
return fail()
}
native, e := engine.filesHash()
if e != nil {
return fail()
}
// Hold Lua inodes as well as the qualification hash; equal-byte replacement
// during a transaction is still a trust change.
for _, name := range []string{"lua/zapret-lib.lua", "lua/zapret-antidpi.lua"} {
path := engine.root + "/" + name
f, e := os.Open(path)
if e != nil {
return fail()
}
lua[path] = f
}
p, rendered, e := s.historicalPlan("")
if e != nil {
return fail()
}
for _, run := range s.matrix.Runs {
if run.Attestation.LuaHash != native {
return fail()
}
}
s.rendered = rendered
s.ref = managedConfigRef{Kind: "generic", Registry: managedHash(store.snapshots[planRegistryName]), Matrix: managedHash(store.snapshots[planMatrixName]), Source: s.source.fingerprint, Plan: planDigest(p.Assignments), Config: managedHash(rendered)}
s.guard = func() error {
if store.guard() != nil || managedCheckIdentity(&managedTransaction{Dir: d.directory, Name: "config"}) != nil {
return errResourcePlan
}
current, e := os.Lstat(config)
held, he := sourceFile.Stat()
b, re := io.ReadAll(io.NewSectionReader(sourceFile, 0, 65537))
fresh, se := readResourceMonitorSource(config, d.lookup, store.files.key)
hash, ne := engine.filesHash()
if e != nil || he != nil || re != nil || se != nil || ne != nil || !os.SameFile(current, held) || resourcePrivateEntry(sourceFile) != nil || !bytes.Equal(b, sourceBytes) || fresh.fingerprint != s.source.fingerprint || hash != native {
return errResourcePlan
}
for path, f := range lua {
current, e := os.Lstat(path)
held, he := f.Stat()
if e != nil || he != nil || !os.SameFile(current, held) {
return errResourcePlan
}
}
return nil
}
if s.guard() != nil {
return fail()
}
return s, nil
}
func resourceApplyHealth(ctx context.Context, s *resourceApplySnapshot, checks resourceCheckDeps) (string, error) {
guard := checks.guard
checks.guard = func() error {
if s.guard() != nil {
return errResourcePlan
}
return guard()
}
passed := true
for _, service := range s.source.config.Services {
for _, spec := range service.Resources {
for _, tls := range []string{"tls12", "tls13"} {
result, e := runResourceProtocol(ctx, service.ID, spec, tls, s.source.server, s.source.proxy, checks)
if e != nil || ctx.Err() != nil {
return "DEGRADED", errResourcePlan
}
passed = passed && result.Outcome == "pass"
}
}
}
if s.guard() != nil {
return "BLOCKED", errResourcePlan
}
if !passed {
return "DEGRADED", errResourcePlan
}
return "OK", nil
}
func resourceApplyMode(config string, d managedDeps) error {
if os.Geteuid() != 0 || os.Getppid() != 1 || config == "" {
return errResourcePlan
}
if _, e := managedRuntimeGuard(d); e != nil {
return errResourcePlan
}
cmd, e := os.ReadFile("/proc/1/cmdline")
if e != nil || string(cmd) != "/bin/sh\x00/opt/managed-target-entrypoint.sh\x00" {
return errResourcePlan
}
env, e := os.ReadFile("/proc/1/environ")
if e != nil {
return errResourcePlan
}
values := map[string]string{}
for _, item := range strings.Split(string(env), "\x00") {
k, v, _ := strings.Cut(item, "=")
values[k] = v
}
if values["ZAPRET_AUTOTUNE_RESOURCE_APPLY"] != "1" || values["ZAPRET_AUTOTUNE_RESOURCE_CONFIG"] != config || values["ZAPRET_AUTOTUNE_QUALIFY_CONFIG"] != "" || values["ZAPRET_AUTOTUNE_STRATEGY_CONFIG"] != "" {
return errResourcePlan
}
requested, _ := d.lookup(resourceStrategyIDsEnv)
if _, e := parseResourceStrategyIDs(requested); e != nil || requested != values[resourceStrategyIDsEnv] {
return errResourcePlan
}
for _, ns := range []string{"pid", "net"} {
a, e := os.Readlink("/proc/1/ns/" + ns)
b, f := os.Readlink("/proc/self/ns/" + ns)
if e != nil || f != nil || a != b {
return errResourcePlan
}
}
if !resourceProcessOwned("/proc", os.Getpid(), "/usr/local/bin/zapret-autotune", []string{"/usr/local/bin/zapret-autotune", "managed", "resource-apply", "--resource-config", config}) {
return errResourcePlan
}
dir, e := os.Open("/run/zapret-autotune-ss")
if e != nil {
return errResourcePlan
}
defer dir.Close()
f, e := resourcePrivateFile(dir, "monitor-ready")
if e != nil {
return errResourcePlan
}
defer f.Close()
b, e := io.ReadAll(io.LimitReader(f, 32))
if e != nil || string(b) != strconv.Itoa(os.Getpid())+"\n" {
return errResourcePlan
}
return nil
}
func resourceApplyCLI(config string, stdout, stderr io.Writer, d managedDeps) int {
fmt.Fprintln(stderr, "controller-startup: resource_apply=entry")
blocked := func() int {
fmt.Fprintln(stderr, "BLOCKED: generic apply trust/health incomplete; deployment preserved or prepared for recovery")
return 78
}
if resourceApplyMode(config, d) != nil {
return blocked()
}
lock, e := acquireManagedMonitorSingleton(d)
if e != nil {
return blocked()
}
defer lock.close()
if qualificationNoWorker() != nil {
return blocked()
}
s, e := openResourceApplySnapshot(config, d)
if e != nil {
return blocked()
}
defer s.close()
signals, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
ctx, cancel := context.WithTimeout(signals, 15*time.Minute)
defer cancel()
checks := productionResourceCheckDeps()
m := &managedTransaction{Dir: d.directory, Name: "config", resource: s, Reload: func() error { return d.run(d.script, "restart") }}
m.Health = func() (string, error) { return resourceApplyHealth(ctx, s, checks) }
m.RollbackHealth = func() (string, error) {
cleanup, cancel := context.WithTimeout(context.WithoutCancel(ctx), 30*time.Second)
defer cancel()
return resourceApplyHealth(cleanup, s, checks)
}
if resourceSingletonGuard(lock, d.directory) != nil || m.Reconcile() != nil {
return blocked()
}
requested, _ := d.lookup(resourceStrategyIDsEnv)
policy, e := parseResourceStrategyIDs(requested)
if e != nil || s.selectPolicy(policy) != nil {
return blocked()
}
originalGuard := s.guard
s.guard = func() error {
if resourceApplyMode(config, d) != nil || resourceSingletonGuard(lock, d.directory) != nil {
return errResourcePlan
}
return originalGuard()
}
j, e := m.load()
if e != nil {
return blocked()
}
if (j.Version != 4 || j.Ref.Kind == "baseline") && qualificationNoWorker() != nil {
return blocked()
}
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e != nil {
return blocked()
}
fmt.Fprintln(stdout, "generic resource apply: committed; configured resources only, not production READY")
return 0
}
func resourceDeploymentIdentity(config string, d managedDeps) error {
m := &managedTransaction{Dir: d.directory, Name: "config"}
j, e := m.load()
if e != nil {
return errResourcePlan
}
if j.Version != 4 {
return nil
}
if j.State != "committed" {
return errResourcePlan
}
s, e := openResourceApplySnapshot(config, d)
if e != nil {
return errResourcePlan
}
defer s.close()
m.resource = s
return m.validateConfigRef(nil, j.Ref, j.Candidates, j.Hash, j.Scope)
}
+499
View File
@@ -0,0 +1,499 @@
package main
import (
"bytes"
"context"
"encoding/json"
"errors"
"io"
"net"
"os"
"path/filepath"
"strconv"
"strings"
"testing"
"time"
)
func completeApplyFixture(t *testing.T) (resourceMonitorSource, []byte, strategyRegistry, qualificationMatrix, time.Time) {
s, key, r, m, now := planFixture(t)
// e was intentionally unknown in the proposal fixture; this request covers
// only a/b/c/d, including a's cross-service duplicate resource.
s.config.Services = append(s.config.Services[:4:4], s.config.Services[5])
return s, key, r, m, now
}
func TestResourceApplyRender(t *testing.T) {
s, key, r, m, now := completeApplyFixture(t)
p, b, e := resourceApplyPlan(s, key, r, m, now)
if e != nil || !p.ProposalOnly || len(p.Assignments) != 3 {
t.Fatal(p, e)
}
if !strings.Contains(string(b), "--hostlist-domains=^a.example,^c.example") || strings.Count(string(b), " --new ") != 2 {
t.Fatal(string(b))
}
for i := 0; i < 30; i++ {
_, next, e := resourceApplyPlan(s, key, r, m, now)
if e != nil || !bytes.Equal(next, b) {
t.Fatal("nondeterministic render")
}
}
for _, kind := range []string{"negative", "unknown", "incomplete", "stale", "future", "config", "tls", "unsafe"} {
t.Run(kind, func(t *testing.T) {
s, key, r, m, now := completeApplyFixture(t)
switch kind {
case "negative":
m.Runs[0].Checks[len(m.Runs[0].Checks)-1].Outcome = "expectation_failed"
case "unknown":
m.Runs[0].Attestation.Joined = false
case "incomplete":
m.Runs[0].Checks = m.Runs[0].Checks[:len(m.Runs[0].Checks)-1]
case "stale":
now = now.Add(25 * time.Hour)
case "future":
now = now.Add(-time.Hour)
case "config":
s.fingerprint = strings.Repeat("f", 64)
case "tls":
s.config.Services[0].Resources[0].TLS = "tls12"
case "unsafe":
r.Strategies[0].Args = "--lua-desync=execute:cmd=id"
}
if _, _, e := resourceApplyPlan(s, key, r, m, now); e == nil {
t.Fatal("authorized invalid proof")
}
})
}
}
// Uses the production snapshot factory, private HMAC envelopes and shared
// transaction engine; native/HTTP fixtures perform no network operations.
func nativeApplyFixture(t *testing.T) (*managedTransaction, *resourceApplySnapshot, string, string) {
return nativeApplyAlternativesFixture(t, false)
}
func nativeApplyAlternativesFixture(t *testing.T, alternatives bool) (*managedTransaction, *resourceApplySnapshot, string, string) {
return nativeApplyProfilesFixture(t, alternatives, false)
}
func nativeApplyProfilesFixture(t *testing.T, alternatives, sameArgs bool) (*managedTransaction, *resourceApplySnapshot, string, string) {
t.Helper()
root, _, _ := qualificationBinaryFixture(t)
dir := t.TempDir()
if e := os.Chmod(dir, 0700); e != nil {
t.Fatal(e)
}
m := &managedTransaction{Dir: dir, Name: "config", Reload: func() error { return nil }, Health: func() (string, error) { return "OK", nil }}
if e := managedInitialize(m); e != nil {
t.Fatal(e)
}
store, e := openResourcePlanStore(dir, true)
if e != nil {
t.Fatal(e)
}
config := resourceCheckFile(t, strings.Replace(resourceCheckExample, `"tls":"tls12"`, `"tls":"both"`, 1))
d := managedDeps{directory: dir, lookup: resourceCheckLookup}
source, e := readResourceMonitorSource(config, d.lookup, store.files.key)
if e != nil {
t.Fatal(e)
}
_, _, registry, _, _ := planFixture(t)
registry.Strategies = registry.Strategies[:1]
if alternatives {
other := registry.Strategies[0]
other.ID = "zz-alternative"
other.Args = "--lua-desync=split:pos=2"
if sameArgs {
other.Args = registry.Strategies[0].Args
}
registry.Strategies = append(registry.Strategies, other)
}
b, _ := json.Marshal(registry)
if e := store.importDefinitions(b); e != nil {
t.Fatal(e)
}
engine, e := openQualificationEngine(root)
if e != nil {
t.Fatal(e)
}
hash, e := engine.filesHash()
engine.file.Close()
if e != nil {
t.Fatal(e)
}
now := time.Now()
def := registry.Strategies[0]
run := qualificationRun{ID: "run-test", Strategy: def.ID, Definition: planDigest(def), Config: source.fingerprint, Started: now.Add(-time.Minute).UnixMilli(), Completed: now.UnixMilli(), Attestation: qualificationAttestation{Collector: "owned-resource-nfq-v1", Runtime: "nfqws2-1.0.2", Definition: planDigest(def), LuaHash: hash, RulesHash: strings.Repeat("b", 64), WorkerArgsHash: strings.Repeat("c", 64), Queue: managedAssetQueue, PID: 123, StartTime: 456, Restored: true, Joined: true}}
for _, service := range source.config.Services {
for _, spec := range service.Resources {
for _, tls := range []string{"tls12", "tls13"} {
run.Checks = append(run.Checks, qualificationCheck{Service: service.ID, Resource: spec.ID, TLS: tls, Fingerprint: resourceRowFingerprint(store.files.key, source, spec), Outcome: "pass", Attempts: 3})
}
}
}
matrix := qualificationMatrix{Version: 1, Registry: planDigest(registry), Runs: []qualificationRun{run}}
if alternatives {
other := run
other.ID, other.Strategy = "run-alternative", registry.Strategies[1].ID
other.Definition = planDigest(registry.Strategies[1])
other.Attestation.Definition = other.Definition
matrix.Runs = append(matrix.Runs, other)
}
if e := store.saveCompleted(completedQualification{matrix: matrix, seal: planDigest(matrix)}); e != nil {
t.Fatal(e)
}
store.close()
s, e := openResourceApplySnapshotAt(config, d, root)
if e != nil {
t.Fatal(e)
}
t.Cleanup(s.close)
m.resource = s
return m, s, config, root
}
func TestResourceApplyLifecycle(t *testing.T) {
m, s, config, root := nativeApplyFixture(t)
reloads := 0
m.Reload = func() error { reloads++; return nil }
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e != nil {
t.Fatal(e)
}
j, e := m.load()
if e != nil || j.Version != 4 || j.State != "committed" || j.Ref.Kind != "generic" || j.PendingRef != nil || reloads != 1 {
t.Fatal(j, e, reloads)
}
if _, e := m.ActiveCandidates(nil); e == nil {
t.Fatal("generic treated as empty baseline")
}
s.now = func() time.Time { return time.Now().Add(25 * time.Hour) }
if s.fresh() == nil {
t.Fatal("new expired apply authorized")
}
if e := m.Reconcile(); e != nil {
t.Fatal("aged committed proof invalidated", e)
}
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e != nil || reloads != 1 {
t.Fatal("replay changed runtime", e, reloads)
}
before, _ := os.ReadFile(filepath.Join(m.Dir, "config"))
raw, _ := os.ReadFile(config)
if e := os.WriteFile(config, bytes.Replace(raw, []byte("example.com"), []byte("changed.com"), 1), 0600); e != nil {
t.Fatal(e)
}
if e := m.Reconcile(); e == nil {
t.Fatal("source drift reconciled")
}
after, _ := os.ReadFile(filepath.Join(m.Dir, "config"))
if !bytes.Equal(before, after) {
t.Fatal("drift erased deployment")
}
if _, e := openResourceApplySnapshotAt(config, managedDeps{directory: m.Dir, lookup: resourceCheckLookup}, root); e == nil {
t.Fatal("changed source accepted after reopen")
}
}
func TestResourceApplyCrashRecovery(t *testing.T) {
for _, point := range []string{"staged", "backup", "prepared", "replaced", "reloaded", "healthy"} {
t.Run(point, func(t *testing.T) {
m, s, _, _ := nativeApplyFixture(t)
m.fault = func(p string) error {
if p == point {
return errors.New("process death")
}
return nil
}
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e == nil {
t.Fatal("fault ignored")
}
m.fault = nil
if e := m.Reconcile(); e != nil {
t.Fatal(e)
}
b, _ := m.read("config")
j, _ := m.load()
if string(b) != managedBaseline || j.State != "committed" || j.Ref != nil && j.Ref.Kind != "baseline" {
t.Fatal("pending promoted", j, string(b))
}
})
}
}
func TestResourceApplyRollback(t *testing.T) {
for _, failure := range []string{"reload", "health", "rollback-reload", "rollback-health"} {
t.Run(failure, func(t *testing.T) {
m, s, _, _ := nativeApplyFixture(t)
calls := 0
m.Reload = func() error {
calls++
if failure == "reload" && calls == 1 || failure == "rollback-reload" && calls == 2 {
return errors.New("reload")
}
return nil
}
m.Health = func() (string, error) { return "DEGRADED", nil }
m.RollbackHealth = func() (string, error) {
if failure == "rollback-health" {
return "DEGRADED", nil
}
return "OK", nil
}
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e == nil {
t.Fatal("failure committed")
}
j, _ := m.load()
b, _ := m.read("config")
want := "committed"
if strings.HasPrefix(failure, "rollback-") {
want = "prepared"
}
if j.State != want || string(b) != managedBaseline {
t.Fatal(j, string(b))
}
})
}
}
func TestResourceApplyHeldMutation(t *testing.T) {
for _, kind := range []string{"matrix", "source", "lua", "directory"} {
t.Run(kind, func(t *testing.T) {
m, s, config, root := nativeApplyFixture(t)
m.fault = func(point string) error {
if point != "staged" {
return nil
}
path := config
switch kind {
case "matrix":
path = filepath.Join(m.Dir, planMatrixName)
case "lua":
path = filepath.Join(root, "lua/zapret-lib.lua")
case "directory":
return os.Rename(m.Dir, m.Dir+"-moved")
}
b, e := os.ReadFile(path)
if e != nil {
return e
}
if e := os.Remove(path); e != nil {
return e
}
return os.WriteFile(path, b, 0600)
}
reloads := 0
m.Reload = func() error { reloads++; return nil }
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e == nil || reloads != 0 {
t.Fatal("mutation authorized", e, reloads)
}
})
}
}
func TestResourceApplyJournalTags(t *testing.T) {
m, s, _, _ := nativeApplyFixture(t)
for _, kind := range []string{"unknown-version", "mixed", "missing", "unknown-tag", "old-tag"} {
t.Run(kind, func(t *testing.T) {
j := managedJournal{Version: 4, ID: "fixture", State: "committed", Hash: s.ref.Config, Ref: &s.ref}
switch kind {
case "unknown-version":
j.Version = 5
case "mixed":
j.Candidates = []int64{1}
case "missing":
j.Ref = nil
case "unknown-tag":
r := s.ref
r.Kind = "other"
j.Ref = &r
case "old-tag":
j.Version = 3
j.Scope = GooglevideoScopeExact
}
if e := m.journal(j); e != nil {
t.Fatal(e)
}
if _, e := m.load(); e == nil {
t.Fatal("contradictory journal accepted")
}
})
}
}
func TestResourceApplyHealthCollector(t *testing.T) {
m, s, _, _ := nativeApplyFixture(t)
_ = m
for _, mode := range []string{"", "verify", "status500"} {
calls := 0
checks := resourceCheckDeps{guard: func() error { return nil }, resolve: func(context.Context, string, string) ([]net.IP, error) {
return []net.IP{net.ParseIP("93.184.216.34")}, nil
}, runner: func(_ context.Context, args []string, stdin io.Reader, out, diag io.Writer) error {
calls++
config, _ := io.ReadAll(stdin)
if !bytes.Contains(config, []byte("connect-to")) {
t.Fatal("missing pin")
}
raw, err := strconv.Unquote(strings.TrimPrefix(strings.Split(string(config), "\n")[0], "url = "))
if err != nil {
t.Fatal(err)
}
method, tls := "GET", "tls12"
for _, arg := range args {
if arg == "--head" {
method = "HEAD"
}
if arg == "--tlsv1.3" {
tls = "tls13"
}
if arg == "--insecure" {
t.Fatal("TLS bypass")
}
}
configuredWire(raw, method, tls, mode, out, diag)
return nil
}}
status, e := resourceApplyHealth(context.Background(), s, checks)
if calls != 4 || (mode == "" && (e != nil || status != "OK")) || (mode != "" && e == nil) {
t.Fatal(mode, calls, status, e)
}
}
}
func TestResourceApplyUnauthorizedCLI(t *testing.T) {
var out bytes.Buffer
d := managedDeps{}
if code := resourceApplyCLI("/private/SECRET_SENTINEL", &out, &out, d); code != 78 {
t.Fatal(code)
}
if got, want := out.String(), "controller-startup: resource_apply=entry\nBLOCKED: generic apply trust/health incomplete; deployment preserved or prepared for recovery\n"; got != want {
t.Fatalf("unexpected diagnostic: %q", got)
}
}
func TestResourceApplyFreshBeforeStaging(t *testing.T) {
for _, kind := range []string{"expired", "forged", "native", "proof", "unsupported"} {
t.Run(kind, func(t *testing.T) {
m, s, _, root := nativeApplyFixture(t)
ref := s.ref
switch kind {
case "expired":
s.now = func() time.Time { return time.Now().Add(25 * time.Hour) }
case "forged":
ref.Plan = strings.Repeat("a", 64)
case "native":
if e := os.WriteFile(filepath.Join(root, "lua/zapret-antidpi.lua"), []byte("drift"), 0755); e != nil {
t.Fatal(e)
}
case "proof":
if e := os.WriteFile(filepath.Join(m.Dir, planMatrixName), []byte(`{"payload":{},"mac":"forged"}`), 0600); e != nil {
t.Fatal(e)
}
case "unsupported":
s.source.config.Services[0].Resources[0].TLS = "tls13"
}
reloads, stages := 0, 0
m.Reload = func() error { reloads++; return nil }
m.fault = func(point string) error {
if point == "staged" {
stages++
}
return nil
}
if e := m.applyPolicy(s.rendered, nil, nil, &ref); e == nil || stages != 0 || reloads != 0 {
t.Fatal("unqualified staging", e, stages, reloads)
}
b, _ := m.read("config")
j, _ := m.load()
if string(b) != managedBaseline || j.Version != 2 || j.Generation != 0 {
t.Fatal("baseline changed", j)
}
})
}
}
func TestResourceApplyPreparedExpiredRecoveryHealth(t *testing.T) {
m, s, _, _ := nativeApplyFixture(t)
m.fault = func(point string) error {
if point == "healthy" {
return errors.New("crash")
}
return nil
}
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e == nil {
t.Fatal("crash missed")
}
m.fault = nil
s.now = func() time.Time { return time.Now().Add(25 * time.Hour) }
m.RollbackHealth = func() (string, error) { return "DEGRADED", nil }
if e := m.Reconcile(); e == nil {
t.Fatal("unhealthy rollback finalized")
}
j, _ := m.load()
b, _ := m.read("config")
if j.State != "prepared" || string(b) != managedBaseline {
t.Fatal("baseline was not physically restored", j)
}
m.RollbackHealth = func() (string, error) { return "OK", nil }
if e := m.Reconcile(); e != nil {
t.Fatal("expiry blocked recovery", e)
}
}
func TestResourceApplyCancellationJoinsBeforeRollback(t *testing.T) {
m, s, _, _ := nativeApplyFixture(t)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
joined := false
checks := resourceCheckDeps{guard: func() error { return nil }, resolve: func(context.Context, string, string) ([]net.IP, error) {
return []net.IP{net.ParseIP("93.184.216.34")}, nil
}, runner: func(ctx context.Context, _ []string, _ io.Reader, _, _ io.Writer) error {
cancel()
<-ctx.Done()
joined = true
return ctx.Err()
}}
m.Health = func() (string, error) { return resourceApplyHealth(ctx, s, checks) }
reloads := 0
m.Reload = func() error {
reloads++
if reloads == 2 && !joined {
t.Fatal("rollback raced health child")
}
return nil
}
m.RollbackHealth = func() (string, error) {
if !joined {
t.Fatal("rollback health raced child")
}
return "OK", nil
}
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e == nil || reloads != 2 || !joined {
t.Fatal(e, reloads, joined)
}
j, _ := m.load()
if j.State != "committed" || j.Ref.Kind != "baseline" {
t.Fatal(j)
}
}
func TestResourceApplyRefusesActiveLegacy(t *testing.T) {
m, s, _, _ := nativeApplyFixture(t)
db := testStore(t)
id := managedCandidateFixture(t, db, "frontend", "www.youtube.com", "--lua-desync=multisplit", true)
m.resource = nil
if e := m.Apply(db, []int64{id}, GooglevideoScopeExact); e != nil {
t.Fatal(e)
}
before, _ := m.read("config")
journal, _ := m.read(managedJournalName)
m.resource = s
reloads := 0
m.Reload = func() error { reloads++; return nil }
if e := m.applyPolicy(s.rendered, db, nil, &s.ref); e == nil || reloads != 0 {
t.Fatal("active legacy maintenance authorized", e)
}
after, _ := m.read("config")
nextJournal, _ := m.read(managedJournalName)
if !bytes.Equal(before, after) || !bytes.Equal(journal, nextJournal) {
t.Fatal("legacy state changed")
}
}
+80 -10
View File
@@ -124,9 +124,35 @@ func parseResourceChecks(data []byte, lookup func(string) (string, bool)) (resou
}
d = json.NewDecoder(bytes.NewReader(data))
d.DisallowUnknownFields()
if d.Decode(&cfg) != nil || cfg.Version != 1 || len(cfg.Services) == 0 || len(cfg.Services) > 16 {
if d.Decode(&cfg) != nil || (cfg.Version != 1 && cfg.Version != 2) || len(cfg.Services) == 0 || len(cfg.Services) > 16 {
return fail()
}
if cfg.Version == 2 {
// Presence matters: an omitted minimum must not silently become zero.
var fields struct {
Services []struct {
Resources []struct {
Kind *string `json:"kind"`
Expect struct {
Statuses *[]int `json:"statuses"`
ContentType *string `json:"content_type"`
MinBytes *int `json:"min_bytes"`
MaxBytes *int `json:"max_bytes"`
} `json:"expect"`
} `json:"resources"`
} `json:"services"`
}
if json.Unmarshal(data, &fields) != nil {
return fail()
}
for _, s := range fields.Services {
for _, r := range s.Resources {
if r.Kind != nil && *r.Kind == "" || r.Expect.Statuses == nil || r.Expect.ContentType == nil || *r.Expect.ContentType == "" || r.Expect.MinBytes == nil || r.Expect.MaxBytes == nil {
return fail()
}
}
}
}
services, total := map[string]bool{}, 0
for si := range cfg.Services {
s := &cfg.Services[si]
@@ -137,8 +163,11 @@ func parseResourceChecks(data []byte, lookup func(string) (string, bool)) (resou
ids := map[string]bool{}
for ri := range s.Resources {
r := &s.Resources[ri]
if cfg.Version == 2 && r.Kind == "" {
r.Kind = "http"
}
total++
if total > 64 || !resourceID.MatchString(r.ID) || ids[r.ID] || !resourceID.MatchString(r.Group) || r.Kind != "http" {
if total > 64 || !resourceID.MatchString(r.ID) || ids[r.ID] || !resourceID.MatchString(r.Group) || (r.Kind != "http" && (cfg.Version != 2 || r.Kind != "image" && r.Kind != "mp4_range")) {
return fail()
}
ids[r.ID] = true
@@ -213,6 +242,24 @@ func parseResourceChecks(data []byte, lookup func(string) (string, bool)) (resou
return fail()
}
}
if r.Kind != "http" {
status := 200
if r.Kind == "mp4_range" {
status = 206
}
if r.Method != "GET" || len(r.Expect.Statuses) != 1 || r.Expect.Statuses[0] != status {
return fail()
}
if r.Kind == "image" {
// ponytail: public GIF awaits an aggregate frame/pixel-bounded decoder;
// legacy image adapters retain their existing GIF contract.
if r.Expect.ContentType != "image/png" && r.Expect.ContentType != "image/jpeg" {
return fail()
}
} else if (r.Expect.ContentType != "video/mp4" && r.Expect.ContentType != "audio/mp4") || r.Expect.MinBytes < 16 || r.Expect.MaxBytes > 65536 {
return fail()
}
}
if len(r.HeadersEnv) > 16 {
return fail()
}
@@ -326,6 +373,9 @@ func approvedResourceIP(r resourceCheckSpec, ip net.IP) bool {
}
type resourceCheckDeps struct {
// Qualification must distinguish infrastructure from verified peer negatives.
// Monitoring retains its existing coarse outcome behavior.
qualification bool
lookup func(string) (string, bool)
guard func() error
resolve func(context.Context, string, string) ([]net.IP, error)
@@ -371,11 +421,21 @@ type resourceCheckResult struct {
func runResourceProtocol(ctx context.Context, service string, r resourceCheckSpec, protocol, server, proxy string, d resourceCheckDeps) (resourceCheckResult, error) {
result := resourceCheckResult{Service: service, Resource: r.ID, Group: r.Group, TLS: protocol, Outcome: "dns_failed"}
if err := d.guard(); err != nil {
return result, err
return result, qualificationOperation(err, stagePreDNSGuard)
}
probeCtx, cancel := context.WithTimeout(ctx, time.Duration(r.TimeoutMS)*time.Millisecond)
defer cancel()
ips, err := d.resolve(probeCtx, server, r.host)
if d.qualification && err != nil {
var dns *net.DNSError
knownTimeout := probeCtx.Err() != nil && managedRecoveryTimeoutSafe(err)
if !knownTimeout && (!errors.As(err, &dns) || !dns.IsNotFound) {
return result, qualificationOperation(err, stageDNS)
}
}
if d.qualification && err == nil && (len(ips) == 0 || len(ips) > 64) {
return result, qualificationOperation(errResourceConfig, stageDNS)
}
if err == nil && len(ips) > 0 && len(ips) <= 64 {
approved := true
for _, ip := range ips {
@@ -386,23 +446,33 @@ func runResourceProtocol(ctx context.Context, service string, r resourceCheckSpe
result.Outcome = "destination_denied"
if approved {
if err := d.guard(); err != nil {
return result, err
return result, qualificationOperation(err, stagePostDNSGuard)
}
pin := r.host + ":443:" + net.JoinHostPort(ips[0].String(), "443")
options := resourceTransferOptions{generic: true, method: r.Method, limit: r.Expect.MaxBytes, timeout: time.Duration(r.TimeoutMS) * time.Millisecond, config: r.privateConfig + "connect-to = " + privateCurlValue(pin) + "\n"}
transfer, _, err := collectResourceTransferOptions(probeCtx, proxy, r.URL, r.host, protocol, false, d.runner, options)
options := resourceTransferOptions{method: r.Method, limit: r.Expect.MaxBytes, timeout: time.Duration(r.TimeoutMS) * time.Millisecond, config: r.privateConfig + "connect-to = " + privateCurlValue(pin) + "\n", expect: &r.Expect}
switch r.Kind {
case "image":
options.response = resourceResponseImage
case "mp4_range":
options.response, options.mediaType = resourceResponseMP4, r.Expect.ContentType
}
transfer, _, err := collectResourceTransferOptions(probeCtx, proxy, r.URL, r.host, protocol, d.runner, options)
if d.qualification && err != nil && err != errResourceExpectation {
knownTimeout := probeCtx.Err() != nil && managedRecoveryTimeoutSafe(err)
if !knownTimeout && !resourceRejectionOnly(err) && !managedAssetTransferNonpass(err) {
return result, qualificationOperation(err, stageTransfer)
}
}
result.Outcome = "transfer_failed"
if err == nil {
if err == nil || err == errResourceExpectation {
result.Bytes, result.TimeMS, result.Status = len(transfer.Body), transfer.LatencyMS, transfer.Status
result.Outcome = "expectation_failed"
for _, status := range r.Expect.Statuses {
if status == transfer.Status && len(transfer.Body) >= r.Expect.MinBytes && (r.Expect.ContentType == "" || transfer.ContentType == r.Expect.ContentType) {
if err == nil {
result.Outcome = "pass"
}
}
}
}
}
if probeCtx.Err() != nil {
result.Outcome = "timeout"
if ctx.Err() != nil {
+6 -3
View File
@@ -49,6 +49,9 @@ func configuredWire(raw, method, protocol, mode string, out, diag io.Writer) {
if mode == "redirect" {
status = 302
}
if mode == "status500" {
status = 500
}
if mode == "redirect-count" {
redirects = 1
}
@@ -91,7 +94,7 @@ func TestResourceCheckSchema(t *testing.T) {
t.Fatal(err)
}
for _, data := range []string{
resourceCheckExample + ` {}`, strings.Replace(resourceCheckExample, `"version":1`, `"version":2`, 1),
resourceCheckExample + ` {}`, strings.Replace(resourceCheckExample, `"version":1`, `"version":3`, 1),
strings.Replace(resourceCheckExample, `"version":1`, `"version":1,"version":1`, 1),
strings.Replace(resourceCheckExample, `"version":1`, `"unknown":1,"version":1`, 1),
strings.Replace(resourceCheckExample, `"id":"first"`, `"id":"First /"`, 1),
@@ -362,7 +365,7 @@ func TestResourceCheckTransferContract(t *testing.T) {
configuredWire("https://example.com/", method, "tls12", mode, out, diag)
return nil
}
transfer, _, err := collectResourceTransferOptions(context.Background(), "socks5h://127.0.0.1:1080", "https://example.com/", "example.com", "tls12", false, runner, resourceTransferOptions{generic: true, method: method, limit: 100, timeout: time.Second})
transfer, _, err := collectResourceTransferOptions(context.Background(), "socks5h://127.0.0.1:1080", "https://example.com/", "example.com", "tls12", runner, resourceTransferOptions{method: method, limit: 100, timeout: time.Second})
valid := mode == "pass" || mode == "chunked" || method == "HEAD" && mode == "truncated"
if (err == nil) != valid {
t.Fatalf("err=%v", err)
@@ -481,7 +484,7 @@ func TestResourceCheckCurlSOCKSPinnedDestination(t *testing.T) {
}()
ctx, cancel := context.WithTimeout(context.Background(), 4*time.Second)
defer cancel()
_, _, err = collectResourceTransferOptions(ctx, "socks5h://"+listener.Addr().String(), "https://example.com/private?secret=value", "example.com", "tls12", false, runResourceCurl, resourceTransferOptions{generic: true, method: "GET", limit: 100, timeout: 3 * time.Second, config: "connect-to = \"example.com:443:93.184.216.34:443\"\n"})
_, _, err = collectResourceTransferOptions(ctx, "socks5h://"+listener.Addr().String(), "https://example.com/private?secret=value", "example.com", "tls12", runResourceCurl, resourceTransferOptions{method: "GET", limit: 100, timeout: 3 * time.Second, config: "connect-to = \"example.com:443:93.184.216.34:443\"\n"})
if err == nil {
t.Fatal("fixture unexpectedly transferred")
}
+148 -73
View File
@@ -9,6 +9,7 @@ import (
"encoding/json"
"errors"
"fmt"
"image"
"io"
"math"
"mime"
@@ -31,18 +32,28 @@ func runResourceCurl(ctx context.Context, args []string, stdin io.Reader, out, d
return cmd.Run()
}
type resourceError struct{ cause error }
type resourceError struct {
cause error
reason qualificationReason
}
// Created only after proxy, URL and TLS inputs have passed collector validation.
// Rejection is never qualification, nor evidence that local ownership failed.
type resourcePeerTLSRejection struct{ error }
// Created only after origin, verified TLS and HTTP framing have been decoded.
// A configured HTTP peer response rejection is not local ownership failure.
type resourcePeerResponseRejection struct{ error }
func (e *resourcePeerResponseRejection) Error() string { return "resource peer response rejected" }
func (e *resourcePeerResponseRejection) Unwrap() error { return e.error }
func (e *resourcePeerTLSRejection) Error() string { return "resource peer TLS rejected" }
func (e *resourcePeerTLSRejection) Unwrap() error { return e.error }
func resourceRejectionOnly(err error) bool {
switch err.(type) {
case *resourcePeerTLSRejection, imageBodyRejection:
case *resourcePeerTLSRejection, *resourcePeerResponseRejection, imageBodyRejection:
return true
}
if e, ok := err.(*resourceError); ok {
@@ -179,39 +190,17 @@ type mediaCapability struct {
// SABR is deliberately unknown/not supported, never fabricated as readiness.
func playerMediaResource(body []byte) (mediaResource, mediaCapability, error) {
var p struct {
PlayabilityStatus struct {
Status string `json:"status"`
} `json:"playabilityStatus"`
StreamingData struct {
Formats []struct {
URL string `json:"url"`
MimeType string `json:"mimeType"`
} `json:"formats"`
AdaptiveFormats []struct {
URL string `json:"url"`
MimeType string `json:"mimeType"`
} `json:"adaptiveFormats"`
} `json:"streamingData"`
}
if len(body) > playerResponseLimit || json.Unmarshal(body, &p) != nil || p.PlayabilityStatus.Status != "OK" {
return mediaResource{}, mediaCapability{}, errors.New("invalid playable metadata")
}
for _, f := range append(p.StreamingData.Formats, p.StreamingData.AdaptiveFormats...) {
u, err := url.Parse(f.URL)
module, err := requiredYoutubeAdapter()
if err != nil {
continue
return mediaResource{}, mediaCapability{}, err
}
host := u.Hostname()
if !strings.HasSuffix(host, ".googlevideo.com") {
continue
resources, err := module.DiscoverResponse(body)
if err != nil {
return mediaResource{}, mediaCapability{}, err
}
if _, err := privateResourceURL(f.URL, host); err != nil {
continue
}
kind, _, err := mime.ParseMediaType(f.MimeType)
if err == nil && (kind == "video/mp4" || kind == "audio/mp4") {
return mediaResource{f.URL, host, kind}, mediaCapability{OrdinaryURL: true}, nil
for _, r := range resources {
if _, err := privateResourceURL(r.URL, r.Host); err == nil {
return mediaResource{r.URL, r.Host, r.Kind}, mediaCapability{OrdinaryURL: true}, nil
}
}
return mediaResource{}, mediaCapability{}, nil
@@ -219,9 +208,16 @@ func playerMediaResource(body []byte) (mediaResource, mediaCapability, error) {
const resourceTrailer = "\nAUTOTUNE_RESOURCE_METRICS:"
func collectResourceTransfer(ctx context.Context, proxy, raw, host, protocol string, media bool, runner subprocessRunner) (assetTransfer, http.Header, error) {
return collectResourceTransferOptions(ctx, proxy, raw, host, protocol, media, runner, resourceTransferOptions{})
}
type resourceResponseKind uint8
// A completed generic mismatch retains measured facts, unlike unsafe transfers.
var errResourceExpectation = errors.New("resource expectation failed")
const (
resourceResponseGeneric resourceResponseKind = iota
resourceResponseImage
resourceResponseMP4
)
type resourceTransferOptions struct {
method string
@@ -229,12 +225,22 @@ type resourceTransferOptions struct {
timeout time.Duration
// Private curl config only: never placed in argv or public evidence.
config string
generic bool
response resourceResponseKind
mediaType string
expect *resourceCheckExpect
}
func collectResourceTransferOptions(ctx context.Context, proxy, raw, host, protocol string, media bool, runner subprocessRunner, options resourceTransferOptions) (assetTransfer, http.Header, error) {
func collectResourceTransferOptions(ctx context.Context, proxy, raw, host, protocol string, runner subprocessRunner, options resourceTransferOptions) (assetTransfer, http.Header, error) {
generic := options.response == resourceResponseGeneric
configured := generic || options.expect != nil
reason := reasonOrigin
fail := func(err error) (assetTransfer, http.Header, error) {
return assetTransfer{}, nil, resourceFailure(ctx, err)
failure := resourceFailure(ctx, err)
failure.(*resourceError).reason = reason
return assetTransfer{}, nil, failure
}
if options.response > resourceResponseMP4 || (!generic && options.method != "GET") || (options.response == resourceResponseMP4 && options.mediaType != "video/mp4" && options.mediaType != "audio/mp4") {
return fail(errors.New("invalid response contract"))
}
proxy, proxyErr := managedProbeProxy(proxy)
if proxyErr != nil || runner == nil {
@@ -250,22 +256,25 @@ func collectResourceTransferOptions(ctx context.Context, proxy, raw, host, proto
return fail(errors.New("invalid TLS protocol"))
}
limit := 5 << 20
if media {
if options.response == resourceResponseMP4 {
limit = 65536
}
if options.generic {
if configured {
limit = options.limit
}
// Qualification uses HTTP/1.1, not browser HTTP/2. --raw preserves native
// transfer framing so net/http, rather than curl and Go twice, decodes it.
args := []string{"--disable", "--silent", "--show-error", "--verbose", "--http1.1", "--raw", "--noproxy", "", "--proxy", proxy, "--proto", "=https", tlsFlag, "--tls-max", maxTLS, "--max-time", "30", "--max-filesize", strconv.Itoa(limit), "--suppress-connect-headers", "--include", "--write-out", resourceTrailer + "%{json}", "--config", "-"}
if media {
if options.response == resourceResponseMP4 {
args = append(args, "--range", "0-65535")
}
if options.generic {
if configured {
if (options.method != "GET" && options.method != "HEAD") || options.limit < 1 || options.limit > 5<<20 || options.timeout < 100*time.Millisecond || options.timeout > 30*time.Second {
return fail(errors.New("invalid transfer options"))
}
if options.response == resourceResponseMP4 && options.limit > 65536 {
return fail(errors.New("invalid transfer options"))
}
args = append(args, "--max-time", strconv.FormatFloat(options.timeout.Seconds(), 'f', 3, 64))
if options.method == "HEAD" {
args = append(args, "--head", "--max-filesize", "0")
@@ -275,16 +284,18 @@ func collectResourceTransferOptions(ctx context.Context, proxy, raw, host, proto
config := "url = \"" + strings.NewReplacer("\\", "\\\\", "\"", "\\\"").Replace(raw) + "\"\n"
config += options.config
timeout := 35 * time.Second
if options.generic {
if configured {
timeout = options.timeout
}
childCtx, cancel := context.WithTimeout(ctx, timeout)
defer cancel()
var output, diagnostic bytes.Buffer
reason = reasonExecution
out := &playerBoundedWriter{dst: &output, limit: limit + (96 << 10)}
diag := &playerBoundedWriter{dst: &diagnostic, limit: 64 << 10}
err := runner(childCtx, args, strings.NewReader(config), out, diag)
if out.exceeded || diag.exceeded {
reason = reasonOutputBound
err = errors.Join(err, errors.New("resource output exceeds limit"))
}
if err != nil || childCtx.Err() != nil {
@@ -298,6 +309,7 @@ func collectResourceTransferOptions(ctx context.Context, proxy, raw, host, proto
}
return fail(errors.Join(err, childCtx.Err()))
}
reason = reasonMetrics
i := bytes.LastIndex(output.Bytes(), []byte(resourceTrailer))
if i < 0 {
return fail(errors.New("missing transfer metrics"))
@@ -314,9 +326,11 @@ func collectResourceTransferOptions(ctx context.Context, proxy, raw, host, proto
if json.Unmarshal(output.Bytes()[i+len(resourceTrailer):], &metrics) != nil {
return fail(errors.New("invalid transfer metrics"))
}
reason = reasonOrigin
if _, err := privateResourceURL(metrics.URL, host); err != nil || metrics.URL != raw || metrics.Verify == nil || *metrics.Verify != 0 || metrics.Redirects == nil || *metrics.Redirects != 0 {
return fail(errors.New("resource origin verification failed"))
}
reason = reasonTLSEvidence
actualTLS := "SSL connection using TLSv1.2"
if version == 0x0304 {
actualTLS = "SSL connection using TLSv1.3"
@@ -328,6 +342,7 @@ func collectResourceTransferOptions(ctx context.Context, proxy, raw, host, proto
if !strings.Contains(diagnostic.String(), actualTLS) || strings.Contains(diagnostic.String(), otherTLS) {
return fail(errors.New("actual TLS version mismatch"))
}
reason = reasonFraming
wire := output.Bytes()[:i]
headerEnd := bytes.Index(wire, []byte("\r\n\r\n"))
statusEnd := bytes.Index(wire, []byte("\r\n"))
@@ -347,10 +362,7 @@ func collectResourceTransferOptions(ctx context.Context, proxy, raw, host, proto
return fail(errors.New("ambiguous response framing"))
}
r := bufio.NewReader(bytes.NewReader(wire))
var request *http.Request
if options.generic {
request = &http.Request{Method: options.method}
}
request := &http.Request{Method: options.method}
response, err := http.ReadResponse(r, request)
if err != nil {
return fail(errors.New("invalid response headers"))
@@ -358,36 +370,110 @@ func collectResourceTransferOptions(ctx context.Context, proxy, raw, host, proto
defer response.Body.Close()
// CONNECT headers are suppressed by curl. Interim/duplicate responses and
// unrequested versions are outside this contract, never heuristically skipped.
if response.ProtoMajor != 1 || response.ProtoMinor != 1 || response.StatusCode < 200 || (!options.generic && response.Header.Get("Content-Type") == "") {
if response.ProtoMajor != 1 || response.ProtoMinor != 1 || response.StatusCode < 200 || (!configured && response.Header.Get("Content-Type") == "") {
return fail(errors.New("unsupported response contract"))
}
if options.generic && response.StatusCode >= 300 && response.StatusCode < 400 {
return fail(errors.New("redirect rejected"))
if generic && response.StatusCode >= 300 && response.StatusCode < 400 {
return fail(&resourcePeerResponseRejection{error: errors.New("redirect rejected")})
}
reason = reasonBody
body, err := io.ReadAll(io.LimitReader(response.Body, int64(limit+1)))
// With --raw curl counts the wire body, including chunk framing; the image
// limit and durable byte identity apply to the fully decoded payload instead.
wireSize := len(wire) - headerEnd - 4
_, remaining := r.Peek(1)
if err != nil || remaining != io.EOF || (!options.generic && len(body) == 0) || len(body) > limit || metrics.Size != float64(wireSize) || response.StatusCode != metrics.Status {
if err != nil || remaining != io.EOF || (!configured && len(body) == 0) || len(body) > limit || metrics.Size != float64(wireSize) || response.StatusCode != metrics.Status {
return fail(errors.New("incomplete or oversized resource body"))
}
reason = reasonMetrics
for _, n := range []float64{metrics.Time, metrics.Speed} {
if n < 0 || (!options.generic && n == 0) || math.IsNaN(n) || math.IsInf(n, 0) {
if n < 0 || (!configured && n == 0) || math.IsNaN(n) || math.IsInf(n, 0) {
return fail(errors.New("invalid resource metrics"))
}
}
if options.generic && metrics.Time > options.timeout.Seconds()+1 {
if configured && metrics.Time > options.timeout.Seconds()+1 {
return fail(errors.New("resource timing exceeds budget"))
}
reason = reasonMIME
contentType, _, err := mime.ParseMediaType(response.Header.Get("Content-Type"))
if err != nil && (!options.generic || response.Header.Get("Content-Type") != "") {
return fail(errors.New("invalid content type"))
if err != nil {
contentType = ""
}
t := assetTransfer{EffectiveHost: host, SSLVerifyResult: metrics.Verify, Redirects: *metrics.Redirects, TLSVersion: version, Status: metrics.Status, ContentType: contentType, Body: body, LatencyMS: metrics.Time * 1000, SpeedKBPS: metrics.Speed / 1024}
if err != nil && options.expect != nil && (options.response == resourceResponseImage || options.response == resourceResponseMP4) {
// A malformed semantic MIME value is a completed typed mismatch only
// after the common wire/security checks; never retain the raw value.
return t, response.Header, errResourceExpectation
}
if err != nil && (!configured || response.Header.Get("Content-Type") != "") {
return fail(errors.New("invalid content type"))
}
if err := options.validateResponse(&t, response.Header); err != nil {
if err == errResourceExpectation {
return t, response.Header, err
}
return assetTransfer{}, nil, resourceFailure(ctx, err)
}
return t, response.Header, nil
}
// Adapter contracts run only after the common origin/TLS/framing/bounds checks.
// Generic v1 deliberately has no range or decoded-image expectation.
func (options resourceTransferOptions) validateResponse(t *assetTransfer, h http.Header) error {
if err := options.validateContract(t, h); err != nil {
if options.expect != nil {
return errResourceExpectation
}
return err
}
if e := options.expect; e != nil {
for _, status := range e.Statuses {
if t.Status == status && len(t.Body) >= e.MinBytes && len(t.Body) <= e.MaxBytes && (e.ContentType == "" || t.ContentType == e.ContentType) {
return nil
}
}
return errResourceExpectation
}
return nil
}
func (options resourceTransferOptions) validateContract(t *assetTransfer, h http.Header) error {
switch options.response {
case resourceResponseGeneric:
return nil
case resourceResponseImage:
if t.Status != 200 {
return errors.New("image HTTP status")
}
if options.expect != nil {
// Reject GIF before the legacy full decoder can allocate its frames.
_, format, err := image.DecodeConfig(bytes.NewReader(t.Body))
if err != nil || (format != "png" && format != "jpeg") {
return errors.New("invalid public image format")
}
}
return normalizeAssetBody(t)
case resourceResponseMP4:
parts := resourceContentRange.FindStringSubmatch(h.Get("Content-Range"))
var end, total int64
if len(parts) == 3 {
var endErr, totalErr error
end, endErr = strconv.ParseInt(parts[1], 10, 64)
total, totalErr = strconv.ParseInt(parts[2], 10, 64)
if endErr != nil || totalErr != nil {
return errors.New("invalid media range or container")
}
}
if t.Status != 206 || t.ContentType != options.mediaType || len(parts) != 3 || end < 15 || end > 65535 || end+1 != int64(len(t.Body)) || total <= end || len(t.Body) < 16 || string(t.Body[4:8]) != "ftyp" || binary.BigEndian.Uint32(t.Body[:4]) < 16 || int(binary.BigEndian.Uint32(t.Body[:4])) > len(t.Body) {
return errors.New("invalid media range or container")
}
t.Size, t.Format, t.BodyHash = len(t.Body), "mp4", fmt.Sprintf("%x", sha256.Sum256(t.Body))
return nil
default:
return errors.New("invalid response contract")
}
}
// The ordinary 3/3 sampler is shared by image and media qualification.
func sampleResource(ctx context.Context, transfer func(context.Context) (assetTransfer, error)) ([]assetTransfer, error) {
var samples []assetTransfer
@@ -407,14 +493,7 @@ func collectAssetMeasurement(ctx context.Context, proxy, raw, host, protocol, ar
return m, resourceFailure(ctx, errors.New("invalid image endpoint"))
}
samples, err := sampleResource(ctx, func(ctx context.Context) (assetTransfer, error) {
t, _, err := collectResourceTransfer(ctx, proxy, raw, host, protocol, false, runner)
if err == nil {
if t.Status != 200 {
err = errors.New("image HTTP status")
} else {
err = normalizeAssetBody(&t)
}
}
t, _, err := collectResourceTransferOptions(ctx, proxy, raw, host, protocol, runner, resourceTransferOptions{method: "GET", response: resourceResponseImage})
if err != nil {
return assetTransfer{}, resourceFailure(ctx, err)
}
@@ -441,22 +520,18 @@ func collectMediaTransfer(ctx context.Context, proxy string, resource mediaResou
if !strings.HasSuffix(resource.host, ".googlevideo.com") || (resource.kind != "video/mp4" && resource.kind != "audio/mp4") {
return assetTransfer{}, GooglevideoPass{}, resourceFailure(ctx, errors.New("ordinary media unavailable"))
}
t, h, err := collectResourceTransfer(ctx, proxy, resource.url, resource.host, protocol, true, runner)
t, h, err := collectResourceTransferOptions(ctx, proxy, resource.url, resource.host, protocol, runner, resourceTransferOptions{method: "GET", response: resourceResponseMP4, mediaType: resource.kind})
if err != nil {
return assetTransfer{}, GooglevideoPass{}, err
}
parts := resourceContentRange.FindStringSubmatch(h.Get("Content-Range"))
var end, total int64
if len(parts) == 3 {
end, _ = strconv.ParseInt(parts[1], 10, 64)
total, _ = strconv.ParseInt(parts[2], 10, 64)
}
if t.Status != 206 || t.ContentType != resource.kind || len(parts) != 3 || end < 15 || end > 65535 || end+1 != int64(len(t.Body)) || total <= end || len(t.Body) < 16 || string(t.Body[4:8]) != "ftyp" || binary.BigEndian.Uint32(t.Body[:4]) < 16 || int(binary.BigEndian.Uint32(t.Body[:4])) > len(t.Body) {
// The shared core has validated and normalized the complete range contract.
_, totalText, _ := strings.Cut(h.Get("Content-Range"), "/")
total, err := strconv.ParseInt(totalText, 10, 64)
if err != nil {
return assetTransfer{}, GooglevideoPass{}, resourceFailure(ctx, errors.New("invalid media range or container"))
}
// Retain only normalized byte identity; no signed URL or payload.
t.Size, t.Format, t.BodyHash = len(t.Body), "mp4", fmt.Sprintf("%x", sha256.Sum256(t.Body))
t.Body = nil
p := GooglevideoPass{Protocol: protocol, VerifiedTLS: true, Status: t.Status, RangeStart: 0, RangeEnd: end, ResourceBytes: total, Bytes: int64(t.Size), FTYP: true, LatencyMS: t.LatencyMS}
p := GooglevideoPass{Protocol: protocol, VerifiedTLS: true, Status: t.Status, RangeStart: 0, RangeEnd: int64(t.Size) - 1, ResourceBytes: total, Bytes: int64(t.Size), FTYP: true, LatencyMS: t.LatencyMS}
return t, p, nil
}
@@ -0,0 +1,127 @@
package main
import (
"bytes"
"context"
"fmt"
"io"
"reflect"
"strings"
"testing"
"time"
)
func TestResourceCoreMediaRangeIntegers(t *testing.T) {
for _, tc := range []struct {
name, contentRange string
total int64
}{
{"ordinary", "bytes 0-23/100", 100},
{"max-valid", "bytes 0-23/9223372036854775807", 9223372036854775807},
{"total-overflow", "bytes 0-23/9223372036854775808", 0},
{"total-huge", "bytes 0-23/999999999999999999999999999999999999", 0},
{"end-overflow", "bytes 0-9223372036854775808/9223372036854775807", 0},
{"nonnumeric", "bytes 0-23/abc", 0},
{"unknown", "bytes 0-23/*", 0},
{"signed", "bytes 0-23/+100", 0},
{"negative", "bytes 0-23/-100", 0},
{"signed-end", "bytes 0-+23/100", 0},
{"negative-end", "bytes 0--23/100", 0},
{"total-equals-end", "bytes 0-23/23", 0},
{"total-below-end", "bytes 0-23/22", 0},
{"end-body-mismatch", "bytes 0-24/100", 0},
} {
t.Run(tc.name, func(t *testing.T) {
raw := "https://rr1.unit.googlevideo.com/fresh?sig=private-secret"
runner := func(_ context.Context, _ []string, _ io.Reader, out, diag io.Writer) error {
var wire bytes.Buffer
resourceWire(raw, "media", &wire, diag)
fmt.Fprint(out, strings.Replace(wire.String(), "bytes 0-23/100", tc.contentRange, 1))
return nil
}
transfer, _, err := collectResourceTransferOptions(context.Background(), "socks5://127.0.0.1:1080", raw, "rr1.unit.googlevideo.com", "tls12", runner, resourceTransferOptions{method: "GET", response: resourceResponseMP4, mediaType: "video/mp4"})
valid := tc.total != 0
if (err == nil) != valid || !valid && !reflect.DeepEqual(transfer, assetTransfer{}) {
t.Fatalf("core valid=%v transfer=%+v err=%v", valid, transfer, err)
}
transfer, pass, err := collectMediaTransfer(context.Background(), "socks5://127.0.0.1:1080", mediaResource{raw, "rr1.unit.googlevideo.com", "video/mp4"}, "tls12", runner)
if (err == nil) != valid {
t.Fatalf("adapter valid=%v err=%v", valid, err)
}
if valid {
if pass.ResourceBytes != tc.total || pass.Bytes != 24 || pass.RangeEnd != 23 || transfer.Size != 24 {
t.Fatalf("incorrect normalized facts: %+v %+v", transfer, pass)
}
} else if !reflect.DeepEqual(transfer, assetTransfer{}) || !reflect.DeepEqual(pass, GooglevideoPass{}) {
t.Fatalf("rejected range produced evidence: %+v %+v", transfer, pass)
}
})
}
}
// Unlike the adapter golden test, these assertions call the shared execution
// entry directly: strict validation must not be left in a legacy-only branch.
func TestResourceCoreTypedContracts(t *testing.T) {
for _, mode := range []string{"pass", "status", "start", "end", "total", "malformed", "ftyp", "box", "decoder"} {
for _, kind := range []resourceResponseKind{resourceResponseGeneric, resourceResponseImage, resourceResponseMP4} {
t.Run(fmt.Sprintf("%d/%s", kind, mode), func(t *testing.T) {
options := resourceTransferOptions{method: "GET", limit: 1024, timeout: time.Second, response: kind, mediaType: "video/mp4"}
raw := "https://i.ytimg.com/fresh?sig=private-secret"
runner := func(_ context.Context, _ []string, _ io.Reader, out, diag io.Writer) error {
var wire bytes.Buffer
wireMode := ""
if kind == resourceResponseMP4 {
wireMode = "media"
}
if mode == "decoder" && kind != resourceResponseMP4 {
wireMode = "type"
}
resourceWire(raw, wireMode, &wire, diag)
data := wire.String()
switch mode {
case "status":
data = strings.ReplaceAll(data, "206", "200")
if kind != resourceResponseMP4 {
data = strings.ReplaceAll(data, "200", "201")
}
case "start":
data = strings.Replace(data, "bytes 0-23/100", "bytes 1-24/100", 1)
case "end":
data = strings.Replace(data, "bytes 0-23/100", "bytes 0-24/100", 1)
case "total":
data = strings.Replace(data, "bytes 0-23/100", "bytes 0-23/23", 1)
case "malformed":
data = strings.Replace(data, "bytes 0-23/100", "bytes 0-23/*", 1)
case "ftyp":
data = strings.Replace(data, "ftyp", "xxxx", 1)
case "box":
data = strings.Replace(data, string([]byte{0, 0, 0, 24}), string([]byte{0, 0, 0, 25}), 1)
}
fmt.Fprint(out, data)
return nil
}
transfer, _, err := collectResourceTransferOptions(context.Background(), "socks5://127.0.0.1:1080", raw, "i.ytimg.com", "tls12", runner, options)
valid := kind == resourceResponseGeneric || mode == "pass" || kind == resourceResponseImage && mode != "decoder" && mode != "status" || kind == resourceResponseMP4 && mode == "decoder"
if (err == nil) != valid {
t.Fatalf("valid=%v err=%v", valid, err)
}
if valid && kind != resourceResponseGeneric && (transfer.Size != len(transfer.Body) || transfer.BodyHash == "" || transfer.Format == "") {
t.Fatal("strict normalization not performed by core")
}
})
}
}
}
func TestResourceCoreGenericExpectationFacts(t *testing.T) {
raw := "https://i.ytimg.com/private"
runner := func(_ context.Context, _ []string, _ io.Reader, out, diag io.Writer) error {
resourceWire(raw, "", out, diag)
return nil
}
expect := resourceCheckExpect{Statuses: []int{201}, MaxBytes: 1024}
transfer, _, err := collectResourceTransferOptions(context.Background(), "socks5://127.0.0.1:1080", raw, "i.ytimg.com", "tls12", runner, resourceTransferOptions{method: "GET", limit: 1024, timeout: time.Second, expect: &expect})
if err != errResourceExpectation || transfer.Status != 200 || len(transfer.Body) != len(resourceFixture()) || transfer.LatencyMS != 100 {
t.Fatalf("core did not validate generic expectation with intact facts: %+v %v", transfer, err)
}
}
@@ -0,0 +1,193 @@
package main
import (
"bytes"
"context"
"crypto/sha256"
"encoding/json"
"fmt"
"io"
"net"
"reflect"
"strings"
"testing"
)
// Exercise the actual legacy adapters and generic producer with the same wire
// mutations. This test also runs unchanged against the pre-migration source.
func TestResourceCoreAdapterParity(t *testing.T) {
for _, adapter := range []string{"image", "media", "GET", "HEAD"} {
for _, protocol := range []string{"tls12", "tls13"} {
for _, mode := range []string{"pass", "verify", "origin", "redirect", "tls", "framing", "oversize", "range", "ftyp", "decoder"} {
t.Run(adapter+"/"+protocol+"/"+mode, func(t *testing.T) {
host := "i.ytimg.com"
if adapter == "media" {
host = "rr1.unit.googlevideo.com"
}
raw := "https://" + host + "/fresh?sig=private-secret"
calls := 0
runner := func(_ context.Context, args []string, stdin io.Reader, out, diag io.Writer) error {
calls++
argv := strings.Join(args, " ")
for _, required := range []string{"--disable --silent", "--http1.1 --raw", "--noproxy --proxy socks5h://127.0.0.1:1080", "--proto =https", "--suppress-connect-headers", "--config -"} {
if !strings.Contains(argv, required) {
t.Fatalf("missing %s", required)
}
}
if strings.Contains(argv, "private-secret") || strings.Contains(argv, host) || strings.Contains(argv, "--insecure") || strings.Contains(argv, "--location") {
t.Fatal("unsafe argv")
}
if strings.Contains(argv, "--range 0-65535") != (adapter == "media") || strings.Contains(argv, "--head") != (adapter == "HEAD") {
t.Fatal("method/range changed")
}
config, _ := io.ReadAll(stdin)
if !strings.Contains(string(config), raw) {
t.Fatal("URL not in private stdin")
}
if adapter == "GET" || adapter == "HEAD" {
if !strings.Contains(string(config), `connect-to = "i.ytimg.com:443:93.184.216.34:443"`) || !strings.Contains(string(config), "Authorization: private-header") {
t.Fatal("private header or CONNECT pin lost")
}
}
var wire, diagnostic bytes.Buffer
wireMode := ""
if adapter == "media" {
wireMode = "media"
}
if mode == "decoder" && adapter == "image" {
wireMode = "type"
}
resourceWire(raw, wireMode, &wire, &diagnostic)
data := wire.String()
if adapter == "HEAD" {
start := strings.Index(data, "\r\n\r\n") + 4
end := strings.LastIndex(data, resourceTrailer)
data = data[:start] + data[end:]
data = strings.Replace(data, fmt.Sprintf(`"size_download":%d`, len(resourceFixture())), `"size_download":0`, 1)
}
switch mode {
case "verify":
data = strings.Replace(data, `"ssl_verify_result":0`, `"ssl_verify_result":60`, 1)
case "origin":
data = strings.Replace(data, `"url_effective":"https://`, `"url_effective":"https://wrong.`, 1)
case "redirect":
data = strings.Replace(data, `"num_redirects":0`, `"num_redirects":1`, 1)
case "framing":
data = strings.Replace(data, "Content-Length:", "Content-Length: 1\r\nContent-Length:", 1)
case "oversize":
_, err := out.Write(make([]byte, 6<<20))
return err
case "range":
data = strings.Replace(data, "bytes 0-23/100", "bytes 1-24/100", 1)
case "ftyp":
data = strings.Replace(data, "ftyp", "xxxx", 1)
}
fmt.Fprint(out, data)
actual := "TLSv1.2"
if protocol == "tls13" {
actual = "TLSv1.3"
}
if mode == "tls" {
actual = "TLSv1.1"
}
fmt.Fprintln(diag, "SSL connection using "+actual)
return nil
}
valid := mode == "pass" || mode == "range" && adapter != "media" || mode == "ftyp" && adapter != "media" || mode == "decoder" && adapter != "image"
var err error
var transfer assetTransfer
switch adapter {
case "image":
var m assetMeasurement
m, err = collectAssetMeasurement(context.Background(), "socks5://127.0.0.1:1080", raw, host, protocol, "candidate", runner)
if err == nil {
transfer = m.Transfers[0]
if calls != 3 || m.Source != "image-transfer" || m.Endpoint != host || m.Protocol != protocol || m.Args != "candidate" {
t.Fatal("legacy measurement changed")
}
}
case "media":
var pass GooglevideoPass
transfer, pass, err = collectMediaTransfer(context.Background(), "socks5://127.0.0.1:1080", mediaResource{raw, host, "video/mp4"}, protocol, runner)
if err == nil && !reflect.DeepEqual(pass, GooglevideoPass{Protocol: protocol, VerifiedTLS: true, Status: 206, RangeStart: 0, RangeEnd: 23, ResourceBytes: 100, Bytes: 24, FTYP: true, LatencyMS: 100}) {
t.Fatalf("legacy pass changed: %+v", pass)
}
default:
d := resourceCheckDeps{guard: func() error { return nil }, resolve: func(context.Context, string, string) ([]net.IP, error) {
return []net.IP{net.ParseIP("93.184.216.34")}, nil
}, runner: runner}
r := resourceCheckSpec{ID: "r", Group: "g", URL: raw, host: host, Method: adapter, TimeoutMS: 1000, Expect: resourceCheckExpect{Statuses: []int{200}, ContentType: "image/png", MaxBytes: 1024}, privateConfig: "header = \"Authorization: private-header\"\n"}
result, e := runResourceProtocol(context.Background(), "s", r, protocol, "dns", "socks5://127.0.0.1:1080", d)
err = e
if valid {
wantBytes := len(resourceFixture())
if adapter == "HEAD" {
wantBytes = 0
}
if result != (resourceCheckResult{Service: "s", Resource: "r", Group: "g", TLS: protocol, Outcome: "pass", Status: 200, Bytes: wantBytes, TimeMS: 100}) {
t.Fatalf("generic result changed: %+v", result)
}
} else if result.Outcome == "pass" {
t.Fatal("unsafe generic pass")
}
if err == nil && !valid {
err = fmt.Errorf("rejected transfer")
}
}
if (err == nil) != valid {
t.Fatalf("valid=%v err=%v", valid, err)
}
if valid && (adapter == "image" || adapter == "media") {
body, format, status, mime := resourceFixture(), "png", 200, "image/png"
if adapter == "media" {
body, format, status, mime = []byte{0, 0, 0, 24, 'f', 't', 'y', 'p', 'i', 's', 'o', 'm', 0, 0, 0, 0, 'i', 's', 'o', 'm', 'm', 'p', '4', '2'}, "mp4", 206, "video/mp4"
}
version := uint16(0x0303)
if protocol == "tls13" {
version = 0x0304
}
verify := 0
want := assetTransfer{EffectiveHost: host, SSLVerifyResult: &verify, TLSVersion: version, Status: status, ContentType: mime, Size: len(body), Format: format, BodyHash: fmt.Sprintf("%x", sha256.Sum256(body)), LatencyMS: 100, SpeedKBPS: 10000.0 / 1024}
if !reflect.DeepEqual(transfer, want) {
t.Fatalf("normalized facts changed: got %+v want %+v", transfer, want)
}
encoded, _ := json.Marshal(transfer)
if strings.Contains(string(encoded), "private-secret") {
t.Fatal("private evidence")
}
}
})
}
}
}
}
func TestResourceCoreGenericMismatchParity(t *testing.T) {
for _, field := range []string{"status", "mime", "bytes"} {
for _, qualification := range []bool{false, true} {
t.Run(fmt.Sprintf("%s/qualification=%v", field, qualification), func(t *testing.T) {
raw := "https://i.ytimg.com/fresh?sig=private-secret"
r := resourceCheckSpec{ID: "r", Group: "g", URL: raw, host: "i.ytimg.com", Method: "GET", TimeoutMS: 1000, Expect: resourceCheckExpect{Statuses: []int{200}, ContentType: "image/png", MaxBytes: 1024}}
switch field {
case "status":
r.Expect.Statuses = []int{201}
case "mime":
r.Expect.ContentType = "image/jpeg"
case "bytes":
r.Expect.MinBytes = 1000
}
d := resourceCheckDeps{qualification: qualification, guard: func() error { return nil }, resolve: func(context.Context, string, string) ([]net.IP, error) {
return []net.IP{net.ParseIP("93.184.216.34")}, nil
}, runner: func(_ context.Context, _ []string, _ io.Reader, out, diag io.Writer) error {
resourceWire(raw, "", out, diag)
return nil
}}
result, err := runResourceProtocol(context.Background(), "s", r, "tls12", "dns", "socks5://127.0.0.1:1080", d)
want := resourceCheckResult{Service: "s", Resource: "r", Group: "g", TLS: "tls12", Outcome: "expectation_failed", Bytes: len(resourceFixture()), TimeMS: 100, Status: 200}
if err != nil || result != want {
t.Fatalf("result=%+v err=%v", result, err)
}
})
}
}
}
+9
View File
@@ -74,6 +74,9 @@ func readResourceMonitorSource(path string, lookup func(string) (string, bool),
func resourceRowFingerprint(key []byte, source resourceMonitorSource, r resourceCheckSpec) string {
b, _ := json.Marshal(r)
b, _ = json.Marshal([]string{string(b), r.privateConfig, source.server, source.proxy})
if source.config.Version == 2 {
b, _ = json.Marshal([]string{"resource-check-v2", string(b)})
}
return resourceMAC(key, b)
}
func reconcileResourceRows(key []byte, source resourceMonitorSource, prior []resourceMonitorRow) []resourceMonitorRow {
@@ -345,6 +348,9 @@ func resourceMonitorCLIWithCheck(ctx context.Context, path string, stdout, stder
if resourceManagedIdentity(target.directory) != nil {
return errResourceState
}
if resourceDeploymentIdentity(path, target) != nil {
return errResourceState
}
return check.guard()
},
probe: func(ctx context.Context, service string, spec resourceCheckSpec, source resourceMonitorSource, guard func() error) (string, error) {
@@ -394,6 +400,9 @@ func resourceStatusCLIAt(stdout io.Writer, target managedDeps, proc string) int
if err != nil || check.guard() != nil || resourceManagedIdentity(target.directory) != nil {
return blocked()
}
if resourceDeploymentIdentity(path, target) != nil {
return blocked()
}
rows := reconcileResourceRows(store.key, source, state.Rows)
b1, _ := json.Marshal(rows)
b2, _ := json.Marshal(state.Rows)
+339
View File
@@ -0,0 +1,339 @@
package main
import (
"bytes"
"crypto/sha256"
"encoding/hex"
"encoding/json"
"errors"
"io"
"sort"
"strings"
"time"
)
var errResourcePlan = errors.New("invalid private qualification data")
// Input groups are metadata. Only the HTTPS collector's selectors are supported.
type strategySelectors struct {
Transport string `json:"transport"`
Ports string `json:"ports"`
L7 string `json:"l7"`
TLS string `json:"tls"`
}
type strategyDefinition struct {
ID string `json:"id"`
Args string `json:"args"`
Selectors strategySelectors `json:"selectors"`
}
type strategyRegistry struct {
Version int `json:"version"`
Strategies []strategyDefinition `json:"strategies"`
}
func strictPlanJSON(b []byte, dst interface{}) error {
if len(b) > 65536 {
return errResourcePlan
}
d := json.NewDecoder(bytes.NewReader(b))
if uniqueResourceJSON(d, 0) != nil {
return errResourcePlan
}
if _, err := d.Token(); err != io.EOF {
return errResourcePlan
}
d = json.NewDecoder(bytes.NewReader(b))
d.DisallowUnknownFields()
if d.Decode(dst) != nil {
return errResourcePlan
}
return nil
}
func planDigest(v interface{}) string {
b, _ := json.Marshal(v)
h := sha256.Sum256(b)
return hex.EncodeToString(h[:])
}
func canonicalStrategy(d strategyDefinition) (strategyDefinition, error) {
d.Args = strings.Join(strings.Fields(d.Args), " ")
if !resourceID.MatchString(d.ID) || d.Args == "" || len(d.Args) > 4096 || (&candidate{Args: d.Args}).validateArgs() != nil {
return d, errResourcePlan
}
if d.Selectors.Transport != "tcp" || d.Selectors.Ports != "443" || d.Selectors.L7 != "tls" || (d.Selectors.TLS != "both" && d.Selectors.TLS != "tls12" && d.Selectors.TLS != "tls13") {
return d, errResourcePlan
}
for _, arg := range strings.Fields(d.Args) {
option, value, _ := strings.Cut(arg, "=")
if option == "--lua-init" {
return d, errResourcePlan
}
if option != "--lua-desync" {
continue
}
parts := strings.Split(value, ":")
switch parts[0] {
case "fake", "multisplit", "multidisorder", "fakeddisorder", "split", "disorder":
default:
return d, errResourcePlan
}
for _, part := range parts[1:] {
key, v, ok := strings.Cut(part, "=")
if !ok || v == "" || strings.ContainsAny(v, "/@%") || strings.Contains(v, "..") {
return d, errResourcePlan
}
switch key {
case "pos", "repeats", "tcp_ts", "tcp_flags_set", "seqovl":
case "blob":
if v != "fake_default_tls" {
return d, errResourcePlan
}
default:
return d, errResourcePlan
}
}
}
return d, nil
}
func parseStrategyRegistry(b []byte) (strategyRegistry, error) {
var r strategyRegistry
if strictPlanJSON(b, &r) != nil || r.Version != 1 || len(r.Strategies) == 0 || len(r.Strategies) > 32 {
return r, errResourcePlan
}
seen := map[string]bool{}
for i, d := range r.Strategies {
v, err := canonicalStrategy(d)
if err != nil || seen[v.ID] {
return r, errResourcePlan
}
seen[v.ID] = true
r.Strategies[i] = v
}
sort.Slice(r.Strategies, func(i, j int) bool { return r.Strategies[i].ID < r.Strategies[j].ID })
return r, nil
}
type qualificationCheck struct {
Service string `json:"service"`
Resource string `json:"resource"`
Fingerprint string `json:"fingerprint"`
TLS string `json:"tls"`
Outcome string `json:"outcome"`
Attempts int `json:"attempts"`
}
type qualificationRun struct {
ID string `json:"id"`
Strategy string `json:"strategy"`
Definition string `json:"definition"`
Config string `json:"config"`
Started int64 `json:"started_ms"`
Completed int64 `json:"completed_ms"`
Attestation qualificationAttestation `json:"attestation"`
Checks []qualificationCheck `json:"checks"`
}
type qualificationAttestation struct {
Collector string `json:"collector"`
Runtime string `json:"runtime"`
Definition string `json:"definition"`
LuaHash string `json:"lua_hash"`
RulesHash string `json:"rules_hash"`
WorkerArgsHash string `json:"worker_args_hash"`
Queue int `json:"queue"`
PID int `json:"pid"`
StartTime int64 `json:"starttime"`
Restored bool `json:"restored"`
Joined bool `json:"joined"`
}
type qualificationMatrix struct {
Version int `json:"version"`
Registry string `json:"registry"`
Runs []qualificationRun `json:"runs"`
}
// Only the owned producer (or internal test fixtures) constructs this capability.
// JSON imports cannot authorize writes, and post-seal DTO mutation is rejected.
type completedQualification struct {
matrix qualificationMatrix
seal string
}
func hashOK(s string) bool {
b, e := hex.DecodeString(s)
return e == nil && len(b) == 32 && strings.ToLower(s) == s
}
func validateMatrix(m qualificationMatrix, registry strategyRegistry) error {
if m.Version != 1 || m.Registry != planDigest(registry) || len(m.Runs) > 32 {
return errResourcePlan
}
defs := map[string]strategyDefinition{}
for _, d := range registry.Strategies {
defs[d.ID] = d
}
ids, strategies := map[string]bool{}, map[string]bool{}
for _, r := range m.Runs {
d, ok := defs[r.Strategy]
a := r.Attestation
if !ok || !resourceID.MatchString(r.ID) || ids[r.ID] || strategies[r.Strategy] || r.Definition != planDigest(d) || !hashOK(r.Config) || r.Started <= 0 || r.Completed < r.Started || r.Completed-r.Started > 86400000 || len(r.Checks) == 0 || len(r.Checks) > 128 {
return errResourcePlan
}
ids[r.ID], strategies[r.Strategy] = true, true
if a.Collector != "owned-resource-nfq-v1" || a.Runtime != "nfqws2-1.0.2" || a.Definition != r.Definition || !hashOK(a.LuaHash) || !hashOK(a.RulesHash) || !hashOK(a.WorkerArgsHash) || a.Queue != managedAssetQueue || a.PID <= 0 || a.StartTime <= 0 || !a.Restored || !a.Joined {
return errResourcePlan
}
seen := map[string]bool{}
for _, c := range r.Checks {
id := c.Service + "/" + c.Resource + "/" + c.TLS
if !resourceID.MatchString(c.Service) || !resourceID.MatchString(c.Resource) || !hashOK(c.Fingerprint) || seen[id] || c.Attempts != 3 || (c.TLS != "tls12" && c.TLS != "tls13") {
return errResourcePlan
}
seen[id] = true
switch c.Outcome {
case "pass", "dns_failed", "destination_denied", "transfer_failed", "expectation_failed", "timeout":
default:
return errResourcePlan
}
}
}
return nil
}
type domainCompatibility struct {
Host string `json:"host"`
Validated []string `json:"validated"`
Negative []string `json:"negative"`
Unknown []string `json:"unknown"`
}
type runtimeAssignment struct {
Strategy string `json:"strategy"`
Args string `json:"args"`
Selectors strategySelectors `json:"selectors"`
Hosts []string `json:"hosts"`
}
type resourcePlan struct {
ProposalOnly bool `json:"proposal_only"`
Policy string `json:"strategy_ids,omitempty"`
Evidence string `json:"evidence"`
Domains []domainCompatibility `json:"domains"`
Assignments []runtimeAssignment `json:"assignments"`
Unassigned []string `json:"unassigned"`
}
func deriveResourcePlan(source resourceMonitorSource, key []byte, registry strategyRegistry, matrix qualificationMatrix, now time.Time, policy ...string) resourcePlan {
p := resourcePlan{ProposalOnly: true, Evidence: "unknown", Domains: []domainCompatibility{}, Assignments: []runtimeAssignment{}, Unassigned: []string{}}
if len(policy) != 0 {
p.Policy = policy[0]
}
hosts := map[string][]qualificationCheck{}
for _, s := range source.config.Services {
for _, r := range s.Resources {
protocols := []string{r.TLS}
if r.TLS == "both" {
protocols = []string{"tls12", "tls13"}
}
for _, tls := range protocols {
host := strings.ToLower(r.host)
hosts[host] = append(hosts[host], qualificationCheck{Service: s.ID, Resource: r.ID, Fingerprint: resourceRowFingerprint(key, source, r), TLS: tls})
}
}
}
runs := map[string]qualificationRun{}
if validateMatrix(matrix, registry) == nil {
p.Evidence = "authenticated"
for _, r := range matrix.Runs {
if r.Config == source.fingerprint && r.Completed <= now.UnixMilli() && now.UnixMilli()-r.Completed <= 86400000 {
runs[r.Strategy] = r
}
}
}
names := []string{}
for host := range hosts {
names = append(names, host)
}
sort.Strings(names)
for _, host := range names {
row := domainCompatibility{Host: host, Validated: []string{}, Negative: []string{}, Unknown: []string{}}
for _, d := range registry.Strategies {
r, exists := runs[d.ID]
checks := map[string]qualificationCheck{}
for _, c := range r.Checks {
checks[c.Service+"/"+c.Resource+"/"+c.TLS] = c
}
complete, failed := exists, false
for _, expected := range hosts[host] {
c, ok := checks[expected.Service+"/"+expected.Resource+"/"+expected.TLS]
if !ok || c.Fingerprint != expected.Fingerprint || (d.Selectors.TLS != "both" && d.Selectors.TLS != expected.TLS) {
complete = false
continue
}
if c.Outcome != "pass" {
failed = true
}
}
if !complete {
row.Unknown = append(row.Unknown, d.ID)
} else if failed {
row.Negative = append(row.Negative, d.ID)
} else {
row.Validated = append(row.Validated, d.ID)
}
}
p.Domains = append(p.Domains, row)
}
// Merge only equal canonical args/selectors, retaining per-candidate alternatives.
// Greatest uncovered coverage is deterministic greedy, not globally optimal.
profiles := map[string]runtimeAssignment{}
coverage := map[string]map[string]bool{}
for _, d := range registry.Strategies {
if len(policy) != 0 && policy[0] != "" && !strings.Contains(","+policy[0]+",", ","+d.ID+",") {
continue
}
profile := planDigest(struct {
Args string
Selectors strategySelectors
}{d.Args, d.Selectors})
a, exists := profiles[profile]
if !exists || d.ID < a.Strategy {
profiles[profile] = runtimeAssignment{Strategy: d.ID, Args: d.Args, Selectors: d.Selectors}
}
if coverage[profile] == nil {
coverage[profile] = map[string]bool{}
}
for _, row := range p.Domains {
for _, id := range row.Validated {
if id == d.ID {
coverage[profile][row.Host] = true
}
}
}
}
assigned := map[string]bool{}
for {
best := ""
var chosen []string
for profile, a := range profiles {
uncovered := []string{}
for _, host := range names {
if coverage[profile][host] && !assigned[host] {
uncovered = append(uncovered, host)
}
}
if len(uncovered) > len(chosen) || len(uncovered) > 0 && len(uncovered) == len(chosen) && (best == "" || a.Strategy < profiles[best].Strategy) {
best, chosen = profile, uncovered
}
}
if len(chosen) == 0 {
break
}
a := profiles[best]
a.Hosts = chosen
p.Assignments = append(p.Assignments, a)
for _, host := range chosen {
assigned[host] = true
}
}
for _, host := range names {
if !assigned[host] {
p.Unassigned = append(p.Unassigned, host)
}
}
return p
}
+300
View File
@@ -0,0 +1,300 @@
package main
import (
"bytes"
"crypto/hmac"
"crypto/rand"
"encoding/json"
"errors"
"fmt"
"io"
"os"
"syscall"
"time"
)
const planKeyName = ".resource-plan-key"
const planRegistryName = ".resource-plan-strategies.json"
const planMatrixName = ".resource-plan-matrix.json"
// Reuse held-directory/private-entry checks, not monitor publication/lifecycle.
// Entries are write-once O_EXCL: no replacement, repair or clobber of foreign files.
type resourcePlanStore struct {
files *resourceMonitorStore
held map[string]*os.File
snapshots map[string][]byte
}
type planEnvelope struct {
Payload json.RawMessage `json:"payload"`
MAC string `json:"mac"`
}
func openResourcePlanStore(path string, create bool) (*resourcePlanStore, error) {
if managedDirectory(path) != nil {
return nil, errResourcePlan
}
fd, err := syscall.Open(path, syscall.O_RDONLY|syscall.O_DIRECTORY|syscall.O_NOFOLLOW|syscall.O_CLOEXEC, 0)
if err != nil {
return nil, err
}
s := &resourcePlanStore{files: &resourceMonitorStore{path: path, dir: os.NewFile(uintptr(fd), "plan directory")}, held: map[string]*os.File{}, snapshots: map[string][]byte{}}
fail := func(err error) (*resourcePlanStore, error) { s.close(); return nil, err }
for _, name := range []string{planKeyName, planRegistryName, planMatrixName} {
f, b, e := s.files.openEntry(name)
if e != nil && !errors.Is(e, os.ErrNotExist) {
return fail(errResourcePlan)
}
if e == nil {
s.held[name] = f
s.snapshots[name] = b
}
}
key := s.snapshots[planKeyName]
if key == nil && create && s.held[planRegistryName] == nil && s.held[planMatrixName] == nil {
key = make([]byte, 32)
if _, err = rand.Read(key); err != nil {
return fail(errResourcePlan)
}
if s.publish(planKeyName, key) != nil {
return fail(errResourcePlan)
}
}
if len(key) != 32 {
if !create && key == nil && s.held[planRegistryName] == nil && s.held[planMatrixName] == nil {
return fail(os.ErrNotExist)
}
return fail(errResourcePlan)
}
s.files.key = append([]byte(nil), key...)
if s.guard() != nil {
return fail(errResourcePlan)
}
return s, nil
}
func (s *resourcePlanStore) close() {
for _, f := range s.held {
f.Close()
}
s.files.dir.Close()
}
func (s *resourcePlanStore) guard() error {
if s.files.guard() != nil {
return errResourcePlan
}
names := map[string]bool{planKeyName: true, planRegistryName: true, planMatrixName: true}
for name := range s.held {
names[name] = true
}
for name := range names {
f := s.held[name]
if f == nil {
g, _, e := s.files.openEntry(name)
if g != nil {
g.Close()
}
if !errors.Is(e, os.ErrNotExist) {
return errResourcePlan
}
continue
}
b, e := s.files.trustedRead(name, f)
if e != nil || !bytes.Equal(b, s.snapshots[name]) {
return errResourcePlan
}
}
return nil
}
func (s *resourcePlanStore) publish(name string, b []byte) error {
if len(b) > 65536 || s.guard() != nil {
return errResourcePlan
}
fd, e := syscall.Openat(int(s.files.dir.Fd()), name, syscall.O_RDWR|syscall.O_CREAT|syscall.O_EXCL|syscall.O_NOFOLLOW|syscall.O_CLOEXEC, 0600)
if e != nil {
return errResourcePlan
}
f := os.NewFile(uintptr(fd), name)
// Keep incomplete entries on failure: never delete a possible foreign inode.
if _, e = f.Write(b); e != nil {
f.Close()
return errResourcePlan
}
if f.Sync() != nil || s.files.entryMatches(name, f) != nil {
f.Close()
return errResourcePlan
}
read, e := s.files.trustedRead(name, f)
if e != nil || !bytes.Equal(read, b) {
f.Close()
return errResourcePlan
}
s.held[name], s.snapshots[name] = f, append([]byte(nil), b...)
if s.files.dir.Sync() != nil || s.guard() != nil {
return errResourcePlan
}
return nil
}
func (s *resourcePlanStore) writeAuthenticated(name string, v interface{}) error {
b, e := json.Marshal(v)
if e != nil {
return errResourcePlan
}
envelope, e := json.Marshal(planEnvelope{Payload: b, MAC: resourceMAC(s.files.key, b)})
if e != nil {
return errResourcePlan
}
return s.publish(name, envelope)
}
func (s *resourcePlanStore) readAuthenticated(name string, dst interface{}) error {
if s.guard() != nil {
return errResourcePlan
}
if s.held[name] == nil {
return os.ErrNotExist
}
var envelope planEnvelope
if strictPlanJSON(s.snapshots[name], &envelope) != nil || !hmac.Equal([]byte(envelope.MAC), []byte(resourceMAC(s.files.key, envelope.Payload))) || strictPlanJSON(envelope.Payload, dst) != nil {
return errResourcePlan
}
return s.guard()
}
func (s *resourcePlanStore) registry() (strategyRegistry, error) {
var r strategyRegistry
if e := s.readAuthenticated(planRegistryName, &r); e != nil {
return r, e
}
b, _ := json.Marshal(r)
canonical, e := parseStrategyRegistry(b)
if e != nil || planDigest(canonical) != planDigest(r) {
return r, errResourcePlan
}
return r, nil
}
func (s *resourcePlanStore) importDefinitions(b []byte) error {
r, e := parseStrategyRegistry(b)
if e != nil {
return e
}
return s.writeAuthenticated(planRegistryName, r)
}
func (s *resourcePlanStore) saveCompleted(p completedQualification) error {
r, e := s.registry()
if e != nil || p.seal == "" || p.seal != planDigest(p.matrix) || validateMatrix(p.matrix, r) != nil {
return errResourcePlan
}
return s.writeAuthenticated(planMatrixName, p.matrix)
}
func (s *resourcePlanStore) matrix(r strategyRegistry) (qualificationMatrix, error) {
var m qualificationMatrix
e := s.readAuthenticated(planMatrixName, &m)
if e != nil {
return m, e
}
if validateMatrix(m, r) != nil {
return m, errResourcePlan
}
return m, nil
}
func resourcePlanCLI(args []string, stdout, stderr io.Writer, d managedDeps) int {
if os.Geteuid() != 0 {
fmt.Fprintln(stderr, "blocked: root target role required")
return 78
}
if _, err := managedTargetSettings(d.lookup); err != nil || d.directory != managedTargetDirectory {
fmt.Fprintln(stderr, "blocked: managed target required")
return 78
}
if len(args) == 6 && args[0] == "qualify" && args[1] == "--config" && args[3] == "--strategies" && args[5] == "--isolated" {
return qualifyResourceCLI(args[2], args[4], stdout, stderr, d)
}
if len(args) == 3 && args[0] == "qualify" && args[1] == "--config" {
fmt.Fprintln(stderr, "unknown: qualification requires --isolated and PID 1 isolated startup; no runtime changes made")
return 78
}
if len(args) == 5 && args[0] == "qualify" && args[1] == "--config" && args[3] == "--strategies" {
fmt.Fprintln(stderr, "unknown: qualification requires --isolated and PID 1 isolated startup; no runtime changes made")
return 78
}
if len(args) == 3 && args[0] == "strategy-import" && args[1] == "--strategies" {
fd, e := syscall.Open(args[2], syscall.O_RDONLY|syscall.O_NOFOLLOW|syscall.O_NONBLOCK|syscall.O_CLOEXEC, 0)
if e != nil {
fmt.Fprintln(stderr, "configerror: definitions unavailable")
return 64
}
f := os.NewFile(uintptr(fd), "definitions")
defer f.Close()
if resourcePrivateEntry(f) != nil {
fmt.Fprintln(stderr, "configerror: definitions require owned regular 0600 file")
return 64
}
b, e := io.ReadAll(io.LimitReader(f, 65537))
current, statErr := os.Lstat(args[2])
held, heldErr := f.Stat()
if e != nil || statErr != nil || heldErr != nil || !os.SameFile(current, held) || resourcePrivateEntry(f) != nil {
return 64
}
s, e := openResourcePlanStore(d.directory, true)
if e != nil {
fmt.Fprintln(stderr, "blocked: private registry unavailable")
return 78
}
defer s.close()
if s.importDefinitions(b) != nil {
fmt.Fprintln(stderr, "blocked: invalid definitions or immutable registry already exists")
return 78
}
fmt.Fprintln(stdout, "strategy definitions imported; qualification UNKNOWN")
return 0
}
if len(args) != 3 || args[0] != "plan" || args[1] != "--config" {
fmt.Fprintln(stderr, "usage: plan --config FILE | strategy-import --strategies FILE | qualify --config FILE --strategies FILE --isolated")
return 64
}
return resourcePlanProposal(args[2], stdout, stderr, d)
}
// Internal reader separated only to test the real CLI trust boundary in a
// temporary owned directory, without touching the provisioned target.
func resourcePlanProposal(configPath string, stdout, stderr io.Writer, d managedDeps) int {
key := make([]byte, 32)
registry := strategyRegistry{Version: 1, Strategies: []strategyDefinition{}}
var matrix qualificationMatrix
s, e := openResourcePlanStore(d.directory, false)
blocked := e != nil && !errors.Is(e, os.ErrNotExist)
if e == nil {
defer s.close()
key = s.files.key
registry, e = s.registry()
if e != nil {
blocked = !errors.Is(e, os.ErrNotExist)
registry = strategyRegistry{Version: 1, Strategies: []strategyDefinition{}}
}
if e == nil {
matrix, e = s.matrix(registry)
if e != nil {
blocked = !errors.Is(e, os.ErrNotExist)
matrix = qualificationMatrix{}
}
}
}
source, e := readResourceMonitorSource(configPath, d.lookup, key)
if e != nil {
fmt.Fprintln(stderr, "configerror: resource configuration unavailable")
return 64
}
p := deriveResourcePlan(source, key, registry, matrix, time.Now())
if blocked {
p.Evidence = "blocked"
}
if s != nil && s.guard() != nil {
p = deriveResourcePlan(source, key, registry, qualificationMatrix{}, time.Now())
p.Evidence = "blocked"
}
if json.NewEncoder(stdout).Encode(p) != nil {
return 1
}
if blocked || p.Evidence == "blocked" {
return 78
}
return 0
}
+331
View File
@@ -0,0 +1,331 @@
package main
import (
"bytes"
"encoding/json"
"os"
"path/filepath"
"reflect"
"strings"
"testing"
"time"
)
func planFixture(t *testing.T) (resourceMonitorSource, []byte, strategyRegistry, qualificationMatrix, time.Time) {
t.Helper()
key := bytes.Repeat([]byte{7}, 32)
now := time.Unix(1700000000, 0)
source := resourceMonitorSource{server: "127.0.0.1:5353", proxy: "socks5h://127.0.0.1:1080", fingerprint: resourceMAC(key, []byte("config-secret")), config: resourceCheckConfig{Version: 1}}
for i, host := range []string{"a.example", "b.example", "c.example", "d.example", "e.example", "a.example"} {
service := "alpha"
group := "group-a"
if i == 2 {
service = "beta"
group = "group-b"
}
if i == 5 {
service = "gamma"
group = "group-c"
}
source.config.Services = append(source.config.Services, resourceCheckService{ID: service + string(rune('a'+i)), Resources: []resourceCheckSpec{{ID: "resource", Group: group, host: host, TLS: "both", URL: "https://" + host + "/SECRET?token=SECRET", privateConfig: "header = SECRET"}}})
}
registry := strategyRegistry{Version: 1}
for i, args := range []string{"--lua-desync=multisplit:pos=1", "--lua-desync=multidisorder:pos=1", "--lua-desync=fake:repeats=2"} {
registry.Strategies = append(registry.Strategies, strategyDefinition{ID: "s" + string(rune('1'+i)), Args: args, Selectors: strategySelectors{Transport: "tcp", Ports: "443", L7: "tls", TLS: "both"}})
}
matrix := qualificationMatrix{Version: 1, Registry: planDigest(registry), Runs: []qualificationRun{}}
for _, d := range registry.Strategies {
digest := planDigest(d)
r := qualificationRun{ID: "run-" + d.ID, Strategy: d.ID, Definition: digest, Config: source.fingerprint, Started: now.Add(-time.Minute).UnixMilli(), Completed: now.UnixMilli(), Attestation: qualificationAttestation{Collector: "owned-resource-nfq-v1", Runtime: "nfqws2-1.0.2", Definition: digest, LuaHash: strings.Repeat("a", 64), RulesHash: strings.Repeat("b", 64), WorkerArgsHash: strings.Repeat("c", 64), Queue: 200, PID: 123, StartTime: 456, Restored: true, Joined: true}}
for _, s := range source.config.Services {
for _, spec := range s.Resources {
if spec.host == "e.example" {
continue
}
pass := d.ID == "s1" && (spec.host == "a.example" || spec.host == "c.example") || d.ID == "s2" && spec.host == "b.example" || d.ID == "s3" && spec.host == "d.example"
for _, tls := range []string{"tls12", "tls13"} {
outcome := "expectation_failed"
if pass {
outcome = "pass"
}
r.Checks = append(r.Checks, qualificationCheck{Service: s.ID, Resource: spec.ID, Fingerprint: resourceRowFingerprint(key, source, spec), TLS: tls, Outcome: outcome, Attempts: 3})
}
}
}
matrix.Runs = append(matrix.Runs, r)
}
return source, key, registry, matrix, now
}
func cloneMatrix(m qualificationMatrix) qualificationMatrix {
b, _ := json.Marshal(m)
var v qualificationMatrix
json.Unmarshal(b, &v)
return v
}
func TestResourcePlanCoverage(t *testing.T) {
source, key, registry, matrix, now := planFixture(t)
p := deriveResourcePlan(source, key, registry, matrix, now)
want := [][]string{{"a.example", "c.example"}, {"b.example"}, {"d.example"}}
if len(p.Assignments) != 3 || !reflect.DeepEqual(p.Unassigned, []string{"e.example"}) {
t.Fatalf("%+v", p)
}
for i, a := range p.Assignments {
if !reflect.DeepEqual(a.Hosts, want[i]) {
t.Fatalf("%+v", p)
}
}
for i := 0; i < 100; i++ {
if !reflect.DeepEqual(p, deriveResourcePlan(source, key, registry, matrix, now)) {
t.Fatal("nondeterministic")
}
}
b, _ := json.Marshal(matrix)
out, _ := json.Marshal(p)
if strings.Contains(string(b)+string(out), "SECRET") || strings.Contains(string(b), "https://") {
t.Fatal("secret serialization")
}
// One bad URL on the same host in another service/group invalidates only a.
bad := cloneMatrix(matrix)
bad.Runs[0].Checks[len(bad.Runs[0].Checks)-1].Outcome = "expectation_failed"
changed := deriveResourcePlan(source, key, registry, bad, now)
if len(changed.Domains[0].Validated) != 0 || !reflect.DeepEqual(p.Domains[1:], changed.Domains[1:]) {
t.Fatalf("subset failure leaked: %+v", changed)
}
partial := cloneMatrix(matrix)
partial.Runs[0].Checks = partial.Runs[0].Checks[:len(partial.Runs[0].Checks)-1]
changed = deriveResourcePlan(source, key, registry, partial, now)
if len(changed.Domains[0].Validated) != 0 || len(changed.Domains[0].Unknown) != 1 {
t.Fatal("partial TLS qualified")
}
for _, mutate := range []func(*qualificationMatrix){
func(m *qualificationMatrix) { m.Runs[0].Config = strings.Repeat("c", 64) },
func(m *qualificationMatrix) {
m.Runs[0].Completed = now.Add(-25 * time.Hour).UnixMilli()
m.Runs[0].Started = m.Runs[0].Completed - 1
},
func(m *qualificationMatrix) { m.Runs[0].Attestation.Joined = false },
func(m *qualificationMatrix) { m.Runs[0].Checks = append(m.Runs[0].Checks, m.Runs[0].Checks[0]) },
func(m *qualificationMatrix) { m.Runs[1].ID = m.Runs[0].ID },
func(m *qualificationMatrix) { m.Runs[0].Definition = strings.Repeat("c", 64) },
} {
m := cloneMatrix(matrix)
mutate(&m)
if len(deriveResourcePlan(source, key, registry, m, now).Domains[0].Validated) != 0 {
t.Fatal("invalid evidence qualified")
}
}
source.fingerprint = strings.Repeat("d", 64)
if len(deriveResourcePlan(source, key, registry, matrix, now).Assignments) != 0 {
t.Fatal("changed config qualified")
}
}
func TestResourcePlanSelectorsAndAlternatives(t *testing.T) {
source, key, registry, matrix, now := planFixture(t)
registry.Strategies[1].Args = registry.Strategies[0].Args
matrix.Registry = planDigest(registry)
matrix.Runs[1].Definition = planDigest(registry.Strategies[1])
matrix.Runs[1].Attestation.Definition = matrix.Runs[1].Definition
// Add a shared alternative on c without removing original evidence.
for i := range matrix.Runs[1].Checks {
if matrix.Runs[1].Checks[i].Service == "betac" {
matrix.Runs[1].Checks[i].Outcome = "pass"
}
}
p := deriveResourcePlan(source, key, registry, matrix, now)
if len(p.Assignments) != 2 || !reflect.DeepEqual(p.Assignments[0].Hosts, []string{"a.example", "b.example", "c.example"}) || len(p.Domains[2].Validated) != 2 {
t.Fatalf("merge/alternatives: %+v", p)
}
registry.Strategies[1].Selectors.TLS = "tls12"
matrix.Registry = planDigest(registry)
matrix.Runs[1].Definition = planDigest(registry.Strategies[1])
matrix.Runs[1].Attestation.Definition = matrix.Runs[1].Definition
// Hosts a,c require both; host b only tls12.
source.config.Services[1].Resources[0].TLS = "tls12"
for i := range matrix.Runs[1].Checks {
c := &matrix.Runs[1].Checks[i]
if c.Service == "alphab" {
c.Fingerprint = resourceRowFingerprint(key, source, source.config.Services[1].Resources[0])
}
}
p = deriveResourcePlan(source, key, registry, matrix, now)
if len(p.Assignments) != 3 || p.Assignments[0].Selectors == p.Assignments[1].Selectors {
t.Fatalf("selectors merged: %+v", p)
}
}
func TestResourcePlanRegistryValidation(t *testing.T) {
_, _, r, _, _ := planFixture(t)
d := r.Strategies[0]
d.Args = " \t--payload=tls_client_hello\n--lua-desync=multisplit:pos=1 "
v, e := canonicalStrategy(d)
if e != nil || v.Args != "--payload=tls_client_hello --lua-desync=multisplit:pos=1" {
t.Fatalf("canonical: %+v %v", v, e)
}
for _, args := range []string{"--lua-init=@/opt/zapret2/lua/evil.lua", "--qnum=200", "--user=root", "--fwmark=1", "--filter-mark=1", "--lua-desync=execute:cmd=id", "--lua-desync=fake:blob=/etc/passwd", "--lua-desync=fake:blob=secret.lua", "--lua-desync=fake:repeats=1;id"} {
d.Args = args
if _, e := canonicalStrategy(d); e == nil {
t.Fatalf("accepted %s", args)
}
}
for _, b := range []string{`{"version":1,"version":1,"strategies":[]}`, `{"version":1,"strategies":[],"evidence":true}`} {
if _, e := parseStrategyRegistry([]byte(b)); e == nil {
t.Fatal("import accepted proof/duplicate")
}
}
}
func TestResourcePlanPrivateStore(t *testing.T) {
_, _, registry, matrix, _ := planFixture(t)
dir := t.TempDir()
os.Chmod(dir, 0700)
s, e := openResourcePlanStore(dir, true)
if e != nil {
t.Fatal(e)
}
defer s.close()
b, _ := json.Marshal(registry)
if s.importDefinitions(b) != nil {
t.Fatal("import")
}
if s.importDefinitions(b) == nil {
t.Fatal("clobbered immutable registry")
}
proof := completedQualification{matrix: cloneMatrix(matrix), seal: planDigest(matrix)}
proof.matrix.Runs[0].Checks[0].Outcome = "expectation_failed"
if s.saveCompleted(proof) == nil {
t.Fatal("post-seal mutation")
}
if s.saveCompleted(completedQualification{matrix: matrix}) == nil {
t.Fatal("unsealed DTO")
}
if s.saveCompleted(completedQualification{matrix: matrix, seal: planDigest(matrix)}) != nil {
t.Fatal("fixture save")
}
loaded, e := s.matrix(registry)
if e != nil || !reflect.DeepEqual(loaded, matrix) {
t.Fatalf("load: %v", e)
}
second, e := openResourcePlanStore(dir, false)
if e != nil {
t.Fatal(e)
}
second.close()
path := filepath.Join(dir, planMatrixName)
raw, e := os.ReadFile(path)
if e != nil {
t.Fatal(e)
}
raw = bytes.Replace(raw, []byte(`"outcome":"pass"`), []byte(`"outcome":"timeout"`), 1)
os.WriteFile(path, raw, 0600)
if _, e = s.matrix(registry); e == nil {
t.Fatal("held mutation accepted")
}
reopened, e := openResourcePlanStore(dir, false)
if e != nil {
t.Fatal(e)
}
defer reopened.close()
if _, e = reopened.matrix(registry); e == nil {
t.Fatal("forged HMAC accepted")
}
}
func TestResourcePlanForeignEntries(t *testing.T) {
for _, kind := range []string{"file", "symlink", "replacement"} {
t.Run(kind, func(t *testing.T) {
dir := t.TempDir()
os.Chmod(dir, 0700)
path := filepath.Join(dir, planRegistryName)
if kind == "file" {
os.WriteFile(path, []byte("FOREIGN"), 0600)
} else if kind == "symlink" {
os.Symlink("foreign", path)
}
s, e := openResourcePlanStore(dir, true)
if kind != "replacement" {
if e == nil {
s.close()
t.Fatal("adopted foreign registry without key")
}
return
}
if e != nil {
t.Fatal(e)
}
defer s.close()
_, _, r, _, _ := planFixture(t)
b, _ := json.Marshal(r)
if s.importDefinitions(b) != nil {
t.Fatal("import")
}
os.Rename(path, path+".old")
os.WriteFile(path, s.snapshots[planRegistryName], 0600)
if _, e = s.registry(); e == nil {
t.Fatal("adopted identical foreign replacement")
}
if s.importDefinitions(b) == nil {
t.Fatal("clobbered replacement")
}
})
}
}
func TestResourcePlanCLIUnknownAndForged(t *testing.T) {
dir := t.TempDir()
os.Chmod(dir, 0700)
path := filepath.Join(dir, "resources.json")
config := `{"version":1,"services":[{"id":"service","resources":[{"id":"resource","group":"metadata","kind":"http","url":"https://a.example/SECRET","method":"GET","tls":"both","timeout_ms":1000,"expect":{"statuses":[200],"min_bytes":0,"max_bytes":100}}]}]}`
if err := os.WriteFile(path, []byte(config), 0600); err != nil {
t.Fatal(err)
}
env := map[string]string{"ZAPRET_AUTOTUNE_SOCKS_PROXY": "socks5h://127.0.0.1:1080", "ZAPRET_AUTOTUNE_DNS_MODE": "native", "SS_DNS_SERVER": "1.1.1.1"}
d := managedDeps{directory: dir, lookup: func(k string) (string, bool) { v, ok := env[k]; return v, ok }}
var out, diagnostics bytes.Buffer
code := resourcePlanProposal(path, &out, &diagnostics, d)
if code != 0 {
t.Fatalf("unknown reader: %d %s", code, diagnostics.String())
}
var p resourcePlan
if json.Unmarshal(out.Bytes(), &p) != nil || p.Evidence != "unknown" || len(p.Assignments) != 0 || len(p.Unassigned) != 1 || strings.Contains(out.String(), "SECRET") {
t.Fatalf("%s", out.String())
}
s, err := openResourcePlanStore(dir, true)
if err != nil {
t.Fatal(err)
}
_, _, registry, matrix, _ := planFixture(t)
definitions, _ := json.Marshal(registry)
if s.importDefinitions(definitions) != nil {
t.Fatal("import")
}
source, err := readResourceMonitorSource(path, d.lookup, s.files.key)
if err != nil {
t.Fatal(err)
}
for i := range matrix.Runs {
r := &matrix.Runs[i]
r.Config = source.fingerprint
r.Started = time.Now().Add(-time.Minute).UnixMilli()
r.Completed = time.Now().UnixMilli()
r.Checks = nil
for _, tls := range []string{"tls12", "tls13"} {
r.Checks = append(r.Checks, qualificationCheck{Service: "service", Resource: "resource", TLS: tls, Outcome: "pass", Fingerprint: resourceRowFingerprint(s.files.key, source, source.config.Services[0].Resources[0]), Attempts: 3})
}
}
if s.saveCompleted(completedQualification{matrix: matrix, seal: planDigest(matrix)}) != nil {
t.Fatal("sealed fixture")
}
s.close()
out.Reset()
code = resourcePlanProposal(path, &out, &diagnostics, d)
if code != 0 || json.Unmarshal(out.Bytes(), &p) != nil || p.Evidence != "authenticated" || len(p.Assignments) != 1 || len(p.Domains[0].Validated) != 3 {
t.Fatalf("authenticated CLI %d %s", code, out.String())
}
// Public JSON PASS and monitor observations cannot become private proof.
if err := os.WriteFile(filepath.Join(dir, planMatrixName), []byte(`{"pass":true,"health":"HEALTHY"}`), 0600); err != nil {
t.Fatal(err)
}
out.Reset()
diagnostics.Reset()
code = resourcePlanProposal(path, &out, &diagnostics, d)
if code != 78 || !strings.Contains(out.String(), `"evidence":"blocked"`) || strings.Contains(out.String(), `"hosts":[`) {
t.Fatalf("forged CLI %d %s %s", code, out.String(), diagnostics.String())
}
}
+684
View File
@@ -0,0 +1,684 @@
package main
import (
"bytes"
"context"
"crypto/rand"
"encoding/hex"
"encoding/json"
"errors"
"fmt"
"io"
"net"
"os"
"os/exec"
"os/signal"
"path/filepath"
"reflect"
"sort"
"strconv"
"strings"
"syscall"
"time"
)
const qualificationEngine = "/opt/zapret2/nfq2/nfqws2"
var errQualificationCapacity = errors.New("qualification matrix exceeds private capacity")
// A fresh disabled baseline needs no daemon stop/config transaction. Deliberately
// refuse existing active profiles: maintenance pause/restore is not implemented.
// ponytail: isolated startup baseline only; add job-bound profile rollback before
// supporting an already active desired set, never just invoke a daemon stop.
func qualificationBaseline(dir string) error {
if resourceManagedIdentity(dir) != nil {
return errResourcePlan
}
m := &managedTransaction{Dir: dir, Name: "config"}
j, e := m.load()
if e != nil || j.State != "committed" || len(j.Candidates) != 0 {
return errResourcePlan
}
b, e := m.read("config")
if e != nil || string(b) != managedBaseline {
return errResourcePlan
}
return nil
}
func qualificationMode(config, strategies string, d managedDeps) error {
if os.Geteuid() != 0 || os.Getppid() != 1 {
return errResourcePlan
}
if _, e := managedRuntimeGuard(d); e != nil {
return e
}
cmd, e := os.ReadFile("/proc/1/cmdline")
if e != nil || string(cmd) != "/bin/sh\x00/opt/managed-target-entrypoint.sh\x00" {
return errResourcePlan
}
env, e := os.ReadFile("/proc/1/environ")
if e != nil {
return errResourcePlan
}
values := map[string]string{}
for _, s := range strings.Split(string(env), "\x00") {
k, v, _ := strings.Cut(s, "=")
values[k] = v
}
if config == "" || strategies == "" || values["ZAPRET_AUTOTUNE_QUALIFY_CONFIG"] != config || values["ZAPRET_AUTOTUNE_STRATEGY_CONFIG"] != strategies {
return errResourcePlan
}
for _, ns := range []string{"pid", "net"} {
a, e := os.Readlink("/proc/1/ns/" + ns)
b, f := os.Readlink("/proc/self/ns/" + ns)
if e != nil || f != nil || a != b {
return errResourcePlan
}
}
if !resourceProcessOwned("/proc", os.Getpid(), "/usr/local/bin/zapret-autotune", []string{"/usr/local/bin/zapret-autotune", "qualify", "--config", config, "--strategies", strategies, "--isolated"}) {
return errResourcePlan
}
dir, e := os.Open("/run/zapret-autotune-ss")
if e != nil {
return errResourcePlan
}
defer dir.Close()
ready, e := resourcePrivateFile(dir, "monitor-ready")
if e != nil {
return e
}
defer ready.Close()
b, e := io.ReadAll(io.LimitReader(ready, 32))
if e != nil || string(b) != strconv.Itoa(os.Getpid())+"\n" {
return errResourcePlan
}
return qualificationBaseline(d.directory)
}
func qualificationFilesHash() (string, error) {
engine, e := openQualificationEngine("/opt/zapret2")
if e != nil {
return "", e
}
defer engine.file.Close()
return engine.filesHash()
}
func (engine *qualificationBinary) filesHash() (string, error) {
// Fixed image files only. Bind both executable and Lua inputs before and after
// every candidate; never accept a caller-supplied native hash/version label.
if engine.validate() != nil {
return "", errResourcePlan
}
hashes := []string{engine.hash}
for _, path := range []string{filepath.Join(engine.root, "lua/zapret-lib.lua"), filepath.Join(engine.root, "lua/zapret-antidpi.lua")} {
fd, e := syscall.Open(path, syscall.O_RDONLY|syscall.O_NOFOLLOW|syscall.O_NONBLOCK|syscall.O_CLOEXEC, 0)
if e != nil {
return "", errResourcePlan
}
f := os.NewFile(uintptr(fd), path)
info, e := f.Stat()
if e != nil || !info.Mode().IsRegular() || info.Size() > 32<<20 || info.Mode().Perm()&0022 != 0 {
f.Close()
return "", errResourcePlan
}
st, ok := info.Sys().(*syscall.Stat_t)
if !ok || st.Uid != 0 {
f.Close()
return "", errResourcePlan
}
b, e := io.ReadAll(io.LimitReader(f, 32<<20+1))
current, se := os.Lstat(path)
f.Close()
if e != nil || se != nil || !os.SameFile(info, current) || int64(len(b)) != info.Size() {
return "", errResourcePlan
}
hashes = append(hashes, planDigest(b))
}
return planDigest(hashes), nil
}
func (s *resourcePlanStore) protectBaseline() error {
for _, name := range []string{"config", managedMarkerName, managedJournalName, managedBackupName} {
f, b, e := s.files.openEntry(name)
if e != nil {
return errResourcePlan
}
s.held[name], s.snapshots[name] = f, b
}
return s.guard()
}
func qualificationNoWorker() error {
if e := qualificationOnlyWorker(0); e != nil {
return qualificationPredicate(e, qualificationErrorReason(e, reasonWorkerScan))
}
occupied, e := managedAssetQueueReady(0)
if e != nil || occupied {
return qualificationPredicate(errResourcePlan, reasonQueuePresentOrUnreadable)
}
return nil
}
func (q resourceQualifier) restoreBaseline(ctx context.Context, noWorker, baseline func() error) error {
if ctx.Err() != nil {
return qualificationPredicate(errResourcePlan, reasonContext)
}
if e := q.guard(); e != nil {
return qualificationPredicate(errResourcePlan, qualificationErrorReason(e, reasonGuard))
}
if e := noWorker(); e != nil {
return qualificationPlanPredicate(e, reasonUnknown)
}
tables, e := q.runtime.nft(ctx, "list tables\n")
if e != nil {
return qualificationPredicate(errResourcePlan, reasonNFTRead)
}
if strings.Contains(tables, "table inet "+managedAssetTable) {
return qualificationPredicate(errResourcePlan, reasonNFTPresent)
}
if e := baseline(); e != nil {
return qualificationPredicate(e, reasonBaseline)
}
return nil
}
func qualificationOnlyWorker(owned int) error {
return qualificationScanWorkers(owned, os.ReadDir, os.Readlink)
}
func qualificationScanWorkers(owned int, readDir func(string) ([]os.DirEntry, error), readlink func(string) (string, error)) error {
entries, e := readDir("/proc")
if e != nil {
return &qualificationDiagnostic{Err: e, Reason: reasonWorkerScanRead, Errno: qualificationScanErrno(e), OwnedPID: owned}
}
for _, entry := range entries {
pid, e := strconv.Atoi(entry.Name())
if e != nil {
continue
}
exe, e := readlink(filepath.Join("/proc", entry.Name(), "exe"))
if e != nil {
if errors.Is(e, os.ErrNotExist) {
continue
}
return &qualificationDiagnostic{Err: errResourcePlan, Reason: reasonWorkerScanReadlink, Errno: qualificationScanErrno(e), ScanPID: pid, OwnedPID: owned}
}
if filepath.Base(exe) == "nfqws2" && entry.Name() != strconv.Itoa(owned) {
return &qualificationDiagnostic{Err: errResourcePlan, Reason: reasonWorkerScanForeign, ScanPID: pid, OwnedPID: owned}
}
}
return nil
}
func qualificationWorkerArgs(endpoints []string, args string) []string {
base := managedProbeWorkerArgs(endpoints, "")
base = append(base, "--filter-l7=tls") // Supported by pinned 1.0.2; generic caller only.
return append(base, strings.Fields(args)...)
}
// All seams are package-private and supplied by the runtime factory, not JSON.
type resourceQualifier struct {
runtime managedAssetRuntime
checks resourceCheckDeps
source func() (resourceMonitorSource, error)
restore func(context.Context) error
attest func(context.Context, int, []string, strategyDefinition) (qualificationAttestation, error)
guard func() error
now func() time.Time
filesHash func() (string, error)
close func()
}
func newResourceQualifier(config, strategies string, d managedDeps, s *resourcePlanStore) resourceQualifier {
r := newManagedAssetRuntime(d)
q := resourceQualifier{runtime: r, checks: productionResourceCheckDeps(), now: time.Now}
var engine *qualificationBinary
q.close = func() {
if engine != nil {
engine.file.Close()
}
}
q.filesHash = func() (string, error) {
if engine == nil {
var e error
engine, e = openQualificationEngine("/opt/zapret2")
if e != nil {
return "", e
}
}
return engine.filesHash()
}
start, queue := r.start, r.queue
r.start = func(args []string) (managedAssetWorker, error) {
if engine == nil || engine.validate() != nil {
return managedAssetWorker{}, errResourcePlan
}
return start(args)
}
r.queue = func(pid int) (bool, error) {
if engine == nil || engine.validate() != nil {
return false, errResourcePlan
}
ready, e := queue(pid)
// withOwnedWorker checks READY before creating the nft table.
if e == nil && ready && pid != 0 && engine.running(fmt.Sprintf("/proc/%d/exe", pid)) != nil {
return false, errResourcePlan
}
return ready, e
}
q.runtime = r
q.source = func() (resourceMonitorSource, error) { return readResourceMonitorSource(config, d.lookup, s.files.key) }
q.guard = func() error {
if e := qualificationMode(config, strategies, d); e != nil {
return qualificationPredicate(errResourcePlan, qualificationErrorReason(e, reasonMode))
}
if e := s.guard(); e != nil {
return qualificationPredicate(errResourcePlan, qualificationErrorReason(e, reasonPlanStore))
}
if e := r.guard(); e != nil {
return qualificationPredicate(e, qualificationErrorReason(e, reasonRuntime))
}
return nil
}
q.restore = func(ctx context.Context) error {
return q.restoreBaseline(ctx, qualificationNoWorker, func() error { return qualificationBaseline(d.directory) })
}
q.attest = func(ctx context.Context, pid int, args []string, definition strategyDefinition) (qualificationAttestation, error) {
if e := q.guard(); e != nil {
return qualificationAttestation{}, qualificationPredicate(errResourcePlan, qualificationErrorReason(e, reasonGuard))
}
if e := qualificationOnlyWorker(pid); e != nil {
return qualificationAttestation{}, qualificationPlanPredicate(e, reasonWorkerScan)
}
if engine == nil || engine.validate() != nil || engine.running(fmt.Sprintf("/proc/%d/exe", pid)) != nil {
return qualificationAttestation{}, qualificationPredicate(errResourcePlan, reasonEngine)
}
cmd, e := os.ReadFile(fmt.Sprintf("/proc/%d/cmdline", pid))
if e != nil || string(cmd) != strings.Join(append([]string{qualificationEngine}, args...), "\x00")+"\x00" {
return qualificationAttestation{}, qualificationPredicate(errResourcePlan, reasonArgv)
}
stat, e := os.ReadFile(fmt.Sprintf("/proc/%d/stat", pid))
if e != nil {
return qualificationAttestation{}, qualificationPredicate(e, reasonParent)
}
end := strings.LastIndex(string(stat), ") ")
if end < 0 {
return qualificationAttestation{}, qualificationPredicate(errResourcePlan, reasonParent)
}
tail := strings.Fields(string(stat)[end+2:])
if len(tail) < 2 || tail[1] != strconv.Itoa(os.Getpid()) {
return qualificationAttestation{}, qualificationPredicate(errResourcePlan, reasonParent)
}
start, e := readMonitorProcessStarttime(pid)
if e != nil {
return qualificationAttestation{}, qualificationPredicate(e, reasonStarttime)
}
ready, e := r.queue(pid)
if e != nil || !ready {
return qualificationAttestation{}, qualificationPredicate(errResourcePlan, reasonQueue)
}
hash, e := q.filesHash()
if e != nil {
return qualificationAttestation{}, qualificationPredicate(e, reasonFiles)
}
versionCtx, cancel := context.WithTimeout(ctx, 5*time.Second)
defer cancel()
var output bytes.Buffer
version := exec.CommandContext(versionCtx, engine.path, "--version")
version.Stdout = &playerBoundedWriter{dst: &output, limit: 4096}
version.Stderr = io.Discard
if version.Run() != nil {
return qualificationAttestation{}, qualificationPredicate(errResourcePlan, reasonVersion)
}
fields := strings.Fields(output.String())
valid := false
for i, f := range fields {
if f == "version" && i+1 < len(fields) && (fields[i+1] == "1.0.2" || fields[i+1] == "v1.0.2") {
valid = true
}
}
if !valid {
return qualificationAttestation{}, qualificationPredicate(errResourcePlan, reasonVersion)
}
rules, e := r.nft(ctx, "list table inet "+managedAssetTable+"\n")
if e != nil || strings.TrimSpace(rules) == "" {
return qualificationAttestation{}, qualificationPredicate(errResourcePlan, reasonRulesRead)
}
return qualificationAttestation{Collector: "owned-resource-nfq-v1", Runtime: "nfqws2-1.0.2", Definition: planDigest(definition), LuaHash: hash, RulesHash: planDigest(rules), WorkerArgsHash: planDigest(args), Queue: managedAssetQueue, PID: pid, StartTime: start}, nil
}
return q
}
func (q resourceQualifier) complete(ctx context.Context, s *resourcePlanStore, registry strategyRegistry) (completedQualification, error) {
fail := func(e error) (completedQualification, error) { return completedQualification{}, e }
if q.guard == nil || q.restore == nil || q.attest == nil || q.source == nil || q.now == nil || q.filesHash == nil || q.guard() != nil {
return fail(errResourcePlan)
}
if s.held[planMatrixName] != nil {
return fail(errResourcePlan)
}
trustedRegistry, registryErr := s.registry()
if registryErr != nil || planDigest(trustedRegistry) != planDigest(registry) {
return fail(errResourcePlan)
}
source, e := q.source()
if e != nil {
return fail(e)
}
hosts := map[string]bool{}
for _, service := range source.config.Services {
for _, spec := range service.Resources {
if !validDomain(spec.host) || net.ParseIP(spec.host) != nil || spec.host != strings.ToLower(spec.host) {
return fail(errResourcePlan)
}
hosts[spec.host] = true
}
}
endpoints := []string{}
for host := range hosts {
endpoints = append(endpoints, host)
}
sort.Strings(endpoints)
matrix := qualificationMatrix{Version: 1, Registry: planDigest(registry), Runs: []qualificationRun{}}
// Check the worst-case immutable envelope size before launching any worker.
estimate := qualificationMatrix{Version: 1, Registry: matrix.Registry}
for _, definition := range registry.Strategies {
run := qualificationRun{ID: "run-" + strings.Repeat("a", 32), Strategy: definition.ID, Definition: planDigest(definition), Config: source.fingerprint, Started: 9223372036854775807, Completed: 9223372036854775807, Attestation: qualificationAttestation{Collector: "owned-resource-nfq-v1", Runtime: "nfqws2-1.0.2", Definition: planDigest(definition), LuaHash: strings.Repeat("a", 64), RulesHash: strings.Repeat("a", 64), WorkerArgsHash: strings.Repeat("a", 64), Queue: 200, PID: 2147483647, StartTime: 9223372036854775807, Restored: true, Joined: true}}
for _, service := range source.config.Services {
for _, spec := range service.Resources {
protocols := []string{spec.TLS}
if spec.TLS == "both" {
protocols = []string{"tls12", "tls13"}
}
for _, protocol := range protocols {
run.Checks = append(run.Checks, qualificationCheck{Service: service.ID, Resource: spec.ID, TLS: protocol, Fingerprint: resourceRowFingerprint(s.files.key, source, spec), Outcome: "expectation_failed", Attempts: 3})
}
}
}
estimate.Runs = append(estimate.Runs, run)
}
b, _ := json.Marshal(estimate)
// Include the exact authenticated envelope overhead, using maximal timestamp
// widths and the longest permitted outcome. Args/selectors are fixed digests.
envelope, _ := json.Marshal(planEnvelope{Payload: b, MAC: strings.Repeat("a", 64)})
if len(envelope) > 65536 {
return fail(errQualificationCapacity)
}
stable := func() error {
if ctx.Err() != nil {
return ctx.Err()
}
if q.guard() != nil {
return errResourcePlan
}
fresh, e := q.source()
if e != nil || fresh.fingerprint != source.fingerprint {
return errResourcePlan
}
return nil
}
for candidateIndex, definition := range registry.Strategies {
if _, e := canonicalStrategy(definition); e != nil {
return fail(e)
}
if e := stable(); e != nil {
return fail(e)
}
nativeBefore, nativeErr := q.filesHash()
if nativeErr != nil || !hashOK(nativeBefore) {
return fail(errResourcePlan)
}
id := make([]byte, 16)
if _, e := rand.Read(id); e != nil {
return fail(e)
}
run := qualificationRun{ID: "run-" + hex.EncodeToString(id), Strategy: definition.ID, Definition: planDigest(definition), Config: source.fingerprint, Started: q.now().UnixMilli()}
runtime := q.runtime
originalStart := runtime.start
pid := 0
var args []string
runtime.start = func(actual []string) (managedAssetWorker, error) {
if !reflect.DeepEqual(actual, managedProbeWorkerArgs(endpoints, definition.Args)) {
return managedAssetWorker{}, errResourcePlan
}
args = qualificationWorkerArgs(endpoints, definition.Args)
worker, e := originalStart(args)
pid = worker.pid
return worker, e
}
err := runtime.withOwnedWorker(ctx, endpoints, definition.Args, func(captureCtx context.Context) error {
if !reflect.DeepEqual(args, qualificationWorkerArgs(endpoints, definition.Args)) {
return errResourcePlan
}
a, e := q.attest(captureCtx, pid, args, definition)
if e != nil {
return qualificationOperation(e, stageAttestation)
}
if a.LuaHash != nativeBefore {
return &qualificationDiagnostic{Err: errResourcePlan, Stage: stageAttestation, Reason: reasonFiles}
}
run.Attestation = a
checks := q.checks
checks.qualification = true
originalGuard := checks.guard
checks.guard = func() error {
if e := stable(); e != nil {
return e
}
if e := originalGuard(); e != nil {
return e
}
current, e := q.attest(captureCtx, pid, args, definition)
if e != nil {
return qualificationOperation(errors.Join(errResourcePlan, e), stageAttestation)
}
if current != a {
return &qualificationDiagnostic{Err: errResourcePlan, Stage: stageAttestation, Reason: attestationMismatch(a, current)}
}
return nil
}
resourceIndex := 0
for _, service := range source.config.Services {
for _, spec := range service.Resources {
resourceIndex++
protocols := []string{spec.TLS}
if spec.TLS == "both" {
protocols = []string{"tls12", "tls13"}
}
for _, protocol := range protocols {
// Unsupported TLS policy is unknown for this candidate, never PASS. Do
// still measure the full required set; planner applies selector eligibility.
cell := qualificationCheck{Service: service.ID, Resource: spec.ID, Fingerprint: resourceRowFingerprint(s.files.key, source, spec), TLS: protocol, Outcome: "pass"}
for repeat := 0; repeat < 3; repeat++ {
result, e := runResourceProtocol(captureCtx, service.ID, spec, protocol, source.server, source.proxy, checks)
if e == nil && captureCtx.Err() != nil {
e = captureCtx.Err()
}
if e == nil && result.Outcome == "cancelled" {
e = context.Canceled
}
if e != nil {
tls := uint8(12)
if protocol == "tls13" {
tls = 13
}
return &qualificationDiagnostic{Err: e, Candidate: candidateIndex + 1, Resource: resourceIndex, Attempt: repeat + 1, TLS: tls}
}
if result.Outcome != "pass" {
cell.Outcome = result.Outcome
}
cell.Attempts++
}
run.Checks = append(run.Checks, cell)
}
}
}
return stable()
})
cleanup, cancel := context.WithTimeout(context.WithoutCancel(ctx), 30*time.Second)
restoreErr := q.restore(cleanup)
if restoreErr != nil {
restoreErr = &qualificationDiagnostic{Err: restoreErr, Stage: stageRestore, Candidate: candidateIndex + 1}
}
cancel()
if err != nil || restoreErr != nil {
err = qualificationOperation(err, stageUnknown)
if diagnostic, ok := err.(*qualificationDiagnostic); ok {
diagnostic.Candidate = candidateIndex + 1
}
if restoreErr != nil {
return fail(&qualificationCleanupFailure{Err: errors.Join(err, restoreErr)})
}
return fail(errors.Join(err, restoreErr))
}
hash, hashErr := q.filesHash()
if hashErr != nil || hash != run.Attestation.LuaHash {
return fail(errResourcePlan)
}
if e := stable(); e != nil {
return fail(e)
}
run.Attestation.Joined, run.Attestation.Restored = true, true
run.Completed = q.now().UnixMilli()
matrix.Runs = append(matrix.Runs, run)
}
if e := stable(); e != nil {
return fail(e)
}
if validateMatrix(matrix, registry) != nil {
return fail(errResourcePlan)
}
return completedQualification{matrix: matrix, seal: planDigest(matrix)}, nil
}
func qualifyResourceCLI(config, strategies string, stdout, stderr io.Writer, d managedDeps) (code int) {
if qualificationMode(config, strategies, d) != nil {
fmt.Fprintln(stderr, "blocked: qualification requires PID 1 isolated startup with disabled baseline")
return 78
}
// Refuse existing/foreign proof state before even creating a singleton inode.
prior, priorErr := openResourcePlanStore(d.directory, false)
if priorErr == nil {
exists := prior.held[planMatrixName] != nil
prior.close()
if exists {
fmt.Fprintln(stderr, "blocked: immutable matrix already exists; no runtime changes made")
return 78
}
} else if !errors.Is(priorErr, os.ErrNotExist) {
fmt.Fprintln(stderr, "blocked: qualification private state")
return 78
}
lock, e := acquireManagedMonitorSingleton(d)
if e != nil {
fmt.Fprintln(stderr, "blocked: another controller owns target")
return 78
}
defer lock.close()
if qualificationNoWorker() != nil {
fmt.Fprintln(stderr, "blocked: conflicting NFQ worker/queue")
return 78
}
s, e := openResourcePlanStore(d.directory, true)
if e != nil {
fmt.Fprintln(stderr, "blocked: qualification private state")
return 78
}
defer s.close()
if s.held[planMatrixName] != nil {
fmt.Fprintln(stderr, "blocked: immutable matrix already exists; no runtime changes made")
return 78
}
if s.protectBaseline() != nil {
fmt.Fprintln(stderr, "blocked: known baseline snapshot unavailable")
return 78
}
fd, e := syscall.Open(strategies, syscall.O_RDONLY|syscall.O_NOFOLLOW|syscall.O_NONBLOCK|syscall.O_CLOEXEC, 0)
if e != nil {
return 64
}
f := os.NewFile(uintptr(fd), "definitions")
defer f.Close()
if resourcePrivateEntry(f) != nil {
return 64
}
b, e := io.ReadAll(io.LimitReader(f, 65537))
info, se := os.Lstat(strategies)
held, he := f.Stat()
if e != nil || se != nil || he != nil || !os.SameFile(info, held) {
return 64
}
registry, e := parseStrategyRegistry(b)
if e != nil {
return 64
}
if s.held[planRegistryName] == nil {
if s.importDefinitions(b) != nil {
return 78
}
} else {
existing, e := s.registry()
if e != nil || planDigest(existing) != planDigest(registry) {
return 78
}
}
signals, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
ctx, cancel := context.WithTimeout(signals, 15*time.Minute)
defer cancel()
q := newResourceQualifier(config, strategies, d, s)
defer q.close()
var sessionErr error
defer func() {
if code != 0 && code != 78 {
return
}
if e := qualificationFinalize(sessionErr, q.restore, s.guard, func() error {
return writeQualificationCompletion(qualificationCompletionDir, s.files.key, os.Getpid(), code)
}); e != nil {
writeQualificationDiagnostics(stderr, e)
code = 78
}
}()
originalGuard := q.guard
q.guard = func() error {
current, e := os.Lstat(strategies)
heldInfo, he := f.Stat()
read, re := io.ReadAll(io.NewSectionReader(f, 0, 65537))
if e != nil || he != nil || re != nil || !os.SameFile(current, heldInfo) || resourcePrivateEntry(f) != nil || !bytes.Equal(read, b) {
return errResourcePlan
}
return originalGuard()
}
proof, e := q.complete(ctx, s, registry)
sessionErr = e
if errors.Is(e, errQualificationCapacity) {
fmt.Fprintln(stderr, "unknown: qualification matrix exceeds 64 KiB capacity; no worker launched")
return 64
}
if e != nil || ctx.Err() != nil {
writeQualificationDiagnostics(stderr, e)
fmt.Fprintln(stderr, "unknown: qualification incomplete or ownership/cleanup blocked; no completed matrix published")
if signals.Err() != nil && managedCancellationOnly(e, signals.Err()) {
return 0
}
return 78
}
fresh, sourceErr := q.source()
if sourceErr != nil || q.guard() != nil || ctx.Err() != nil {
fmt.Fprintln(stderr, "unknown: source changed before publication")
return 78
}
for _, run := range proof.matrix.Runs {
if run.Config != fresh.fingerprint {
fmt.Fprintln(stderr, "unknown: source changed before publication")
return 78
}
}
if s.saveCompleted(proof) != nil {
fmt.Fprintln(stderr, "blocked: completed matrix publication refused")
return 78
}
return resourcePlanProposal(config, stdout, stderr, d)
}
@@ -0,0 +1,133 @@
package main
import (
"io"
"os"
"path/filepath"
"syscall"
)
// Only the packaged nfq2 alias is followed. All target parents and the final
// binary remain no-symlink, root-owned and not group/other writable. As with
// the runtime contract, a malicious root racing checks is outside the boundary.
type qualificationBinary struct {
root, path string
file *os.File
info, alias os.FileInfo
hash string
}
func qualificationTrusted(info os.FileInfo) bool {
st, ok := info.Sys().(*syscall.Stat_t)
return ok && st.Uid == 0 && info.Mode().Perm()&0022 == 0
}
func qualificationParents(path, boundary string) error {
for dir := filepath.Dir(path); ; dir = filepath.Dir(dir) {
info, e := os.Lstat(dir)
if e != nil || !info.IsDir() || !qualificationTrusted(info) {
return errResourcePlan
}
if dir == boundary {
return nil
}
if dir == "/" {
return errResourcePlan
}
}
}
func qualificationEnginePath(root string) (string, os.FileInfo, error) {
alias := filepath.Join(root, "nfq2/nfqws2")
boundary := root // Private fixture seam; production also checks /opt and /.
if root == "/opt/zapret2" {
boundary = "/"
}
if qualificationParents(alias, boundary) != nil {
return "", nil, errResourcePlan
}
info, e := os.Lstat(alias)
if e != nil || info.Mode()&os.ModeSymlink == 0 {
return "", nil, errResourcePlan
}
st, ok := info.Sys().(*syscall.Stat_t)
if !ok || st.Uid != 0 {
return "", nil, errResourcePlan
}
link, e := os.Readlink(alias)
if e != nil {
return "", nil, errResourcePlan
}
// Check the raw spelling before Join can erase symlink/.. traversal. Only
// the platform observed in the pinned qualification package is supported.
if link != "../binaries/linux-x86_64/nfqws2" {
return "", nil, errResourcePlan
}
path := filepath.Join(root, "binaries/linux-x86_64/nfqws2")
if qualificationParents(path, boundary) != nil {
return "", nil, errResourcePlan
}
return path, info, nil
}
func qualificationBinaryHash(f *os.File) (os.FileInfo, string, error) {
info, e := f.Stat()
if e != nil || !info.Mode().IsRegular() || !qualificationTrusted(info) || info.Mode().Perm()&0111 == 0 || info.Size() > 32<<20 {
return nil, "", errResourcePlan
}
b, e := io.ReadAll(io.NewSectionReader(f, 0, 32<<20+1))
if e != nil || int64(len(b)) != info.Size() {
return nil, "", errResourcePlan
}
return info, planDigest(b), nil
}
func openQualificationEngine(root string) (*qualificationBinary, error) {
path, alias, e := qualificationEnginePath(root)
if e != nil {
return nil, e
}
fd, e := syscall.Open(path, syscall.O_RDONLY|syscall.O_NOFOLLOW|syscall.O_NONBLOCK|syscall.O_CLOEXEC, 0)
if e != nil {
return nil, errResourcePlan
}
f := os.NewFile(uintptr(fd), path)
info, hash, e := qualificationBinaryHash(f)
b := &qualificationBinary{root: root, path: path, alias: alias, file: f, info: info, hash: hash}
if e != nil || b.validate() != nil {
f.Close()
return nil, errResourcePlan
}
return b, nil
}
func (b *qualificationBinary) validate() error {
path, alias, e := qualificationEnginePath(b.root)
if e != nil || path != b.path || !os.SameFile(alias, b.alias) {
return errResourcePlan
}
current, e := os.Lstat(path)
if e != nil || !current.Mode().IsRegular() || !os.SameFile(current, b.info) || !qualificationTrusted(current) {
return errResourcePlan
}
info, hash, e := qualificationBinaryHash(b.file)
if e != nil || !os.SameFile(info, b.info) || hash != b.hash {
return errResourcePlan
}
return nil
}
func (b *qualificationBinary) running(procExe string) error {
// /proc/PID/exe is a kernel-owned link, not a catalogue path. Compare the
// opened running inode and bytes, never the alias spelling from readlink.
f, e := os.Open(procExe)
if e != nil {
return errResourcePlan
}
defer f.Close()
info, hash, e := qualificationBinaryHash(f)
if e != nil || !os.SameFile(info, b.info) || hash != b.hash {
return errResourcePlan
}
return nil
}
@@ -0,0 +1,265 @@
package main
import (
"context"
"errors"
"fmt"
"os"
"os/exec"
"path/filepath"
"testing"
)
func qualificationBinaryFixture(t *testing.T) (string, string, string) {
t.Helper()
if os.Geteuid() != 0 {
t.Skip("root-owned package fixture requires root; run with sudo")
}
root := t.TempDir()
for _, dir := range []string{"nfq2", "binaries/linux-x86_64", "lua"} {
if e := os.MkdirAll(filepath.Join(root, dir), 0755); e != nil {
t.Fatal(e)
}
}
path := filepath.Join(root, "binaries/linux-x86_64/nfqws2")
for _, name := range []string{path, filepath.Join(root, "lua/zapret-lib.lua"), filepath.Join(root, "lua/zapret-antidpi.lua")} {
if e := os.WriteFile(name, []byte("fixed package bytes"), 0755); e != nil {
t.Fatal(e)
}
}
alias := filepath.Join(root, "nfq2/nfqws2")
if e := os.Symlink("../binaries/linux-x86_64/nfqws2", alias); e != nil {
t.Fatal(e)
}
return root, path, alias
}
func TestQualificationPackagedAlias(t *testing.T) {
root, path, _ := qualificationBinaryFixture(t)
b, e := openQualificationEngine(root)
if e != nil {
t.Fatal(e)
}
defer b.file.Close()
want := planDigest([]string{planDigest([]byte("fixed package bytes")), planDigest([]byte("fixed package bytes")), planDigest([]byte("fixed package bytes"))})
if hash, e := b.filesHash(); e != nil || hash != want || b.path != path {
t.Fatal("packaged alias hash", hash, e)
}
proc := filepath.Join(root, "proc-exe")
if e := os.Symlink(path, proc); e != nil {
t.Fatal(e)
}
if b.running(proc) != nil {
t.Fatal("canonical running inode rejected")
}
other := filepath.Join(root, "other")
if e := os.WriteFile(other, []byte("fixed package bytes"), 0755); e != nil {
t.Fatal(e)
}
if b.running(other) == nil {
t.Fatal("same hash, different running inode accepted")
}
}
func TestQualificationPackagedAliasRejects(t *testing.T) {
for _, mode := range []string{"escape", "final-symlink", "writable", "untrusted", "parent-writable", "parent-symlink", "missing"} {
t.Run(mode, func(t *testing.T) {
root, path, alias := qualificationBinaryFixture(t)
var e error
switch mode {
case "escape":
e = os.Remove(alias)
if e == nil {
e = os.Symlink("/bin/nfqws2", alias)
}
case "final-symlink":
e = os.Remove(path)
if e == nil {
e = os.Symlink("/bin/true", path)
}
case "writable":
e = os.Chmod(path, 0775)
case "untrusted":
e = os.Chown(path, 1000, 1000)
case "parent-writable":
e = os.Chmod(filepath.Dir(path), 0777)
case "parent-symlink":
dir := filepath.Dir(path)
e = os.Rename(dir, dir+"-real")
if e == nil {
e = os.Symlink(dir+"-real", dir)
}
case "missing":
e = os.Remove(path)
}
if e != nil {
t.Fatal(e)
}
if b, e := openQualificationEngine(root); e == nil {
b.file.Close()
t.Fatal("unsafe package accepted")
}
})
}
}
func TestQualificationPackagedAliasRawSpelling(t *testing.T) {
for _, link := range []string{
"../binaries/linux-x86_64/./nfqws2",
"../binaries//linux-x86_64/nfqws2",
"../binaries/linux-x86_64/../linux-x86_64/nfqws2",
"../binaries/unsupported/nfqws2",
"../binaries/linux-x86_64/nested/nfqws2",
} {
t.Run(link, func(t *testing.T) {
root, _, alias := qualificationBinaryFixture(t)
if e := os.Remove(alias); e != nil {
t.Fatal(e)
}
if e := os.Symlink(link, alias); e != nil {
t.Fatal(e)
}
if b, e := openQualificationEngine(root); e == nil {
b.file.Close()
t.Fatal("non-package spelling accepted")
}
})
}
}
func TestQualificationConcealedTraversalBeforeWorker(t *testing.T) {
root, _, alias := qualificationBinaryFixture(t)
attacker := t.TempDir()
marker := filepath.Join(attacker, "executed")
for _, dir := range []string{"subdir", "linux-x86_64"} {
if e := os.Mkdir(filepath.Join(attacker, dir), 0755); e != nil {
t.Fatal(e)
}
}
outside := filepath.Join(attacker, "linux-x86_64/nfqws2")
if e := os.WriteFile(outside, []byte(fmt.Sprintf("#!/bin/sh\n: > %q\n", marker)), 0755); e != nil {
t.Fatal(e)
}
if e := os.Symlink(filepath.Join(attacker, "subdir"), filepath.Join(root, "binaries/redirect")); e != nil {
t.Fatal(e)
}
if e := os.Remove(alias); e != nil {
t.Fatal(e)
}
if e := os.Symlink("../binaries/redirect/../linux-x86_64/nfqws2", alias); e != nil {
t.Fatal(e)
}
resolved, e := os.Stat(alias)
outsideInfo, outsideErr := os.Stat(outside)
if e != nil || outsideErr != nil || !os.SameFile(resolved, outsideInfo) {
t.Fatal("fixture must resolve outside package", e, outsideErr)
}
q, s, registry, events, calls := qualifierFixture(t)
q.filesHash = func() (string, error) {
b, e := openQualificationEngine(root)
if e != nil {
return "", e
}
defer b.file.Close()
return b.filesHash()
}
starts := 0
q.runtime.start = func([]string) (managedAssetWorker, error) {
starts++
if e := exec.Command(alias).Run(); e != nil {
t.Fatal(e)
}
return managedAssetWorker{}, errResourcePlan
}
proof, e := q.complete(context.Background(), s, registry)
if !errors.Is(e, errResourcePlan) || starts != 0 || *calls != 0 || len(*events) != 0 || proof.seal != "" {
t.Fatalf("traversal rejection too late: err=%v starts=%d calls=%d events=%v", e, starts, *calls, *events)
}
if _, e := os.Stat(marker); !errors.Is(e, os.ErrNotExist) {
t.Fatal("outside executable ran", e)
}
}
func TestQualificationPackagedAliasProcExe(t *testing.T) {
root, path, alias := qualificationBinaryFixture(t)
bytes, e := os.ReadFile("/bin/sleep")
if e != nil {
t.Fatal(e)
}
if e := os.WriteFile(path, bytes, 0755); e != nil {
t.Fatal(e)
}
b, e := openQualificationEngine(root)
if e != nil {
t.Fatal(e)
}
defer b.file.Close()
cmd := exec.Command(alias, "60")
if e := cmd.Start(); e != nil {
t.Fatal(e)
}
defer func() { cmd.Process.Kill(); cmd.Wait() }()
if b.running(fmt.Sprintf("/proc/%d/exe", cmd.Process.Pid)) != nil {
t.Fatal("real packaged worker inode rejected")
}
if _, e := b.filesHash(); e != nil {
t.Fatal("valid pinned alias changed after start", e)
}
if b.running("/proc/self/exe") == nil {
t.Fatal("foreign running executable accepted")
}
}
func TestQualificationPackagedAliasMutationBeforeStart(t *testing.T) {
for _, mode := range []string{"resolution", "alias-replaced", "binary-replaced", "bytes", "lua-symlink", "lua-writable"} {
t.Run(mode, func(t *testing.T) {
root, path, alias := qualificationBinaryFixture(t)
b, e := openQualificationEngine(root)
if e != nil {
t.Fatal(e)
}
defer b.file.Close()
if _, e := b.filesHash(); e != nil {
t.Fatal(e)
}
switch mode {
case "resolution", "alias-replaced":
link := "../binaries/linux-x86_64/nfqws2"
if mode == "resolution" {
link = "../binaries/other/nfqws2"
if e := os.Mkdir(filepath.Join(root, "binaries/other"), 0755); e != nil {
t.Fatal(e)
}
if e := os.WriteFile(filepath.Join(root, "binaries/other/nfqws2"), []byte("fixed package bytes"), 0755); e != nil {
t.Fatal(e)
}
}
e = os.Rename(alias, alias+"-old")
if e == nil {
e = os.Symlink(link, alias)
}
case "binary-replaced":
e = os.Rename(path, path+"-old")
if e == nil {
e = os.WriteFile(path, []byte("fixed package bytes"), 0755)
}
case "bytes":
e = os.WriteFile(path, []byte("mutated package"), 0755)
case "lua-symlink":
lua := filepath.Join(root, "lua/zapret-lib.lua")
e = os.Remove(lua)
if e == nil {
e = os.Symlink(path, lua)
}
case "lua-writable":
e = os.Chmod(filepath.Join(root, "lua/zapret-lib.lua"), 0666)
}
if e != nil {
t.Fatal(e)
}
if _, e := b.filesHash(); e == nil {
t.Fatal("changed package accepted before start")
}
})
}
}
@@ -0,0 +1,178 @@
package main
import (
"bytes"
"context"
"errors"
"io"
"os"
"strconv"
"strings"
"syscall"
"time"
)
const qualificationCompletionName = "qualification-completed.json"
const qualificationCompletionDir = "/run/zapret-autotune-ss"
// This acknowledges controller cleanup, NOT candidate qualification. It cannot
// authorize a PASS, matrix write or acceptance of ordinary monitor exit 78.
type qualificationCompletion struct {
Version int `json:"version"`
PID int `json:"pid"`
Code int `json:"code"`
CleanupComplete bool `json:"cleanup_complete"`
}
type qualificationCleanupFailure struct{ Err error }
func (e *qualificationCleanupFailure) Error() string { return "qualification cleanup unverified" }
func (e *qualificationCleanupFailure) Unwrap() error { return e.Err }
func qualificationCompletionStore(path string, key []byte) (*resourcePlanStore, error) {
if managedDirectory(path) != nil || len(key) != 32 {
return nil, errResourcePlan
}
fd, e := syscall.Open(path, syscall.O_RDONLY|syscall.O_DIRECTORY|syscall.O_NOFOLLOW|syscall.O_CLOEXEC, 0)
if e != nil {
return nil, e
}
s := &resourcePlanStore{files: &resourceMonitorStore{path: path, dir: os.NewFile(uintptr(fd), "qualification completion directory"), key: append([]byte(nil), key...)}, held: map[string]*os.File{}, snapshots: map[string][]byte{}}
f, b, e := s.files.openEntry(qualificationCompletionName)
if e == nil {
s.held[qualificationCompletionName], s.snapshots[qualificationCompletionName] = f, b
} else if !errors.Is(e, os.ErrNotExist) {
s.close()
return nil, errResourcePlan
}
return s, nil
}
func writeQualificationCompletion(path string, key []byte, pid, code int) error {
if pid <= 1 || (code != 0 && code != 78) {
return errResourcePlan
}
s, e := qualificationCompletionStore(path, key)
if e != nil {
return e
}
defer s.close()
return s.writeAuthenticated(qualificationCompletionName, qualificationCompletion{Version: 1, PID: pid, Code: code, CleanupComplete: true})
}
func readQualificationCompletion(path string, key []byte, pid, code int) error {
s, e := qualificationCompletionStore(path, key)
if e != nil {
return e
}
defer s.close()
var ack qualificationCompletion
if s.readAuthenticated(qualificationCompletionName, &ack) != nil || ack.Version != 1 || ack.PID != pid || pid <= 1 || ack.Code != code || (code != 0 && code != 78) || !ack.CleanupComplete {
return errResourcePlan
}
return nil
}
func qualificationCleanupVerified(err error, restore func(context.Context) error) bool {
var ownedCleanup *managedAssetCleanupFailure
var restoration *qualificationCleanupFailure
if errors.As(err, &ownedCleanup) || errors.As(err, &restoration) {
return false
}
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
return restore(ctx) == nil
}
// Same cleanup gate and ordering as the controller's completion defer. Keeping
// operations separate makes ACK refusal diagnosable without exposing errors.
func qualificationFinalize(err error, restore func(context.Context) error, guard, ack func() error) error {
var restoreErr error
if !qualificationCleanupVerified(err, func(ctx context.Context) error {
restoreErr = restore(ctx)
return restoreErr
}) {
if restoreErr == nil {
return qualificationOperation(qualificationPredicate(errResourcePlan, reasonPriorCleanupVeto), stageRestore)
}
return qualificationOperation(restoreErr, stageRestore)
}
if e := guard(); e != nil {
return qualificationOperation(e, stageFinalStoreGuard)
}
return qualificationOperation(ack(), stageACKWrite)
}
func qualificationCompletionCLI(args []string, d managedDeps) int {
if len(args) != 2 || os.Geteuid() != 0 || os.Getppid() != 1 {
return 78
}
pid, e := strconv.Atoi(args[0])
code, ce := strconv.Atoi(args[1])
if e != nil || ce != nil || strconv.Itoa(pid) != args[0] || strconv.Itoa(code) != args[1] {
return 78
}
cmd, e := os.ReadFile("/proc/1/cmdline")
if e != nil || !bytes.Equal(cmd, []byte("/bin/sh\x00/opt/managed-target-entrypoint.sh\x00")) {
return 78
}
env, e := os.ReadFile("/proc/1/environ")
if e != nil {
return 78
}
values := map[string]string{}
for _, field := range strings.Split(string(env), "\x00") {
k, v, _ := strings.Cut(field, "=")
values[k] = v
}
if values["ZAPRET_AUTOTUNE_QUALIFY_CONFIG"] == "" || values["ZAPRET_AUTOTUNE_STRATEGY_CONFIG"] == "" {
return 78
}
if _, e := managedRuntimeGuard(d); e != nil {
return 78
}
lock, e := acquireManagedMonitorSingleton(d)
if e != nil {
return 78
}
defer lock.close()
s, e := openResourcePlanStore(d.directory, false)
if e != nil {
return 78
}
defer s.close()
if s.protectBaseline() != nil || qualificationBaseline(d.directory) != nil {
return 78
}
dns, e := parseManagedDNSConfig(d.lookup)
if e != nil || verifyResourceDNSRuntime(dns) != nil || qualificationNoWorker() != nil {
return 78
}
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
defer cancel()
r := newManagedAssetRuntime(d)
tables, e := r.nft(ctx, "list tables\n")
if e != nil || strings.Contains(tables, "table inet "+managedAssetTable) {
return 78
}
// The private session ACK ties completion to this PID 1 launch, not a stale
// target-volume status. The completion itself lives only on this boot's tmpfs.
dir, e := os.Open(qualificationCompletionDir)
if e != nil {
return 78
}
defer dir.Close()
f, e := resourcePrivateFile(dir, "monitor-ready")
if e != nil {
return 78
}
defer f.Close()
b, e := io.ReadAll(io.LimitReader(f, 32))
if e != nil || string(b) != strconv.Itoa(pid)+"\n" {
return 78
}
if readQualificationCompletion(qualificationCompletionDir, s.files.key, pid, code) != nil || s.guard() != nil {
return 78
}
return 0
}
@@ -0,0 +1,102 @@
package main
import (
"bytes"
"context"
"errors"
"os"
"path/filepath"
"strings"
"testing"
)
func TestResourceQualifierCompletionOwnership(t *testing.T) {
dir := t.TempDir()
if e := os.Chmod(dir, 0700); e != nil {
t.Fatal(e)
}
key := bytes.Repeat([]byte{9}, 32)
if e := writeQualificationCompletion(dir, key, 123, 78); e != nil {
t.Fatal(e)
}
if e := readQualificationCompletion(dir, key, 123, 78); e != nil {
t.Fatal(e)
}
for _, mismatch := range []struct {
pid, code int
key []byte
}{{124, 78, key}, {123, 0, key}, {123, 78, bytes.Repeat([]byte{8}, 32)}} {
if readQualificationCompletion(dir, mismatch.key, mismatch.pid, mismatch.code) == nil {
t.Fatal("accepted foreign launch/status/key")
}
}
if writeQualificationCompletion(dir, key, 123, 78) == nil {
t.Fatal("clobbered write-once completion")
}
path := filepath.Join(dir, qualificationCompletionName)
raw, e := os.ReadFile(path)
if e != nil {
t.Fatal(e)
}
if e := os.WriteFile(path, bytes.Replace(raw, []byte(`"cleanup_complete":true`), []byte(`"cleanup_complete":false`), 1), 0600); e != nil {
t.Fatal(e)
}
if readQualificationCompletion(dir, key, 123, 78) == nil {
t.Fatal("accepted forged completion")
}
}
func TestResourceQualifierCompletionCleanupVeto(t *testing.T) {
for _, tc := range []struct {
err error
safe bool
}{
{nil, true}, {errors.New("source changed, proof UNKNOWN"), true}, {context.Canceled, true},
{&managedRecoveryFatal{Err: &managedAssetCleanupFailure{Err: errors.New("worker join failed")}}, false},
{errors.Join(context.Canceled, &qualificationCleanupFailure{Err: errors.New("baseline restoration failed")}), false},
} {
called := false
got := qualificationCleanupVerified(tc.err, func(context.Context) error { called = true; return nil })
if got != tc.safe || called != tc.safe {
t.Fatalf("cleanup ACK=%v called=%v for %T", got, called, tc.err)
}
}
if qualificationCleanupVerified(nil, func(context.Context) error { return errors.New("live baseline guard failed") }) {
t.Fatal("acknowledged unverified cleanup")
}
}
func TestResourceQualifierCompletionForeignFile(t *testing.T) {
dir := t.TempDir()
os.Chmod(dir, 0700)
path := filepath.Join(dir, qualificationCompletionName)
if e := os.WriteFile(path, []byte("FOREIGN"), 0600); e != nil {
t.Fatal(e)
}
key := bytes.Repeat([]byte{9}, 32)
if writeQualificationCompletion(dir, key, 123, 78) == nil || readQualificationCompletion(dir, key, 123, 78) == nil {
t.Fatal("adopted foreign completion")
}
b, e := os.ReadFile(path)
if e != nil || string(b) != "FOREIGN" {
t.Fatal("changed foreign completion")
}
}
func TestResourceQualifierOwnedCleanupFailureCannotAcknowledge(t *testing.T) {
q, s, r, _, _ := qualifierFixture(t)
nft := q.runtime.nft
q.runtime.nft = func(ctx context.Context, input string) (string, error) {
if strings.HasPrefix(input, "delete table") {
return "", errors.New("owned nft cleanup failed")
}
return nft(ctx, input)
}
proof, err := q.complete(context.Background(), s, r)
if err == nil || proof.seal != "" {
t.Fatal("unsafe lifecycle sealed proof")
}
if qualificationCleanupVerified(err, q.restore) {
t.Fatal("unsafe owned cleanup authorized completed ACK")
}
}
@@ -0,0 +1,332 @@
package main
import (
"context"
"encoding/json"
"errors"
"io"
"net"
"os"
"os/exec"
"strconv"
"strings"
"testing"
"time"
)
func qualifierFixture(t *testing.T, candidates ...int) (resourceQualifier, *resourcePlanStore, strategyRegistry, *[]string, *int) {
t.Helper()
dir := t.TempDir()
os.Chmod(dir, 0700)
store, e := openResourcePlanStore(dir, true)
if e != nil {
t.Fatal(e)
}
t.Cleanup(store.close)
raw := `{"version":1,"services":[{"id":"one","resources":[{"id":"good","group":"a","kind":"http","url":"https://a.example/good?SECRET","method":"GET","tls":"both","timeout_ms":1000,"expect":{"statuses":[200],"min_bytes":1,"max_bytes":1000}}]},{"id":"two","resources":[{"id":"other","group":"b","kind":"http","url":"https://a.example/other?SECRET","method":"GET","tls":"both","timeout_ms":1000,"expect":{"statuses":[200],"min_bytes":1,"max_bytes":1000}}]}]}`
cfg, e := parseResourceChecks([]byte(raw), func(string) (string, bool) { return "", false })
if e != nil {
t.Fatal(e)
}
source := resourceMonitorSource{config: cfg, server: "127.0.0.1:5353", proxy: "socks5h://127.0.0.1:1080", fingerprint: resourceMAC(store.files.key, []byte(raw))}
_, _, registry, _, _ := planFixture(t)
count := 1
if len(candidates) > 0 {
count = candidates[0]
}
registry.Strategies = registry.Strategies[:count]
b, _ := json.Marshal(registry)
if store.importDefinitions(b) != nil {
t.Fatal("registry import")
}
runtime, events, _ := assetRuntimeFixture(t)
calls := 0
q := resourceQualifier{runtime: runtime, source: func() (resourceMonitorSource, error) { return source, nil }, guard: func() error { return nil }, now: time.Now, filesHash: func() (string, error) { return strings.Repeat("a", 64), nil }}
q.restore = func(ctx context.Context) error { *events = append(*events, "restore"); return ctx.Err() }
q.attest = func(_ context.Context, pid int, args []string, d strategyDefinition) (qualificationAttestation, error) {
if !strings.Contains(strings.Join(args, " "), "--filter-l7=tls") {
t.Fatal("generic TLS selector not enforced")
}
*events = append(*events, "attest")
return qualificationAttestation{Collector: "owned-resource-nfq-v1", Runtime: "nfqws2-1.0.2", Definition: planDigest(d), LuaHash: strings.Repeat("a", 64), RulesHash: planDigest(managedAssetRules), WorkerArgsHash: planDigest(args), Queue: 200, PID: pid, StartTime: 456}, nil
}
q.checks = resourceCheckDeps{guard: func() error { return nil }, resolve: func(context.Context, string, string) ([]net.IP, error) { return []net.IP{net.ParseIP("8.8.8.8")}, nil }, runner: func(_ context.Context, args []string, stdin io.Reader, out, diagnostics io.Writer) error {
calls++
*events = append(*events, "capture")
b, e := io.ReadAll(stdin)
if e != nil {
return e
}
if !strings.Contains(string(b), "connect-to =") || strings.Contains(strings.Join(args, " "), "SECRET") {
t.Fatal("private pin/argv violated")
}
raw, e := strconv.Unquote(strings.TrimPrefix(strings.Split(string(b), "\n")[0], "url = "))
if e != nil {
return e
}
protocol := "tls12"
if strings.Contains(strings.Join(args, " "), "--tlsv1.3") {
protocol = "tls13"
}
configuredWire(raw, "GET", protocol, "pass", out, diagnostics)
return nil
}}
return q, store, registry, events, &calls
}
func TestResourceQualifierCompletedProducer(t *testing.T) {
q, s, r, events, calls := qualifierFixture(t)
proof, e := q.complete(context.Background(), s, r)
if e != nil {
t.Fatal(e)
}
if *calls != 12 || len(proof.matrix.Runs) != 1 || len(proof.matrix.Runs[0].Checks) != 4 {
t.Fatalf("real repeats %d %+v", *calls, proof)
}
for _, c := range proof.matrix.Runs[0].Checks {
if c.Outcome != "pass" || c.Attempts != 3 {
t.Fatalf("%+v", c)
}
}
last := -1
for _, event := range []string{"start", "ready", "activate", "attest", "capture", "delete", "join", "restore"} {
found := -1
for i := last + 1; i < len(*events); i++ {
if (*events)[i] == event {
found = i
break
}
}
if found < 0 {
t.Fatalf("ordering: %v", *events)
}
last = found
}
if !proof.matrix.Runs[0].Attestation.Joined || !proof.matrix.Runs[0].Attestation.Restored {
t.Fatal("premature seal")
}
if s.saveCompleted(proof) != nil {
t.Fatal("producer publication")
}
before := *calls
if _, e := q.complete(context.Background(), s, r); e == nil || *calls != before {
t.Fatal("retested immutable matrix")
}
}
func TestResourceQualifierCapacityBeforeWorkers(t *testing.T) {
q, _, registry, events, calls := qualifierFixture(t)
dir := t.TempDir()
if e := os.Chmod(dir, 0700); e != nil {
t.Fatal(e)
}
s, e := openResourcePlanStore(dir, true)
if e != nil {
t.Fatal(e)
}
defer s.close()
template := registry.Strategies[0]
registry.Strategies = nil
for i := 0; i < 32; i++ {
definition := template
definition.ID = "strategy-" + strconv.Itoa(i)
registry.Strategies = append(registry.Strategies, definition)
}
b, _ := json.Marshal(registry)
registry, e = parseStrategyRegistry(b)
if e != nil {
t.Fatal(e)
}
if e = s.importDefinitions(b); e != nil {
t.Fatal(e)
}
source, e := q.source()
if e != nil {
t.Fatal(e)
}
spec := source.config.Services[0].Resources[0]
cfg := resourceCheckConfig{Version: 1, Services: []resourceCheckService{{ID: "service"}}}
for i := 0; i < 64; i++ {
resource := spec
resource.ID = "resource-" + strconv.Itoa(i)
cfg.Services[0].Resources = append(cfg.Services[0].Resources, resource)
}
b, _ = json.Marshal(cfg)
cfg, e = parseResourceChecks(b, func(string) (string, bool) { return "", false })
if e != nil {
t.Fatal(e)
}
source.config = cfg
source.fingerprint = resourceMAC(s.files.key, b)
q.source = func() (resourceMonitorSource, error) { return source, nil }
starts := 0
start := q.runtime.start
q.runtime.start = func(args []string) (managedAssetWorker, error) { starts++; return start(args) }
proof, e := q.complete(context.Background(), s, registry)
if !errors.Is(e, errQualificationCapacity) || starts != 0 || *calls != 0 || len(*events) != 0 || proof.seal != "" || s.held[planMatrixName] != nil {
t.Fatalf("capacity rejection too late: err=%v starts=%d calls=%d events=%v", e, starts, *calls, *events)
}
}
func TestResourceQualifierSequentialCandidates(t *testing.T) {
q, s, r, events, calls := qualifierFixture(t, 2)
proof, err := q.complete(context.Background(), s, r)
if err != nil {
t.Fatal(err)
}
if *calls != 24 || len(proof.matrix.Runs) != 2 {
t.Fatal("incomplete sequential batch", *calls)
}
stages := strings.Join(*events, ",")
if strings.Count(stages, "start") != 2 || strings.Count(stages, "restore") != 2 || !strings.Contains(stages, "delete,join,restore,inventory,start") {
t.Fatal("next candidate began before previous cleanup", stages)
}
if s.saveCompleted(proof) != nil {
t.Fatal("sequential producer publication")
}
}
func TestResourceQualifierNegativeAndUnknown(t *testing.T) {
for _, mode := range []string{"negative", "status500", "tls-peer", "timeout", "timeout-infrastructure", "infrastructure", "guard", "cancel", "source", "restore", "hash", "mutation"} {
t.Run(mode, func(t *testing.T) {
q, s, r, _, calls := qualifierFixture(t)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
original := q.checks.runner
if strings.HasPrefix(mode, "timeout") {
oldSource := q.source
source, e := oldSource()
if e != nil {
t.Fatal(e)
}
for i := range source.config.Services {
for j := range source.config.Services[i].Resources {
source.config.Services[i].Resources[j].TimeoutMS = 100
}
}
q.source = func() (resourceMonitorSource, error) { return source, nil }
}
q.checks.runner = func(ctx context.Context, args []string, stdin io.Reader, out, diag io.Writer) error {
if mode == "tls-peer" {
(*calls)++
return exec.CommandContext(ctx, "/bin/sh", "-c", "exit 35").Run()
}
if strings.HasPrefix(mode, "timeout") {
(*calls)++
<-ctx.Done()
if mode == "timeout-infrastructure" {
return errors.New("local safety failure coincident with timeout")
}
return ctx.Err()
}
if mode == "infrastructure" {
return errors.New("unknown local collector infrastructure")
}
if mode == "cancel" {
cancel()
return context.Canceled
}
if (mode == "negative" || mode == "status500") && *calls == 6 { // First repeat of other URL; still do all repeats.
*calls++
b, e := io.ReadAll(stdin)
if e != nil {
return e
}
raw, e := strconv.Unquote(strings.TrimPrefix(strings.Split(string(b), "\n")[0], "url = "))
if e != nil {
return e
}
wireMode := "redirect"
if mode == "status500" {
wireMode = "status500"
}
configuredWire(raw, "GET", "tls12", wireMode, out, diag)
return nil
}
return original(ctx, args, stdin, out, diag)
}
if mode == "guard" {
q.checks.guard = func() error { return errors.New("ownership lost") }
}
if mode == "source" {
source := q.source
q.source = func() (resourceMonitorSource, error) {
v, e := source()
if *calls > 0 {
v.fingerprint = strings.Repeat("d", 64)
}
return v, e
}
}
if mode == "restore" {
q.restore = func(context.Context) error { return errors.New("unsafe restore") }
}
if mode == "hash" {
q.filesHash = func() (string, error) { return strings.Repeat("b", 64), nil }
}
proof, e := q.complete(ctx, s, r)
if mode == "negative" || mode == "status500" || mode == "timeout" || mode == "tls-peer" {
if e != nil || *calls != 12 {
t.Fatalf("negative incomplete: %v %d", e, *calls)
}
source, _ := q.source()
if mode == "status500" && proof.matrix.Runs[0].Checks[2].Outcome != "expectation_failed" {
t.Fatal("500 must be a complete configured expectation failure")
}
if mode == "timeout" {
for _, c := range proof.matrix.Runs[0].Checks {
if c.Attempts != 3 || c.Outcome != "timeout" {
t.Fatal("timeout contract", c)
}
}
}
p := deriveResourcePlan(source, s.files.key, r, proof.matrix, time.Now())
if len(p.Assignments) != 0 || len(p.Domains[0].Negative) != 1 {
t.Fatalf("bad required URL qualified: %+v", p)
}
return
}
if mode == "mutation" {
if e != nil {
t.Fatal(e)
}
proof.matrix.Runs[0].Checks[0].Outcome = "timeout"
if s.saveCompleted(proof) == nil {
t.Fatal("post-seal mutation")
}
return
}
if e == nil || proof.seal != "" || s.held[planMatrixName] != nil {
t.Fatalf("published incomplete %s: %v", mode, e)
}
})
}
}
func TestResourceQualifierTypedPeerResponseBoundary(t *testing.T) {
for _, mode := range []string{"redirect", "extra"} {
t.Run(mode, func(t *testing.T) {
runner := func(_ context.Context, _ []string, _ io.Reader, out, diag io.Writer) error {
configuredWire("https://a.example/", "GET", "tls12", mode, out, diag)
return nil
}
_, _, err := collectResourceTransferOptions(context.Background(), "socks5h://127.0.0.1:1080", "https://a.example/", "a.example", "tls12", runner, resourceTransferOptions{method: "GET", limit: 100, timeout: time.Second})
var peer *resourcePeerResponseRejection
var fatal *managedRecoveryFatal
if mode == "redirect" {
if !errors.As(err, &peer) || errors.As(err, &fatal) || !resourceRejectionOnly(err) {
t.Fatalf("redirect cause should be typed peer rejection, got %T", err)
}
} else if err == nil || resourceRejectionOnly(err) || !errors.As(err, &fatal) {
t.Fatal("malformed body/evidence became peer negative")
}
})
}
}
func TestResourceQualifierRejectsRegularTarget(t *testing.T) {
d := managedDependencies(runPlayerCurl)
if qualificationMode("resources", "strategies", d) == nil {
t.Fatal("regular process authorized isolated qualification")
}
}
@@ -0,0 +1,278 @@
package main
import (
"bytes"
"errors"
"os"
"path/filepath"
"reflect"
"strings"
"testing"
"time"
)
func TestResourceStrategyIDs(t *testing.T) {
for _, value := range []string{"a,a", "a,", " a", "../a", "a\n", strings.Repeat("a,", 32) + "b", strings.Repeat("a", 4097)} {
if _, e := parseResourceStrategyIDs(value); e == nil {
t.Fatalf("accepted %q", value)
}
}
if got, e := parseResourceStrategyIDs("b,a"); e != nil || got != "a,b" {
t.Fatal(got, e)
}
}
func TestResourceSelectionCoverage(t *testing.T) {
s, key, r, m, now := completeApplyFixture(t)
// Equal profiles must not inherit coverage from an excluded candidate.
r.Strategies[1].Args = r.Strategies[0].Args
m.Registry = planDigest(r)
m.Runs[1].Definition = planDigest(r.Strategies[1])
m.Runs[1].Attestation.Definition = m.Runs[1].Definition
defaultPlan, _, e := resourceApplyPlan(s, key, r, m, now)
if e != nil {
t.Fatal(e)
}
p, _, e := resourceApplyPlan(s, key, r, m, now, "s1,s2,s3")
if e != nil || p.Policy != "s1,s2,s3" || len(p.Assignments) != 2 {
t.Fatal(p, e)
}
if !reflect.DeepEqual(p.Domains, defaultPlan.Domains) {
t.Fatal("hidden alternatives")
}
for _, policy := range []string{"s1", "s2,s3", "s1,s3", "unknown", "s2,s1,s3"} {
if _, _, e := resourceApplyPlan(s, key, r, m, now, policy); e == nil {
t.Fatal("invalid coverage/policy", policy)
}
}
for _, kind := range []string{"m3", "fingerprint", "unknown", "tls"} {
bad := cloneMatrix(m)
switch kind {
case "m3":
bad.Runs[0].Checks[0].Attempts = 2
case "fingerprint":
bad.Runs[0].Checks[0].Fingerprint = strings.Repeat("a", 64)
case "unknown":
bad.Runs[0].Attestation.Joined = false
case "tls":
r.Strategies[0].Selectors.TLS = "tls12"
bad.Registry = planDigest(r)
bad.Runs[0].Definition = planDigest(r.Strategies[0])
bad.Runs[0].Attestation.Definition = bad.Runs[0].Definition
}
if _, _, e := resourceApplyPlan(s, key, r, bad, now, "s1,s2,s3"); e == nil {
t.Fatal("invalid selected evidence", kind)
}
}
}
func TestResourceGenericRotation(t *testing.T) {
m, s, config, root := nativeApplyAlternativesFixture(t, true)
proofs := map[string][]byte{}
for _, name := range []string{planRegistryName, planMatrixName} {
b, e := os.ReadFile(filepath.Join(m.Dir, name))
if e != nil {
t.Fatal(e)
}
proofs[name] = b
}
reloads := 0
m.Reload = func() error { reloads++; return nil }
apply := func(policy string) {
t.Helper()
if e := s.selectPolicy(policy); e != nil {
t.Fatal(e)
}
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e != nil {
t.Fatal(e)
}
}
apply("")
a, _ := m.load()
old, _ := m.read("config")
apply("zz-alternative")
b, _ := m.load()
if *a.Ref == *b.Ref || a.Hash == b.Hash || b.Ref.Policy != "zz-alternative" {
t.Fatal(a, b)
}
// Reopen with a different startup request: identity is the stored policy.
d := managedDeps{directory: m.Dir, lookup: func(k string) (string, bool) {
if k == resourceStrategyIDsEnv {
return "unknown", true
}
return resourceCheckLookup(k)
}}
reopened, e := openResourceApplySnapshotAt(config, d, root)
if e != nil {
t.Fatal(e)
}
defer reopened.close()
m.resource = reopened
if e := m.Reconcile(); e != nil {
t.Fatal(e)
}
m.resource = s
s.now = func() time.Time { return time.Now().Add(25 * time.Hour) }
apply("zz-alternative") // historical replay, no reload
if reloads != 2 {
t.Fatal(reloads)
}
if e := s.selectPolicy(""); e != nil {
t.Fatal(e)
}
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e == nil {
t.Fatal("stale rotation")
}
if reloads != 2 {
t.Fatal(reloads)
}
s.now = time.Now
apply("")
got, _ := m.read("config")
if !bytes.Equal(old, got) || reloads != 3 {
t.Fatal("A not restored", reloads)
}
for _, policy := range []string{"unknown", "zz-alternative,zz-alternative", "zz-alternative,"} {
if s.selectPolicy(policy) == nil {
t.Fatal("invalid policy", policy)
}
}
bad := *a.Ref
bad.Policy = "zz-alternative"
if s.validate(&bad, bad.Config) == nil {
t.Fatal("policy forged render")
}
bad = managedConfigRef{Kind: "baseline", Policy: "zz-alternative"}
if validManagedConfigRef(&bad, nil, "", managedHash([]byte(managedBaseline))) {
t.Fatal("baseline policy")
}
for name, before := range proofs {
after, e := os.ReadFile(filepath.Join(m.Dir, name))
if e != nil || !bytes.Equal(before, after) {
t.Fatal("immutable proof changed", name, e)
}
}
}
func TestResourceSameBytesPolicyRequiresFreshProof(t *testing.T) {
m, s, _, _ := nativeApplyProfilesFixture(t, true, true)
policyA := s.registry.Strategies[0].ID
if e := s.selectPolicy(policyA); e != nil {
t.Fatal(e)
}
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e != nil {
t.Fatal(e)
}
oldRef := s.ref
oldBytes := append([]byte(nil), s.rendered...)
before := map[string][]byte{}
for _, name := range []string{m.Name, managedJournalName, managedBackupName, planRegistryName, planMatrixName} {
b, e := os.ReadFile(filepath.Join(m.Dir, name))
if e != nil {
t.Fatal(e)
}
before[name] = b
}
reloads, checkpoints := 0, 0
m.Reload = func() error { reloads++; return nil }
m.fault = func(string) error { checkpoints++; return nil }
assertUnchanged := func() {
t.Helper()
if reloads != 0 || checkpoints != 0 {
t.Fatal("request reached staging/reload", checkpoints, reloads)
}
for name, want := range before {
got, e := os.ReadFile(filepath.Join(m.Dir, name))
if e != nil || !bytes.Equal(got, want) {
t.Fatal("request mutated deployment/proof", name, e)
}
}
j, e := m.load()
if e != nil || j.State != "committed" || j.Ref == nil || *j.Ref != oldRef || j.PendingRef != nil {
t.Fatal("committed policy changed", j, e)
}
}
aged := time.Now().Add(25 * time.Hour)
s.now = func() time.Time { return aged }
if e := s.selectPolicy("zz-alternative"); e != nil {
t.Fatal(e)
}
if s.ref.Policy == oldRef.Policy || s.ref.Config != oldRef.Config || !bytes.Equal(s.rendered, oldBytes) {
t.Fatal("fixture must change policy but preserve rendered bytes", oldRef, s.ref)
}
if e := s.validate(&s.ref, s.ref.Config); e != nil {
t.Fatal("alternative lacks valid historical proof", e)
}
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); !errors.Is(e, errResourcePlan) {
t.Fatal("same-byte rotation bypassed freshness", e)
}
assertUnchanged()
unknown := s.ref
unknown.Policy = "unknown"
if e := m.validateConfigRef(nil, &unknown, nil, unknown.Config, ""); e == nil {
t.Fatal("unknown policy ref authorized")
}
assertUnchanged()
if e := s.selectPolicy(policyA); e != nil {
t.Fatal(e)
}
if s.fresh() == nil {
t.Fatal("fixture proof must be expired")
}
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e != nil {
t.Fatal("identical committed policy replay rejected after expiry", e)
}
assertUnchanged()
}
func TestResourceRotationRecovery(t *testing.T) {
for _, point := range []string{"staged", "backup", "prepared", "replaced", "reloaded", "healthy", "health-failure", "rollback-health"} {
t.Run(point, func(t *testing.T) {
m, s, _, _ := nativeApplyAlternativesFixture(t, true)
if e := s.selectPolicy("zz-alternative"); e != nil {
t.Fatal(e)
}
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e != nil {
t.Fatal(e)
}
old, _ := m.load()
oldBytes, _ := m.read("config")
if e := s.selectPolicy(""); e != nil {
t.Fatal(e)
}
m.fault = func(p string) error {
if p == point {
return errors.New("death")
}
return nil
}
if point == "health-failure" || point == "rollback-health" {
m.Health = func() (string, error) { return "DEGRADED", nil }
}
if point == "rollback-health" {
m.RollbackHealth = func() (string, error) { return "DEGRADED", nil }
}
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e == nil {
t.Fatal("fault ignored")
}
j, _ := m.load()
if *j.Ref != *old.Ref {
t.Fatal("old policy lost", j)
}
if point == "rollback-health" && j.State != "prepared" {
t.Fatal(j)
}
m.fault = nil
m.RollbackHealth = func() (string, error) { return "OK", nil }
s.now = func() time.Time { return time.Now().Add(25 * time.Hour) }
if e := m.Reconcile(); e != nil {
t.Fatal(e)
}
j, _ = m.load()
got, _ := m.read("config")
if j.State != "committed" || *j.Ref != *old.Ref || !bytes.Equal(got, oldBytes) {
t.Fatal("pending promoted", j)
}
})
}
}
+468
View File
@@ -0,0 +1,468 @@
package main
import (
"bytes"
"context"
"encoding/binary"
"encoding/json"
"errors"
"fmt"
"image"
"image/gif"
"image/jpeg"
"io"
"net"
"strconv"
"strings"
"testing"
"time"
)
const resourceV2Example = `{"version":2,"services":[{"id":"media","resources":[{"id":"image","group":"images","kind":"image","url":"https://example.com/image?SECRET","method":"GET","tls":"both","timeout_ms":700,"expect":{"statuses":[200],"content_type":"image/png","min_bytes":1,"max_bytes":1024},"headers_env":{"Authorization":"CHECK_TOKEN"}},{"id":"prefix","group":"media","kind":"mp4_range","url":"https://example.com/prefix?SECRET","method":"GET","tls":"both","timeout_ms":700,"expect":{"statuses":[206],"content_type":"video/mp4","min_bytes":16,"max_bytes":65536}}]}]}`
func resourceV2Source(t *testing.T) resourceMonitorSource {
t.Helper()
cfg, err := parseResourceChecks([]byte(resourceV2Example), resourceCheckLookup)
if err != nil {
t.Fatal(err)
}
return resourceMonitorSource{config: cfg, server: "127.0.0.1:5353", proxy: "socks5h://127.0.0.1:1080", fingerprint: strings.Repeat("a", 64)}
}
func TestResourceV2Schema(t *testing.T) {
for _, tc := range []struct{ from, to string }{
{`"version":2`, `"version":3`}, {`"version":2`, `"version":1`},
{`"kind":"image"`, `"kind":"unknown"`}, {`"method":"GET"`, `"method":"HEAD"`},
{`"kind":"image"`, `"kind":""`},
{`"statuses":[200]`, `"statuses":[200,201]`}, {`"statuses":[206]`, `"statuses":[200]`},
{`"image/png"`, `"image/gif"`}, {`"image/png"`, `"video/mp4"`},
{`"video/mp4"`, `"image/png"`}, {`"min_bytes":16`, `"min_bytes":15`},
{`"max_bytes":65536`, `"max_bytes":65537`}, {`"min_bytes":1,`, ``},
{`"content_type":"image/png",`, ``}, {`"content_type":"image/png"`, `"content_type":null`},
{`"Authorization"`, `"rAnGe"`}, {`"Authorization"`, `"Accept-Encoding"`},
{`"version":2`, `"version":2,"version":2`}, {`"kind":"image"`, `"module_version":1,"kind":"image"`},
} {
if _, err := parseResourceChecks([]byte(strings.Replace(resourceV2Example, tc.from, tc.to, 1)), resourceCheckLookup); err == nil {
t.Fatalf("accepted %s -> %s", tc.from, tc.to)
}
}
legacyMissing := strings.Replace(resourceCheckExample, `"kind":"http",`, "", 1)
if _, err := parseResourceChecks([]byte(legacyMissing), resourceCheckLookup); err == nil {
t.Fatal("v1 defaulted kind")
}
for _, kind := range []string{"", `"kind":"http",`} {
raw := strings.ReplaceAll(resourceCheckExample, `"version":1`, `"version":2`)
raw = strings.ReplaceAll(raw, `"kind":"http",`, kind)
raw = strings.Replace(raw, `"statuses":[200],"min_bytes":0`, `"statuses":[200],"content_type":"text/plain","min_bytes":0`, 1)
cfg, err := parseResourceChecks([]byte(raw), resourceCheckLookup)
if err != nil || cfg.Services[0].Resources[0].Kind != "http" {
t.Fatal("v2 default/explicit http", err)
}
}
}
// Real HTTP/1.1 framing and curl JSON, but no network and no child secrets.
func resourceV2Checks(t *testing.T, mode string, calls *int) resourceCheckDeps {
t.Helper()
return resourceCheckDeps{qualification: true, guard: func() error { return nil }, resolve: func(context.Context, string, string) ([]net.IP, error) { return []net.IP{net.ParseIP("8.8.8.8")}, nil }, runner: func(ctx context.Context, args []string, stdin io.Reader, out, diag io.Writer) error {
(*calls)++
config, err := io.ReadAll(stdin)
if err != nil {
return err
}
argv := strings.Join(args, " ")
if strings.Contains(argv, "SECRET") || strings.Contains(argv, "private-token") || !bytes.Contains(config, []byte("connect-to =")) {
t.Fatal("secret/pin boundary")
}
if !strings.Contains(argv, "--max-time 0.700") {
t.Fatal("configured curl timeout", argv)
}
deadline, ok := ctx.Deadline()
if !ok || time.Until(deadline) > 700*time.Millisecond {
t.Fatal("configured context timeout")
}
raw, err := strconv.Unquote(strings.TrimPrefix(strings.Split(string(config), "\n")[0], "url = "))
if err != nil {
return err
}
body, mimeType, status, extra := resourceFixture(), "image/png", 200, ""
if strings.Contains(raw, "/prefix") {
if !strings.Contains(argv, "--range 0-65535") {
t.Fatal("fixed core range missing")
}
size := 24
if mode == "min-range" {
size = 16
}
if mode == "max-range" {
size = 65536
}
body = make([]byte, size)
binary.BigEndian.PutUint32(body, 16)
copy(body[4:], "ftypisom")
mimeType, status = "video/mp4", 206
extra = fmt.Sprintf("Content-Range: bytes 0-%d/100000\r\n", size-1)
if mode == "bad-range" {
extra = "Content-Range: bytes 0-23/9223372036854775808\r\n"
}
if mode == "bad-body" {
copy(body[4:], "xxxx")
}
} else {
if !bytes.Contains(config, []byte("Authorization: Bearer private-token")) {
t.Fatal("secret header not on stdin")
}
if mode == "jpeg" {
var b bytes.Buffer
if err := jpeg.Encode(&b, image.NewRGBA(image.Rect(0, 0, 1, 1)), nil); err != nil {
return err
}
body, mimeType = b.Bytes(), "image/jpeg"
}
if mode == "gif" {
var b bytes.Buffer
if err := gif.Encode(&b, image.NewRGBA(image.Rect(0, 0, 1, 1)), nil); err != nil {
return err
}
body = b.Bytes()
}
if mode == "bad-body" {
body = []byte("not an image")
}
if mode == "truncated" {
body = body[:len(body)/2]
}
}
if mode == "status" {
status = 500
}
if mode == "mime" {
mimeType = "text/html"
}
if mode == "malformed-mime" {
mimeType += "; broken"
}
fmt.Fprintf(out, "HTTP/1.1 %d result\r\nContent-Type: %s\r\nContent-Length: %d\r\n%s\r\n", status, mimeType, len(body), extra)
out.Write(body)
metrics, _ := json.Marshal(map[string]interface{}{"url_effective": raw, "ssl_verify_result": 0, "num_redirects": 0, "http_code": status, "time_total": 0.1, "speed_download": 10000, "size_download": len(body)})
if mode == "metrics" {
metrics = []byte("invalid")
}
fmt.Fprint(out, resourceTrailer, string(metrics))
version := "1.2"
if strings.Contains(argv, "--tlsv1.3") {
version = "1.3"
}
fmt.Fprintln(diag, "SSL connection using TLSv"+version)
if mode == "mixed" {
return errors.Join(errors.New("execution failed"), errors.New("invalid media range or container"))
}
return nil
}}
}
func TestResourceV2SharedPublicPaths(t *testing.T) {
for _, mode := range []string{"pass", "jpeg", "min-range", "max-range", "bad-body", "bad-range", "status", "mime", "malformed-mime", "gif", "truncated", "metrics", "mixed"} {
t.Run(mode, func(t *testing.T) {
source := resourceV2Source(t)
if mode == "jpeg" {
source.config.Services[0].Resources[0].Expect.ContentType = "image/jpeg"
}
calls := 0
checks := resourceV2Checks(t, mode, &calls)
negative := mode == "bad-body" || mode == "bad-range" || mode == "status" || mode == "mime" || mode == "malformed-mime" || mode == "gif" || mode == "truncated"
unknown := mode == "metrics" || mode == "mixed"
for _, spec := range source.config.Services[0].Resources {
for _, protocol := range []string{"tls12", "tls13"} {
result, err := runResourceProtocol(context.Background(), "media", spec, protocol, source.server, source.proxy, checks)
if unknown {
if err == nil {
t.Fatal("infrastructure became negative")
}
continue
}
want := "pass"
if negative && (mode != "bad-range" || spec.Kind == "mp4_range") && (mode != "gif" && mode != "truncated" || spec.Kind == "image") {
want = "expectation_failed"
}
if err != nil || result.Outcome != want || result.Bytes == 0 {
t.Fatalf("%s %s: %+v %v", spec.Kind, protocol, result, err)
}
if mode == "malformed-mime" {
status, size := 200, len(resourceFixture())
if spec.Kind == "mp4_range" {
status, size = 206, 24
}
if result.Status != status || result.Bytes != size || result.TimeMS != 100 {
t.Fatalf("lost malformed-MIME facts: %+v", result)
}
}
}
}
if unknown {
return
}
for _, spec := range source.config.Services[0].Resources {
_, err := probeResourceMonitor(context.Background(), "media", spec, source, func() error { return nil }, checks)
if err != nil {
t.Fatal(err)
}
}
snapshot := &resourceApplySnapshot{source: source, guard: func() error { return nil }}
status, err := resourceApplyHealth(context.Background(), snapshot, checks)
if (!negative && (err != nil || status != "OK")) || (negative && err == nil) {
t.Fatal("apply", status, err)
}
})
}
}
func TestResourceV2QualifierCompletion(t *testing.T) {
for _, mode := range []string{"pass", "bad-body", "bad-range", "malformed-mime", "metrics"} {
t.Run(mode, func(t *testing.T) {
q, store, registry, _, _ := qualifierFixture(t)
source := resourceV2Source(t)
q.source = func() (resourceMonitorSource, error) { return source, nil }
calls := 0
q.checks = resourceV2Checks(t, mode, &calls)
proof, err := q.complete(context.Background(), store, registry)
if mode == "metrics" {
if err == nil {
t.Fatal("sealed unknown infrastructure")
}
return
}
if err != nil || calls != 12 {
t.Fatalf("completion %d %v", calls, err)
}
for _, cell := range proof.matrix.Runs[0].Checks {
want := "pass"
if mode == "bad-body" || mode == "malformed-mime" || mode == "bad-range" && cell.Resource == "prefix" {
want = "expectation_failed"
}
if cell.Attempts != 3 || cell.Outcome != want {
t.Fatalf("incomplete/wrong cell %+v", cell)
}
}
})
}
}
func TestResourceCoreMalformedMIMEClassification(t *testing.T) {
for _, kind := range []resourceResponseKind{resourceResponseGeneric, resourceResponseImage, resourceResponseMP4} {
for _, configured := range []bool{false, true} {
t.Run(fmt.Sprintf("kind-%d/configured-%t", kind, configured), func(t *testing.T) {
raw := "https://i.ytimg.com/private?SECRET"
options := resourceTransferOptions{method: "GET", response: kind, mediaType: "video/mp4", limit: 1024, timeout: time.Second}
if configured {
options.expect = &resourceCheckExpect{Statuses: []int{200}, MaxBytes: 1024}
}
runner := func(_ context.Context, _ []string, _ io.Reader, out, diag io.Writer) error {
var wire bytes.Buffer
mode := ""
if kind == resourceResponseMP4 {
mode = "media"
}
resourceWire(raw, mode, &wire, diag)
data := strings.Replace(wire.String(), "Content-Type: image/png\r\n", "Content-Type: image/png; broken\r\n", 1)
data = strings.Replace(data, "Content-Type: video/mp4\r\n", "Content-Type: video/mp4; broken\r\n", 1)
_, err := io.WriteString(out, data)
return err
}
transfer, _, err := collectResourceTransferOptions(context.Background(), "socks5h://127.0.0.1:1080", raw, "i.ytimg.com", "tls12", runner, options)
if configured && kind != resourceResponseGeneric {
status, size := 200, len(resourceFixture())
if kind == resourceResponseMP4 {
status, size = 206, 24
}
if err != errResourceExpectation || transfer.Status != status || len(transfer.Body) != size || transfer.LatencyMS != 100 || transfer.ContentType != "" {
t.Fatalf("typed facts/classification: %+v %v", transfer, err)
}
} else {
var fatal *managedRecoveryFatal
if !errors.As(err, &fatal) || err == errResourceExpectation || resourceRejectionOnly(err) || transfer.Status != 0 || len(transfer.Body) != 0 {
t.Fatalf("generic/private malformed MIME relaxed: %+v %v", transfer, err)
}
var resource *resourceError
if !errors.As(err, &resource) || resource.reason != reasonMIME || fatal.Err.Error() != "invalid content type" {
t.Fatalf("legacy MIME failure changed: %v", err)
}
}
})
}
}
}
func TestResourceV2FingerprintCompatibility(t *testing.T) {
key := []byte("fingerprint-test-key")
cfg, err := parseResourceChecks([]byte(resourceCheckExample), resourceCheckLookup)
if err != nil {
t.Fatal(err)
}
source := resourceMonitorSource{config: cfg, server: "dns", proxy: "proxy"}
spec := cfg.Services[0].Resources[0]
b, _ := json.Marshal(spec)
b, _ = json.Marshal([]string{string(b), spec.privateConfig, source.server, source.proxy})
old := resourceMAC(key, b)
if resourceRowFingerprint(key, source, spec) != old {
t.Fatal("historic v1 serialization changed")
}
source.config.Version = 2
v2 := resourceRowFingerprint(key, source, spec)
if v2 == old {
t.Fatal("v2 reused v1 proof namespace")
}
for _, mutate := range []func(*resourceCheckSpec){func(r *resourceCheckSpec) { r.Kind = "image" }, func(r *resourceCheckSpec) { r.Expect.MinBytes++ }, func(r *resourceCheckSpec) { r.Expect.MaxBytes++ }, func(r *resourceCheckSpec) { r.Expect.Statuses = []int{201} }} {
r := spec
mutate(&r)
if resourceRowFingerprint(key, source, r) == v2 {
t.Fatal("semantic change reused row proof")
}
}
prior := []resourceMonitorRow{{Service: cfg.Services[0].ID, Resource: spec.ID, Fingerprint: old, Health: "OK", Outcome: "pass"}}
rows := reconcileResourceRows(key, source, prior)
if rows[0].Health != "UNKNOWN" {
t.Fatal("v1 health survived v2 semantics")
}
}
func TestResourceV2ConfiguredBounds(t *testing.T) {
for _, kind := range []string{"image", "mp4_range"} {
for _, mode := range []string{"exact", "minimum", "overflow"} {
t.Run(kind+"/"+mode, func(t *testing.T) {
source := resourceV2Source(t)
spec := source.config.Services[0].Resources[0]
size := len(resourceFixture())
if kind == "mp4_range" {
spec = source.config.Services[0].Resources[1]
size = 24
}
spec.Expect.MinBytes, spec.Expect.MaxBytes = size, size
if mode == "minimum" {
spec.Expect.MinBytes++
spec.Expect.MaxBytes++
}
if mode == "overflow" {
spec.Expect.MinBytes--
spec.Expect.MaxBytes--
}
calls := 0
checks := resourceV2Checks(t, "pass", &calls)
runner := checks.runner
checks.runner = func(ctx context.Context, args []string, stdin io.Reader, out, diag io.Writer) error {
for i, a := range args {
if a == "--max-filesize" && args[i+1] != strconv.Itoa(spec.Expect.MaxBytes) {
t.Fatal("ignored configured limit", args)
}
}
return runner(ctx, args, stdin, out, diag)
}
result, err := runResourceProtocol(context.Background(), "media", spec, "tls12", source.server, source.proxy, checks)
switch mode {
case "exact":
if err != nil || result.Outcome != "pass" {
t.Fatal(result, err)
}
case "minimum":
if err != nil || result.Outcome != "expectation_failed" {
t.Fatal(result, err)
}
case "overflow":
if err == nil {
t.Fatal("body-bound violation became completed expectation", result)
}
}
})
}
}
}
func TestResourceV2CancelCannotSeal(t *testing.T) {
q, store, registry, _, _ := qualifierFixture(t)
source := resourceV2Source(t)
q.source = func() (resourceMonitorSource, error) { return source, nil }
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
joined := false
calls := 0
q.checks = resourceV2Checks(t, "pass", &calls)
q.checks.runner = func(child context.Context, _ []string, _ io.Reader, _ io.Writer, _ io.Writer) error {
cancel()
<-child.Done()
joined = true
return child.Err()
}
proof, err := q.complete(ctx, store, registry)
if err == nil || !joined || store.saveCompleted(proof) == nil {
t.Fatal("cancelled batch acquired proof", err, joined)
}
}
func TestResourceV2OldMatrixCannotPromote(t *testing.T) {
source, key, registry, matrix, now := planFixture(t)
if len(deriveResourcePlan(source, key, registry, matrix, now).Assignments) == 0 {
t.Fatal("invalid baseline fixture")
}
source.config.Version = 2
for si := range source.config.Services {
for ri := range source.config.Services[si].Resources {
source.config.Services[si].Resources[ri].Kind = "image"
}
}
// Even retaining the old source MAC cannot bypass per-cell semantic binding.
if len(deriveResourcePlan(source, key, registry, matrix, now).Assignments) != 0 {
t.Fatal("old HTTP matrix promoted typed images")
}
}
func TestResourceV2CLIAndSourceBinding(t *testing.T) {
for _, mode := range []string{"pass", "bad-body"} {
t.Run(mode, func(t *testing.T) {
calls := 0
checks := resourceV2Checks(t, mode, &calls)
checks.lookup = resourceCheckLookup
var out, diag bytes.Buffer
code := resourceCheckCLI(context.Background(), resourceCheckFile(t, resourceV2Example), &out, &diag, checks)
want := 0
if mode != "pass" {
want = 1
}
if code != want || calls != 4 || bytes.Contains(out.Bytes(), []byte("SECRET")) || bytes.Contains(out.Bytes(), []byte("private-token")) || diag.Len() != 0 {
t.Fatal(code, calls, out.String(), diag.String())
}
})
}
calls := 0
checks := resourceV2Checks(t, "pass", &calls)
checks.lookup = resourceCheckLookup
checks.resolve = func(context.Context, string, string) ([]net.IP, error) {
t.Fatal("invalid typed HEAD reached DNS")
return nil, nil
}
var out, diag bytes.Buffer
if resourceCheckCLI(context.Background(), resourceCheckFile(t, strings.Replace(resourceV2Example, `"method":"GET"`, `"method":"HEAD"`, 1)), &out, &diag, checks) != 2 || calls != 0 {
t.Fatal("invalid typed config executed")
}
legacy, key := resourceMonitorFixture(t)
b, _ := json.Marshal(legacy.config)
parts := []string{string(b), legacy.server, legacy.proxy}
for _, s := range legacy.config.Services {
for _, r := range s.Resources {
parts = append(parts, r.privateConfig)
}
}
b, _ = json.Marshal(parts)
if legacy.fingerprint != resourceMAC(key, b) {
t.Fatal("v1 source serialization changed")
}
raw := strings.Replace(resourceCheckExample, `"version":1`, `"version":2`, 1)
raw = strings.Replace(raw, `"statuses":[200],"min_bytes":0`, `"statuses":[200],"content_type":"text/plain","min_bytes":0`, 1)
v2, err := readResourceMonitorSource(resourceCheckFile(t, raw), resourceCheckLookup, key)
if err != nil || v2.fingerprint == legacy.fingerprint {
t.Fatal("version unbound source", err)
}
implicit, err := readResourceMonitorSource(resourceCheckFile(t, strings.ReplaceAll(raw, `"kind":"http",`, "")), resourceCheckLookup, key)
if err != nil || implicit.fingerprint != v2.fingerprint {
t.Fatal("default kind not canonical", err)
}
}
+54
View File
@@ -0,0 +1,54 @@
package main
import (
"errors"
"sync"
"zapret-autotune/internal/adapter"
"zapret-autotune/internal/adapter/youtube"
)
// Trusted compiled-in dispatch, deliberately not a public config/plugin API.
// An empty registry is a supported fail-closed seam; no HTTP fallback exists.
type serviceAdapterKey struct {
kind string
version int
}
type serviceAdapterRegistry map[serviceAdapterKey]adapter.Module
var errServiceAdapterUnavailable = errors.New("required service adapter unavailable")
// The registry is immutable in production. The lock permits race-free test
// replacement while request callbacks/owned lifecycle children are running.
var serviceAdapters = struct {
sync.RWMutex
modules serviceAdapterRegistry
}{modules: serviceAdapterRegistry{{youtube.Kind, youtube.Version}: youtube.Module{}}}
func requiredYoutubeAdapter() (adapter.Module, error) {
serviceAdapters.RLock()
m, err := serviceAdapters.modules.lookup(youtube.Kind, youtube.Version)
serviceAdapters.RUnlock()
if err != nil || m.Binding() != (youtube.Module{}).Binding() {
return nil, &managedRecoveryFatal{Err: errServiceAdapterUnavailable}
}
return m, nil
}
func (r serviceAdapterRegistry) lookup(kind string, version int) (adapter.Module, error) {
m, ok := r[serviceAdapterKey{kind, version}]
if !ok || m == nil {
return nil, errServiceAdapterUnavailable
}
b := m.Binding()
if b.Kind != kind || b.Version != version {
return nil, errServiceAdapterUnavailable
}
return m, nil
}
func lookupServiceAdapter(kind string, version int) (adapter.Module, error) {
serviceAdapters.RLock()
defer serviceAdapters.RUnlock()
return serviceAdapters.modules.lookup(kind, version)
}
@@ -0,0 +1,284 @@
package main
import (
"context"
"encoding/json"
"errors"
"io"
"testing"
"time"
"zapret-autotune/internal/adapter"
"zapret-autotune/internal/adapter/youtube"
)
func replaceServiceAdapters(t *testing.T, modules serviceAdapterRegistry) {
t.Helper()
serviceAdapters.Lock()
previous := serviceAdapters.modules
serviceAdapters.modules = modules
serviceAdapters.Unlock()
t.Cleanup(func() {
serviceAdapters.Lock()
serviceAdapters.modules = previous
serviceAdapters.Unlock()
})
}
func TestServiceAdapterDispatchFailClosed(t *testing.T) {
for _, key := range []serviceAdapterKey{{"unknown", 1}, {youtube.Kind, 0}, {youtube.Kind, 2}} {
if _, err := lookupServiceAdapter(key.kind, key.version); err == nil {
t.Fatal("unsupported adapter accepted")
}
}
if _, err := (serviceAdapterRegistry{}).lookup(youtube.Kind, youtube.Version); err == nil {
t.Fatal("absent adapter accepted")
}
replaceServiceAdapters(t, serviceAdapterRegistry{})
if _, _, err := playerMediaResource([]byte(`{"playabilityStatus":{"status":"OK"}}`)); err == nil {
t.Fatal("discovery bypassed registry")
}
r, c, resources, events := googlevideoRuntimeFixture(t)
if _, err := r.verifyGooglevideo(context.Background(), c, resources); err == nil || len(*events) != 0 {
t.Fatal("qualification bypassed registry")
}
}
type mismatchedServiceAdapter struct{ adapter.Module }
func (m mismatchedServiceAdapter) Binding() adapter.Binding {
b := m.Module.Binding()
b.Version++
return b
}
func TestServiceAdapterIdentityBeforeExecution(t *testing.T) {
replaceServiceAdapters(t, serviceAdapterRegistry{{youtube.Kind, youtube.Version}: mismatchedServiceAdapter{youtube.Module{}}})
r, c, resources, events := googlevideoRuntimeFixture(t)
if _, err := r.verifyGooglevideo(context.Background(), c, resources); err == nil || len(*events) != 0 {
t.Fatal("identity mismatch reached worker")
}
}
func TestServiceAdapterStoredProofRefusals(t *testing.T) {
r, c, resources, _ := googlevideoRuntimeFixture(t)
p, err := r.verifyGooglevideo(context.Background(), c, resources)
if err != nil {
t.Fatal(err)
}
db := testStore(t)
tx, err := db.Begin()
if err != nil {
t.Fatal(err)
}
defer tx.Rollback()
id, err := insertStrategyCandidate(tx, c.Group, c.Endpoints, c.Args, nil, c.Mode)
if err != nil {
t.Fatal(err)
}
if err := saveAdaptiveGooglevideoEvidence(tx, id, p); err != nil {
t.Fatal(err)
}
if err := tx.Commit(); err != nil {
t.Fatal(err)
}
front := managedCandidateFixture(t, db, "frontend", "www.youtube.com", "--lua-desync=multisplit", true)
for _, mode := range []string{"kind", "version", "digest", "missing", "unknown-proof", "downgrade", "attestation"} {
t.Run(mode, func(t *testing.T) {
raw, _ := json.Marshal(p)
var proof GooglevideoQualificationProof
if err := json.Unmarshal(raw, &proof); err != nil {
t.Fatal(err)
}
switch mode {
case "kind":
proof.Adapter.Kind = "unknown"
case "version":
proof.Adapter.Version++
case "digest":
proof.Adapter.PolicySHA256 = "changed"
case "missing":
proof.Adapter = nil
case "unknown-proof":
proof.Version = 3
case "downgrade":
proof.Version = 1
case "attestation":
proof.Attestation = nil
}
if proof.validateStored(c) == nil {
t.Fatal("malformed proof reconstructed")
}
raw, _ = json.Marshal(proof)
if _, err := db.Exec("UPDATE googlevideo_scope_evidence SET proof=? WHERE candidate_id=?", string(raw), id); err != nil {
t.Fatal(err)
}
m, reloads := managedFixture(t)
if err := m.Apply(db, []int64{front, id}, GooglevideoScopeSuffix); err == nil || *reloads != 0 {
t.Fatal("invalid proof reached apply", err, *reloads)
}
})
}
}
func TestServiceAdapterProofBoundary(t *testing.T) {
r, c, resources, _ := googlevideoRuntimeFixture(t)
p, err := r.verifyGooglevideo(context.Background(), c, resources)
if err != nil {
t.Fatal(err)
}
if p.Version != 2 || p.Adapter == nil || *p.Adapter != (youtube.Module{}).Binding() {
t.Fatal("missing module identity")
}
seal := googlevideoProofSeal(p)
for _, mutate := range []func(){
func() { p.Adapter.Kind = "unknown" },
func() { p.Adapter.Version++ },
func() { p.Adapter.PolicySHA256 = "changed" },
} {
original := *p.Adapter
mutate()
if p.validateStored(c) == nil || googlevideoProofSeal(p) == seal {
t.Fatal("module boundary not bound to evidence")
}
*p.Adapter = original
}
// Reads retain the original v1 contract, including fresh apply eligibility.
p.Version, p.Adapter = 1, nil
raw, err := json.Marshal(p)
if err != nil {
t.Fatal(err)
}
legacy, err := decodeGooglevideoProof(string(raw))
if err != nil || legacy.validateStored(c) != nil {
t.Fatal("legacy reconstruction lost", err)
}
if legacy.validate(c) != nil {
t.Fatal("fresh trusted legacy adaptive proof lost apply eligibility")
}
legacy.Attestation.CheckedAt = time.Now().Add(-25 * time.Hour)
if legacy.validate(c) == nil {
t.Fatal("expired legacy adaptive proof authorized new apply")
}
replaceServiceAdapters(t, serviceAdapterRegistry{})
if legacy.validateStored(c) != nil {
t.Fatal("historical reconstruction depends on current execution registry")
}
}
type changedPolicyServiceAdapter struct{ adapter.Module }
func (m changedPolicyServiceAdapter) Binding() adapter.Binding {
b := m.Module.Binding()
b.PolicySHA256 = "changed"
return b
}
func TestServiceAdapterDiscoveryPreflight(t *testing.T) {
for _, mode := range []string{"missing", "version", "digest"} {
t.Run(mode, func(t *testing.T) {
modules := serviceAdapterRegistry{}
if mode == "version" {
modules[serviceAdapterKey{youtube.Kind, youtube.Version}] = mismatchedServiceAdapter{youtube.Module{}}
}
if mode == "digest" {
modules[serviceAdapterKey{youtube.Kind, youtube.Version}] = changedPolicyServiceAdapter{youtube.Module{}}
}
calls := 0
player := func(context.Context, []string, io.Reader, io.Writer, io.Writer) error { calls++; return nil }
request := func(context.Context) ([]byte, error) { calls++; return nil, nil }
target, _, _ := managedCLIFixture(t)
target.player = player
target.verifySOCKS = func(int) error { calls++; return nil }
r, _, _ := recoveryFixture(t, false)
bound, err := managedRecoveryDependencies(target, r.db, r.scan, r.check, r.run)
if err != nil {
t.Fatal(err)
}
calls = 0 // Construction guard is allowed before removing the module.
replaceServiceAdapters(t, modules)
cfg := healthConfig{VideoID: "dQw4w9WgXcQ", SOCKSProxy: "socks5://127.0.0.1:1080"}
assertBlocked := func(err error) {
t.Helper()
var fatal *managedRecoveryFatal
if !errors.Is(err, errServiceAdapterUnavailable) || !errors.As(err, &fatal) || errors.Is(err, errNoQualifiedStrategy) || errors.Is(err, errGooglevideoInsufficientSamples) || managedResourceTransferNonpass(err, true, true) {
t.Fatal("adapter configuration became a retryable absence", err)
}
if calls != 0 {
t.Fatal("preflight made network calls", calls)
}
}
_, err = requestPlayer(cfg.VideoID, cfg.SOCKSProxy, player)
assertBlocked(err)
if managedResourceTransferNonpass(errors.Join(err, context.DeadlineExceeded), true, true) || managedRecoveryCooldown(errors.Join(err, errNoQualifiedStrategy), time.Minute) != time.Minute {
t.Fatal("mixed fatal adapter failure became transfer-negative/2h absence")
}
_, err = bound.discover(context.Background())
assertBlocked(err)
_, err = bound.health(context.Background())
assertBlocked(err)
_, err = managedGooglevideoSamples([]string{cfg.VideoID}, func(ctx context.Context, _ string) ([]byte, error) { return request(ctx) })(context.Background())
assertBlocked(err)
inventory := managedServiceResources(cfg, request, player)
_, err = inventory(context.Background())
assertBlocked(err)
_, err = managedServiceHealth(cfg.SOCKSProxy, func(context.Context) (string, error) { calls++; return "OK", nil }, inventory, func(context.Context, []string, io.Reader, io.Writer, io.Writer) error { calls++; return nil })(context.Background())
assertBlocked(err)
if code := discoverYoutube([]string{"--video-id", cfg.VideoID, "--proxy", cfg.SOCKSProxy}, io.Discard, io.Discard, player); code != 1 || calls != 0 {
t.Fatal("legacy CLI bypassed preflight", code, calls)
}
_, err = managedRecoveryDependencies(target, r.db, r.scan, r.check, r.run)
assertBlocked(err)
assertBlocked(managedExperimentalSuffix(nil, managedDeps{}))
assertBlocked(managedActivate(nil, managedDeps{}, nil, nil))
for _, saved := range []bool{false, true} {
code := googlevideoProposal([]string{"--video-id", cfg.VideoID, "--proxy", cfg.SOCKSProxy}, io.Discard, io.Discard, player, func(context.Context, string, string, io.Writer, io.Writer) (string, error) { calls++; return "", nil }, func(context.Context, string, string, string, string, io.Writer, io.Writer) error { calls++; return nil }, saved)
if code != 1 || calls != 0 {
t.Fatal("proposal pipeline bypassed preflight", code, calls)
}
}
runtime, c, resources, events := googlevideoRuntimeFixture(t)
_, err = runtime.verifyGooglevideo(context.Background(), c, resources)
assertBlocked(err)
if len(*events) != 0 {
t.Fatal("preflight launched worker")
}
})
}
}
func TestServiceAdapterLegacyNewWriteRefused(t *testing.T) {
r, c, resources, _ := googlevideoRuntimeFixture(t)
p, err := r.verifyGooglevideo(context.Background(), c, resources)
if err != nil {
t.Fatal(err)
}
p.Version, p.Adapter = 1, nil
if p.validate(c) != nil {
t.Fatal("trusted legacy read invalid")
}
// Even an internally constructed seal cannot authorize NEW v1 publication.
seal := googlevideoProofSeal(p)
p.producerSeal = &seal
db := testStore(t)
tx, err := db.Begin()
if err != nil {
t.Fatal(err)
}
defer tx.Rollback()
id, err := insertStrategyCandidate(tx, c.Group, c.Endpoints, c.Args, nil, c.Mode)
if err != nil {
t.Fatal(err)
}
if saveAdaptiveGooglevideoEvidence(tx, id, p) == nil {
t.Fatal("new sealed v1 write accepted")
}
raw, _ := json.Marshal(p)
var dto GooglevideoQualificationProof
if err := json.Unmarshal(raw, &dto); err != nil {
t.Fatal(err)
}
if saveAdaptiveGooglevideoEvidence(tx, id, dto) == nil {
t.Fatal("unsealed v1 DTO write accepted")
}
}
@@ -0,0 +1,168 @@
package main
// Portable trusted-history fixtures: run unchanged on archived d324b41 too.
// Direct SQL below models an existing trusted v1 database, not an import API.
import (
"bytes"
"context"
"database/sql"
"encoding/json"
"errors"
"reflect"
"testing"
"time"
)
func legacyYoutubeV1Fixture(t *testing.T, db *sql.DB) (int64, GooglevideoQualificationProof) {
t.Helper()
r, c, resources, _ := googlevideoRuntimeFixture(t)
p, err := r.verifyGooglevideo(context.Background(), c, resources)
if err != nil {
t.Fatal(err)
}
raw, _ := json.Marshal(p)
var fields map[string]json.RawMessage
if err := json.Unmarshal(raw, &fields); err != nil {
t.Fatal(err)
}
delete(fields, "adapter")
fields["version"] = json.RawMessage(`1`)
raw, _ = json.Marshal(fields)
legacy, err := decodeGooglevideoProof(string(raw))
if err != nil || legacy.validate(c) != nil {
t.Fatal("trusted v1 contract invalid", err)
}
tx, err := db.Begin()
if err != nil {
t.Fatal(err)
}
defer tx.Rollback()
id, err := insertStrategyCandidate(tx, c.Group, c.Endpoints, c.Args, nil, c.Mode)
if err != nil {
t.Fatal(err)
}
if _, err := tx.Exec("INSERT INTO googlevideo_scope_evidence(candidate_id,proof) VALUES (?,?)", id, string(raw)); err != nil {
t.Fatal(err)
}
if err := tx.Commit(); err != nil {
t.Fatal(err)
}
return id, legacy
}
func expireLegacyYoutubeV1(t *testing.T, db *sql.DB, id int64, proof GooglevideoQualificationProof) {
t.Helper()
proof.Attestation.CheckedAt = time.Now().Add(-25 * time.Hour)
raw, _ := json.Marshal(proof)
if _, err := db.Exec("UPDATE googlevideo_scope_evidence SET proof=? WHERE candidate_id=?", string(raw), id); err != nil {
t.Fatal(err)
}
}
func TestLegacyYoutubeV1FreshApplyEligibility(t *testing.T) {
db := testStore(t)
id, proof := legacyYoutubeV1Fixture(t, db)
c, err := readStrategyCandidate(db, id)
if err != nil || !c.Qualified {
t.Fatal("fresh stored v1 lost qualification", err)
}
saved, err := listQualifiedCandidates(db, "googlevideo", 10)
if err != nil || len(saved) != 1 || saved[0].ID != id {
t.Fatal("fresh stored v1 disappeared from shortlist", err)
}
front := managedCandidateFixture(t, db, "frontend", "www.youtube.com", "--lua-desync=multisplit", true)
ids := []int64{front, id} // Native suffix still does not require assets.
m, reloads := managedFixture(t)
if err := m.Apply(db, ids, GooglevideoScopeSuffix); err != nil {
t.Fatal("fresh v1 apply rejected", err)
}
config, err := m.read(m.Name)
if err != nil {
t.Fatal(err)
}
expireLegacyYoutubeV1(t, db, id, proof)
c, err = readStrategyCandidate(db, id)
if err != nil || c.Qualified {
t.Fatal("expired v1 retained fresh qualification", err)
}
before := *reloads
if err := m.Apply(db, ids, GooglevideoScopeSuffix); err == nil || *reloads != before {
t.Fatal("expired v1 activated")
}
rendered, err := renderStoredManagedConfigScope(db, ids, GooglevideoScopeSuffix)
if err != nil || !bytes.Equal(config, rendered) {
t.Fatal("expired historical reconstruction changed", err)
}
}
func TestLegacyYoutubeV1PreparedRollback(t *testing.T) {
for _, point := range []string{"staged", "backup", "prepared", "replaced", "reloaded", "healthy"} {
t.Run(point, func(t *testing.T) {
db := testStore(t)
id, proof := legacyYoutubeV1Fixture(t, db)
front := managedCandidateFixture(t, db, "frontend", "www.youtube.com", "--lua-desync=multisplit", true)
ids := []int64{front, id}
m, reloads := managedFixture(t)
if err := m.Apply(db, ids, GooglevideoScopeSuffix); err != nil {
t.Fatal(err)
}
before, err := m.load()
if err != nil {
t.Fatal(err)
}
journal, err := m.read(managedJournalName)
if err != nil {
t.Fatal(err)
}
config, err := m.read(m.Name)
if err != nil {
t.Fatal(err)
}
nextFront := managedCandidateFixture(t, db, "frontend", "www.youtube.com", "--lua-desync=fake", true)
m.fault = func(stage string) error {
if stage == point {
return errors.New("fixture crash")
}
return nil
}
if err := m.Apply(db, []int64{nextFront, id}, GooglevideoScopeSuffix); err == nil {
t.Fatal("fault missed")
}
if point != "staged" {
backup, err := m.read(managedBackupName)
if err != nil || !bytes.Equal(backup, config) {
t.Fatal("legacy backup changed", err)
}
}
expireLegacyYoutubeV1(t, db, id, proof)
m.fault = nil
priorReloads := *reloads
if err := m.Reconcile(db); err != nil {
t.Fatal("expired v1 prepared recovery failed", err)
}
after, err := m.load()
if err != nil {
t.Fatal(err)
}
wantGeneration, wantReloads := before.Generation+1, priorReloads+1
if point == "staged" || point == "backup" {
wantGeneration, wantReloads = before.Generation, priorReloads
}
if after.State != "committed" || after.Hash != before.Hash || after.Scope != before.Scope || !reflect.DeepEqual(after.Candidates, before.Candidates) || len(after.PendingCandidates) != 0 || after.PendingScope != "" || after.Generation != wantGeneration || *reloads != wantReloads {
t.Fatal("legacy rollback contract changed", after, *reloads)
}
restored, err := m.read(m.Name)
if err != nil || !bytes.Equal(restored, config) {
t.Fatal("rollback config bytes changed", err)
}
if point == "staged" || point == "backup" {
afterJournal, err := m.read(managedJournalName)
if err != nil || !bytes.Equal(journal, afterJournal) {
t.Fatal("pre-prepare journal changed", err)
}
}
assertActiveCandidates(t, m, db, ids)
t.Logf("legacy restored config SHA256=%s", after.Hash)
})
}
}
@@ -0,0 +1,133 @@
package main
// This corpus also runs unmodified against archived d324b41. It uses only
// pre-extraction APIs, with explicit expected normalized facts, not wrappers.
import (
"bytes"
"context"
"encoding/json"
"fmt"
"strings"
"testing"
"time"
)
func TestYoutubeLegacyParserParity(t *testing.T) {
good := "https://rr1---sn-alpha.googlevideo.com/videoplayback?expire=1800000000&sig=private-secret"
format := func(raw, mime string) map[string]string { return map[string]string{"url": raw, "mimeType": mime} }
for _, tc := range []struct {
name string
formats, adaptive []map[string]string
want string
}{
{"ordinary-first", []map[string]string{format(good, "audio/mp4; codecs=mp4a")}, []map[string]string{format(good+"&second=1", "video/mp4")}, "audio/mp4"},
{"nonmp4-fallback", []map[string]string{format(good, "video/webm")}, []map[string]string{format(good, "video/mp4")}, "video/mp4"},
{"userinfo", []map[string]string{format(strings.Replace(good, "https://", "https://user@", 1), "video/mp4")}, nil, ""},
{"fragment", []map[string]string{format(good+"#secret", "video/mp4")}, nil, ""},
{"http", []map[string]string{format(strings.Replace(good, "https:", "http:", 1), "video/mp4")}, nil, ""},
{"invalid-dns", []map[string]string{format(strings.Replace(good, "rr1---sn-alpha", "bad_host", 1), "video/mp4")}, nil, ""},
{"sabr-only", nil, nil, ""},
} {
t.Run(tc.name, func(t *testing.T) {
body, _ := json.Marshal(map[string]interface{}{"playabilityStatus": map[string]string{"status": "OK"}, "streamingData": map[string]interface{}{"formats": tc.formats, "adaptiveFormats": tc.adaptive, "serverAbrStreamingUrl": good}})
r, capability, err := playerMediaResource(body)
if err != nil || r.kind != tc.want || capability.OrdinaryURL != (tc.want != "") || capability.SABRSupported {
t.Fatal(r.kind, capability, err)
}
if tc.want != "" && (r.url != good || r.host != "rr1---sn-alpha.googlevideo.com") {
t.Fatal("normalized resource changed")
}
})
}
for _, body := range [][]byte{[]byte(`{"secret":`), bytes.Repeat([]byte("x"), playerResponseLimit+1)} {
if _, _, err := playerMediaResource(body); err == nil || err.Error() != "invalid playable metadata" {
t.Fatal("metadata error contract changed", err)
}
}
for _, delta := range []time.Duration{-time.Hour, 19 * time.Minute, 21 * time.Minute} {
r := mediaResource{fmt.Sprintf("https://rr1---sn-alpha.googlevideo.com/videoplayback?expire=%d&lsig=secret", time.Now().Add(delta).Unix()), "rr1---sn-alpha.googlevideo.com", "video/mp4"}
if freshGooglevideoSample(r) != (delta > managedGooglevideoSampleTTL) {
t.Fatal("inventory freshness changed", delta)
}
}
}
func TestYoutubeLegacyJournalParity(t *testing.T) {
r, c, resources, _ := googlevideoRuntimeFixture(t)
p, err := r.verifyGooglevideo(context.Background(), c, resources)
if err != nil {
t.Fatal(err)
}
db := testStore(t)
tx, err := db.Begin()
if err != nil {
t.Fatal(err)
}
defer tx.Rollback()
id, err := insertStrategyCandidate(tx, c.Group, c.Endpoints, c.Args, nil, c.Mode)
if err != nil {
t.Fatal(err)
}
if err := saveAdaptiveGooglevideoEvidence(tx, id, p); err != nil {
t.Fatal(err)
}
if err := tx.Commit(); err != nil {
t.Fatal(err)
}
front := managedCandidateFixture(t, db, "frontend", "www.youtube.com", "--lua-desync=multisplit", true)
m, _ := managedFixture(t)
ids := []int64{front, id} // No new asset requirement for native suffix policy.
if err := m.Apply(db, ids, GooglevideoScopeSuffix); err != nil {
t.Fatal(err)
}
before, err := m.load()
if err != nil {
t.Fatal(err)
}
journalBytes, err := m.read(managedJournalName)
if err != nil {
t.Fatal(err)
}
config, err := m.read(m.Name)
if err != nil {
t.Fatal(err)
}
// Convert the actual producer facts to the exact historical v1 wire shape.
p.Attestation.CheckedAt = time.Now().Add(-25 * time.Hour)
raw, _ := json.Marshal(p)
var fields map[string]json.RawMessage
if err := json.Unmarshal(raw, &fields); err != nil {
t.Fatal(err)
}
delete(fields, "adapter")
fields["version"] = json.RawMessage(`1`)
raw, _ = json.Marshal(fields)
if _, err := db.Exec("UPDATE googlevideo_scope_evidence SET proof=? WHERE candidate_id=?", string(raw), id); err != nil {
t.Fatal(err)
}
rendered, err := renderStoredManagedConfigScope(db, ids, GooglevideoScopeSuffix)
if err != nil || !bytes.Equal(config, rendered) || managedHash(rendered) != before.Hash {
t.Fatal("historical render/hash changed", err)
}
if err := m.Reconcile(db); err != nil {
t.Fatal(err)
}
active, err := m.ActiveCandidates(db)
if err != nil || len(active) != 2 || active[0] != front || active[1] != id {
t.Fatal(active, err)
}
after, err := m.load()
if err != nil {
t.Fatal(err)
}
a, _ := json.Marshal(before)
b, _ := json.Marshal(after)
if !bytes.Equal(a, b) {
t.Fatal("historical journal changed")
}
afterBytes, err := m.read(managedJournalName)
if err != nil || !bytes.Equal(journalBytes, afterBytes) {
t.Fatal("historical journal bytes changed", err)
}
t.Logf("historical config SHA256=%s", before.Hash)
}
+43
View File
@@ -27,7 +27,16 @@ with tempfile.TemporaryDirectory(prefix='target-config-') as tmp:
ZAPRET_AUTOTUNE_YOUTUBE_SERVICE='0', ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE='cold',
ZAPRET_AUTOTUNE_VIDEO_ID='dQw4w9WgXcQ',
ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS='["dQw4w9WgXcQ"]',
ZAPRET_AUTOTUNE_TARGET_ROLE='managed-test',
ZAPRET_AUTOTUNE_TARGET_ID='zapret-autotune-managed-v1',
ZAPRET_AUTOTUNE_TARGET_DIR='/var/lib/zapret-autotune/target',
ZAPRET_CONFIG='/var/lib/zapret-autotune/target/config',
SOCKS_PORT='1080',
ZAPRET_AUTOTUNE_SOCKS_PROXY='socks5h://127.0.0.1:1080')
for name in ('ZAPRET_AUTOTUNE_RESOURCE_APPLY', 'ZAPRET_AUTOTUNE_RESOURCE_CONFIG',
'ZAPRET_AUTOTUNE_QUALIFY_CONFIG', 'ZAPRET_AUTOTUNE_STRATEGY_CONFIG',
'ZAPRET_AUTOTUNE_RESOURCE_STRATEGY_IDS'):
base.pop(name, None)
for dns, valid in (('', True), ('8.8.8.8', True), ('2001:4860:4860::8888', True),
('example.org', False), ('8.8.8.8\n', False), ('-d 8.8.8.8', False)):
for path in private.iterdir():
@@ -61,6 +70,28 @@ with tempfile.TemporaryDirectory(prefix='target-config-') as tmp:
assert base['SS_PASSWORD'].encode() not in result.stdout + result.stderr
if not valid:
assert not list(private.iterdir())
for overrides, valid in (
({'ZAPRET_AUTOTUNE_RESOURCE_STRATEGY_IDS': 's1'}, False),
({'ZAPRET_AUTOTUNE_RESOURCE_CONFIG': '/private/resources.json',
'ZAPRET_AUTOTUNE_RESOURCE_STRATEGY_IDS': 's2,s1'}, True),
({'ZAPRET_AUTOTUNE_RESOURCE_CONFIG': '/private/resources.json',
'ZAPRET_AUTOTUNE_RESOURCE_STRATEGY_IDS': 's1,s1'}, False),
({'ZAPRET_AUTOTUNE_RESOURCE_APPLY': '1'}, False),
({'ZAPRET_AUTOTUNE_RESOURCE_APPLY': 'yes'}, False),
({'ZAPRET_AUTOTUNE_RESOURCE_APPLY': '1', 'ZAPRET_AUTOTUNE_RESOURCE_CONFIG': '/private/resources.json'}, True),
({'ZAPRET_AUTOTUNE_RESOURCE_APPLY': '1', 'ZAPRET_AUTOTUNE_RESOURCE_CONFIG': '/private/resources.json',
'ZAPRET_AUTOTUNE_TARGET_ROLE': ''}, False),
({'ZAPRET_AUTOTUNE_RESOURCE_APPLY': '1', 'ZAPRET_AUTOTUNE_RESOURCE_CONFIG': '/private/resources.json',
'SOCKS_PORT': '1081'}, False),
({'ZAPRET_AUTOTUNE_RESOURCE_APPLY': '1', 'ZAPRET_AUTOTUNE_RESOURCE_CONFIG': '/private/resources.json',
'ZAPRET_AUTOTUNE_QUALIFY_CONFIG': '/private/resources.json', 'ZAPRET_AUTOTUNE_STRATEGY_CONFIG': '/private/strategies.json'}, False),
):
for path in private.iterdir():
path.unlink()
result = subprocess.run(['sh', str(entry)], env=dict(base, **overrides), capture_output=True, timeout=5)
assert result.returncode == (0 if valid else 64), result.stderr
if not valid:
assert not list(private.iterdir()), 'Invalid apply mode created service configs'
# Render privately, never print the config/password; config does not deploy.
compose_env = {k: v for k, v in base.items() if not k.startswith('ZAPRET_AUTOTUNE_')}
factory_cases = []
@@ -100,6 +131,18 @@ with tempfile.TemporaryDirectory(prefix='target-config-') as tmp:
factory_cases.append({'environment': {k: v for k, v in settings.items()
if k.startswith('ZAPRET_AUTOTUNE_')}, 'ids': ids})
# Validate the actual rendered DNS environment, not an invented equivalent.
result = subprocess.run(['docker', 'compose', '--env-file', '/dev/null', '-f',
str(ROOT / 'deploy/managed-target/compose.yaml'), 'config', '--format', 'json'],
env=dict(compose_env, ZAPRET_AUTOTUNE_RESOURCE_APPLY='1',
ZAPRET_AUTOTUNE_RESOURCE_STRATEGY_IDS='s2,s1',
ZAPRET_AUTOTUNE_RESOURCE_CONFIG='/private/resources.json',
ZAPRET_AUTOTUNE_STOP_GRACE_PERIOD='90s'), capture_output=True, timeout=10)
assert result.returncode == 0, 'Generic Compose config failed (output suppressed)'
target = json.loads(result.stdout)['services']['target']
assert target['environment']['ZAPRET_AUTOTUNE_RESOURCE_APPLY'] == '1'
assert target['environment']['ZAPRET_AUTOTUNE_RESOURCE_STRATEGY_IDS'] == 's2,s1'
assert target['environment']['ZAPRET_AUTOTUNE_RESOURCE_CONFIG'] == '/private/resources.json'
assert target['stop_grace_period'] == '1m30s'
# Compose must not inject Google bootstrap into a custom-provider request.
for service in ('0', '1'):
for provider in ({}, {'ZAPRET_AUTOTUNE_DOH_URL': 'https://secret.invalid/token'}):
@@ -0,0 +1,177 @@
"""Offline deterministic startup guards: python3 deploy/managed-target/check_startup_diagnostics.py."""
from pathlib import Path
import os
import stat
import subprocess
import tempfile
import time
ROOT = Path(__file__).resolve().parents[2]
entry = (ROOT / 'managed-target-entrypoint.sh').read_text()
capture = entry[entry.index('capture_monitor_session() {'):entry.index('# Wait with a watchdog:')]
launch = entry[entry.index('launch_controller() {'):entry.index('run_cycle() {')]
fixture_launch = launch.replace(' 8<&- 9<&- &', ' 8<&- 9<&-').replace(
'monitor_ack_deadline=$(($(date +%s) + 5))', 'monitor_ack_deadline=100')
secret = 'SECRET_SENTINEL'
roles = ('managed resource-apply --resource-config', 'managed monitor', 'qualify --isolated')
# Injected empty/malformed ACKs must still fail closed, not retry the read.
for role in roles:
for scenario, diagnostic in (
('empty', 'ack=read_failed'), ('malformed', 'ack=invalid_pid'),
('wrong', 'ack=identity_mismatch'), ('session', 'launch=session_mismatch'),
('gone', 'launch=child_gone'), ('timeout', 'launch=ack_timeout'),
):
with tempfile.TemporaryDirectory(prefix='startup-diagnostics-') as tmp:
script = f'''set -eu
config_dir='{tmp}'
monitor_sid=
scenario={scenario}
date() {{ echo 100; }}
setsid() {{
case "$scenario" in
empty) : > "$config_dir/monitor-ready" ;;
malformed) echo '{secret}' > "$config_dir/monitor-ready" ;;
wrong) echo 1 > "$config_dir/monitor-ready" ;;
session) echo "$!" > "$config_dir/monitor-ready" ;;
esac
}}
kill() {{ [ "$scenario" != gone ]; }}
signal_tree() {{ :; }}
proc_identity() {{ proc_sid=1; }}
{capture}
sleep 30 & seed=$!
{fixture_launch}
code=0
launch_controller {role} '{secret}' || code=$?
[ "$code" -eq 78 ]
[ -z "$monitor_sid" ]
command kill "$seed"
wait "$seed" 2>/dev/null || true
'''
result = subprocess.run(['sh', '-c', script], capture_output=True, timeout=3)
assert result.returncode == 0, (role, scenario, result.stderr)
assert result.stderr == f'controller-startup: {diagnostic}\n'.encode(), result.stderr
assert secret.encode() not in result.stdout + result.stderr
helper = (ROOT / 'managed-target-supervision.sh').read_text()
for role in roles:
with tempfile.TemporaryDirectory(prefix='delayed-ack-') as tmp:
root = Path(tmp)
wrapper_path = root / 'wrapper.sh'
write = ' printf \'%s\\n\' "$$" > "$ack_temp" || return 1'
assert helper.count(write) == 1
wrapper_path.write_text(helper.replace(write, ''' : > "$config_dir/barrier"
while [ ! -e "$config_dir/release" ]; do sleep 0.01; done
''' + write))
real_launch = launch.replace('/opt/managed-target-supervision.sh', str(wrapper_path)).replace(
'/usr/local/bin/zapret-autotune', '/bin/true')
process = subprocess.Popen(['sh', '-c', f'''set -eu
config_dir='{tmp}'
monitor_sid=
. '{ROOT / 'managed-target-supervision.sh'}'
{capture}
{real_launch}
launch_controller {role} '{secret}'
[ "$monitor_sid" = "$monitor_pid" ]
wait "$monitor_pid"
# An owned completed launch can be replaced for the next controller.
monitor_sid=
launch_controller {role} '{secret}'
[ "$monitor_sid" = "$monitor_pid" ]
wait "$monitor_pid"
'''], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
try:
deadline = time.monotonic() + 2
while not (root / 'barrier').exists():
assert process.poll() is None
assert time.monotonic() < deadline
time.sleep(0.01)
assert not (root / 'monitor-ready').exists()
pending, = root.glob('.monitor-ready.*')
assert pending.read_bytes() == b''
assert stat.S_IMODE(pending.stat().st_mode) == 0o600
(root / 'release').touch()
stdout, stderr = process.communicate(timeout=3)
assert process.returncode == 0, stderr
assert stderr == b'controller-startup: wrapper=exec_pending\n' * 2, stderr
ready = root / 'monitor-ready'
assert ready.read_text().strip().isdigit()
assert ready.read_bytes().endswith(b'\n')
assert ready.stat().st_nlink == 1
assert ready.stat().st_uid == os.getuid()
assert stat.S_IMODE(ready.stat().st_mode) == 0o600
assert not list(root.glob('.monitor-ready.*'))
assert secret.encode() not in stdout + stderr
finally:
(root / 'release').touch()
if process.poll() is None:
process.kill()
process.communicate(timeout=3)
wrapper = helper[helper.index('if [ "${1-}" = --monitor ]; then'):helper.index('elif [ "${1-}" = --watchdog ]; then')] + '\nfi\n'
for identity, diagnostic, code in (
('return 1', 'identity_unavailable', 78),
('proc_sid=1', 'not_session_leader', 78),
('proc_sid=$$', 'exec_pending', 0),
):
with tempfile.TemporaryDirectory(prefix='wrapper-diagnostics-') as tmp:
result = subprocess.run(['sh', '-c', f'proc_identity() {{ {identity}; }}\n{wrapper}',
'fixture', '--monitor', tmp, '/bin/true', secret], capture_output=True, timeout=3)
assert result.returncode == code, result.stderr
assert result.stderr == f'controller-startup: wrapper={diagnostic}\n'.encode(), result.stderr
ready = Path(tmp) / 'monitor-ready'
assert ready.exists() == (code == 0)
assert secret.encode() not in result.stdout + result.stderr
# Both publication and parent cleanup preserve unknown artifacts, including links.
for artifact in ('file', 'symlink', 'directory'):
with tempfile.TemporaryDirectory(prefix='foreign-ack-') as tmp:
root = Path(tmp)
ready = root / 'monitor-ready'
if artifact == 'file':
ready.write_text(secret)
elif artifact == 'symlink':
ready.symlink_to(root / 'absent')
else:
ready.mkdir()
before = ready.lstat()
result = subprocess.run(['sh', '-c', f'proc_identity() {{ proc_sid=$$; }}\n{wrapper}',
'fixture', '--monitor', tmp, '/bin/true', secret], capture_output=True, timeout=3)
assert result.returncode == 78, result.stderr
assert result.stderr == b'controller-startup: wrapper=ack_publish_failed\n', result.stderr
result = subprocess.run(['sh', '-c', f'''set -eu
config_dir='{tmp}'
{launch}
launch_controller managed monitor
'''], capture_output=True, timeout=3)
assert result.returncode == 78, result.stderr
assert result.stderr == b'controller-startup: launch=ack_conflict\n', result.stderr
assert ready.lstat() == before
if artifact == 'file':
assert ready.read_text() == secret
assert not list(root.glob('.monitor-ready.*'))
with tempfile.TemporaryDirectory(prefix='replaced-ack-') as tmp:
result = subprocess.run(['sh', '-c', f'''set -eu
config_dir='{tmp}'
printf '123\n' > "$config_dir/monitor-ready"
monitor_ready_identity=$(stat -c '%d:%i:%u:%a:%h' "$config_dir/monitor-ready")
monitor_ready_pid=123
printf '{secret}' > "$config_dir/replacement"
mv "$config_dir/replacement" "$config_dir/monitor-ready"
{launch}
launch_controller managed monitor
'''], capture_output=True, timeout=3)
assert result.returncode == 78, result.stderr
assert result.stderr == b'controller-startup: launch=ack_conflict\n'
assert (Path(tmp) / 'monitor-ready').read_text() == secret
with tempfile.TemporaryDirectory(prefix='unsafe-ack-dir-') as tmp:
os.chmod(tmp, 0o755)
result = subprocess.run(['sh', '-c', f'proc_identity() {{ proc_sid=$$; }}\n{wrapper}',
'fixture', '--monitor', tmp, '/bin/true'], capture_output=True, timeout=3)
assert result.returncode == 78, result.stderr
assert result.stderr == b'controller-startup: wrapper=ack_publish_failed\n'
assert not list(Path(tmp).iterdir())
print('startup diagnostic fixtures passed (shared apply/monitor/qualify launch and atomic wrapper)')
+19 -2
View File
@@ -4,6 +4,19 @@ from pathlib import Path
directory = Path(__file__).resolve().parent
root = directory.parents[1]
# ponytail: guard the current literal COPY and deny-all/allowlist ignore shape;
# use a Dockerfile/ignore parser if these become general patterns or multi-module.
ignore = [line for line in (root / ".dockerignore").read_text().splitlines()
if line and not line.startswith("#")]
assert ignore[0] == "*" and all(line.startswith("!") for line in ignore[1:])
package_roots = {path.relative_to(root).parts[0] for path in root.rglob("*.go")}
for name in ("Dockerfile", "Dockerfile.target"):
builder = (root / name).read_text().split("\nFROM ", 1)[0]
before_build = builder[:builder.index("go build")]
for package in package_roots:
assert f"COPY {package}/ ./{package}/\n" in before_build, (name, package)
assert f"!{package}/" in ignore and f"!{package}/**" in ignore, package
compose = (directory / "compose.yaml").read_text()
entrypoint = (directory.parents[1] / "managed-target-entrypoint.sh").read_text()
readme = (directory / "README.md").read_text()
@@ -55,7 +68,8 @@ assert "Docker administrators/root can still read container" in readme
assert "removed\nfrom the entrypoint environment" in readme
assert "No host networking" in readme
assert ' restart: "no"\n' in compose
assert " stop_grace_period: 240s\n" in compose
assert " stop_grace_period: ${ZAPRET_AUTOTUNE_STOP_GRACE_PERIOD-240s}\n" in compose
assert 'ZAPRET_AUTOTUNE_RESOURCE_APPLY: ${ZAPRET_AUTOTUNE_RESOURCE_APPLY-0}' in compose
assert 'bounded_wait "$monitor_pid" 45' in entrypoint
assert '/bin/sh /opt/managed-target-supervision.sh --watchdog' in entrypoint
assert 'setsid /bin/sh /opt/managed-target-supervision.sh --monitor' in entrypoint
@@ -77,6 +91,9 @@ for command, seconds in (("/usr/local/bin/zapret-autotune managed init", 45),
assert entrypoint.index("managed reconcile || return 78") < entrypoint.index("zapret2 start || return 78")
experimental = 'managed apply --googlevideo-experiment native-circular --googlevideo-scope experimental_googlevideo_suffix --googlevideo-request startup || exit 78'
assert entrypoint.index('zapret2 start || return 78') < entrypoint.index(experimental)
assert entrypoint.index(experimental) < entrypoint.index('setsid /bin/sh /opt/managed-target-supervision.sh --monitor')
assert entrypoint.index(experimental) < entrypoint.index('launch_controller "$@" || return 78')
generic_apply = 'launch_controller managed resource-apply --resource-config "$ZAPRET_AUTOTUNE_RESOURCE_CONFIG"'
assert entrypoint.index('managed reconcile || return 78') < entrypoint.index(generic_apply) < entrypoint.index('zapret2 start || return 78')
assert entrypoint.index(generic_apply) < entrypoint.index('set -- managed monitor')
assert entrypoint.index('managed experimental-request-check || return 78') < entrypoint.index('managed init || return 78')
assert 'rm "${ZAPRET_CONFIG%/*}/.autotune-experimental-suffix-request"' not in entrypoint
+250 -4
View File
@@ -24,7 +24,7 @@ DNS_CLI = Path(CLI_TMP.name) / 'autotune'
subprocess.run(['go', 'build', '-o', str(DNS_CLI), './cmd/zapret-autotune'],
cwd=ROOT, env=dict(os.environ, GOPROXY='off', GOTOOLCHAIN='local'), check=True)
FAKE = r'''#!/usr/bin/env python3
import ctypes, os, pathlib, signal, subprocess, sys, time
import ctypes, json, os, pathlib, signal, subprocess, sys, time
root = pathlib.Path(os.environ['FIXTURE'])
name = pathlib.Path(sys.argv[0]).name
scenario = os.environ['SCENARIO']
@@ -38,7 +38,7 @@ def increment(key):
return n
def linger(code=0):
def stop(*_):
if os.environ.get('ZAPRET_AUTOTUNE_DNS_MODE', 'doh') == 'doh' and scenario not in ('dns-death', 'dns-killed', 'dns-replace', 'dns-replace-symlink', 'dns-prestart'):
if os.environ.get('ZAPRET_AUTOTUNE_DNS_MODE', 'doh') == 'doh' and scenario not in ('dns-death', 'dns-killed', 'dns-replace', 'dns-replace-symlink', 'dns-prestart', 'generic-apply-dns-loss', 'generic-apply-dns-restart'):
os.kill(int((root / 'dns-pid').read_text()), 0)
sys.exit(code)
signal.signal(signal.SIGTERM, stop)
@@ -67,10 +67,26 @@ if name == 'awk':
sys.exit(0)
if name == 'date':
n = increment('clock')
print(100000 + (n * 21601 if scenario == 'window' else n + (21601 if (root / 'budget-pause').exists() else 0)))
elapsed = int((root / 'elapsed').read_text()) if (root / 'elapsed').exists() else 0
print(100000 + elapsed + (n * 21601 if scenario == 'window' else n + (21601 if (root / 'budget-pause').exists() else 0)))
sys.exit(0)
if name == 'sleep':
seconds = int(sys.argv[1])
if seconds == 60 and scenario.startswith('generic-apply-'):
# Three former 60s retry windows; the failed producer becomes healthy
# after its first call, so any unintended retry would reach native READY.
n = increment('blocked_windows')
(root / 'elapsed').write_text(str(n * 60))
log('blocked-window ' + str(n))
if n == 3:
os.kill(os.getppid(), signal.SIGTERM)
time.sleep(.01)
sys.exit(0)
if scenario.startswith('qualify-') and (root / 'qualified').exists():
log('safe-idle')
os.kill(os.getppid(), signal.SIGTERM)
time.sleep(.02)
sys.exit(0)
if scenario.startswith('generic-budget-') and seconds != 60:
os.execv(os.environ['REAL_SLEEP'], [os.environ['REAL_SLEEP'], *sys.argv[1:]])
if seconds == 60 and scenario in ('generic-ss', 'generic-unsafe', 'generic-budget-monitor', 'generic-budget-reconcile', 'generic-budget-ack'):
@@ -105,6 +121,8 @@ if name in ('ss-server', 'ss-local'):
os.kill(int((root / 'dns-pid').read_text()), 0)
assert (pathlib.Path(sys.argv[2]).stat().st_mode & 0o777) == 0o600
log(name)
if name == 'ss-local':
(root / 'ss-pid').write_text(str(os.getpid()))
if scenario.startswith('generic-budget-'):
signal.signal(signal.SIGTERM, signal.SIG_IGN)
while True:
@@ -125,8 +143,32 @@ if name == 'zapret2':
sys.exit(1)
sys.exit(0)
assert name == 'autotune'
if sys.argv[1] == 'qualification-race-barrier':
# Deterministic interleaving: child has exited, shell has not reached wait.
pid = int(sys.argv[2])
deadline = time.monotonic() + 5
while True:
stat = pathlib.Path('/proc/' + str(pid) + '/stat')
if not stat.exists() or stat.read_text().rsplit(') ', 1)[1].split()[0] == 'Z':
break
assert time.monotonic() < deadline, 'qualifier did not exit at barrier'
time.sleep(.001)
log('term-before-parent-reap')
os.kill(os.getppid(), signal.SIGTERM)
sys.exit(0)
if sys.argv[1] == 'qualification-completion-check':
ack = root / 'completion-ack'
if not ack.exists() or ack.is_symlink() or ack.stat().st_mode & 0o777 != 0o600:
sys.exit(78)
payload = json.loads(ack.read_text())
assert payload == {'pid': int(sys.argv[2]), 'code': int(sys.argv[3]), 'cleanup_complete': True}
log('completion-checked')
sys.exit(0)
if sys.argv[1:] == ['target-config-check']:
sys.exit(0)
if sys.argv[1:] == ['apply-launch']:
log('apply-launch')
sys.exit(0)
if sys.argv[1] == 'dns-forwarder':
if sys.argv[2] in ('readiness-check', 'readiness-cleanup'):
# exec preserves PID1's actual inherited anchors; no simulated unlink.
@@ -194,6 +236,40 @@ if sys.argv[1] == 'dns-forwarder':
os.kill(os.getpid(), signal.SIGKILL)
while True:
time.sleep(.01)
if sys.argv[1] == 'qualify':
assert sys.argv[2:] == ['--config', '/operator/resources.json', '--strategies', '/operator/strategies.json', '--isolated']
assert os.getsid(0) == os.getpid(), 'qualifier lacks owned session'
assert 'SS_PASSWORD' not in os.environ
events = (root / 'events').read_text().splitlines()
assert 'start' not in events and 'stop' not in events
assert events[-1] == 'reconcile', events
log('qualify')
assert increment('qualification_runs') == 1, 'qualification restarted'
def acknowledge(code):
ack = root / 'completion-ack'
ack.write_text(json.dumps({'pid': os.getpid(), 'code': code, 'cleanup_complete': True}))
ack.chmod(0o600)
if not scenario.endswith('-race'):
(root / 'qualified').touch()
if scenario in ('qualify-unsafe', 'qualify-unsafe-race'):
if not scenario.endswith('-race'):
(root / 'qualified').touch()
sys.exit(78)
if scenario in ('qualify-blocked', 'qualify-blocked-race'):
acknowledge(78)
sys.exit(78)
if scenario == 'qualify-cancel':
def restored(*_):
log('qualification-restored')
acknowledge(0)
sys.exit(0)
signal.signal(signal.SIGTERM, restored)
os.kill(os.getppid(), signal.SIGTERM)
while True:
time.sleep(.01)
log('qualification-restored')
acknowledge(0)
sys.exit(0)
command = sys.argv[2]
if command == 'init':
log('init')
@@ -224,6 +300,14 @@ elif command == 'monitor':
f.write(str(os.getpid()) + '\n')
n = increment('monitor_runs')
log('monitor ' + str(n))
if n == 1 and scenario == 'generic-apply-monitor-restart':
sys.exit(75)
if n == 1 and scenario == 'generic-apply-ss-restart':
os.kill(int((root / 'ss-pid').read_text()), signal.SIGKILL)
linger()
if n == 1 and scenario == 'generic-apply-dns-restart':
os.kill(int((root / 'dns-pid').read_text()), signal.SIGKILL)
linger()
assert os.getsid(0) == os.getpid(), 'monitor not in its own verified session'
if scenario.startswith('generic-budget-'):
time.sleep(.03)
@@ -265,6 +349,40 @@ elif command == 'monitor':
time.sleep(.03)
os.kill(os.getppid(), signal.SIGTERM)
linger()
elif command == 'resource-apply':
assert sys.argv[3:] == ['--resource-config', '/operator/resources.json']
assert os.environ['ZAPRET_AUTOTUNE_RESOURCE_APPLY'] == '1'
assert os.environ['ZAPRET_AUTOTUNE_RESOURCE_STRATEGY_IDS'] == 's2,s1'
assert os.getsid(0) == os.getpid(), 'apply lacks owned session'
assert (root / 'private' / 'monitor-ready').read_text() == str(os.getpid()) + '\n'
events = (root / 'events').read_text().splitlines()
assert events[-2:] == ['reconcile', 'apply-launch'] and 'start' not in events
log('resource-apply')
n = increment('apply_runs')
if scenario == 'generic-apply-blocked' and n == 1:
sys.exit(78)
if scenario == 'generic-apply-abrupt' and n == 1:
sys.exit(23)
if scenario == 'generic-apply-cleanup' and n == 1:
log('resource-returned-zero')
sys.exit(0)
if scenario == 'generic-apply-signal' and n == 1:
os.kill(os.getpid(), signal.SIGKILL)
if scenario in ('generic-apply-dns-loss', 'generic-apply-ss-loss') and n == 1:
os.kill(int((root / ('dns-pid' if scenario.endswith('dns-loss') else 'ss-pid')).read_text()), signal.SIGKILL)
signal.signal(signal.SIGTERM, lambda *_: sys.exit(0))
while True:
time.sleep(.01)
if scenario == 'generic-apply-cancel':
def restored(*_):
log('resource-restored')
sys.exit(0)
signal.signal(signal.SIGTERM, restored)
os.kill(os.getppid(), signal.SIGTERM)
while True:
time.sleep(.01)
log('resource-applied')
sys.exit(0)
'''
@@ -284,6 +402,8 @@ def run(scenario, expected, dns='', mode='native'):
(binary / name).unlink()
configs = root / 'private'
configs.mkdir()
if scenario == 'generic-apply-disabled':
(root / 'prepared').touch()
if scenario.startswith('dns-stale'):
if scenario == 'dns-stale-symlink':
(configs / 'dns-ready').symlink_to(configs / 'missing')
@@ -304,6 +424,25 @@ def run(scenario, expected, dns='', mode='native'):
text = text.replace('config_dir=/run/zapret-autotune-ss', f'config_dir={configs}')
text = text.replace('/usr/local/bin/zapret-autotune', str(binary / 'autotune'))
text = text.replace('/opt/zapret2/init.d/sysv/zapret2', str(binary / 'zapret2'))
if scenario.startswith('generic-apply-'):
launch = ' launch_controller managed resource-apply'
assert launch in text
text = text.replace(launch, f' {binary / "autotune"} apply-launch\n' + launch)
if scenario == 'generic-apply-launch-fail':
text = text.replace(' launch_controller managed resource-apply', ' false || return 78\n launch_controller managed resource-apply')
if scenario == 'generic-apply-ack-fail':
broken_helper = root / 'ack-helper.sh'
broken_helper.write_text(helper.read_text().replace(' ack_publish ||', ' false && ack_publish ||'))
text = text.replace(str(helper), str(broken_helper))
if scenario == 'generic-apply-cleanup':
text = text.replace('cleanup_session || { echo', "cleanup_session() { return 78; }\n cleanup_session || { echo", 1)
if scenario.endswith('-race'):
barrier = f'{binary / "autotune"} qualification-race-barrier "$monitor_pid"\n'
loop = 'while dns_alive && kill -0 "$server_pid" && kill -0 "$local_pid" && kill -0 "$monitor_pid"; do'
assert loop in text
# The apply child has its own wait loop before this monitor loop.
before, after = text.rsplit(loop, 1)
text = before + barrier + loop + after
if realtime:
# Private fixture scaling only: production has no configurable grace.
text = text.replace('$(date +%s) + 80', '$(date +%s) + 12')
@@ -317,6 +456,17 @@ def run(scenario, expected, dns='', mode='native'):
env['ZAPRET_AUTOTUNE_RESOURCE_CONFIG'] = '/operator/resources.json'
else:
env.pop('ZAPRET_AUTOTUNE_RESOURCE_CONFIG', None)
env.pop('ZAPRET_AUTOTUNE_QUALIFY_CONFIG', None)
env.pop('ZAPRET_AUTOTUNE_STRATEGY_CONFIG', None)
env.pop('ZAPRET_AUTOTUNE_RESOURCE_APPLY', None)
env.pop('ZAPRET_AUTOTUNE_RESOURCE_STRATEGY_IDS', None)
if scenario.startswith('generic-apply-') and scenario != 'generic-apply-disabled':
env['ZAPRET_AUTOTUNE_RESOURCE_APPLY'] = '1'
env['ZAPRET_AUTOTUNE_RESOURCE_STRATEGY_IDS'] = 's2,s1'
if scenario.startswith('qualify-'):
env['ZAPRET_AUTOTUNE_QUALIFY_CONFIG'] = '/operator/resources.json'
if scenario != 'qualify-missing':
env['ZAPRET_AUTOTUNE_STRATEGY_CONFIG'] = '/operator/strategies.json'
if mode is None:
env.pop('ZAPRET_AUTOTUNE_DNS_MODE', None)
else:
@@ -341,7 +491,12 @@ def run(scenario, expected, dns='', mode='native'):
peer.wait(timeout=2)
events = (root / 'events').read_text().splitlines() if (root / 'events').exists() else []
assert result.returncode == expected, (scenario, result.returncode, result.stderr, events)
assert b'Illegal number' not in result.stderr, (scenario, result.stderr)
assert b'ab' * 32 not in result.stdout + result.stderr
if scenario in ('generic-apply-blocked', 'generic-apply-abrupt', 'generic-apply-cleanup'):
code = {'generic-apply-blocked': 78, 'generic-apply-abrupt': 23, 'generic-apply-cleanup': 0}[scenario]
assert f'controller-startup: apply_wait_exit={code}\n'.encode() in result.stderr, result.stderr
assert (b'controller-startup: apply_cleanup=failed\n' in result.stderr) == (scenario == 'generic-apply-cleanup'), result.stderr
for pid in ((root / 'children').read_text().splitlines() if (root / 'children').exists() else []):
assert not Path(f'/proc/{pid}').exists(), (scenario, 'child not joined', pid)
if scenario == 'orphans':
@@ -357,7 +512,7 @@ def run(scenario, expected, dns='', mode='native'):
# Every activation and stop must immediately follow successful reconcile.
for i, event in enumerate(events):
if event in ('start', 'stop'):
assert events[i - 1] == 'reconcile', (scenario, events)
assert events[i - 1] == 'reconcile' or event == 'start' and events[i - 1] == 'resource-applied', (scenario, events)
foreign = scenario in ('dns-symlink', 'dns-preack-foreign', 'dns-replace', 'dns-replace-symlink', 'dns-stale', 'dns-stale-symlink')
if foreign:
ready = configs / 'dns-ready'
@@ -371,6 +526,15 @@ def run(scenario, expected, dns='', mode='native'):
assert not (configs / 'dns-ready').exists(), (scenario, 'stale DNS readiness')
if mode == 'native':
assert 'dns-start' not in events
if scenario.startswith('generic-apply-') and 'blocked-window 1' in events:
assert events.count('apply-launch') == 1, (scenario, events)
assert int((root / 'elapsed').read_text()) == 180, (scenario, events)
assert events.count('dns-start') == events.count('ss-server') == events.count('ss-local') == 1, (scenario, events)
assert 'start' not in events and not any(e.startswith('monitor ') for e in events), (scenario, events)
assert 'cooldown 15' not in events and 'cooldown 30' not in events, (scenario, events)
assert events[events.index('blocked-window 1'):] == ['blocked-window 1', 'blocked-window 2', 'blocked-window 3'], (scenario, events)
if scenario != 'generic-apply-dns-loss':
assert events.index('dns-stop') < events.index('blocked-window 1'), (scenario, events)
return events
@@ -401,6 +565,32 @@ assert events.count('generic-pause') == 1, events
assert events.count('dns-start') == events.count('dns-stop') == 4, events
events = run('generic-unsafe', 0)
assert 'start' not in events and events.count('generic-pause') == 1, events
events = run('generic-apply-pass', 0, mode='doh')
assert events.count('resource-apply') == 1 and events.count('start') == 1, events
assert events.index('resource-applied') < events.index('start') < events.index('monitor 1'), events
events = run('generic-apply-cancel', 0, mode='doh')
assert 'resource-restored' in events and 'start' not in events and 'monitor 1' not in events, events
assert events.index('resource-restored') < events.index('stop') < events.index('dns-stop'), events
events = run('generic-apply-blocked', 0, mode='doh')
assert events.count('resource-apply') == 1 and 'blocked-window 3' in events, events
for scenario, code in (('generic-apply-abrupt', 0), ('generic-apply-cleanup', 78)):
events = run(scenario, code, mode='doh')
assert 'start' not in events and 'monitor 1' not in events and 'resource-applied' not in events, events
for scenario in ('signal', 'dns-loss', 'ss-loss', 'launch-fail', 'ack-fail'):
events = run('generic-apply-' + scenario, 78 if scenario == 'ack-fail' else 0, mode='doh')
assert 'blocked-window 3' in events and 'resource-applied' not in events, events
assert events.count('resource-apply') == (0 if scenario in ('launch-fail', 'ack-fail') else 1), events
for scenario in ('monitor-restart', 'ss-restart', 'dns-restart'):
events = run('generic-apply-' + scenario, 0, mode='doh')
assert events.count('resource-apply') == events.count('apply-launch') == 1, events
assert events.count('start') == events.count('stop') == 2 and 'monitor 2' in events, events
assert events.count('dns-start') == 2 and events.count('dns-stop') == (1 if scenario == 'dns-restart' else 2), events
# Independent fresh PID 1 invocations reset the RAM latch; flag=0 still repairs
# prepared bytes by reconcile, without launching apply.
events = run('generic-apply-pass', 0, mode='doh')
assert events.count('resource-apply') == 1, events
events = run('generic-apply-disabled', 0, mode='doh')
assert 'resource-apply' not in events and 'start' in events, events
for scenario in ('generic-budget-monitor', 'generic-budget-reconcile', 'generic-budget-ack'):
events = run(scenario, 78, mode='doh')
assert events.count('start') == 1 and 'stop' not in events, (scenario, events)
@@ -432,6 +622,25 @@ for scenario in ('dns-term', 'dns-int'):
events = run(scenario, 0, mode='doh')
assert events == ['dns-start', 'dns-stop'], events
for scenario in ('qualify-pass', 'qualify-negative', 'qualify-blocked', 'qualify-cancel'):
events = run(scenario, 0, mode='doh')
assert events.count('qualify') == 1 and not any(e.startswith('monitor ') for e in events), events
assert 'start' not in events and 'stop' not in events, events
assert events.index('dns-ready') < events.index('qualify') < events.index('dns-stop'), events
if scenario != 'qualify-blocked':
assert events.index('qualification-restored') < events.index('dns-stop'), events
events = run('qualify-missing', 64, mode='doh')
assert not events, events
for scenario, code in (('qualify-blocked-race', 0), ('qualify-unsafe-race', 78), ('qualify-unsafe', 78)):
events = run(scenario, code, mode='doh')
assert events.count('qualify') == 1 and 'start' not in events and 'stop' not in events, events
if scenario.endswith('-race'):
assert events.count('term-before-parent-reap') == 1, events
if code == 0:
assert events.index('term-before-parent-reap') < events.index('completion-checked') < events.index('dns-stop'), events
else:
assert 'completion-checked' not in events, events
def check_watchdog_traps(shell):
"""Same bounded_wait from normal flow, plain EXIT, and TERM -> EXIT traps."""
@@ -476,10 +685,47 @@ esac
assert not Path(f'/proc/{pid}').exists(), (shell, mode, 'orphan sleeper', pid)
def check_session_deadline(shell):
"""Startup sweep has no aggregate deadline; termination must keep its cap."""
entrypoint = (ROOT / 'managed-target-entrypoint.sh').read_text()
cleanup_session = entrypoint[entrypoint.index('cleanup_session() {'):entrypoint.index('capture_monitor_session() {')]
for deadline, live_until, expected, elapsed in (('', 102, 0, 2), ('180', 102, 0, 2), ('101', 110, 78, 1)):
with tempfile.TemporaryDirectory(prefix='session-deadline-') as tmp:
script = Path(tmp) / 'check.sh'
script.write_text(f'''set -eu
generic_mode=1
generic_cleanup_deadline='{deadline}'
monitor_sid=42
clock=100
date() {{ printf '%s\\n' "$clock"; }}
pause() {{ clock=$((clock + $1)); }}
signal_session() {{
[ "$1" = 42 ] || return 78
case "$2" in
TERM|KILL) ;;
CHECK) session_live=1; [ "$clock" -lt {live_until} ] || session_live=0 ;;
*) return 78 ;;
esac
}}
{cleanup_session}
code=0
cleanup_session || code=$?
[ "$code" -eq {expected} ]
[ "$clock" -eq {100 + elapsed} ]
[ "$generic_cleanup_deadline" = '{deadline}' ]
''')
result = subprocess.run([*shell, str(script)], capture_output=True, timeout=2)
assert result.returncode == 0, (shell, deadline, result.stderr)
expected_stderr = b'error: owned monitor session survived cleanup\n' if expected else b''
assert result.stderr == expected_stderr, (shell, deadline, result.stderr)
check_watchdog_traps(['/bin/sh'])
check_session_deadline(['/bin/sh'])
busybox = shutil.which('busybox')
if busybox:
check_watchdog_traps([busybox, 'ash'])
check_session_deadline([busybox, 'ash'])
for pid, seconds in (('1', '5'), ('0', '5'), ('-1', '5'), ('42', '0'), ('42', 'bad')):
result = subprocess.run(['/bin/sh', str(ROOT / 'managed-target-supervision.sh'), '--watchdog', pid, seconds, '/'], capture_output=True, timeout=2)
assert result.returncode == 64
+7 -1
View File
@@ -19,6 +19,11 @@ services:
ZAPRET_AUTOTUNE_DOH_TIMEOUT_SEC: ${ZAPRET_AUTOTUNE_DOH_TIMEOUT_SEC-5}
ZAPRET_AUTOTUNE_YOUTUBE_SERVICE: ${ZAPRET_AUTOTUNE_YOUTUBE_SERVICE-0}
ZAPRET_AUTOTUNE_RESOURCE_CONFIG: ${ZAPRET_AUTOTUNE_RESOURCE_CONFIG-}
ZAPRET_AUTOTUNE_RESOURCE_APPLY: ${ZAPRET_AUTOTUNE_RESOURCE_APPLY-0}
ZAPRET_AUTOTUNE_RESOURCE_STRATEGY_IDS: ${ZAPRET_AUTOTUNE_RESOURCE_STRATEGY_IDS-}
# INTRUSIVE isolated startup batch. Never route working traffic here.
ZAPRET_AUTOTUNE_QUALIFY_CONFIG: ${ZAPRET_AUTOTUNE_QUALIFY_CONFIG-}
ZAPRET_AUTOTUNE_STRATEGY_CONFIG: ${ZAPRET_AUTOTUNE_STRATEGY_CONFIG-}
ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE: ${ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE-cold}
ZAPRET_AUTOTUNE_VIDEO_ID: ${ZAPRET_AUTOTUNE_VIDEO_ID-dQw4w9WgXcQ}
ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS: '${ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS:-}'
@@ -42,7 +47,8 @@ services:
- /run/zapret-autotune-ss:rw,noexec,nosuid,nodev,size=1m,mode=0700,uid=0,gid=0
networks: [target]
restart: "no"
stop_grace_period: 240s
# Generic mode uses 90s; retain the legacy adapter's longer default.
stop_grace_period: ${ZAPRET_AUTOTUNE_STOP_GRACE_PERIOD-240s}
sysctls:
net.ipv4.ip_unprivileged_port_start: "0"
networks:
+35
View File
@@ -0,0 +1,35 @@
// Package adapter defines the closed internal contract used by compiled-in
// service adapters. It is not the public resource-check configuration API.
package adapter
import "time"
type Binding struct {
Kind string `json:"kind"`
Version int `json:"version"`
PolicySHA256 string `json:"policy_sha256"`
}
// Resource and Plan are ephemeral; neither private inputs nor policy caps belong
// in evidence. Only core-validated, measured facts can become a sealed proof.
type Resource struct {
URL string `json:"-"`
Host string `json:"-"`
Kind string `json:"-"`
}
type Plan struct {
Hosts []string `json:"-"`
Protocols [2]string `json:"-"`
WarmupLimit int `json:"-"`
WarmupTimeout time.Duration `json:"-"`
Samples int `json:"-"`
MinBytes int64 `json:"-"`
}
type Module interface {
Binding() Binding
DiscoverResponse([]byte) ([]Resource, error)
ValidateHosts([]string) error
PlanQualification([]string, []Resource, time.Time) (Plan, error)
}
+29
View File
@@ -0,0 +1,29 @@
# Internal YouTube adapter
The compiled-in `youtube/1` module parses bounded Player JSON and supplies the
ordinary signed MP4 qualification policy and ordered host/TLS sample plan.
The existing YouTube execution paths select it explicitly through the core's
kind/version registry. Unsupported or absent modules fail closed.
The core retains origin validation, HTTPS/SNI/certificate checks, transport
framing/body bounds, retries/cancellation, worker ownership, cleanup/join and
proof sealing. Module output is not evidence or permission to claim PASS.
Signed URLs remain ephemeral and cannot serialize through the internal contract.
New adaptive proof v2 binds the module kind, version and immutable policy digest
inside its producer seal. Unbound proof v1 explicitly means the historical
YouTube v1 policy: fresh trusted stored proofs retain their original apply
eligibility; expired proofs reconstruct history but cannot authorize fresh apply.
Only new producer/persistence writes require sealed module-bound v2 evidence.
Cold historical evidence is unchanged. Discovery entrypoints preflight the module
before requests; absence/mismatched identity or policy is a fatal configuration
failure, not unavailable inventory or a no-strategy cooldown.
Legacy routing remains SOCKS5h with encrypted-resolver ownership guards and
origin/SNI/certificate/TLS checks, not generic all-answer approval/IP pinning.
Full request-policy capabilities and parity are prerequisites to generic cutover.
This is a real internal legacy-execution integration, not a fully generic
controller, public v3 configuration, native addon, runtime plugin or readiness
claim. Another compiled-in module can implement the closed internal contract;
public GET/HEAD v1/v2 configuration remains unchanged.
+120
View File
@@ -0,0 +1,120 @@
// Package youtube contains pure service semantics, not transport or worker ownership.
package youtube
import (
"crypto/sha256"
"encoding/json"
"errors"
"fmt"
"mime"
"net/url"
"strconv"
"strings"
"time"
"zapret-autotune/internal/adapter"
)
const Kind = "youtube"
const Version = 1
const InventoryTTL = 20 * time.Minute
// Immutable v1 policy. Changes require another module version, not a new meaning
// for existing evidence. Legacy proof v1 uses this same historical policy.
const policyV1 = "youtube/1;ggc>=3;roots>=2;prefixes>=2;ordinary-signed-mp4;fresh>120s;inventory>20m;tls12,tls13;warmup<=10,120s;steady=3/protocol;min=1024"
type Binding = adapter.Binding
type Module struct{}
func (Module) Binding() Binding {
return Binding{Kind: Kind, Version: Version, PolicySHA256: fmt.Sprintf("%x", sha256.Sum256([]byte(policyV1)))}
}
// Resource is ephemeral: never serialize signed URLs or inventory metadata.
type Resource = adapter.Resource
// DiscoverResponse preserves player format order. The core must validate every
// returned origin before selecting or fetching it. SABR is not supported.
func (Module) DiscoverResponse(body []byte) ([]Resource, error) {
type format struct {
URL string `json:"url"`
MimeType string `json:"mimeType"`
}
var p struct {
PlayabilityStatus struct {
Status string `json:"status"`
} `json:"playabilityStatus"`
StreamingData struct {
Formats []format `json:"formats"`
AdaptiveFormats []format `json:"adaptiveFormats"`
} `json:"streamingData"`
}
if len(body) > 1<<20 || json.Unmarshal(body, &p) != nil || p.PlayabilityStatus.Status != "OK" {
return nil, errors.New("invalid playable metadata")
}
var resources []Resource
for _, f := range append(p.StreamingData.Formats, p.StreamingData.AdaptiveFormats...) {
u, err := url.Parse(f.URL)
if err != nil || !strings.HasSuffix(u.Hostname(), ".googlevideo.com") {
continue
}
kind, _, err := mime.ParseMediaType(f.MimeType)
if err == nil && (kind == "video/mp4" || kind == "audio/mp4") {
resources = append(resources, Resource{URL: f.URL, Host: u.Hostname(), Kind: kind})
}
}
return resources, nil
}
// ValidateHosts checks service diversity; DNS/origin policy remains in the core.
func (Module) ValidateHosts(hosts []string) error {
seen, roots, prefixes := map[string]bool{}, map[string]bool{}, map[string]bool{}
for _, host := range hosts {
prefix, root, ok := strings.Cut(strings.TrimSuffix(host, ".googlevideo.com"), "---sn-")
if !ok || !strings.HasSuffix(host, ".googlevideo.com") || host != strings.ToLower(host) || seen[host] || root == "" || len(prefix) < 3 || !strings.HasPrefix(prefix, "rr") || strings.Trim(prefix[2:], "0123456789") != "" {
return errors.New("insufficient representative GGC hosts")
}
seen[host], roots[strings.Split(root, "-")[0]], prefixes[prefix] = true, true, true
}
if len(hosts) < 3 || len(roots) < 2 || len(prefixes) < 2 {
return errors.New("insufficient representative GGC hosts")
}
return nil
}
type Plan = adapter.Plan
// PlanQualification grants no PASS capability. Core validates origins first,
// executes the ordered schedule, validates measured facts, and seals after join.
func (m Module) PlanQualification(hosts []string, resources []Resource, now time.Time) (Plan, error) {
p := Plan{Hosts: append([]string(nil), hosts...), Protocols: [2]string{"tls12", "tls13"}, WarmupLimit: 10, WarmupTimeout: 120 * time.Second, Samples: 3, MinBytes: 1024}
if err := m.ValidateHosts(hosts); err != nil {
return Plan{}, err
}
seen := map[string]bool{}
for _, r := range resources {
if !SignedSampleFresh(r, now, 120*time.Second) {
return Plan{}, errors.New("insufficient fresh signed media samples")
}
seen[r.Host] = true
}
for _, host := range hosts {
if !seen[host] {
return Plan{}, errors.New("missing representative sample")
}
}
return p, nil
}
// SignedSampleFresh checks service semantics only, not permission to fetch.
// The core independently validates HTTPS origins before calling this helper.
func SignedSampleFresh(r Resource, now time.Time, ttl time.Duration) bool {
u, err := url.Parse(r.URL)
if err != nil || u.Path != "/videoplayback" || (r.Kind != "video/mp4" && r.Kind != "audio/mp4") {
return false
}
q := u.Query()
expire, err := strconv.ParseInt(q.Get("expire"), 10, 64)
return err == nil && expire > now.Add(ttl).Unix() && (q.Get("sig") != "" || q.Get("signature") != "" || q.Get("lsig") != "")
}
+90
View File
@@ -0,0 +1,90 @@
package youtube
import (
"encoding/json"
"fmt"
"strings"
"testing"
"time"
)
func TestDiscoverResponse(t *testing.T) {
m := Module{}
resources, err := m.DiscoverResponse([]byte(`{"playabilityStatus":{"status":"OK"},"streamingData":{"formats":[{"url":"https://rr1.unit.googlevideo.com/videoplayback?sig=secret","mimeType":"audio/mp4"}],"adaptiveFormats":[{"url":"https://rr2.unit.googlevideo.com/videoplayback?sig=secret","mimeType":"video/mp4; codecs=avc1"}]}}`))
if err != nil || len(resources) != 2 || resources[0].Kind != "audio/mp4" || resources[1].Kind != "video/mp4" {
t.Fatal(resources, err)
}
resources, err = m.DiscoverResponse([]byte(`{"playabilityStatus":{"status":"OK"},"streamingData":{"serverAbrStreamingUrl":"https://rr1.unit.googlevideo.com/sabr?sig=secret"}}`))
if err != nil || len(resources) != 0 {
t.Fatal("SABR fabricated ordinary readiness")
}
for _, body := range []string{"{", strings.Repeat(" ", (1<<20)+1), `{"playabilityStatus":{"status":"ERROR"}}`} {
if _, err := m.DiscoverResponse([]byte(body)); err == nil || strings.Contains(err.Error(), "secret") {
t.Fatal("invalid metadata accepted or disclosed")
}
}
}
func TestQualificationPlan(t *testing.T) {
m, now := Module{}, time.Unix(1800000000, 0)
if m.Binding().PolicySHA256 != "2cfc521a9784c574ceee16430aa485b9c0e9e64795486a23df5117b6265d89d5" {
t.Fatal("immutable youtube/1 policy changed: introduce a new version")
}
hosts := []string{"rr1---sn-alpha.googlevideo.com", "rr2---sn-beta.googlevideo.com", "rr5---sn-gamma.googlevideo.com"}
var resources []Resource
for _, host := range hosts {
resources = append(resources, Resource{URL: fmt.Sprintf("https://%s/videoplayback?expire=%d&sig=secret", host, now.Unix()+121), Host: host, Kind: "video/mp4"})
}
p, err := m.PlanQualification(hosts, resources, now)
if err != nil || strings.Join(p.Hosts, ",") != strings.Join(hosts, ",") || p.Protocols != [2]string{"tls12", "tls13"} || p.Samples != 3 || p.MinBytes != 1024 || p.WarmupLimit != 10 || p.WarmupTimeout != 120*time.Second {
t.Fatal(p, err)
}
for _, mode := range []string{"expired", "unsigned", "sabr", "missing", "diversity"} {
t.Run(mode, func(t *testing.T) {
r, h := append([]Resource(nil), resources...), append([]string(nil), hosts...)
switch mode {
case "expired":
r[0].URL = strings.Replace(r[0].URL, "1800000121", "1800000120", 1)
case "unsigned":
r[0].URL = strings.Split(r[0].URL, "&sig=")[0]
case "sabr":
r[0].URL = strings.Replace(r[0].URL, "videoplayback", "sabr", 1)
case "missing":
r = r[:2]
case "diversity":
h[1] = h[0]
}
if _, err := m.PlanQualification(h, r, now); err == nil {
t.Fatal("invalid sample accepted")
}
})
}
encoded, err := json.Marshal(struct {
Resources []Resource
Plan Plan
}{resources, p})
if err != nil || strings.Contains(string(encoded), "secret") || strings.Contains(string(encoded), hosts[0]) {
t.Fatal("ephemeral data serialized", string(encoded), err)
}
}
func TestSignedSampleFresh(t *testing.T) {
now := time.Unix(1800000000, 900000000)
for _, tc := range []struct {
query string
want bool
}{
{"expire=1800000121&sig=secret", true},
{"expire=1800000120&sig=secret", false},
{"expire=1800000121&signature=secret", true},
{"expire=1800000121&lsig=secret", true},
{"expire=1800000121", false},
{"expire=invalid&sig=secret", false},
{"expire=9223372036854775808&sig=secret", false},
} {
r := Resource{URL: "https://rr1---sn-alpha.googlevideo.com/videoplayback?" + tc.query, Kind: "video/mp4"}
if SignedSampleFresh(r, now, 120*time.Second) != tc.want {
t.Fatal("signed expiry boundary changed")
}
}
}
+112 -19
View File
@@ -18,6 +18,25 @@ dns_mode=$(/usr/local/bin/zapret-autotune dns-forwarder mode) || exit 64
dns_address=$(/usr/local/bin/zapret-autotune dns-forwarder address) || exit 64
generic_mode=0
[ -z "${ZAPRET_AUTOTUNE_RESOURCE_CONFIG-}" ] || generic_mode=1
qualification_mode=0
qualification_attempted=0
# RAM-only authorization: a new PID 1 may attempt again, never a runtime retry.
resource_apply_state=unattempted
if [ -n "${ZAPRET_AUTOTUNE_QUALIFY_CONFIG-}${ZAPRET_AUTOTUNE_STRATEGY_CONFIG-}" ]; then
[ -n "${ZAPRET_AUTOTUNE_QUALIFY_CONFIG-}" ] && [ -n "${ZAPRET_AUTOTUNE_STRATEGY_CONFIG-}" ] || {
echo 'error: isolated qualification requires both config and strategy paths' >&2; exit 64;
}
qualification_mode=1
generic_mode=1
echo 'INTRUSIVE ISOLATED qualification target: do not route working traffic here' >&2
fi
case "${ZAPRET_AUTOTUNE_RESOURCE_APPLY-0}" in
0|'') ;;
1) [ "$generic_mode" -eq 1 ] && [ "$qualification_mode" -eq 0 ] && [ -n "${ZAPRET_AUTOTUNE_RESOURCE_CONFIG-}" ] || {
echo 'error: resource apply requires generic config and excludes qualification' >&2; exit 64;
} ;;
*) echo 'error: invalid resource apply flag' >&2; exit 64 ;;
esac
case "$dns_mode" in doh|native) ;; *) exit 64 ;; esac
config_dir=/run/zapret-autotune-ss
awk '$2 == "/run/zapret-autotune-ss" && $3 == "tmpfs" { found=1 } END { exit !found }' /proc/mounts || {
@@ -94,7 +113,7 @@ cleanup_session() {
for session_signal in TERM KILL; do
signal_session "$monitor_sid" "$session_signal" || return 78
session_deadline=$(($(date +%s) + 3))
if [ "$generic_mode" -eq 1 ] && [ "$session_deadline" -gt "$generic_cleanup_deadline" ]; then
if [ "$generic_mode" -eq 1 ] && [ -n "$generic_cleanup_deadline" ] && [ "$session_deadline" -gt "$generic_cleanup_deadline" ]; then
session_deadline=$generic_cleanup_deadline
fi
while :; do
@@ -113,9 +132,13 @@ cleanup_session() {
capture_monitor_session() {
[ -z "$monitor_sid" ] || return 0
if [ -e "$config_dir/monitor-ready" ]; then
read -r candidate_sid < "$config_dir/monitor-ready" || return 78
case "$candidate_sid" in ''|*[!0-9]*) return 78 ;; esac
[ "$candidate_sid" = "$monitor_pid" ] && [ "$candidate_sid" -gt 1 ] && [ "$candidate_sid" != "$$" ] || return 78
{ read -r candidate_sid < "$config_dir/monitor-ready"; } 2>/dev/null || {
echo 'controller-startup: ack=read_failed' >&2; return 78;
}
case "$candidate_sid" in ''|*[!0-9]*) echo 'controller-startup: ack=invalid_pid' >&2; return 78 ;; esac
[ "$candidate_sid" = "$monitor_pid" ] && [ "$candidate_sid" -gt 1 ] && [ "$candidate_sid" != "$$" ] || {
echo 'controller-startup: ack=identity_mismatch' >&2; return 78;
}
monitor_sid=$candidate_sid
elif proc_identity "$monitor_pid" && [ "$proc_sid" = "$monitor_pid" ] && [ "$proc_sid" -gt 1 ]; then
monitor_sid=$proc_sid
@@ -196,9 +219,15 @@ cleanup_cycle() {
if [ -n "$monitor_pid" ]; then
capture_monitor_session || cleanup_code=78
kill -TERM "$monitor_pid" 2>/dev/null || true
bounded_wait "$monitor_pid" 45
if [ "$qualification_mode" -eq 1 ]; then bounded_wait "$monitor_pid" 75; else bounded_wait "$monitor_pid" 45; fi
# Cancellation must restore safely even when SS failure requested retry.
if [ "$qualification_mode" -eq 1 ]; then
# Exit 78 is an outcome only with a producer-owned completed-cleanup
# ACK and a fresh baseline/worker/nft check, while SS+DNS still live.
cleanup_command 10 /usr/local/bin/zapret-autotune qualification-completion-check "$monitor_pid" "$waited_code" || cleanup_code=78
else
case "$waited_code" in 0|143) ;; *) echo "error: monitor cleanup status $waited_code" >&2; cleanup_code=78 ;; esac
fi
[ "$join_timed_out" -eq 0 ] || { echo 'error: monitor join timeout' >&2; cleanup_code=78; }
# SIGTERM may have arrived before the launch acknowledgement was read.
capture_monitor_session || cleanup_code=78
@@ -309,7 +338,47 @@ start_dns() {
pause 1
done
}
launch_controller() {
# Only an ACK retained from our previous launch is owned stale state.
# Unknown files (including dangling links) are evidence, not debris.
if [ -e "$config_dir/monitor-ready" ] || [ -L "$config_dir/monitor-ready" ]; then
if [ -z "${monitor_ready_identity-}" ] || [ -L "$config_dir/monitor-ready" ] ||
[ ! -f "$config_dir/monitor-ready" ] ||
[ "$(stat -c '%d:%i:%u:%a:%h' "$config_dir/monitor-ready")" != "$monitor_ready_identity" ] ||
! { read -r stale_pid < "$config_dir/monitor-ready"; } 2>/dev/null ||
[ "$stale_pid" != "${monitor_ready_pid-}" ]; then
echo 'controller-startup: launch=ack_conflict' >&2
return 78
fi
rm -f "$config_dir/monitor-ready" || return 78
fi
monitor_ready_identity=
setsid /bin/sh /opt/managed-target-supervision.sh --monitor "$config_dir" /usr/local/bin/zapret-autotune "$@" 8<&- 9<&- &
monitor_pid=$!
monitor_ack_deadline=$(($(date +%s) + 5))
while [ ! -e "$config_dir/monitor-ready" ]; do
if ! kill -0 "$monitor_pid" 2>/dev/null || [ "$(date +%s)" -ge "$monitor_ack_deadline" ]; then
if ! kill -0 "$monitor_pid" 2>/dev/null; then
echo 'controller-startup: launch=child_gone' >&2
else
echo 'controller-startup: launch=ack_timeout' >&2
fi
signal_tree "$monitor_pid" KILL
return 78
fi
done
capture_monitor_session || return 78
monitor_ready_identity=$(stat -c '%d:%i:%u:%a:%h' "$config_dir/monitor-ready") || return 78
monitor_ready_pid=$monitor_pid
if proc_identity "$monitor_pid" && [ "$proc_sid" != "$monitor_sid" ]; then
echo 'controller-startup: launch=session_mismatch' >&2
monitor_sid=
return 78
fi
}
run_cycle() {
[ "$resource_apply_state" != failed ] || return 78
[ "$qualification_attempted" -eq 0 ] || return 78
start_dns || return $?
set -- ss-server -c "$config_dir/server.json"
# DNS is not a secret. Native -d avoids assuming support for a JSON DNS key.
@@ -338,8 +407,24 @@ fi
# Never activate pending or unknown bytes. Any reconciliation failure is fatal,
# not a transient (75) restart request. Reload restores known bytes first.
timeout -k 5 45 /usr/local/bin/zapret-autotune managed reconcile || return 78
if [ "$qualification_mode" -eq 0 ]; then
zapret_started=1
if [ "${ZAPRET_AUTOTUNE_RESOURCE_APPLY-0}" = 1 ] && [ "$resource_apply_state" = unattempted ]; then
resource_apply_state=failed
launch_controller managed resource-apply --resource-config "$ZAPRET_AUTOTUNE_RESOURCE_CONFIG" || return 78
while dns_alive && kill -0 "$server_pid" && kill -0 "$local_pid" && kill -0 "$monitor_pid"; do pause 1; done
{ dns_alive && kill -0 "$server_pid" && kill -0 "$local_pid"; } || return 75
apply_code=0
wait "$monitor_pid" || apply_code=$?
echo "controller-startup: apply_wait_exit=$apply_code" >&2
monitor_pid=
cleanup_session || { echo 'controller-startup: apply_cleanup=failed' >&2; return 78; }
[ "$apply_code" -eq 0 ] || return 78
resource_apply_state=succeeded
fi
# Idempotent apply may have only verified the already committed config.
timeout -k 5 30 /opt/zapret2/init.d/sysv/zapret2 start || return 78
fi
# Private operator request, consumed only after a successful guarded transaction.
# Never suppress qualification gates globally or race the owned monitor.
@@ -351,27 +436,31 @@ fi
# fork a monitor we already need to cancel: an early TERM can race interpreter
# startup before its cancellation handler exists. Clean this cycle directly.
{ dns_alive && kill -0 "$server_pid" 2>/dev/null && kill -0 "$local_pid" 2>/dev/null; } || return 75
rm -f "$config_dir/monitor-ready"
setsid /bin/sh /opt/managed-target-supervision.sh --monitor "$config_dir" /usr/local/bin/zapret-autotune managed monitor 8<&- 9<&- &
monitor_pid=$!
monitor_ack_deadline=$(($(date +%s) + 5))
while [ ! -e "$config_dir/monitor-ready" ]; do
if ! kill -0 "$monitor_pid" 2>/dev/null || [ "$(date +%s)" -ge "$monitor_ack_deadline" ]; then
signal_tree "$monitor_pid" KILL
return 78
fi
done
capture_monitor_session || return 78
if proc_identity "$monitor_pid" && [ "$proc_sid" != "$monitor_sid" ]; then
monitor_sid=
return 78
if [ "$qualification_mode" -eq 1 ]; then
qualification_attempted=1
set -- qualify --config "$ZAPRET_AUTOTUNE_QUALIFY_CONFIG" --strategies "$ZAPRET_AUTOTUNE_STRATEGY_CONFIG" --isolated
else
set -- managed monitor
fi
launch_controller "$@" || return 78
while dns_alive && kill -0 "$server_pid" && kill -0 "$local_pid" && kill -0 "$monitor_pid"; do
pause 1
done
if ! kill -0 "$monitor_pid" 2>/dev/null; then
code=0
wait "$monitor_pid" || code=$?
if [ "$qualification_mode" -eq 1 ]; then
if ! timeout -k 1 10 /usr/local/bin/zapret-autotune qualification-completion-check "$monitor_pid" "$code"; then
generic_cleanup_failed=1
fi
monitor_pid=
# One finite batch only, including negative/blocked completion. No live
# daemon and no automatic retest. Keep owned SS/DoH alive for inspection.
echo "isolated qualification finished (status $code); target safe-idle" >&2
cleanup_session || return 78
while dns_alive && kill -0 "$server_pid" && kill -0 "$local_pid"; do pause 1; done
return 78
fi
monitor_pid=
echo 'error: target managed monitor exited' >&2
case "$code" in
@@ -417,6 +506,10 @@ while :; do
fi
code=78
fi
if [ "$resource_apply_state" = failed ]; then
echo 'BLOCKED: resource apply failed/incomplete; operator repair/restart required' >&2
while :; do pause 60; done
fi
if [ "$generic_mode" -eq 1 ] && [ "$code" -eq 78 ]; then
echo 'BLOCKED: generic target trust/startup failure; checks paused' >&2
pause 60
+21 -2
View File
@@ -61,11 +61,30 @@ if [ "${1-}" = --monitor ]; then
[ "$#" -ge 3 ] || exit 64
config_dir=$2
shift 2
proc_identity "$$" && [ "$proc_sid" = "$$" ] && [ "$$" -gt 1 ] || exit 78
proc_identity "$$" || { echo 'controller-startup: wrapper=identity_unavailable' >&2; exit 78; }
[ "$proc_sid" = "$$" ] && [ "$$" -gt 1 ] || {
echo 'controller-startup: wrapper=not_session_leader' >&2; exit 78;
}
# Acknowledge the private session before exec, so even an immediate monitor
# crash leaves its ownership recorded. No worker can precede this handshake.
trap 'exit 143' INT TERM
printf '%s\n' "$$" > "$config_dir/monitor-ready"
ack_temp=
trap '[ -z "$ack_temp" ] || rm -f "$ack_temp"' EXIT
ack_publish() {
# Private same-UID directory: no untrusted writer can race the absent
# destination check. Rename exposes complete bytes with a single link.
[ -d "$config_dir" ] && [ ! -L "$config_dir" ] &&
[ "$(stat -c '%u:%a' "$config_dir")" = "$(id -u):700" ] || return 1
umask 077
ack_temp=$(mktemp "$config_dir/.monitor-ready.XXXXXX") || return 1
printf '%s\n' "$$" > "$ack_temp" || return 1
[ ! -e "$config_dir/monitor-ready" ] && [ ! -L "$config_dir/monitor-ready" ] || return 1
mv -T "$ack_temp" "$config_dir/monitor-ready" || return 1
ack_temp=
}
ack_publish || { echo 'controller-startup: wrapper=ack_publish_failed' >&2; exit 78; }
trap - EXIT
echo 'controller-startup: wrapper=exec_pending' >&2
exec "$@"
elif [ "${1-}" = --watchdog ]; then
set -eu
+30
View File
@@ -0,0 +1,30 @@
{
"version": 2,
"services": [
{
"id": "example",
"resources": [
{
"id": "image",
"group": "images",
"kind": "image",
"url_env": "RESOURCE_IMAGE_URL",
"method": "GET",
"tls": "both",
"timeout_ms": 10000,
"expect": {"statuses": [200], "content_type": "image/png", "min_bytes": 1, "max_bytes": 5242880}
},
{
"id": "prefix",
"group": "media",
"kind": "mp4_range",
"url_env": "RESOURCE_MP4_URL",
"method": "GET",
"tls": "both",
"timeout_ms": 10000,
"expect": {"statuses": [206], "content_type": "video/mp4", "min_bytes": 16, "max_bytes": 65536}
}
]
}
]
}