Compare commits
21
Commits
2242643000
...
88564b340a
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
88564b340a | ||
|
|
e270810f33 | ||
|
|
f4853173b3 | ||
|
|
d324b4137b | ||
|
|
cd05e75189 | ||
|
|
9ab9fd73a9 | ||
|
|
45d28e3733 | ||
|
|
2149a4abc8 | ||
|
|
8d43dd9363 | ||
|
|
5d0be0288f | ||
|
|
643d0c6220 | ||
|
|
36aace02da | ||
|
|
ade3b89137 | ||
|
|
de453ed2c8 | ||
|
|
3743cdbd3e | ||
|
|
5cdc0cd767 | ||
|
|
83921350fd | ||
|
|
63cdc7d490 | ||
|
|
610916ce66 | ||
|
|
a1a76bff76 | ||
|
|
573da3f797 |
No files matched your search
@@ -3,6 +3,8 @@
|
||||
!go.sum
|
||||
!cmd/
|
||||
!cmd/**
|
||||
!internal/
|
||||
!internal/**
|
||||
!Dockerfile
|
||||
!autotune-entrypoint.sh
|
||||
!Dockerfile.target
|
||||
|
||||
@@ -4,6 +4,7 @@ ENV CGO_ENABLED=0 GOTOOLCHAIN=local
|
||||
COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
COPY cmd/ ./cmd/
|
||||
COPY internal/ ./internal/
|
||||
RUN --mount=type=cache,target=/root/.cache/go-build go build -trimpath -o /out/zapret-autotune ./cmd/zapret-autotune
|
||||
|
||||
FROM vernette/ss-zapret2@sha256:ba158180f27cd71a291ee521c291490374fd3fcf2f81eb1434ddfb19d35de45b
|
||||
|
||||
@@ -4,6 +4,7 @@ ENV CGO_ENABLED=0 GOTOOLCHAIN=local
|
||||
COPY go.mod go.sum ./
|
||||
RUN go mod download
|
||||
COPY cmd/ ./cmd/
|
||||
COPY internal/ ./internal/
|
||||
RUN --mount=type=cache,target=/root/.cache/go-build go build -trimpath -o /out/zapret-autotune ./cmd/zapret-autotune
|
||||
|
||||
FROM vernette/ss-zapret2@sha256:ba158180f27cd71a291ee521c291490374fd3fcf2f81eb1434ddfb19d35de45b
|
||||
|
||||
+96
-2
@@ -1,6 +1,6 @@
|
||||
# Periodic configured resource checks
|
||||
|
||||
`managed monitor --resource-config FILE` runs resource-config v1 periodically.
|
||||
`managed monitor --resource-config FILE` runs resource-config v1 or v2 periodically.
|
||||
PID 1 selects the same mode when `ZAPRET_AUTOTUNE_RESOURCE_CONFIG` is nonempty;
|
||||
unset preserves the existing YouTube monitor. The modes share one singleton
|
||||
lock. Supply an absolute container-visible config path (for example mount a
|
||||
@@ -19,9 +19,69 @@ not playback health, generic recovery or whole-production READY: HEALTHY is
|
||||
base passed native full/race checks, independent security review and isolated
|
||||
remote fault acceptance for `4efc066` plus ownership-status fix `5c85f3f`.
|
||||
See [PROGRESS.md](PROGRESS.md) for supplied evidence and its narrow scope.
|
||||
Version1 is bounded to 16 services / 64 resources with stable configured IDs;
|
||||
Both versions are bounded to 16 services / 64 resources with stable configured IDs;
|
||||
valid configuration/state does not imply an automatic strategy assignment.
|
||||
|
||||
V1 remains HTTP GET/HEAD only, with explicit `kind: http`. See
|
||||
`resource-check-v2.example.json` for v2: omitted kind defaults to `http`; all
|
||||
expectations explicitly declare statuses/content_type/min_bytes/max_bytes.
|
||||
`image` is GET-only, [200], PNG/JPEG with full bounded decode (5 MiB maximum,
|
||||
16 MiPixels); public GIF is deferred pending an aggregate frame/pixel budget,
|
||||
without changing legacy GIF adapters. `mp4_range` is GET-only, [206], audio/mp4
|
||||
or video/mp4, 16–65536 bytes, checked Content-Range and ftyp. Range stays fixed
|
||||
at 0-65535; a smaller max_bytes bounds the actual response, not the requested end.
|
||||
Operator Range/Accept-Encoding headers remain forbidden in both versions.
|
||||
Timeouts (100–30000 ms) and declared byte limits apply to every kind. Typed
|
||||
wire-valid semantic failures degrade checks; malformed/unsafe execution cannot
|
||||
qualify a candidate. V2 version/kind/expectations invalidate old HMAC rows/proofs;
|
||||
historic v1 fingerprints remain unchanged. MP4 prefix health is not playback or
|
||||
YouTube proof: three discovered CDNs, >=1024-byte producer samples, warmup,
|
||||
suffix authorization and adaptive policy still belong to the legacy adapter.
|
||||
The private compiled-in YouTube module and core `executeAdapterPlan[T]` bridge
|
||||
now drive the existing owned qualifier's ordered warmup and steady collection.
|
||||
They do not add public module configuration or change this monitor's schemas,
|
||||
controller, suffix authorization or legacy SOCKS5h routing boundary. Generic
|
||||
controller cutover still needs dynamic-inventory fingerprints and warmup-bearing
|
||||
generic proof binding; images/avatars and full YouTube READY remain unresolved.
|
||||
Local source/closure review and injected-wire checks are
|
||||
recorded alongside scoped managed v2 field observations below; full acceptance
|
||||
is not implied.
|
||||
|
||||
### Scoped managed v2 monitor observations (2026-10-15)
|
||||
|
||||
Independent closeout at **2026-10-15T12:28:52Z** records **SCOPED_PASS** for source
|
||||
`cd05e75189f90c32e1346536c50f0973e7105cb3`, audit runtime unchanged. See
|
||||
[RESOURCE_PLAN.md](RESOURCE_PLAN.md#managed-v2-scoped_pass-observations-2026-10-15)
|
||||
for the root-private report/audit pointers, qualification/apply counts, dated
|
||||
cleanup and full evidence limits. Official owned PID1 SS/DoH/public v2 paths
|
||||
qualified PNG/JPEG and video/audio MP4 prefixes on both TLS versions (16 M3
|
||||
cells / 48 TLS transfers); eight apply and eight initial monitor checks passed.
|
||||
|
||||
Five HMAC-authenticated snapshots, including initial state, show real host-fixture
|
||||
PNG truncation and MP4 total overflow degrading only the affected row to
|
||||
`DEGRADED / expectation_failed`, followed by all-HEALTHY repair. The SAME
|
||||
controller PID 379/start 62777447 and journal/config/proof/key/input hashes were
|
||||
retained, without restart or requalification. HEALTHY is still CHECKS_ONLY.
|
||||
Both official stops exited 0; dated audit found all five owned containers,
|
||||
owned network and 174 recorded process identities/namespaces gone, and three
|
||||
foreign raw inspect snapshots unchanged. Historical Xray equality is only a
|
||||
harness assertion without retained initial raw process inventory.
|
||||
|
||||
Typed semantics are supported by real production validation, authenticated
|
||||
outcomes and fixture metadata, **not independent re-decoding of retained bodies**:
|
||||
bodies were deliberately not retained for privacy. Live executable/held-FD
|
||||
snapshots were also not retained; PID/start/parent/netns/native argv checks and
|
||||
reviewed ACK enforcement are not a durable live-FD certificate. The initial
|
||||
DoH-forwarder/worker observer misclassification was corrected in the harness,
|
||||
retaining logs and the same qualified proof, not fixing a product failure.
|
||||
|
||||
No new public-v1 managed field run, negative M3/cancel/freshness/old-matrix remote
|
||||
campaign, actual YouTube discovery/three-CDN/suffix/adaptive/warmup/avatar/playback
|
||||
was run. Local v1 parity and unit rejection coverage do not fill those gaps.
|
||||
The static test image did not replace the user's working SS service. Missing
|
||||
body/live-FD evidence and untested cases remain limits before full acceptance
|
||||
and YouTube adapter cutover; there is no full safety, migration or READY claim.
|
||||
|
||||
Invalid resource JSON initially or after reload is BLOCKED, with no DNS or
|
||||
transfers until repaired. Bad static target role, proxy or DNS settings still
|
||||
fail startup validation; resource JSON is deliberately not part of
|
||||
@@ -70,3 +130,37 @@ within a 90-second Docker stop limit (the supplied Compose grants 240 seconds).
|
||||
Bounded final reaps/worst-case cleanup were verified offline; measured remote
|
||||
normal stops were 1.45s, 0.59s and 0.57s, not an 80-second runtime worst-case test.
|
||||
These changes do not claim to fix legacy YouTube exit 78.
|
||||
|
||||
### Internal service-adapter boundary (local extraction, field acceptance pending)
|
||||
|
||||
Legacy YouTube execution now explicitly selects compiled-in `youtube/1` via
|
||||
`cmd/zapret-autotune/service_adapters.go`. `internal/adapter/youtube` parses
|
||||
bounded Player media JSON and supplies signed-MP4 freshness/diversity semantics
|
||||
and the owned qualification plan. Unknown/absent kind or version fails closed,
|
||||
without a generic HTTP fallback. The shared `internal/adapter` DTOs are internal
|
||||
only: signed URLs/inventory and plan caps have `json:"-"` fields.
|
||||
|
||||
The existing controller still owns discovery requests/retries, transport,
|
||||
origin/SNI/certificate/TLS checks, worker guards/cleanup and sealing after join.
|
||||
Legacy probes retain SOCKS5h and the owned encrypted-resolver guard; they do not
|
||||
run the generic all-A/AAAA approval/connect-to IP pinning path. Full request-policy
|
||||
capabilities (including future metadata POST) and parity are prerequisites before
|
||||
generic cutover, not guarantees supplied by this minimal extraction.
|
||||
Public generic checks and their HMAC/state formats have not migrated to this
|
||||
module interface. Suffix/experimental authorization is separate from measured
|
||||
endpoints, and avatar/thumbnail health remains independent. A module plan is
|
||||
neither proof nor permission to claim READY.
|
||||
|
||||
New adaptive proof v2 seals adapter kind/version/policy digest along with the
|
||||
measured facts. Unknown identities, mismatched policy, missing attestation and
|
||||
unsealed DTOs cannot publish adaptive evidence. Legacy v1 is explicitly the
|
||||
immutable historical YouTube policy for journal reconstruction, not a request
|
||||
to reinterpret old evidence using the latest module. Expired historical proofs
|
||||
can reconstruct their owned config. Fresh trusted stored v1 retains its original
|
||||
apply eligibility; expiry still blocks fresh apply. Only new producer writes
|
||||
require sealed module-bound v2; public/unsealed DTOs cannot publish evidence.
|
||||
Discovery/asset/health entrypoints preflight the required kind/version/policy
|
||||
before requests. Missing/mismatched modules are fatal configuration failures,
|
||||
not retryable inventory absence or a genuine-no-strategy cooldown.
|
||||
No public module field, dynamic plugin ABI, controller cutover, same-UID sandbox,
|
||||
remote YouTube acceptance, or fix for legacy exit 78 is claimed here.
|
||||
@@ -0,0 +1,849 @@
|
||||
# Resource compatibility planner and isolated qualifier
|
||||
|
||||
## First collector migration (shared-core parity verified; managed v2 scoped observations)
|
||||
|
||||
Private typed response contracts now select generic GET/HEAD, full decoded image,
|
||||
or bounded MP4 range validation inside the existing shared HTTPS transfer core.
|
||||
The redundant legacy forwarding wrapper and media boolean are removed; image and
|
||||
media adapters retain their measurements and GooglevideoPass DTOs. Generic v1
|
||||
status/MIME/byte expectations use the same validation entry, with a distinct
|
||||
completed-mismatch result retaining the same measured facts. No public Range config, schema,
|
||||
proof importer, registry or dependency was added.
|
||||
|
||||
Injected-wire tests exercise the real generic producer and legacy callers, both
|
||||
TLS versions, normalized byte/hash/timing facts, private stdin/pinning, origin,
|
||||
redirect, framing, bounds and strict range/ftyp/image decoder rejection. Direct
|
||||
core tests ensure strict validation is not an unused legacy-only helper.
|
||||
MP4 requests use Range `0-65535` (a 64 KiB upper bound, not an exact response
|
||||
size requirement). The core accepts validated 16–65536-byte ranges; the existing
|
||||
producer additionally requires at least 1024 measured bytes. Content-Range end
|
||||
and total must parse without overflow, with total greater than end; rejected or
|
||||
unknown totals cannot produce a media PASS DTO.
|
||||
This is collector migration only: public schemas and the legacy YouTube producer
|
||||
remain unchanged, including three discovered CDNs, adaptive A/E, warmup then 3/3
|
||||
samples on each TLS version, explicit googlevideo.com suffix authorization, fresh
|
||||
discovery, and independent qualification of image/avatar assets. The legacy path
|
||||
is retained. Generic GET success is never full YouTube qualification.
|
||||
|
||||
Recorded local verification: archived `2149a4a` versus current source passed 86
|
||||
parity subcases; 14 new integration cases, 11 focused EUID-0 serial/race tests and
|
||||
full Go tests passed (full Go about 55s). Python static/config/startup checks passed;
|
||||
the long supervision check timed out at 18s and again at 120s and is unresolved.
|
||||
Do not call all tests green or infer a regression from that timeout: entrypoint
|
||||
code was unchanged and later full Go entry tests passed. These are prior results,
|
||||
not reruns in this docs-only update.
|
||||
|
||||
### Scoped remote shared-core parity (2026-10-15)
|
||||
|
||||
Privileged-read evidence: `/home/agent/core-runtime-parity-45d28e3-20261015/report.md`.
|
||||
Exact-source baseline `2149a4a` and current
|
||||
`45d28e37330069ddf0efa3120fec8a558d37ba7f` each passed 48 real curl/HTTPS/SOCKS5h
|
||||
fixture subtests (24 cases × TLS1.2/1.3). Of the normalized rows, 46 matched and
|
||||
two intentionally differed: overflowing Content-Range total, formerly saturated
|
||||
by ParseInt, now rejects without a PASS DTO. Known negative classifications are
|
||||
preserved; invalid/unknown/overflow totals are not proof.
|
||||
|
||||
Coverage includes GET/HEAD expectations and retained mismatch facts, bounds,
|
||||
redirect/framing/trust rejection, real PNG/JPEG/GIF decode and rejection, strict
|
||||
206 range/ftyp checks, and stdin/argv/result privacy sentinels. Real 65536-byte
|
||||
and 1024-byte media responses passed the same Range request. Each version made
|
||||
60 SOCKS5h connections/SNI handshakes and 58 authenticated HTTP requests, including
|
||||
18 actual Range requests; two untrusted-certificate attempts stopped before HTTP.
|
||||
Original hostname/SNI was retained; curl verified a fresh private fixture CA,
|
||||
without TLS bypass. Timing/speed were positive but excluded from exact normalized
|
||||
parity hashes; this is not a performance comparison or fatal-reason-tree parity.
|
||||
|
||||
The proxy maps only allowlisted fixture names to a loopback TLS listener: it is
|
||||
not an SS proxy or DoH resolver. **NOT RUN:** normal public CLI under owned PID1
|
||||
SS/DoH, managed ownership/guard checks and probe orchestration, NFQ/native behavior,
|
||||
real YouTube discovery/three-CDN qualification, or candidate/apply/monitor flow.
|
||||
No DoH validation or production qualification follows from helper tests. No
|
||||
candidate/PASS/DB/JSON proof imports occurred. Helpers exited and joined listeners;
|
||||
no owned containers/networks were created. Root-private artifacts, keys, manifests
|
||||
and hashes remain retained. Selected foreign identity/running/start facts matched
|
||||
before/after, not arbitrary foreign application-data hashes.
|
||||
|
||||
### Public typed checks (v2; scoped managed field verification)
|
||||
|
||||
`resource-check-v2.example.json` adds closed `kind` values `http`, `image` and
|
||||
`mp4_range`; omitted v2 kind canonicalizes to `http`. V1 still requires explicit
|
||||
`http` and retains its serialization/fingerprints. V2 requires explicit statuses,
|
||||
content_type, min_bytes and max_bytes. HTTP retains GET/HEAD semantics; typed
|
||||
checks require GET. Image requires exactly [200] and image/png or image/jpeg,
|
||||
full decoding, positive dimensions at most 16 MiPixels and a configured body
|
||||
ceiling at most 5 MiB. Public GIF/omitted image MIME are deliberately deferred
|
||||
until a decoder with an aggregate frame/pixel budget exists; legacy GIF remains.
|
||||
MP4 requires exactly [206], video/mp4 or audio/mp4, and 16 <= min <= max <= 65536.
|
||||
The core always requests bytes 0-65535, even with a smaller max_bytes; actual
|
||||
returned bytes must fit that smaller bound and the validated Content-Range/ftyp.
|
||||
This proves a bounded MP4 prefix, **not playback or YouTube qualification**.
|
||||
|
||||
All kinds use configured 100–30000 ms timeouts and byte ceilings, private stdin,
|
||||
strict framing/TLS/origin and all-answer DNS policy through the same public check,
|
||||
periodic monitor, isolated M=3 qualifier and apply-health path. Wire-valid typed
|
||||
semantic failures are complete expectation negatives; unsafe output, malformed
|
||||
metrics/framing, cancellation and ownership failures cannot become benign proof.
|
||||
V2 source and row HMACs bind version, normalized kind and expectations, so old
|
||||
plain-HTTP evidence/state cannot promote typed checks. V1 historical journal
|
||||
reconstruction is unchanged. No public proof import or module framework exists.
|
||||
|
||||
Local verification for this v2 slice: full Go tests (`-p 1 -parallel 1`), affected
|
||||
resource race tests, vet, CGO-disabled build, four managed-target Python fixtures
|
||||
and shell syntax passed. Current and archived `45d28e3` each passed the existing
|
||||
86 legacy adapter/generic-mismatch parity subtests. Privileged tests ran as EUID 0
|
||||
inside the current user namespace (UID 0 maps to host UID 100000), not host-initial
|
||||
root. These are local injected-wire/fixture results, not remote network acceptance.
|
||||
|
||||
Next let the YouTube adapter supply qualified exact endpoints/dynamic inventory into
|
||||
the generic registry, preserving fresh discovery and the three-CDN contract.
|
||||
Keep measured endpoint proof separate from explicit suffix/adaptive-learning
|
||||
authorization: both capabilities are prerequisites to complete managed migration.
|
||||
No plugin framework, legacy deletion or full controller/journal/profile migration
|
||||
is implied. Full runtime acceptance remains incomplete; avatars and legacy exit78 remain
|
||||
unresolved. No production READY claim.
|
||||
|
||||
### Managed v2 SCOPED_PASS observations (2026-10-15)
|
||||
|
||||
Source `cd05e75189f90c32e1346536c50f0973e7105cb3`. Privileged reads of
|
||||
`/home/agent/managed-v2-cd05e75-20261015/report.md` and
|
||||
`field-private/independent-audit.safe.json` record independent closeout at
|
||||
**2026-10-15T12:28:52Z**, verdict **SCOPED_PASS**, with
|
||||
`runtime_state_changed_by_audit: false`. These are bounded positive observations,
|
||||
not unequivocal whole-criterion acceptance, full safety or READY. The static test
|
||||
binary/image is not a deployment to the user's working SS service. The audit
|
||||
matched 137 archived product files and the pinned native NFQ engine (1.0.2).
|
||||
|
||||
- Official public v2 CLI under owned PID1 SS/DoH exercised PNG/JPEG full decode
|
||||
and video/mp4 + audio/mp4 bounded prefixes, each over TLS1.2/1.3 with verified
|
||||
fixture CA/SNI. Two candidates × four resources × both TLS versions produced
|
||||
16 M=3 cells / 48 real qualification TLS transfers. Queue-200 worker argv and
|
||||
PID/start matched; exact-payload registry/matrix HMACs, v2 source and semantic
|
||||
row fingerprints were independently recomputed, not imported PASS evidence.
|
||||
- Official apply reused that proof without requalification: queue 300, canonical
|
||||
`multidisorder`, three exact fixture hosts, journal v4 / generation 1 / committed
|
||||
generic. Eight actual typed apply checks and eight initial monitor checks passed.
|
||||
Source fingerprint:
|
||||
`cc5a2622133daed58df0b2c22254f6112a3d9da7fbd9391ce7d1c6f467761886`;
|
||||
committed config SHA-256:
|
||||
`27f9a8ffd56a4c19d4bd522dd9f5a5e8b49b18b0ae48d41431ef9b22975ea772`.
|
||||
- Five authenticated monitor snapshots (including initial state) showed truncated
|
||||
PNG and overflowing video MP4 Content-Range each degrading **only** its affected
|
||||
row to `DEGRADED / expectation_failed`, then repair restoring all four HEALTHY.
|
||||
Host fixture changes produced real responses; controller PID 379/start 62777447,
|
||||
journal/config/proofs/key/input hashes stayed the same, without requalification.
|
||||
Apply/monitor wire evidence totals 48 transfers (33 healthy-mode, 7 PNG-fault,
|
||||
8 MP4-fault); fixture timing/size/status is not durable product DTO telemetry.
|
||||
- Dated closeout independently rechecked both official target stops exited 0;
|
||||
all five recorded owned containers, owned network and 174 recorded PID/start
|
||||
identities/namespaces were absent. Three foreign raw Docker inspect snapshots
|
||||
remained byte-identical: `f6c9d42858e1` running, `4ad54175a54c` and `8cc36c813d6f`
|
||||
stopped. Historical Xray inventory equality is a harness assertion only: its
|
||||
initial raw process inventory was not retained, so it is not independently proved.
|
||||
|
||||
**Evidence limits:** production strict validation, authenticated outcomes and actual
|
||||
fixture metadata establish this scoped typed-response observation. Bodies were
|
||||
intentionally not retained to avoid data exposure and were **not independently
|
||||
re-decoded**. Actual PID/start/parent/netns/native argv were independently checked;
|
||||
live executable/held-FD snapshots were **not retained**. ACK/snapshot guards are
|
||||
reviewed code/log enforcement, not a durable live-FD certificate. These missing
|
||||
body/live-FD certificates prevent an unqualified full acceptance claim.
|
||||
|
||||
The initial observer misclassified the intentionally live DoH forwarder as an
|
||||
unjoined worker. Original logs/harness remain; the corrected continuation used
|
||||
the SAME proof, with no new qualification. This is a harness assertion correction,
|
||||
not an observed product bug.
|
||||
|
||||
**NOT RUN in this field campaign:** public v1 managed GET/HEAD (86 current/old
|
||||
local parity cases passed, not a remote v1 run); negative M3 qualification,
|
||||
cancellation, freshness/24h expiry or old-matrix rejection; actual YouTube
|
||||
discovery/three-CDN, suffix authorization, adaptive learning, warmup, avatar or
|
||||
playback. New kind/version cannot promote old proofs by local unit coverage,
|
||||
not a fresh remote rejection campaign. MP4 proves a prefix, not playback or an
|
||||
exact 64-KiB body; public GIF remains deferred and legacy GIF retained. Prior full
|
||||
Go/race/Python results are recorded above, not rerun for this docs-only change.
|
||||
|
||||
**Next minimum slice (design, NOT implemented):** a real YouTube/core adapter
|
||||
boundary wrapping existing discovery and strict qualification calls, not a full
|
||||
framework. Core owns networking, limits, proof, persistence and apply; YouTube
|
||||
owns fresh URLs, three-CDN warmup/sampling, per-host adaptive learning and explicit
|
||||
suffix policy. Plan a minimal compiled-in Go registry keyed by kind + version,
|
||||
fail closed for unknown contracts; bind module/policy version into source/proof
|
||||
and preserve authenticated historical restore under its original contract. No
|
||||
dynamic plugin/RPC API exists. Do not generalize public POST/discovery until an
|
||||
actual adapter needs that request contract. Retain legacy functionality until
|
||||
parity and the remaining acceptance gaps are closed before full adapter cutover.
|
||||
|
||||
`plan` remains **proposal-only** and never changes runtime or config.
|
||||
Explicit generic startup apply/replay and the per-PID-1 failure latch have passed
|
||||
bounded remote acceptance with independent privileged verification (see below).
|
||||
This is not automatic recovery, full concurrency acceptance or production READY.
|
||||
Explicit isolated qualification temporarily installs an owned NFQ worker and
|
||||
queue rules, but never changes legacy SQL groups, renderer or apply journals.
|
||||
Monitor HEALTHY, old SQL measurements and public check JSON are not proof.
|
||||
Without a completed owned qualification batch, compatibility is UNKNOWN and
|
||||
hosts are unassigned. No PASS/evidence import is exposed.
|
||||
|
||||
Commands require root and the existing `managed-test` target settings:
|
||||
|
||||
```
|
||||
zapret-autotune strategy-import --strategies /private/strategies.json
|
||||
zapret-autotune plan --config /private/resources.json
|
||||
zapret-autotune qualify --config /private/resources.json --strategies /private/strategies.json --isolated
|
||||
```
|
||||
|
||||
`strategy-import` requires an owned, regular, single-link 0600 file. Only
|
||||
definitions are accepted; duplicate keys, unknown fields and evidence fields
|
||||
are rejected. Example:
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"strategies": [
|
||||
{
|
||||
"id": "split-one",
|
||||
"args": "--payload=tls_client_hello --lua-desync=multisplit:pos=1",
|
||||
"selectors": {"transport":"tcp","ports":"443","l7":"tls","tls":"both"}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
The registry is one immutable batch (1–32 unique IDs). This conservative slice
|
||||
accepts only TCP/443/TLS selectors and `tls12`, `tls13`, or `both` policy.
|
||||
Argument whitespace is canonicalized; option/Lua stage order is preserved.
|
||||
Existing payload/range validation is reused. Lua stages are restricted to
|
||||
fake, multisplit, multidisorder, fakeddisorder, split and disorder; parameters
|
||||
to pos, repeats, tcp_ts, tcp_flags_set, seqovl and blob=fake_default_tls.
|
||||
File/inline Lua, queue/user/fwmark/filter options and arbitrary stages are
|
||||
rejected. Acceptance of a definition is **not** runtime verification.
|
||||
|
||||
The private target directory holds `.resource-plan-key` (32 random bytes),
|
||||
`.resource-plan-strategies.json` and, when an owned producer exists,
|
||||
`.resource-plan-matrix.json`. JSON envelopes have `payload` and an HMAC-SHA256
|
||||
`mac`. Each file is write-once, owned, 0600, no-follow, single-link and bounded
|
||||
to 64 KiB. Held directory/file identities and exact snapshots are checked;
|
||||
foreign entries, replacement inodes and modified bytes are never clobbered or
|
||||
silently adopted. Interrupted writes remain blocked for explicit operator
|
||||
repair. There is no automatic rotation/update/repair in this slice. The private
|
||||
key and same-UID/root boundary must be protected: HMAC is not protection from an
|
||||
actor who can read the key and replace the whole target state between processes.
|
||||
|
||||
Matrix v1 binds the registry digest and at most one completed run per candidate.
|
||||
Each run binds ID, definition digest (canonical args **and selectors**), keyed
|
||||
config fingerprint, start/completion milliseconds, resource/TLS outcomes and
|
||||
keyed resource fingerprints. Its attestation records collector/runtime,
|
||||
definition digest, composite executable/Lua input hash, installed rules hash,
|
||||
actual worker-argv hash, queue and worker PID/start identity, plus
|
||||
restored/joined completion. No URLs, headers, resolved IPs, bodies or raw errors
|
||||
are persisted. A nonserialized producer seal protects the completed in-memory
|
||||
snapshot against mutation before publication. Only the owned producer and
|
||||
test-internal fixtures create seals; JSON DTOs cannot authorize publication.
|
||||
Every resource/protocol cell records exactly three actual attempts; any nonpass
|
||||
in those attempts makes the cell negative (never an average or partial PASS).
|
||||
|
||||
`plan` loads only authenticated private state and the existing resource-config
|
||||
snapshot (including resolved environment secrets, DNS/proxy and private headers
|
||||
in keyed fingerprints). Every resource on an exact host, across all services
|
||||
and input group labels, must have matching complete results for every required
|
||||
TLS protocol. Complete failed checks are negative; missing, expired (24 hours),
|
||||
future, mismatched or partial checks are unknown. Changed config invalidates old
|
||||
run evidence. Malformed/unauthenticated state yields blocked, unassigned output
|
||||
and exit 78; valid missing evidence yields unknown output and exit 0.
|
||||
|
||||
Output contains `proposal_only`, `evidence` (authentication status, **not**
|
||||
health), per-host `validated`/`negative`/`unknown` candidate alternatives,
|
||||
`assignments` and `unassigned`. Compatible equal args/selectors merge coverage
|
||||
across input labels. Greedy greatest-uncovered coverage, then lexical candidate
|
||||
ID, produces deterministic disjoint exact-host assignments. It does not claim
|
||||
global optimality or suffix coverage. Explicit legacy YouTube suffix policy is
|
||||
unchanged.
|
||||
|
||||
## INTRUSIVE / ISOLATED startup only
|
||||
|
||||
Set **both** `ZAPRET_AUTOTUNE_QUALIFY_CONFIG` and
|
||||
`ZAPRET_AUTOTUNE_STRATEGY_CONFIG` to container paths on a dedicated target.
|
||||
Do not send working user traffic to this target's published SOCKS port. This is
|
||||
not noninterrupting background search. Provision the private resource and 0600
|
||||
strategy files separately; compose forwards paths but does not mount files for
|
||||
you. Start with the managed disabled baseline, not an existing active profile.
|
||||
Qualification v1 accepts lowercase DNS hostnames, not literal IP targets; the
|
||||
ordinary resource-check command's broader literal-IP policy is unchanged.
|
||||
|
||||
PID 1 starts its owned SS+DoH services, initializes/reconciles known state, then
|
||||
launches one finite `qualify ... --isolated` session **instead of** starting the
|
||||
live zapret daemon or monitor. Qualifier checks root/target namespace, PID 1's
|
||||
actual entrypoint/environment, direct-child executable/arguments/session ACK,
|
||||
owned SS and DNS readiness anchors. A flag or docker-exec environment alone
|
||||
does not authorize qualification. Missing either startup path fails closed.
|
||||
Without `--isolated`, `qualify` remains UNKNOWN/exit 78 without runtime changes.
|
||||
|
||||
The shared monitor singleton excludes legacy and generic controllers. Any
|
||||
existing NFQ worker/queue, active profile, unknown config/journal or existing
|
||||
immutable matrix is refused. Known baseline config/journal/backup/identity
|
||||
inodes and bytes remain held unchanged throughout the job. This first producer
|
||||
does **not** pause/stop/restart existing active profiles; job-bound maintenance
|
||||
rollback is deferred rather than pretending a path-only daemon stop is safe.
|
||||
|
||||
Each candidate uses the existing worker-before-nft/queue-ready lifecycle on
|
||||
queue 200, fixed Lua imports and fwmark 0x10000000, plus native
|
||||
`--filter-l7=tls`. The actual child executable, argv, parent, start identity,
|
||||
queue socket ownership, executable/Lua hashes, native `--version` output 1.0.2
|
||||
and installed nft rule bytes are attested before capture and checked again at
|
||||
each collector guard. DoH's existing 0x50000000 socket mark bypass remains.
|
||||
TLS policy is collector/eligibility policy, not a nonexistent native TLS-version
|
||||
filter. All configured protocols are measured; incompatible selector policies
|
||||
remain UNKNOWN in the proposal.
|
||||
|
||||
The existing GET/HEAD generic collector preserves TLS origin verification,
|
||||
configured body/status/content-type expectations, private stdin and IP pinning.
|
||||
Completed expectation failures and typed peer response/TLS/transport negatives
|
||||
do not stop the remaining repeats/cells. Guard/source/unknown infrastructure or
|
||||
batch cancellation veto the entire batch; no partial positive matrix is saved.
|
||||
Config (including resolved environment/header secrets) and strategy-source
|
||||
snapshots are checked after DNS/before transfer and before sealing/publication.
|
||||
Workers are joined and temporary nft queues removed before verifying unchanged
|
||||
disabled baseline/runtime and sealing. The batch has a fixed 15-minute deadline
|
||||
with separate bounded cleanup; PID 1 keeps SS+DoH alive through cleanup.
|
||||
The bounded 64 KiB matrix size (including its authenticated envelope and
|
||||
worst-case field widths) is checked before any worker/nft operation; excessive
|
||||
catalogue/resource combinations return UNKNOWN/exit 64 even within individual
|
||||
count caps (32 strategies, 64 resources).
|
||||
|
||||
Successful batches, including complete negatives, return the assignment proposal.
|
||||
Incomplete/unsafe jobs return UNKNOWN/blocked exit 78 without proof. PID 1 then
|
||||
stays safe-idle with SS+DoH for inspection, for either success or blocked
|
||||
completion: no qualification retry loop, duplicate controller or automatic
|
||||
generic monitor start. Retests require future explicit immutable snapshot
|
||||
rotation, not overwriting prior evidence.
|
||||
|
||||
Completed controller cleanup is acknowledged separately from qualification in
|
||||
an exclusive, 0600 HMAC-authenticated `qualification-completed.json` on the
|
||||
current boot's SS tmpfs. It binds the launched PID, exit outcome (0 or 78) and
|
||||
verified cleanup; it is **not** strategy proof. PID 1 checks that ACK against its
|
||||
private session launch and revalidates the disabled baseline, singleton,
|
||||
SS/DoH ownership and absence of NFQ workers/temporary nft queues before shutting
|
||||
down dependencies. Thus TERM after a clean blocked qualifier has exited but
|
||||
before its parent reaps it can exit cleanly. Worker/nft/join/restoration failure,
|
||||
missing/foreign/mutated ACK or unverified early failure still veto clean shutdown
|
||||
with exit 78. Ordinary legacy/generic monitor exit 78 remains a safety veto.
|
||||
|
||||
No unsupported `--filter-mark` or blockcheck embedded/stream/progress/deadline
|
||||
flags are used. Background noninterrupting search, active-profile maintenance
|
||||
and immutable snapshot rotation for repeated runs remain deferred.
|
||||
|
||||
## Explicit generic transactional startup apply (local implementation)
|
||||
|
||||
On the target with its already completed private qualification batch, unset both
|
||||
qualification startup paths and set:
|
||||
|
||||
```
|
||||
ZAPRET_AUTOTUNE_RESOURCE_CONFIG=/private/resources.json
|
||||
ZAPRET_AUTOTUNE_RESOURCE_APPLY=1
|
||||
ZAPRET_AUTOTUNE_STOP_GRACE_PERIOD=90s
|
||||
```
|
||||
|
||||
Compose forwards the apply flag and configurable stop grace (legacy default
|
||||
remains 240s). Generic PID 1 cleanup has one shared 80-second budget. No external
|
||||
manager, new proof importer, SQL frontend surrogate or parallel journal exists.
|
||||
PID 1 starts owned SS/DoH, reconciles known bytes, launches the acknowledged
|
||||
`managed resource-apply --resource-config FILE` child before the checks-only
|
||||
monitor, and starts the native daemon only with verified committed config.
|
||||
This is at most one apply launch per PID 1 lifetime. A RAM latch is marked failed
|
||||
before launch and succeeds only after child wait=0 and owned-session cleanup.
|
||||
Launch/ACK failure, nonzero/signal exit, SS/DoH loss, or incomplete cleanup leaves
|
||||
it failed: after authoritative cleanup PID 1 remains cancellably BLOCKED, without
|
||||
restarting SS/DoH, native zapret, apply or monitoring, even across former 60s retry
|
||||
windows. Cleanup still stops SS/DoH (DNS last); BLOCKED does not keep them alive.
|
||||
Successful apply is skipped on later monitor/SS restarts, but every runtime cycle
|
||||
still reconciles committed refs before native start. Qualification and legacy
|
||||
retry behavior are unchanged.
|
||||
|
||||
**Ceiling:** this is not a durable manual-intent latch. A fresh PID 1 (including
|
||||
an external Docker restart policy) resets authorization and, with flag=1, permits
|
||||
one new attempt. Operators must disable unattended external restart/recreation
|
||||
for this workflow. Set `ZAPRET_AUTOTUNE_RESOURCE_APPLY=0` for explicit repair-only
|
||||
startup: prepared state is reconciled, never promoted or reapplied. Identical
|
||||
committed requests still resume by authenticated refs without a new transaction.
|
||||
The command refuses standalone/docker-exec callers: actual PID 1 environment,
|
||||
entrypoint, namespace, direct-child executable/argv and held launch ACK are
|
||||
required, plus the existing controller singleton and no pre-existing NFQ worker.
|
||||
|
||||
First apply supports **disabled baseline → generic**. Explicit PID 1 startup
|
||||
selection also permits generic → generic rotation among already verified
|
||||
alternatives in the **same immutable authenticated registry/matrix/source**.
|
||||
Active legacy profiles, changed source/catalogue/proofs and uninterrupted live
|
||||
rotation remain unsupported. Every configured resource must require `both` TLS protocols and
|
||||
every assigned exact host must have complete, candidate-bound passing proof for
|
||||
all its resources across all input labels. New apply requires completion within
|
||||
24 hours, never in the future. Negative/unknown/unassigned hosts cannot authorize
|
||||
apply. Registry definitions may still describe single-TLS proposals, but native
|
||||
apply rejects them instead of widening unsupported TLS-version selectors.
|
||||
|
||||
Canonical ordered args/selectors define profiles separated by `--new`; each
|
||||
contains `--filter-tcp=443 --filter-l7=tls --hostlist-domains=^a,^b`. Exact-host
|
||||
semantics use the pinned native source
|
||||
`https://raw.githubusercontent.com/bol-van/zapret2/4df94497869e18ed4fdf1272141427b2fbfc6604/nfq2/hostlist.c`:
|
||||
inline `AppendHostlistItem` uses `addpool`, leading `^` sets
|
||||
`HOSTLIST_POOL_FLAG_STRICT_MATCH`, and `SearchHostList` rejects suffix fallback
|
||||
when that flag is set and `!bHostFull`. Existing YouTube suffix policy is unchanged.
|
||||
The packaged executable/Lua hash must match private attestation; held native,
|
||||
Lua, source, marker and proof identities/bytes are guarded through the transaction.
|
||||
No active queue version probe or mark/filter change is made (DoH500/probe100/live400).
|
||||
|
||||
The **existing** journal gains v4 independently tagged committed/pending refs
|
||||
(`baseline`, `legacy`, `generic`). Generic refs exclude legacy IDs/scopes and bind
|
||||
exact authenticated registry/matrix envelope hashes, source fingerprint,
|
||||
internally derived canonical assignment digest and complete config hash.
|
||||
Optional scalar `strategy_ids` retains the canonical selection policy on each
|
||||
generic ref; missing/empty means the original default greedy policy. No policy
|
||||
is permitted on baseline/legacy refs. This remains v4: older strict-decoding
|
||||
engines reject the new field, so downgrade is blocked, not silently reinterpreted.
|
||||
Unknown versions/tags, mixed refs and unresolved historical reconstruction block
|
||||
without adopting or erasing foreign state; v1–v3 legacy behavior remains.
|
||||
Staging/backup/prepared/replace/fixed restart/health/commit reuse the same engine.
|
||||
All required resources/TLS are checked through the existing synchronous owned
|
||||
SS/DoH collector (all-answer SSRF validation, CONNECT pin, SNI/cert validation,
|
||||
private stdin). No Player API health or public check JSON authorizes commit.
|
||||
|
||||
Committed identical requests are idempotent: historical proof expiry alone does
|
||||
not trigger reload, reapply or rejection on restart. Authenticated integrity,
|
||||
source and native drift still block. Prepared recovery **never** promotes pending
|
||||
bytes, including after successful health: it restores prior bytes and reloads.
|
||||
Generic rollback additionally checks all resources before finalizing the journal.
|
||||
**Strict first-slice ceiling:** a DPI-broken disabled baseline can legitimately
|
||||
fail these checks; restored bytes then remain `prepared`/BLOCKED until operator
|
||||
repair. Physical restoration is not falsely reported as healthy rollback.
|
||||
Legacy rollback has no new health requirement. Cancellation joins synchronous
|
||||
checks before rollback; PID 1 bounds the owned session and retains prepared state
|
||||
on unsafe/unfinished cleanup. `ActiveCandidates` refuses generic deployments;
|
||||
managed/resource status validates generic refs without inventing legacy groups.
|
||||
|
||||
### Explicit startup selection (implemented; bounded positive field observations)
|
||||
|
||||
Set `ZAPRET_AUTOTUNE_RESOURCE_STRATEGY_IDS=s1,s2` with
|
||||
`ZAPRET_AUTOTUNE_RESOURCE_CONFIG` and `ZAPRET_AUTOTUNE_RESOURCE_APPLY=1` on a
|
||||
fresh container PID 1. Empty retains default greedy selection. The bounded list
|
||||
accepts at most 32 unique registry IDs, canonicalized in lexical order; malformed,
|
||||
duplicate, unknown or incomplete-coverage requests cannot authorize a transaction.
|
||||
Eligibility is filtered **before** equal-args/selectors coverage merging; excluded
|
||||
candidates cannot lend proof to a selected profile. Compatibility evidence and
|
||||
all alternatives remain visible; the policy is not proof or a new qualification.
|
||||
No registry/matrix files are written by rotation. Exact hosts, both TLS versions,
|
||||
all required cells/M=3, native/source integrity and the 24-hour freshness gate
|
||||
remain required for every new selection. Identical committed selection replays
|
||||
without reload even after expiry. Distinct redundant allowlists are distinct
|
||||
policies and may create a transaction even if rendered bytes coincide.
|
||||
|
||||
PID 1 forwards the setting in environment only; the apply child authenticates it
|
||||
against PID 1's original environment, not CLI arguments. Syntax is validated
|
||||
even with APPLY=0; nonempty policy requires generic resource config. With APPLY=0
|
||||
selection is ignored for reconciliation/monitoring (registry membership is not
|
||||
resolved): committed and pending refs reconstruct independently from their own
|
||||
stored policies and historical completion horizon, never the requested setting.
|
||||
Prepared recovery restores the old generic bytes **and policy**, never promotes
|
||||
the pending side; failed rollback health retains prepared state for repair.
|
||||
Startup still allows just one attempt per PID 1; external restart policies reset
|
||||
that authorization. There is no monitor autotuning, new qualification, proof
|
||||
import, snapshot overwrite or uninterrupted maintenance in this slice.
|
||||
|
||||
Local root fixture tests cover the production snapshot factory and shared engine;
|
||||
independent local review and deterministic 180-second retry-window fixtures passed.
|
||||
The earlier bounded baseline gates below cover guarded apply/replay, strict rollback,
|
||||
prepared repair and the per-PID-1 failure latch; they do not establish rotation fault
|
||||
acceptance or authorize live rollout.
|
||||
|
||||
### Verified-profile rotation observations — source `8d43dd9`
|
||||
|
||||
**Positive field observations only; partial native acceptance, not full rotation
|
||||
safety acceptance or READY.** Saved-artifact privileged audits observed default A
|
||||
generation 1 → selected multisplit B generation 2 → identical B replay generation 2
|
||||
→ A return generation 3. No repeat positive sequence is required for the next gate.
|
||||
|
||||
- A config SHA-256 was `27f9a8ffd56a4c19d4bd522dd9f5a5e8b49b18b0ae48d41431ef9b22975ea772`;
|
||||
B was `51faacf53e76fa0ce95fead1f02da3846c1706ecb9edc8ddaba095c089ea2c82`.
|
||||
Replay retained transaction `09dfd03e9741abdf8e6b9f3bd44b2118`; A return used
|
||||
`49a29d560f086a11f9de770913777e07` and restored the exact original A bytes.
|
||||
Historical prepared snapshots held actual old-A backup before B and actual old-B
|
||||
backup before A. Sampled old-committed → prepared → new-committed transitions
|
||||
demonstrate these observations, not crash safety or failed-rotation rollback.
|
||||
- Fourteen saved registry/matrix snapshots passed exact-payload HMAC and byte
|
||||
invariance checks; source fingerprint was unchanged. The two original runs,
|
||||
16 M=3 cells and 48 prior both-TLS qualification transfers were reused, about
|
||||
12 hours old at startup, **not requalified**. Native A/B argument hashes differed;
|
||||
A return matched original A, with native queue 300 socket-inode/PID linkage
|
||||
observed. Early A-return process inventory is missing; full phase-path native
|
||||
argv acceptance is not established.
|
||||
- Six authenticated monitor snapshots each showed four HEALTHY resources and zero
|
||||
failures. Saved fixture records showed 64 successful GETs, TLS 32/32, matching
|
||||
SNI/Host and expected-header presence; A return contributed 32, TLS 16/16.
|
||||
This proves fixture compatibility, **not causal DPI benefit** or continuously live
|
||||
change. Original A-return official stop exited 0; fixture stop exited 2, not an
|
||||
observed product failure in this run.
|
||||
- Evidence base: `/home/agent/rotation-8d43dd9/field-private/` (root-private).
|
||||
`report.safe.json` covers only generations 1/2/2, not final completion. Generation
|
||||
3 is supported separately by `a-return-timeline.private.json`, retained state and
|
||||
`final-passive-evidence/{before,stopped}/` snapshots. The initial harness ended
|
||||
with KeyboardInterrupt; two continuation-construction bugs and observer filename
|
||||
alias corrections are disclosed in `harness-path-alias.safe.json`. Corrections
|
||||
affected host observer artifacts only, not proof inputs.
|
||||
- Final saved cleanup verification is
|
||||
`final-passive-evidence/final-cleanup-verification.safe.json`, timestamp
|
||||
`1792051407108`: both exact owned stage containers and the owned network absent,
|
||||
eight recorded PID paths absent, both known network namespaces without PIDs,
|
||||
and three foreign full-inspect hashes unchanged. Owned containers/network were
|
||||
explicitly removed with orchestrator approval; private state/evidence/keys were
|
||||
retained. Foreign SS remained running and the two older containers unchanged.
|
||||
Xray was not mutated; no fresh Xray inventory or exhaustive process audit is claimed.
|
||||
- The first independent forensic audit passed the bounded 14-HMAC positive evidence,
|
||||
but the automated grader rejected missing independently refreshed dated evidence
|
||||
and exact native argv phase paths. Do not upgrade this to full native/safety
|
||||
acceptance. Unknown-ID refusal and old-profile fault/repair observations were
|
||||
subsequently recorded below, with narrower attribution than a branch-specific
|
||||
refusal proof. Crash safety remains untested. Natural 24-hour expiry remains
|
||||
unit-only, without a 24-hour wait.
|
||||
- Prior local verification recorded 17 EUID-0 fixtures in one race/parallel pass
|
||||
with 57 subrecords; namespace UID 0 mapped to host UID 100000, not host-initial
|
||||
root. Full producer Go/race and four Python checks passed serially after parallel
|
||||
resource-contention failures; this does not establish a contention-free baseline.
|
||||
These are prior checks, not tests rerun for this documentation-only update.
|
||||
|
||||
### Subsequent old-generic rotation faults (bounded saved-artifact acceptance)
|
||||
|
||||
Actual remote source was `8d43dd9`, **not** shared-core `45d28e3`; these separate
|
||||
results do not establish end-to-end acceptance of a combined release. Privileged
|
||||
reads verified `/home/agent/rotation-fault-8d43dd9/field-private/ACCEPTANCE.safe.md`
|
||||
and `standaloneprivilegedaudit.safe.json`, dated `1792054235151`
|
||||
(2026-10-15 08:50:35.151 UTC), verdict
|
||||
`PASS_BOUNDED_SAVED_ARTIFACTS_AND_CURRENT_CLEANUP`.
|
||||
|
||||
- All 13 snapshots passed exact-serialized-payload HMAC/immutable proof checks.
|
||||
Old A is generic with empty/default policy, not baseline; B selects multisplit.
|
||||
Historical old/new refs reconstruct independently.
|
||||
- One-shot: eight B apply checks (one HTTP500), eight successful A rollback checks,
|
||||
A committed/pending clear, no automatic reapply through 80s; official stop0.
|
||||
- Persistent: 24 checks across apply/rollback/cleanup reconciliation, three
|
||||
HTTP500; A bytes physically restored with PREPARED old-A/pending-B retained,
|
||||
no reapply through 80s; stop78 retained the unsafe-cleanup latch. Healthy fixture
|
||||
plus only APPLY=1→0 (requested B still set) gave eight reconcile + eight monitor
|
||||
successes, old A committed/pending clear and four HEALTHY resources; stop0.
|
||||
- Total: 56 authenticated requests, TLS28/28, four controlled HTTP500. At the
|
||||
dated audit all 150 recorded owned PID/start identities were gone, recorded netns
|
||||
had no processes, five owned containers and network were removed, and three
|
||||
foreign raw-inspect hashes were unchanged. Root-private evidence was retained;
|
||||
sampled identities are not an exhaustive all-helper inventory.
|
||||
- Valid-syntax `unknown-safe-id` reached the actual CLI apply entry, which exited
|
||||
78 and latched BLOCKED, with zero requests. Before/stopped config, journal,
|
||||
backup and proof bytes were unchanged; no staging was sampled. This is an
|
||||
**observed refusal with an unknown-ID request**, not uniquely attributed to the
|
||||
unknown-ID branch: the generic diagnostic cannot exclude another trust error.
|
||||
Source validation order supports an inference, not a specific logged reason
|
||||
or complete pre-mutation proof. Official stop0 is separate from CLI exit78.
|
||||
|
||||
Startup remains one-shot per PID1, without automatic apply retry. These fixture
|
||||
faults do not establish crash/cancellation safety, continuously live rotation,
|
||||
causal DPI benefit or READY. Next migration work is the minimal typed generic
|
||||
image/media capability above; owned managed v2 scoped observations are recorded
|
||||
above, with full runtime acceptance still incomplete.
|
||||
New-batch/versioned stores, uninterrupted rotation, strategy breadth and Prometheus
|
||||
are separate future work, not prerequisites to documenting this collector slice.
|
||||
|
||||
## Bounded generic apply/replay and failure-latch acceptance
|
||||
|
||||
**DONE within the stated bounds**, not full generic autotuning or service READY.
|
||||
Fault acceptance used source `643d0c6`; the earlier healthy committed replay used
|
||||
`36aace0` after the atomic launch-ACK correction. Independent privileged audit
|
||||
returned `PASS_BOUNDED_GENERIC_APPLY_FAULTS` for the observed A/B/C safety gates.
|
||||
Root-private evidence references (no private payloads reproduced here):
|
||||
|
||||
- `/home/agent/generic-latch-643d0c6/field-private/field-report.md`
|
||||
- `/home/agent/generic-latch-643d0c6/field-private/independent-audit.safe.json`
|
||||
- `/home/agent/ack-recreate-36aace0/field-private/field-report.md`
|
||||
|
||||
### Observed acceptance
|
||||
|
||||
- Real apply health reused authentic, still-fresh qualification: 16 M=3 cells,
|
||||
48 prior TLS transfers, two strategies and four resources. No new qualification,
|
||||
timestamp rewrite, fake ACK or imported proof was used. Pinned native 1.0.2/Lua
|
||||
stayed unchanged. The earlier healthy start/recreate each made eight TLS checks
|
||||
with four authenticated HEALTHY resources; v4 committed generation 1, refs,
|
||||
config and proof bytes remained unchanged. Both official stops exited 0.
|
||||
- **A — one HTTP 500:** eight apply plus eight rollback requests, one 500 total.
|
||||
Rollback committed prior baseline generation 1 and cleared pending. Request
|
||||
count, journal and config stayed frozen at 45.096/90.084/135.124 seconds;
|
||||
one apply only, no automatic retry. PID 1 stayed alive/BLOCKED with SS/DNS and
|
||||
other owned runtime stopped. Official stop exited 0.
|
||||
- **B — persistent HTTP 500:** 24 requests and three 500s: eight apply, eight
|
||||
rollback and eight cleanup-reconcile checks, not a second apply. Prior baseline
|
||||
bytes were physically restored, but failed health retained PREPARED committed
|
||||
baseline/pending generic refs. State stayed unchanged for 80.097 seconds with
|
||||
no reapply; official stop exited 78. After repairing only fixture behavior and
|
||||
changing startup APPLY=1 to APPLY=0, the same state/source/environment completed
|
||||
eight reconcile and eight monitor checks. Baseline became committed, pending
|
||||
cleared, and generic was never promoted; official stop exited 0.
|
||||
- **C — TERM during a real slow request:** one canceled request plus eight passing
|
||||
rollback checks. The canceled curl was joined before the first rollback request;
|
||||
baseline generation 1 committed and pending cleared, with no false generic
|
||||
commit. Actual PID 1 exit was **78**, with `monitor cleanup status 78`, not 0.
|
||||
Apply health cancellation is collapsed into `errPlan`/CLI 78; outside the
|
||||
qualification-specific completion path, the supervisor accepts only 0/143 and
|
||||
retains a sticky failure even after verified rollback. This is conservative
|
||||
**outcome conflation**, not an observed rollback-health failure or cleanup
|
||||
defect. Do not blanket-allow 78 or manufacture a completion ACK. A normal-0
|
||||
path would require authenticated completion plus fresh guards; that optional
|
||||
future refinement is not a blocker for this bounded P0 safety slice.
|
||||
|
||||
### Bounds and remaining work
|
||||
|
||||
- A/B/repair/C totalled 65 actual HTTPS requests. All 171 observed owned PID/start
|
||||
identities disappeared, owned netns were empty, and owned targets/network were
|
||||
removed. Foreign SS/container full-inspect hashes stayed unchanged. Passive
|
||||
100ms inventories do not cover every shorter-lived helper; no exhaustive argv
|
||||
or packet-level privacy audit is claimed. Same-root HMAC-key mutation is outside
|
||||
the trust boundary.
|
||||
- The latch is one RAM authorization per PID 1, **not durable manual intent**.
|
||||
Fresh startup/external restart policy reauthorizes ENV=1; keep unattended
|
||||
external restart/recreation disabled. ENV=0 repairs/reconciles only. Historical
|
||||
committed replay/repair allows proof expiry; a new apply still requires fresh,
|
||||
complete source-bound qualification within 24 hours. Natural remote 24-hour
|
||||
expiry remains unit-covered only; remote qualifier singleton conflict is still
|
||||
untested.
|
||||
- Use an execution-free remote harness: stage inputs before startup; no target
|
||||
`exec`, `cp` or `nsenter` during active guards. Observe host metadata/private
|
||||
files and Docker logs/inspect; official stop is the cancellation operation.
|
||||
Earlier scan EACCES nonrecurrence does not establish its cause.
|
||||
- No production READY, causal DPI benefit, active-profile maintenance, plan or
|
||||
snapshot rotation, automatic recovery or monitor autotuning is accepted here.
|
||||
Next roadmap work is the missing rotation fault/repair gate above, then dynamic
|
||||
profile/plan rotation and compatible selectors,
|
||||
generic resource tuning and bounded strategy-DB collections; bounded-ID
|
||||
Prometheus remains a separate planned function. No new framework, probes or
|
||||
native/live upgrade was introduced. Legacy YouTube exit 78 and unfinished
|
||||
images/avatars remain NOT READY.
|
||||
|
||||
## Bounded remote acceptance — source `5cdc0cd`
|
||||
|
||||
The isolated producer/registry/matrix/planner slice is **bounded-qualified**.
|
||||
Independent privileged evidence audits passed the observed positive, fault and
|
||||
active-state refusal gates. **Full remote concurrency acceptance is PARTIAL**;
|
||||
this is not generic apply, active-profile maintenance or service READY.
|
||||
|
||||
Root-private proof references (retained reports, not logs copied into this doc):
|
||||
|
||||
- `/home/agent/zapret2-autotune-matrix-5cdc0cd/execution-free-private/execution-free-report.md`
|
||||
- `/home/agent/zapret2-autotune-matrix-5cdc0cd/fault-final-private/qualification-fault-report.md`
|
||||
- `/home/agent/zapret2-autotune-matrix-5cdc0cd/refusal-gap-private/remaining-refusal-reports.md`
|
||||
|
||||
### Observed positive and fault gates
|
||||
|
||||
- The execution-free private batch made 60 actual transfers: two candidates,
|
||||
five resources, both TLS protocols, three attempts per cell. Its 20 aggregated
|
||||
cells contain 16 PASS and four `expectation_failed`: mandatory resource d's
|
||||
HTTP 500 fails for both candidates/protocols, leaving d unassigned. Exact
|
||||
hosts a/b/c receive the greedy multidisorder proposal with authentic multisplit
|
||||
alternatives. Input groups remain metadata, not assignment boundaries;
|
||||
there is no generic suffix coverage or automatic apply.
|
||||
- Independent verification authenticated the exact stored payload HMAC, not a
|
||||
reserialized JSON equivalent. Distinct native arguments/worker-argv hashes,
|
||||
executable/Lua/rules evidence, queue 200 and NFQ 1.0.2 attestations bind the
|
||||
two joined/restored runs. Disabled baseline bytes stayed unchanged; sampled
|
||||
workers, queue and temporary NFT rules were gone. Official PID 1 shutdown
|
||||
exited 0 in 1.31s; foreign inspect hashes stayed unchanged.
|
||||
- Mid-batch cancellation and effective semantic config drift each stopped
|
||||
after two requests, published no matrix and preserved the native baseline.
|
||||
Official PID 1 exits were 0 in 2.46s and 0.44s respectively; the deliberately
|
||||
changed config source was retained, not silently rolled back.
|
||||
- Unsafe Lua/queue/fwmark/user/file definitions and evidence/schema imports
|
||||
were rejected without transfers. Oversized capacity was rejected before
|
||||
worker launch; unauthorized isolated startup was refused. Corrupt MAC blocked
|
||||
planning while preserving payload bytes; changed effective config produced
|
||||
unknown/unassigned output without changing the original authenticated matrix.
|
||||
Immutable catalogue replacement and completed-matrix retest were refused
|
||||
without transfers or overwriting original state.
|
||||
- A genuine archived v3 committed journal with two candidates passed official
|
||||
`reconcile`/`ActiveCandidates` validation before qualification refused active
|
||||
profile state. Profile/config, journal, last-good and target identity remained
|
||||
byte-identical; zero transfers/workers and no matrix. Actual PID 1 exit 78 was
|
||||
an expected early preflight refusal, **not** an exit-0 cleanup ACK. This did
|
||||
not activate or maintain a running active-profile NFQ daemon; that remains
|
||||
unsupported. Whole-DB byte preservation is not claimed.
|
||||
|
||||
### Explicit limits and testing lesson
|
||||
|
||||
- Remote qualifier singleton conflict is **NOT RUN**. The earlier planted-lock
|
||||
fixture failed initialization before that branch and does not count. The
|
||||
shared-lock unit second-acquire `EWOULDBLOCK` check passed, but is not remote
|
||||
qualifier concurrency acceptance. No deterministic official lifecycle seam
|
||||
exists for this bounded test; no test framework or racy injection was added.
|
||||
- Natural 24-hour remote expiry is **NOT RUN**. The existing unit check rejects
|
||||
a structurally valid 25-hour-old proof; this is not observed remote expiry.
|
||||
- Privacy sentinel absence was checked in tested logs/state and sampled fault/
|
||||
refusal argv. Positive active argv and exhaustive DNS packet-level privacy
|
||||
were not proved. Configured/observed encrypted DNS is not exhaustive proof
|
||||
against transient fallback.
|
||||
- Compatibility on this baseline is not causal DPI-desync benefit. Historical
|
||||
worker-scan EACCES origin remains unknown; nonrecurrence in the execution-free
|
||||
batch does not identify its cause. Earlier harness failures remain recorded
|
||||
in the referenced reports, not erased or relabeled as product acceptance.
|
||||
- Stage all inputs before target startup. From startup through qualification
|
||||
completion, prohibit target `exec`, `cp` and `nsenter` (even passive exec starts
|
||||
a process). Observe host `/proc` metadata and Docker logs/inspect only; inspect
|
||||
runtime and run `plan` after finalization/cleanup. The explicit official stop
|
||||
used for cancellation is the documented exception, not an observer exec.
|
||||
- Owned test containers/network were removed and private evidence retained.
|
||||
Foreign SS1080 stayed running and legacy11082 stayed stopped with exit 78.
|
||||
No live upgrade or push occurred; unresolved legacy exit 78, YouTube/image
|
||||
readiness, transactional generic apply/profile-journal migration, immutable
|
||||
rotation and noninterrupting search remain outside this producer acceptance;
|
||||
the later bounded apply/replay gates are recorded separately above.
|
||||
|
||||
## Internal YouTube adapter slice (local; field acceptance pending)
|
||||
|
||||
`internal/adapter` is a small closed compiled-in contract (`Binding`, ephemeral
|
||||
`Resource`/`Plan`, `Module`); `internal/adapter/youtube` implements `youtube/1`.
|
||||
The explicit kind/version registry in `service_adapters.go` has no init hooks,
|
||||
dynamic loading, public configuration, HTTP fallback or module privileges.
|
||||
Player media parsing (`playerMediaResource`) and owned native qualification
|
||||
(`verifyGooglevideo`) actually call it. Discovery retry/rotation/cancellation and
|
||||
representative selection remain in the legacy controller; signed MP4 freshness
|
||||
and the host/TLS qualification plan are pure service semantics in the module.
|
||||
|
||||
The next private execution bridge is now wired: `verifyGooglevideo` runs the
|
||||
actual owned-worker collector through `executeAdapterPlan[T]`. Core independently
|
||||
pins the known module binding and exact three-host/TLS1.2+1.3, 10-attempt/120-second
|
||||
warmup, three steady samples/protocol and 1024-byte policy before worker startup
|
||||
and probes. Invalid, duplicate, overflowing or weakened plans fail closed. The
|
||||
executor preserves host order, alternating warmups, discarded learning successes,
|
||||
strict measured steady facts and negative/fatal classification. It has no worker,
|
||||
goroutine, persistence or seal capability; existing core range/ftyp/TLS validation
|
||||
and post-worker-join proof validation/sealing remain the authority.
|
||||
|
||||
This is a **private execution bridge**, not public adapter configuration or a new
|
||||
generic service/matrix proof. Dynamic-inventory fingerprints and warmup-bearing
|
||||
generic proof binding are still required before generic-controller cutover.
|
||||
The inherited legacy SOCKS5h boundary below is unchanged, not an all-DNS-answer
|
||||
pinning guarantee. Three-CDN qualification does not itself grant suffix scope;
|
||||
images/avatars and runtime acceptance remain separate and not READY.
|
||||
|
||||
Player JSON now travels through bounded private stdin (`curl --json @-`), not
|
||||
process argv; the shared runner passes no stdin for metadata GETs. Video IDs are
|
||||
validated before execution, POST bytes and headers remain unchanged, and opaque
|
||||
public errors retain typed cancellation/unsafe-output causes. This is a private
|
||||
Player transport change, not generic public POST/discovery configuration.
|
||||
|
||||
Bridge-local checks: an unchanged fixture run against archived `f485317` and the
|
||||
wired producer has normalized proof SHA256
|
||||
`fb380324e320a78f1ceb011c59b82b9de4ff4f31f5f032f469a181292987e3b2`.
|
||||
Only warmup duration, attestation time and private seals are excluded; exact
|
||||
probe order and cleanup/join are checked. Baseline and focused serial/race logs:
|
||||
`/tmp/opencode/adapter-executor-{baseline,focused}.log`. These are injected local
|
||||
checks, not network/native acceptance or a full-suite rerun.
|
||||
|
||||
Cancellation review follow-up: both warmup and steady probe boundaries now use
|
||||
`managedRecoveryContextError`, preserving concurrent unsafe I/O/output overflow
|
||||
alongside cancellation/deadline causes. Production collector regressions check
|
||||
fatal/non-restartable classification, full cleanup/join and no attestation/seal;
|
||||
plain cancellation remains timeout-safe. A pre-fix Go-overlay negative control
|
||||
fails all eight mixed-failure cases; the fixed producer retains the hash above.
|
||||
Actual `sudo` EUID-0 serial full Go checks passed (1806 test/subtest pass events),
|
||||
as did 227 affected serial/race events, separate module race tests, vet and a
|
||||
CGO-disabled build. Logs: `/tmp/opencode/adapter-executor-root-full.json`,
|
||||
`adapter-executor-context-root-race.json`, `adapter-executor-context-focused.log`
|
||||
and `adapter-executor-context-negative-control.log` in that same directory.
|
||||
The reviewer-reported earlier `SavedFirstAndFailClosed/nonpass` freshness failure
|
||||
did not recur in ten focused repeats or this full run; no freshness guards or
|
||||
fixtures were changed, and its earlier cause is not established as a baseline
|
||||
flake. Python checks were not rerun in this follow-up. These are local tests,
|
||||
not remote runtime acceptance, a commit/deployment or a READY claim.
|
||||
|
||||
Core origin/TLS validation, shared image/media transfer code, bounded worker
|
||||
lifecycle and post-cleanup/join sealing remain authoritative. Legacy media and
|
||||
metadata routing retains SOCKS5h with the owned encrypted-resolver guard and
|
||||
exact origin/SNI/certificate/redirect checks. It does **not** use public generic
|
||||
`runResourceProtocol`'s all-A/AAAA approval and connect-to IP pinning. That inherited
|
||||
ceiling is unchanged, not a pinning guarantee: full request-policy capabilities
|
||||
(including future metadata POST) and parity are prerequisites to generic cutover.
|
||||
No new DNS/IP routing policy is introduced by this extraction. Both TLS
|
||||
versions must learn separately, then each has three measured MP4 samples per
|
||||
representative host, minimum 1024 bytes (not an exact 64 KiB requirement).
|
||||
Measured endpoints do not authorize a suffix: existing exact/suffix/experimental
|
||||
policy and independent thumbnail/avatar qualification are unchanged. The module
|
||||
cannot manufacture evidence, request wildcard apply or claim YouTube READY.
|
||||
|
||||
New adaptive proof **v2** includes adapter kind/version/immutable policy SHA256
|
||||
inside the producer seal. Public JSON cannot recreate that seal. Missing or
|
||||
mismatched binding/version/digest fails closed; v1 with v2 fields is rejected.
|
||||
Historical proof **v1** explicitly uses the pinned legacy YouTube v1 contract,
|
||||
including historical freshness-independent journal reconstruction. Its config
|
||||
bytes/hash and journal refs remain stable. Fresh stored trusted v1 proofs retain
|
||||
their original apply/qualification eligibility, including unchanged attestation
|
||||
freshness and media gates. Expired v1 can reconstruct but cannot authorize fresh
|
||||
apply. Only **new production/persistence** requires sealed, module-bound v2;
|
||||
neither sealed-new-v1 nor public unsealed v1/v2 DTOs can publish new evidence.
|
||||
Cold v1 evidence, SQL schema 7, native arguments,
|
||||
public GET/HEAD v1/v2, state keys and HMAC formats are unchanged.
|
||||
|
||||
This slice is not full generic-controller migration, native-engine runtime
|
||||
acceptance, a security sandbox (including against same-UID actors), or a fix for
|
||||
unresolved avatars/legacy exit 78. No legacy path is deleted. Local tests and
|
||||
archived baseline parity are recorded separately; no remote validation or commit
|
||||
is part of this extraction review.
|
||||
|
||||
Discovery preflight resolves the required module identity and policy digest
|
||||
before any Player/WWW/avatar/front/media requests, including injected callbacks,
|
||||
legacy proposal discovery and managed discovery hooks. Missing/mismatched modules
|
||||
produce a fixed private sentinel wrapped as a fatal configuration failure, not
|
||||
retryable unavailable inventory, a transfer-negative or genuine no-strategy pause.
|
||||
The compiled production registry remains fixed; synchronized test replacement
|
||||
does not add public runtime module configuration or loading.
|
||||
|
||||
### Local verification for this extraction (2026-10-15)
|
||||
|
||||
- Fresh serial `go test -json -p 1 ./... -parallel 1 -count=1 -timeout 600s`:
|
||||
399 top-level / 1737 including subtest PASS events, zero failures.
|
||||
- Affected serial race run (`-race -p 1 -parallel 1 -count=1`) covers adapters,
|
||||
Player/discovery, owned qualification/proof gates, historical journal parity,
|
||||
independent assets and generic resource core: 61 top-level / 307 including
|
||||
subtest PASS events, zero failures. Full regex and logs are retained at
|
||||
`/tmp/opencode/youtube-adapter-fixes/{full,race}.json`.
|
||||
- Archived `d324b4137b2e95f1c5f4d6c8240470e3f8749f0c` and current builds each
|
||||
passed 275 affected test/subtest events. The portable legacy parser/journal
|
||||
corpus ran **unmodified** on both: file SHA256
|
||||
`7b8eea9c9bbcdd8f0fadde3f78596d41e9d032a0a4f7b2ca90d1223d106584d5`.
|
||||
Both reconstructed historical config SHA256
|
||||
`172c7377388334e4baa2dc1009270478dc271a7b2f00d28cfada631f44a35364`,
|
||||
retaining journal bytes/refs after reconcile and ActiveCandidates validation.
|
||||
An additional unchanged portable v1 corpus passed 2 top-level / 8 including
|
||||
subtest events on **each** build: fresh stored v1 shortlist/apply eligibility,
|
||||
expired fresh-apply refusal, and staged/backup/prepared/replaced/reloaded/healthy
|
||||
crash rollback with expired historical proofs. Backup/config bytes and hashes
|
||||
are stable; pre-prepare journal bytes are unchanged and post-prepare generation/
|
||||
reload/ref transitions follow the original contract. Corpus SHA256:
|
||||
`6df53b12af5d80d164ea3201a614a5fd5723561dc7b2d6bc2d2aa37f845f053d`.
|
||||
New negative tests separately reject even internally sealed NEW v1 writes and
|
||||
public unsealed DTOs; missing/changed module identity/policy makes zero Player,
|
||||
WWW/avatar/media/front/worker calls and does not become a two-hour absence.
|
||||
- All four `python3 deploy/managed-target/check_{static,config,startup_diagnostics,supervision}.py`
|
||||
checks passed serially; supervision was bounded with `timeout 600s`, no retry.
|
||||
`go vet ./...`, `CGO_ENABLED=0 go build`, `sh -n` on all three entrypoint/
|
||||
supervision scripts, and `git diff --check` passed. Build SHA256:
|
||||
`8327faf86098145bb41306b3de895cdd622aeccdcda90f8ec73e19a759c8f98a`.
|
||||
- Ownership-sensitive final runs used `sudo -n` EUID 0 **inside the existing
|
||||
user namespace**, mapping UID 0 to parent/host UID 100000, not initial-host
|
||||
root. The initial extra single-UID `unshare -Ur` run failed ownership fixtures
|
||||
(`chown: invalid argument` / `invalid private qualification data`); a stale
|
||||
test assertion also expected newly produced proof v1 and was updated to v2.
|
||||
Those prior extraction failures are retained in
|
||||
`/tmp/opencode/youtube-adapter-review/go-full.json`, not reported as passes. Vet's
|
||||
initial unkeyed shared-DTO literals were fixed before final successful checks.
|
||||
|
||||
Reproduction commands (run serially; root is only for offline ownership fixtures):
|
||||
|
||||
```sh
|
||||
pattern='Test(YoutubeLegacy|ManagedGooglevideo|ManagedAdaptive|ExperimentalPromotion|Googlevideo|Resource(Core|Discovery|Contract)|Player|Discover|ManagedAsset|Asset)'
|
||||
sudo -n env GOPROXY=off GOTOOLCHAIN=local GOMODCACHE="$(go env GOMODCACHE)" "$(command -v go)" test -json -p 1 ./... -parallel 1 -count=1 -timeout 600s
|
||||
sudo -n env GOPROXY=off GOTOOLCHAIN=local GOMODCACHE="$(go env GOMODCACHE)" "$(command -v go)" test -race -json -p 1 ./internal/adapter/... ./cmd/zapret-autotune -parallel 1 -run "${pattern}|Test(ServiceAdapter|LegacyYoutubeV1|QualificationPlan|SignedSample|DiscoverResponse)" -count=1 -timeout 600s
|
||||
# Baseline/current parity: the same $pattern, without -race, on both trees.
|
||||
# Baseline tree: git archive d324b4137b2e95f1c5f4d6c8240470e3f8749f0c;
|
||||
# add identical youtube_legacy_parity_test.go and youtube_legacy_apply_parity_test.go.
|
||||
# The additional portable v1 corpus runs separately with -run '^TestLegacyYoutubeV1'.
|
||||
```
|
||||
@@ -0,0 +1,125 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"time"
|
||||
|
||||
"zapret-autotune/internal/adapter"
|
||||
"zapret-autotune/internal/adapter/youtube"
|
||||
)
|
||||
|
||||
var errAdapterPlan = errors.New("invalid private adapter execution plan")
|
||||
|
||||
// This is core policy, not module-supplied caps. Adding a binding requires an
|
||||
// audited execution policy here; even a correctly identified module is untrusted.
|
||||
func validateAdapterPlan(binding adapter.Binding, plan adapter.Plan, resources map[string]mediaResource) error {
|
||||
if binding != (youtube.Module{}).Binding() || len(plan.Hosts) != 3 ||
|
||||
plan.Protocols != [2]string{"tls12", "tls13"} || plan.Samples != 3 ||
|
||||
plan.MinBytes != 1024 || plan.WarmupLimit != 10 || plan.WarmupTimeout != 120*time.Second {
|
||||
return errAdapterPlan
|
||||
}
|
||||
if (youtube.Module{}).ValidateHosts(plan.Hosts) != nil {
|
||||
return errAdapterPlan
|
||||
}
|
||||
for _, host := range plan.Hosts {
|
||||
resource, ok := resources[host]
|
||||
if !ok || !validDomain(host) || resource.host != host {
|
||||
return errAdapterPlan
|
||||
}
|
||||
if _, err := privateResourceURL(resource.url, host); err != nil {
|
||||
return errAdapterPlan
|
||||
}
|
||||
if !youtube.SignedSampleFresh(youtube.Resource{URL: resource.url, Host: host, Kind: resource.kind}, time.Now(), 120*time.Second) {
|
||||
return errAdapterPlan
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Typed measurements stay private and ephemeral. Only core callbacks collect
|
||||
// and classify them; modules cannot supply callbacks, attestations or seals.
|
||||
type adapterMeasurement[T any] struct {
|
||||
value T
|
||||
bytes int64
|
||||
}
|
||||
|
||||
type adapterHostMeasurements[T any] struct {
|
||||
host string
|
||||
warmupAttempts, warmupFailures int
|
||||
warmupLimit int
|
||||
warmupMS float64
|
||||
samples []T
|
||||
}
|
||||
|
||||
// executeAdapterPlan owns scheduling only. Its caller owns the worker, transport
|
||||
// validation and post-join proof validation/sealing. No goroutines or persistence.
|
||||
func executeAdapterPlan[T any](ctx context.Context, binding adapter.Binding, plan adapter.Plan, resources map[string]mediaResource,
|
||||
probe func(context.Context, mediaResource, string) (adapterMeasurement[T], error), nonpass func(error) bool,
|
||||
) ([]adapterHostMeasurements[T], error) {
|
||||
if err := validateAdapterPlan(binding, plan, resources); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
hosts := make([]adapterHostMeasurements[T], len(plan.Hosts))
|
||||
for i, host := range plan.Hosts {
|
||||
hosts[i].host = host
|
||||
}
|
||||
for i := range hosts {
|
||||
h := &hosts[i]
|
||||
h.warmupLimit = plan.WarmupLimit
|
||||
start := time.Now()
|
||||
warmCtx, cancel := context.WithTimeout(ctx, plan.WarmupTimeout)
|
||||
learned := map[string]bool{}
|
||||
for h.warmupAttempts < h.warmupLimit && len(learned) < len(plan.Protocols) && warmCtx.Err() == nil {
|
||||
protocol := plan.Protocols[h.warmupAttempts%len(plan.Protocols)]
|
||||
h.warmupAttempts++
|
||||
measurement, err := probe(warmCtx, resources[h.host], protocol)
|
||||
err = managedRecoveryContextError(warmCtx, err)
|
||||
if err != nil {
|
||||
h.warmupFailures++
|
||||
if !nonpass(err) && (warmCtx.Err() == nil || !managedRecoveryTimeoutSafe(err)) {
|
||||
cancel()
|
||||
return hosts, err
|
||||
}
|
||||
} else if measurement.bytes < plan.MinBytes {
|
||||
h.warmupFailures++
|
||||
} else {
|
||||
learned[protocol] = true
|
||||
}
|
||||
}
|
||||
h.warmupMS = float64(time.Since(start).Microseconds()) / 1000
|
||||
cancel()
|
||||
if err := ctx.Err(); err != nil {
|
||||
return hosts, err
|
||||
}
|
||||
if len(learned) != len(plan.Protocols) || h.warmupMS > float64(plan.WarmupTimeout/time.Millisecond) {
|
||||
return hosts, errCandidateNonpass
|
||||
}
|
||||
for _, protocol := range plan.Protocols {
|
||||
for n := 0; n < plan.Samples; n++ {
|
||||
if err := ctx.Err(); err != nil {
|
||||
return hosts, err
|
||||
}
|
||||
measurement, err := probe(ctx, resources[h.host], protocol)
|
||||
err = managedRecoveryContextError(ctx, err)
|
||||
if ctx.Err() != nil {
|
||||
return hosts, err
|
||||
}
|
||||
if err != nil {
|
||||
if nonpass(err) {
|
||||
return hosts, errCandidateNonpass
|
||||
}
|
||||
return hosts, err
|
||||
}
|
||||
if measurement.bytes < plan.MinBytes {
|
||||
return hosts, errCandidateNonpass
|
||||
}
|
||||
h.samples = append(h.samples, measurement.value)
|
||||
}
|
||||
}
|
||||
}
|
||||
return hosts, nil
|
||||
}
|
||||
@@ -0,0 +1,148 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Also runnable unchanged against archived f485317. Only wall-clock duration,
|
||||
// attestation time and process-private seals are excluded from the golden hash.
|
||||
func TestAdapterExecutorLegacyParity(t *testing.T) {
|
||||
r, c, resources, events := googlevideoRuntimeFixture(t)
|
||||
runner := r.curl
|
||||
var order []string
|
||||
r.curl = func(ctx context.Context, args []string, stdin io.Reader, out, diagnostic io.Writer) error {
|
||||
protocol := "tls12"
|
||||
if strings.Contains(strings.Join(args, " "), "--tlsv1.3") {
|
||||
protocol = "tls13"
|
||||
}
|
||||
order = append(order, protocol)
|
||||
if len(order) <= 2 {
|
||||
return context.DeadlineExceeded
|
||||
}
|
||||
return runner(ctx, args, stdin, out, diagnostic)
|
||||
}
|
||||
p, err := r.verifyGooglevideo(context.Background(), c, resources)
|
||||
if err != nil || p.producerSeal == nil {
|
||||
t.Fatal(err, p)
|
||||
}
|
||||
if !strings.HasSuffix(strings.Join(*events, ","), "delete,join") {
|
||||
t.Fatal(*events)
|
||||
}
|
||||
expected := []string{"tls12", "tls13"}
|
||||
for range c.Endpoints {
|
||||
expected = append(expected, "tls12", "tls13", "tls12", "tls12", "tls12", "tls13", "tls13", "tls13")
|
||||
}
|
||||
if !reflect.DeepEqual(order, expected) {
|
||||
t.Fatal(order, expected)
|
||||
}
|
||||
for i := range p.Hosts {
|
||||
p.Hosts[i].WarmupMS = 0
|
||||
}
|
||||
p.Attestation.CheckedAt = time.Unix(1700000000, 0).UTC()
|
||||
raw, err := json.Marshal(p)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
hash := fmt.Sprintf("%x", sha256.Sum256(raw))
|
||||
t.Log("normalized proof sha256:", hash)
|
||||
const baseline = "fb380324e320a78f1ceb011c59b82b9de4ff4f31f5f032f469a181292987e3b2"
|
||||
if hash != baseline {
|
||||
t.Fatalf("proof parity changed: %s\n%s", hash, raw)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAdapterExecutorProductionCancellationJoinsWithoutSeal(t *testing.T) {
|
||||
for _, at := range []int{1, 3} {
|
||||
t.Run(fmt.Sprint(at), func(t *testing.T) {
|
||||
r, c, resources, events := googlevideoRuntimeFixture(t)
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
runner, calls := r.curl, 0
|
||||
r.curl = func(probeCtx context.Context, args []string, stdin io.Reader, out, diagnostic io.Writer) error {
|
||||
calls++
|
||||
if calls == at {
|
||||
cancel()
|
||||
return ctx.Err()
|
||||
}
|
||||
return runner(probeCtx, args, stdin, out, diagnostic)
|
||||
}
|
||||
p, err := r.verifyGooglevideo(ctx, c, resources)
|
||||
if !errors.Is(err, context.Canceled) || calls != at || p.Attestation != nil || p.producerSeal != nil ||
|
||||
!strings.HasSuffix(strings.Join(*events, ","), "delete,join") {
|
||||
t.Fatal(p, err, calls, *events)
|
||||
}
|
||||
if !managedRecoveryTimeoutSafe(err) {
|
||||
t.Fatal("plain cancellation became unsafe", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestAdapterExecutorProductionCancellationPreservesUnsafeTransfer(t *testing.T) {
|
||||
unsafeIO := errors.New("unsafe collector IO")
|
||||
for _, at := range []int{1, 3} { // first warmup or first steady after both TLS versions learn
|
||||
for _, deadline := range []bool{false, true} {
|
||||
for _, overflow := range []bool{false, true} {
|
||||
t.Run(fmt.Sprintf("probe%d/deadline%t/overflow%t", at, deadline, overflow), func(t *testing.T) {
|
||||
r, c, resources, events := googlevideoRuntimeFixture(t)
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
wantContext := context.Canceled
|
||||
if deadline {
|
||||
var stop context.CancelFunc
|
||||
ctx, stop = context.WithTimeout(ctx, time.Second)
|
||||
defer stop()
|
||||
wantContext = context.DeadlineExceeded
|
||||
}
|
||||
runner, calls := r.curl, 0
|
||||
r.curl = func(probeCtx context.Context, args []string, stdin io.Reader, out, diagnostic io.Writer) error {
|
||||
calls++
|
||||
if calls != at {
|
||||
return runner(probeCtx, args, stdin, out, diagnostic)
|
||||
}
|
||||
if deadline {
|
||||
<-ctx.Done()
|
||||
} else {
|
||||
cancel()
|
||||
}
|
||||
if overflow {
|
||||
// Exercise the real bounded collector writer, not a forged fatal DTO.
|
||||
_, writeErr := out.Write(make([]byte, 256<<10))
|
||||
return errors.Join(writeErr, ctx.Err())
|
||||
}
|
||||
return errors.Join(unsafeIO, ctx.Err())
|
||||
}
|
||||
p, err := r.verifyGooglevideo(ctx, c, resources)
|
||||
var fatal *managedRecoveryFatal
|
||||
if !errors.Is(err, wantContext) || !errors.As(err, &fatal) || managedRecoveryTimeoutSafe(err) || managedResourceTransferNonpass(err, false, false) {
|
||||
t.Fatalf("lost mixed safety/context error: %v", err)
|
||||
}
|
||||
if overflow {
|
||||
var collected *resourceError
|
||||
if !errors.As(err, &collected) || collected.reason != reasonOutputBound {
|
||||
t.Fatal("lost collector overflow", err)
|
||||
}
|
||||
} else if !errors.Is(err, unsafeIO) {
|
||||
t.Fatal("lost unsafe IO cause", err)
|
||||
}
|
||||
if calls != at || p.Attestation != nil || p.producerSeal != nil ||
|
||||
!strings.HasSuffix(strings.Join(*events, ","), "delete,join") {
|
||||
t.Fatal(p, err, calls, *events)
|
||||
}
|
||||
if at == 3 && (p.Hosts[0].WarmupAttempts != 2 || len(p.Hosts[0].Passes) != 0) {
|
||||
t.Fatal("learning leaked into steady proof", p.Hosts)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,248 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"math"
|
||||
"reflect"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"zapret-autotune/internal/adapter"
|
||||
"zapret-autotune/internal/adapter/youtube"
|
||||
)
|
||||
|
||||
func adapterExecutorFixture(t *testing.T) (adapter.Binding, adapter.Plan, map[string]mediaResource) {
|
||||
t.Helper()
|
||||
hosts := []string{"rr1---sn-alpha.googlevideo.com", "rr2---sn-beta.googlevideo.com", "rr5---sn-gamma.googlevideo.com"}
|
||||
resources := map[string]mediaResource{}
|
||||
var samples []youtube.Resource
|
||||
for _, host := range hosts {
|
||||
url := "https://" + host + "/videoplayback?expire=4102444800&sig=private"
|
||||
resources[host] = mediaResource{host: host, url: url, kind: "video/mp4"}
|
||||
samples = append(samples, youtube.Resource{Host: host, URL: url, Kind: "video/mp4"})
|
||||
}
|
||||
m := youtube.Module{}
|
||||
p, err := m.PlanQualification(hosts, samples, time.Now())
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return m.Binding(), p, resources
|
||||
}
|
||||
|
||||
func TestAdapterExecutorRejectsUntrustedPlanBeforeProbe(t *testing.T) {
|
||||
mutations := map[string]func(*adapter.Binding, *adapter.Plan, map[string]mediaResource){
|
||||
"binding-kind": func(b *adapter.Binding, _ *adapter.Plan, _ map[string]mediaResource) { b.Kind = "other" },
|
||||
"binding-version": func(b *adapter.Binding, _ *adapter.Plan, _ map[string]mediaResource) { b.Version++ },
|
||||
"binding-policy": func(b *adapter.Binding, _ *adapter.Plan, _ map[string]mediaResource) { b.PolicySHA256 = "" },
|
||||
"missing-host": func(_ *adapter.Binding, p *adapter.Plan, _ map[string]mediaResource) { p.Hosts = p.Hosts[:2] },
|
||||
"extra-host": func(_ *adapter.Binding, p *adapter.Plan, _ map[string]mediaResource) {
|
||||
p.Hosts = append(p.Hosts, "rr9---sn-delta.googlevideo.com")
|
||||
},
|
||||
"duplicate-host": func(_ *adapter.Binding, p *adapter.Plan, _ map[string]mediaResource) { p.Hosts[1] = p.Hosts[0] },
|
||||
"host-url": func(_ *adapter.Binding, p *adapter.Plan, _ map[string]mediaResource) {
|
||||
p.Hosts[0] = "https://example.com"
|
||||
},
|
||||
"host-case": func(_ *adapter.Binding, p *adapter.Plan, _ map[string]mediaResource) {
|
||||
p.Hosts[0] = "RR1---sn-alpha.googlevideo.com"
|
||||
},
|
||||
"host-suffix": func(_ *adapter.Binding, p *adapter.Plan, _ map[string]mediaResource) {
|
||||
p.Hosts[0] = "rr1---sn-alpha.example.com"
|
||||
},
|
||||
"missing-resource": func(_ *adapter.Binding, p *adapter.Plan, r map[string]mediaResource) { delete(r, p.Hosts[0]) },
|
||||
"resource-host": func(_ *adapter.Binding, p *adapter.Plan, r map[string]mediaResource) {
|
||||
v := r[p.Hosts[0]]
|
||||
v.host = p.Hosts[1]
|
||||
r[p.Hosts[0]] = v
|
||||
},
|
||||
"resource-url": func(_ *adapter.Binding, p *adapter.Plan, r map[string]mediaResource) {
|
||||
v := r[p.Hosts[0]]
|
||||
v.url = "https://evil.example/"
|
||||
r[p.Hosts[0]] = v
|
||||
},
|
||||
"missing-protocol": func(_ *adapter.Binding, p *adapter.Plan, _ map[string]mediaResource) { p.Protocols[1] = "" },
|
||||
"duplicate-protocol": func(_ *adapter.Binding, p *adapter.Plan, _ map[string]mediaResource) { p.Protocols[1] = p.Protocols[0] },
|
||||
"unknown-protocol": func(_ *adapter.Binding, p *adapter.Plan, _ map[string]mediaResource) { p.Protocols[1] = "http" },
|
||||
"protocol-order": func(_ *adapter.Binding, p *adapter.Plan, _ map[string]mediaResource) {
|
||||
p.Protocols = [2]string{"tls13", "tls12"}
|
||||
},
|
||||
}
|
||||
for _, n := range []int{math.MinInt, -1, 0, 1, 4, math.MaxInt} {
|
||||
value := n
|
||||
mutations["samples-"+time.Duration(n).String()] = func(_ *adapter.Binding, p *adapter.Plan, _ map[string]mediaResource) { p.Samples = value }
|
||||
}
|
||||
for _, n := range []int{-1, 0, 1, 11, math.MaxInt} {
|
||||
value := n
|
||||
mutations["warmup-"+time.Duration(n).String()] = func(_ *adapter.Binding, p *adapter.Plan, _ map[string]mediaResource) { p.WarmupLimit = value }
|
||||
}
|
||||
for _, n := range []int64{math.MinInt64, -1, 0, 1, 1025, math.MaxInt64} {
|
||||
value := n
|
||||
mutations["bytes-"+time.Duration(n).String()] = func(_ *adapter.Binding, p *adapter.Plan, _ map[string]mediaResource) { p.MinBytes = value }
|
||||
}
|
||||
for _, n := range []time.Duration{-1, 0, time.Second, 121 * time.Second, time.Duration(math.MaxInt64)} {
|
||||
value := n
|
||||
mutations["timeout-"+n.String()] = func(_ *adapter.Binding, p *adapter.Plan, _ map[string]mediaResource) { p.WarmupTimeout = value }
|
||||
}
|
||||
for name, mutate := range mutations {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
b, p, resources := adapterExecutorFixture(t)
|
||||
mutate(&b, &p, resources)
|
||||
_, err := executeAdapterPlan(context.Background(), b, p, resources,
|
||||
func(context.Context, mediaResource, string) (adapterMeasurement[int], error) {
|
||||
t.Fatal("invalid plan probed")
|
||||
return adapterMeasurement[int]{}, nil
|
||||
},
|
||||
func(error) bool { t.Fatal("invalid plan classified"); return false })
|
||||
if !errors.Is(err, errAdapterPlan) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestAdapterExecutorOrderedWarmupAndTypedSamples(t *testing.T) {
|
||||
b, p, resources := adapterExecutorFixture(t)
|
||||
var order []string
|
||||
results, err := executeAdapterPlan(context.Background(), b, p, resources,
|
||||
func(ctx context.Context, r mediaResource, protocol string) (adapterMeasurement[int], error) {
|
||||
if _, ok := ctx.Deadline(); len(order)%8 < 2 && !ok {
|
||||
t.Fatal("warmup lacks deadline")
|
||||
}
|
||||
order = append(order, r.host+"/"+protocol)
|
||||
return adapterMeasurement[int]{value: len(order), bytes: 1024}, nil
|
||||
}, func(error) bool { return false })
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
var expected []string
|
||||
for i, host := range p.Hosts {
|
||||
for _, protocol := range []string{"tls12", "tls13", "tls12", "tls12", "tls12", "tls13", "tls13", "tls13"} {
|
||||
expected = append(expected, host+"/"+protocol)
|
||||
}
|
||||
h := results[i]
|
||||
base := i * 8
|
||||
if h.host != host || h.warmupAttempts != 2 || h.warmupFailures != 0 || h.warmupLimit != 10 || !reflect.DeepEqual(h.samples, []int{base + 3, base + 4, base + 5, base + 6, base + 7, base + 8}) {
|
||||
t.Fatal(h)
|
||||
}
|
||||
}
|
||||
if !reflect.DeepEqual(order, expected) {
|
||||
t.Fatal(order, expected)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAdapterExecutorFailureAndCancellation(t *testing.T) {
|
||||
fatal := errors.New("infrastructure failure")
|
||||
negative := errors.New("known negative")
|
||||
for _, mode := range []string{"warm-negative", "warm-small", "warm-fatal", "warm-mixed", "steady-negative", "steady-small", "steady-fatal", "cancel-before", "cancel-warm", "deadline-warm", "cancel-steady"} {
|
||||
t.Run(mode, func(t *testing.T) {
|
||||
b, p, resources := adapterExecutorFixture(t)
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
want := errCandidateNonpass
|
||||
wantCalls := 10
|
||||
if mode == "cancel-before" {
|
||||
cancel()
|
||||
want = context.Canceled
|
||||
wantCalls = 0
|
||||
}
|
||||
if mode == "warm-fatal" {
|
||||
want = fatal
|
||||
wantCalls = 1
|
||||
}
|
||||
if mode == "warm-mixed" {
|
||||
want = fatal
|
||||
wantCalls = 1
|
||||
}
|
||||
if mode == "steady-negative" || mode == "steady-small" {
|
||||
wantCalls = 3
|
||||
}
|
||||
if mode == "steady-fatal" {
|
||||
want = fatal
|
||||
wantCalls = 3
|
||||
}
|
||||
if mode == "cancel-warm" {
|
||||
want = context.Canceled
|
||||
wantCalls = 1
|
||||
}
|
||||
if mode == "cancel-steady" {
|
||||
want = context.Canceled
|
||||
wantCalls = 3
|
||||
}
|
||||
if mode == "deadline-warm" {
|
||||
var stop context.CancelFunc
|
||||
ctx, stop = context.WithTimeout(ctx, time.Second)
|
||||
defer stop()
|
||||
want = context.DeadlineExceeded
|
||||
wantCalls = 1
|
||||
}
|
||||
calls := 0
|
||||
hosts, err := executeAdapterPlan(ctx, b, p, resources,
|
||||
func(probeCtx context.Context, _ mediaResource, _ string) (adapterMeasurement[int], error) {
|
||||
calls++
|
||||
m := adapterMeasurement[int]{value: calls, bytes: 1024}
|
||||
switch mode {
|
||||
case "warm-negative":
|
||||
return m, negative
|
||||
case "warm-small":
|
||||
m.bytes = 1023
|
||||
case "warm-fatal":
|
||||
return m, fatal
|
||||
case "warm-mixed":
|
||||
return m, errors.Join(negative, fatal)
|
||||
case "steady-negative":
|
||||
if calls == 3 {
|
||||
return m, negative
|
||||
}
|
||||
case "steady-small":
|
||||
if calls == 3 {
|
||||
m.bytes = 0
|
||||
}
|
||||
case "steady-fatal":
|
||||
if calls == 3 {
|
||||
return m, fatal
|
||||
}
|
||||
case "cancel-warm":
|
||||
cancel()
|
||||
return m, ctx.Err()
|
||||
case "deadline-warm":
|
||||
<-probeCtx.Done()
|
||||
return m, probeCtx.Err()
|
||||
case "cancel-steady":
|
||||
if calls == 3 {
|
||||
cancel()
|
||||
return m, ctx.Err()
|
||||
}
|
||||
}
|
||||
return m, nil
|
||||
}, func(err error) bool { return err == negative })
|
||||
if !errors.Is(err, want) || calls != wantCalls {
|
||||
t.Fatal(err, calls, want, wantCalls)
|
||||
}
|
||||
for _, host := range hosts {
|
||||
if len(host.samples) != 0 {
|
||||
t.Fatal("phantom steady sample", host)
|
||||
}
|
||||
}
|
||||
if calls == 10 && (hosts[0].warmupAttempts != 10 || hosts[0].warmupFailures != 10) {
|
||||
t.Fatal(hosts)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
type weakenedExecutorAdapter struct{ youtube.Module }
|
||||
|
||||
func (m weakenedExecutorAdapter) PlanQualification(hosts []string, resources []adapter.Resource, now time.Time) (adapter.Plan, error) {
|
||||
p, err := m.Module.PlanQualification(hosts, resources, now)
|
||||
p.Samples = 1
|
||||
return p, err
|
||||
}
|
||||
|
||||
func TestAdapterExecutorProductionPreflightRejectsWeakenedModule(t *testing.T) {
|
||||
replaceServiceAdapters(t, serviceAdapterRegistry{{youtube.Kind, youtube.Version}: weakenedExecutorAdapter{}})
|
||||
r, c, resources, events := googlevideoRuntimeFixture(t)
|
||||
p, err := r.verifyGooglevideo(context.Background(), c, resources)
|
||||
if !errors.Is(err, errAdapterPlan) || len(*events) != 0 || p.Attestation != nil || p.producerSeal != nil {
|
||||
t.Fatal("module weakened production execution policy", p, err, *events)
|
||||
}
|
||||
}
|
||||
@@ -271,7 +271,9 @@ func TestAssetCLIApplyPartialFailureAndStatus(t *testing.T) {
|
||||
t.Fatalf("requested status: %s %s", out, diagnostic)
|
||||
}
|
||||
if fail {
|
||||
d.player = func(context.Context, []string, io.Writer, io.Writer) error { return errors.New("unit health failure") }
|
||||
d.player = func(context.Context, []string, io.Reader, io.Writer, io.Writer) error {
|
||||
return errors.New("unit health failure")
|
||||
}
|
||||
}
|
||||
code, _, diagnostic := callManaged(t, d, "apply", "--asset-candidate", fmt.Sprintf("%d,%d", a, b))
|
||||
if (code != 0) != fail {
|
||||
|
||||
@@ -9,6 +9,8 @@ import (
|
||||
"math"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"zapret-autotune/internal/adapter/youtube"
|
||||
)
|
||||
|
||||
// Additive only: old proposals are not independently qualified candidates.
|
||||
@@ -351,6 +353,7 @@ type GooglevideoHostProof struct {
|
||||
}
|
||||
|
||||
type GooglevideoQualificationProof struct {
|
||||
Adapter *youtube.Binding `json:"adapter,omitempty"`
|
||||
Version int `json:"version"`
|
||||
Mode QualificationMode `json:"mode"`
|
||||
SourceKind string `json:"source_kind"`
|
||||
@@ -371,7 +374,25 @@ func (p GooglevideoQualificationProof) validate(c storedCandidate) error {
|
||||
}
|
||||
|
||||
func (p GooglevideoQualificationProof) validateStored(c storedCandidate) error {
|
||||
if p.Version != 1 || p.Mode != c.Mode || p.Args != c.Args || len(p.Hosts) != len(c.Endpoints) || len(p.Hosts) < 3 {
|
||||
// Missing binding is explicitly the immutable legacy YouTube v1 policy,
|
||||
// never the current registry default. Retain historical reconstruction.
|
||||
if p.Version == 1 {
|
||||
if p.Adapter != nil {
|
||||
return fmt.Errorf("legacy proof cannot carry adapter binding")
|
||||
}
|
||||
} else {
|
||||
if p.Mode != QualificationAdaptive || p.Adapter == nil {
|
||||
return fmt.Errorf("version-bound adaptive proof required")
|
||||
}
|
||||
module, err := lookupServiceAdapter(p.Adapter.Kind, p.Adapter.Version)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if *p.Adapter != module.Binding() {
|
||||
return fmt.Errorf("adapter policy mismatch")
|
||||
}
|
||||
}
|
||||
if (p.Version != 1 && p.Version != 2) || p.Mode != c.Mode || p.Args != c.Args || len(p.Hosts) != len(c.Endpoints) || len(p.Hosts) < 3 {
|
||||
return fmt.Errorf("invalid representative proof definition")
|
||||
}
|
||||
if (p.Mode == QualificationCold && p.SourceKind != "media-range") || (p.Mode == QualificationAdaptive && p.SourceKind != NativeCircularPostWarmup) || (p.Mode != QualificationCold && p.Mode != QualificationAdaptive) {
|
||||
@@ -380,7 +401,7 @@ func (p GooglevideoQualificationProof) validateStored(c storedCandidate) error {
|
||||
if err := (&candidate{Args: p.Args}).validateArgs(); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := validateGooglevideoHosts(c.Endpoints); err != nil {
|
||||
if err := validateLegacyYoutubeV1Hosts(c.Endpoints); err != nil {
|
||||
return err
|
||||
}
|
||||
if p.Mode == QualificationCold && p.Attestation != nil {
|
||||
@@ -494,7 +515,9 @@ func validateAdaptiveGooglevideoEvidence(db *sql.DB, c storedCandidate, fresh bo
|
||||
// One transactional insert, no cold candidate_evidence summaries. A private
|
||||
// producer seal is required as well as valid DTO facts and stored definition.
|
||||
func saveAdaptiveGooglevideoEvidence(tx *sql.Tx, id int64, proof GooglevideoQualificationProof) error {
|
||||
if proof.producerSeal == nil || *proof.producerSeal != googlevideoProofSeal(proof) {
|
||||
// Existing trusted v1 rows retain their original read/apply contract. Only
|
||||
// this new-write boundary requires the version-bound producer protocol.
|
||||
if proof.Version != 2 || proof.producerSeal == nil || *proof.producerSeal != googlevideoProofSeal(proof) {
|
||||
return fmt.Errorf("completed owned collector attestation required")
|
||||
}
|
||||
var c storedCandidate
|
||||
|
||||
@@ -24,7 +24,7 @@ const playerResponseLimit = 1 << 20
|
||||
var videoIDPattern = regexp.MustCompile(`^[A-Za-z0-9_-]{11}$`)
|
||||
var discoveryDNSLabel = regexp.MustCompile(`^[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?$`)
|
||||
|
||||
type playerRunner func(context.Context, []string, io.Writer, io.Writer) error
|
||||
type playerRunner func(context.Context, []string, io.Reader, io.Writer, io.Writer) error
|
||||
|
||||
// Keep request diagnostics private while retaining typed causes for supervision.
|
||||
type playerRequestError struct{ cause error }
|
||||
@@ -32,8 +32,9 @@ type playerRequestError struct{ cause error }
|
||||
func (e *playerRequestError) Error() string { return "player request failed or exceeded output limit" }
|
||||
func (e *playerRequestError) Unwrap() error { return e.cause }
|
||||
|
||||
func runPlayerCurl(ctx context.Context, args []string, stdout, stderr io.Writer) error {
|
||||
func runPlayerCurl(ctx context.Context, args []string, stdin io.Reader, stdout, stderr io.Writer) error {
|
||||
cmd := exec.CommandContext(ctx, "curl", args...)
|
||||
cmd.Stdin = stdin
|
||||
cmd.Stdout, cmd.Stderr = stdout, stderr
|
||||
return cmd.Run()
|
||||
}
|
||||
@@ -148,6 +149,12 @@ func discoverYoutube(args []string, stdout, stderr io.Writer, runner playerRunne
|
||||
}
|
||||
|
||||
func requestPlayer(videoID, proxy string, runner playerRunner) ([]byte, error) {
|
||||
if _, err := requiredYoutubeAdapter(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if !videoIDPattern.MatchString(videoID) {
|
||||
return nil, errors.New("invalid player video ID")
|
||||
}
|
||||
payload, _ := json.Marshal(map[string]interface{}{
|
||||
"videoId": videoID,
|
||||
"context": map[string]interface{}{"client": map[string]interface{}{
|
||||
@@ -161,8 +168,9 @@ func requestPlayer(videoID, proxy string, runner playerRunner) ([]byte, error) {
|
||||
output := &playerBoundedWriter{dst: &body, limit: playerResponseLimit}
|
||||
diagnostics := &playerBoundedWriter{dst: io.Discard, limit: 64 << 10}
|
||||
// Disable curlrc and NO_PROXY bypasses: the explicit proxy must be used.
|
||||
// Keep the bounded JSON body off argv; curl retains its JSON POST headers.
|
||||
err := runner(ctx, []string{"--disable", "--silent", "--show-error", "--fail", "--max-time", "30",
|
||||
"--noproxy", "", "--proxy", proxy, "--user-agent", playerUserAgent, "--json", string(payload), playerURL}, output, diagnostics)
|
||||
"--noproxy", "", "--proxy", proxy, "--user-agent", playerUserAgent, "--json", "@-", playerURL}, bytes.NewReader(payload), output, diagnostics)
|
||||
if err != nil || output.exceeded || diagnostics.exceeded {
|
||||
if output.exceeded || diagnostics.exceeded {
|
||||
// An output failure remains unsafe even if the child was also canceled.
|
||||
|
||||
@@ -20,13 +20,13 @@ func discoveryArgs() []string {
|
||||
func TestDiscoverYoutubeSuccess(t *testing.T) {
|
||||
var stdout, stderr bytes.Buffer
|
||||
called := false
|
||||
runner := func(ctx context.Context, args []string, out, diagnostics io.Writer) error {
|
||||
runner := func(ctx context.Context, args []string, stdin io.Reader, out, diagnostics io.Writer) error {
|
||||
called = true
|
||||
if _, ok := ctx.Deadline(); !ok {
|
||||
t.Error("request must have a deadline")
|
||||
}
|
||||
prefix := []string{"--disable", "--silent", "--show-error", "--fail", "--max-time", "30", "--noproxy", "", "--proxy", "socks5://localhost:1080", "--user-agent", playerUserAgent, "--json"}
|
||||
if len(args) != len(prefix)+2 || !reflect.DeepEqual(args[:len(prefix)], prefix) || args[len(args)-1] != playerURL {
|
||||
if len(args) != len(prefix)+2 || !reflect.DeepEqual(args[:len(prefix)], prefix) || args[len(prefix)] != "@-" || args[len(args)-1] != playerURL {
|
||||
t.Fatalf("unexpected curl arguments: %q", args)
|
||||
}
|
||||
var payload struct {
|
||||
@@ -42,7 +42,7 @@ func TestDiscoverYoutubeSuccess(t *testing.T) {
|
||||
} `json:"client"`
|
||||
} `json:"context"`
|
||||
}
|
||||
if err := json.Unmarshal([]byte(args[len(prefix)]), &payload); err != nil {
|
||||
if err := json.NewDecoder(stdin).Decode(&payload); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
client := payload.Context.Client
|
||||
@@ -79,7 +79,7 @@ func TestDiscoverYoutubeInvalidArgs(t *testing.T) {
|
||||
}
|
||||
for _, args := range tests {
|
||||
var stdout, stderr bytes.Buffer
|
||||
runner := func(context.Context, []string, io.Writer, io.Writer) error {
|
||||
runner := func(context.Context, []string, io.Reader, io.Writer, io.Writer) error {
|
||||
t.Fatal("invalid arguments must not invoke curl")
|
||||
return nil
|
||||
}
|
||||
@@ -114,7 +114,7 @@ func TestPlayerHostnameFailsClosed(t *testing.T) {
|
||||
}
|
||||
for _, fixture := range fixtures {
|
||||
var stdout, stderr bytes.Buffer
|
||||
runner := func(_ context.Context, _ []string, out, diagnostics io.Writer) error {
|
||||
runner := func(_ context.Context, _ []string, _ io.Reader, out, diagnostics io.Writer) error {
|
||||
io.WriteString(diagnostics, "SECRET")
|
||||
_, err := io.WriteString(out, fixture)
|
||||
return err
|
||||
@@ -136,7 +136,7 @@ func TestDiscoverYoutubeRequestFailures(t *testing.T) {
|
||||
for _, mode := range []string{"runner", "stdout limit", "stderr limit", "write result"} {
|
||||
t.Run(mode, func(t *testing.T) {
|
||||
var stdout, stderr bytes.Buffer
|
||||
runner := func(_ context.Context, _ []string, out, diagnostics io.Writer) error {
|
||||
runner := func(_ context.Context, _ []string, _ io.Reader, out, diagnostics io.Writer) error {
|
||||
switch mode {
|
||||
case "runner":
|
||||
io.WriteString(out, discoveryFixture)
|
||||
|
||||
@@ -44,6 +44,10 @@ func googlevideoProposal(flags []string, stdout, stderr io.Writer, discover play
|
||||
fmt.Fprintln(stderr, "error: tune googlevideo", err)
|
||||
return 2
|
||||
}
|
||||
if _, err := requiredYoutubeAdapter(); err != nil {
|
||||
fmt.Fprintln(stderr, "error: tune googlevideo", err)
|
||||
return 1
|
||||
}
|
||||
started := time.Now()
|
||||
db, err := openStore(databasePath())
|
||||
if err != nil {
|
||||
|
||||
@@ -17,16 +17,20 @@ var cacheTestFlags = []string{"--video-id", "dQw4w9WgXcQ", "--proxy", "socks5://
|
||||
|
||||
func cachePlayer(t *testing.T, order *[]string) playerRunner {
|
||||
t.Helper()
|
||||
return func(_ context.Context, args []string, out, diagnostics io.Writer) error {
|
||||
return func(_ context.Context, args []string, stdin io.Reader, out, diagnostics io.Writer) error {
|
||||
*order = append(*order, "discover")
|
||||
invocation := strings.Join(args, " ")
|
||||
for _, want := range []string{"--proxy socks5://localhost:1080", "21.02.35", "dQw4w9WgXcQ", playerURL} {
|
||||
for _, want := range []string{"--proxy socks5://localhost:1080", "21.02.35", "--json @-", playerURL} {
|
||||
if !strings.Contains(invocation, want) {
|
||||
t.Fatalf("discovery missing %q: %s", want, invocation)
|
||||
}
|
||||
}
|
||||
payload, err := io.ReadAll(stdin)
|
||||
if err != nil || !strings.Contains(string(payload), `"videoId":"dQw4w9WgXcQ"`) || strings.Contains(invocation, "dQw4w9WgXcQ") {
|
||||
t.Fatal("player payload must be supplied only on stdin")
|
||||
}
|
||||
fmt.Fprint(diagnostics, "https://secret.invalid/signed?token=SECRET")
|
||||
_, err := fmt.Fprintf(out, `{"playabilityStatus":{"status":"OK"},"streamingData":{"serverAbrStreamingUrl":"https://%s/videoplayback?token=SECRET"}}`, cacheTestHost)
|
||||
_, err = fmt.Fprintf(out, `{"playabilityStatus":{"status":"OK"},"streamingData":{"serverAbrStreamingUrl":"https://%s/videoplayback?token=SECRET"}}`, cacheTestHost)
|
||||
return err
|
||||
}
|
||||
}
|
||||
@@ -209,7 +213,7 @@ func TestTuneGooglevideoFailures(t *testing.T) {
|
||||
var order []string
|
||||
player := cachePlayer(t, &order)
|
||||
if mode == "discovery" {
|
||||
player = func(context.Context, []string, io.Writer, io.Writer) error {
|
||||
player = func(context.Context, []string, io.Reader, io.Writer, io.Writer) error {
|
||||
order = append(order, "discover")
|
||||
return fmt.Errorf("https://signed.invalid/?SECRET")
|
||||
}
|
||||
|
||||
@@ -125,20 +125,20 @@ func TestRunHealthYoutube(t *testing.T) {
|
||||
t.Setenv("NO_PROXY", "*")
|
||||
var stdout, stderr bytes.Buffer
|
||||
calls := 0
|
||||
runner := func(ctx context.Context, args []string, out, diagnostics io.Writer) error {
|
||||
runner := func(ctx context.Context, args []string, stdin io.Reader, out, diagnostics io.Writer) error {
|
||||
calls++
|
||||
deadline, ok := ctx.Deadline()
|
||||
if !ok || time.Until(deadline) <= 0 || time.Until(deadline) > 35*time.Second {
|
||||
t.Error("request must have bounded deadline")
|
||||
}
|
||||
prefix := []string{"--disable", "--silent", "--show-error", "--fail", "--max-time", "30", "--noproxy", "", "--proxy", proxy, "--user-agent", playerUserAgent, "--json"}
|
||||
if len(args) != len(prefix)+2 || !reflect.DeepEqual(args[:len(prefix)], prefix) || args[len(args)-1] != playerURL {
|
||||
if len(args) != len(prefix)+2 || !reflect.DeepEqual(args[:len(prefix)], prefix) || args[len(prefix)] != "@-" || args[len(args)-1] != playerURL {
|
||||
t.Fatalf("unexpected proxy/API arguments: %q", args)
|
||||
}
|
||||
var payload struct {
|
||||
VideoID string `json:"videoId"`
|
||||
}
|
||||
if err := json.Unmarshal([]byte(args[len(prefix)]), &payload); err != nil || payload.VideoID != "abcdefghijk" {
|
||||
if err := json.NewDecoder(stdin).Decode(&payload); err != nil || payload.VideoID != "abcdefghijk" {
|
||||
t.Fatalf("unexpected video payload: %+v %v", payload, err)
|
||||
}
|
||||
io.WriteString(diagnostics, "SECRET")
|
||||
@@ -158,7 +158,7 @@ func TestHealthYoutubeFailures(t *testing.T) {
|
||||
t.Run(mode, func(t *testing.T) {
|
||||
var stdout, stderr bytes.Buffer
|
||||
calls := 0
|
||||
runner := func(_ context.Context, _ []string, out, diagnostics io.Writer) error {
|
||||
runner := func(_ context.Context, _ []string, _ io.Reader, out, diagnostics io.Writer) error {
|
||||
calls++
|
||||
io.WriteString(diagnostics, "SECRET")
|
||||
switch mode {
|
||||
@@ -192,7 +192,7 @@ func TestHealthYoutubeFailures(t *testing.T) {
|
||||
|
||||
func TestHealthDispatchAndConfigFailWithoutRequest(t *testing.T) {
|
||||
setHealthEnvironment(t)
|
||||
runner := func(context.Context, []string, io.Writer, io.Writer) error {
|
||||
runner := func(context.Context, []string, io.Reader, io.Writer, io.Writer) error {
|
||||
t.Fatal("invalid command/config must not request or fall back")
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -9,6 +9,12 @@ import (
|
||||
const usage = `Usage: zapret-autotune <command>
|
||||
|
||||
Commands:
|
||||
strategy-import --strategies FILE
|
||||
Import immutable definitions only into the private target registry
|
||||
plan --config FILE
|
||||
Authenticated domain compatibility proposal (never apply)
|
||||
qualify --config FILE --strategies FILE --isolated
|
||||
Intrusive finite batch; only PID 1 isolated startup on disabled baseline
|
||||
dns-forwarder mode|address|serve
|
||||
Managed DNS configuration or loopback DoH forwarder
|
||||
dns-forwarder readiness-check|readiness-cleanup <expected-dns-pid>
|
||||
@@ -45,6 +51,12 @@ func run(args []string, stdout, stderr io.Writer) int {
|
||||
}
|
||||
|
||||
func runWithPlayer(args []string, stdout, stderr io.Writer, player playerRunner) int {
|
||||
if len(args) > 0 && args[0] == "qualification-completion-check" {
|
||||
return qualificationCompletionCLI(args[1:], managedDependencies(player))
|
||||
}
|
||||
if len(args) > 0 && (args[0] == "plan" || args[0] == "strategy-import" || args[0] == "qualify") {
|
||||
return resourcePlanCLI(args, stdout, stderr, managedDependencies(player))
|
||||
}
|
||||
if len(args) == 3 && args[0] == "check" && args[1] == "--config" {
|
||||
return configuredResourceCheck(args[2], stdout, stderr)
|
||||
}
|
||||
@@ -53,6 +65,14 @@ func runWithPlayer(args []string, stdout, stderr io.Writer, player playerRunner)
|
||||
}
|
||||
if len(args) == 1 && args[0] == "target-config-check" {
|
||||
path, _ := os.LookupEnv("ZAPRET_AUTOTUNE_RESOURCE_CONFIG")
|
||||
policy, _ := os.LookupEnv(resourceStrategyIDsEnv)
|
||||
if _, err := parseResourceStrategyIDs(policy); err != nil || policy != "" && path == "" {
|
||||
fmt.Fprintln(stderr, "configerror: invalid generic strategy selection")
|
||||
return 64
|
||||
}
|
||||
if path == "" {
|
||||
path, _ = os.LookupEnv("ZAPRET_AUTOTUNE_QUALIFY_CONFIG")
|
||||
}
|
||||
if path != "" {
|
||||
port, err := managedTargetSettings(os.LookupEnv)
|
||||
if err != nil || port != 1080 {
|
||||
|
||||
@@ -234,12 +234,12 @@ func (r managedAssetRuntime) withOwnedWorker(ctx context.Context, endpoints []st
|
||||
}
|
||||
occupied, e := r.queue(0)
|
||||
if e != nil || occupied {
|
||||
return &managedRecoveryFatal{Err: errors.Join(e, errors.New("asset queue unavailable"))}
|
||||
return &managedRecoveryFatal{Err: qualificationOperation(errors.Join(e, errors.New("asset queue unavailable")), stageWorkerQueue)}
|
||||
}
|
||||
args := managedProbeWorkerArgs(endpoints, strategyArgs)
|
||||
worker, e := r.start(args)
|
||||
if e != nil {
|
||||
return &managedRecoveryFatal{Err: e}
|
||||
return &managedRecoveryFatal{Err: qualificationOperation(e, stageWorkerStart)}
|
||||
}
|
||||
joined, table := false, false
|
||||
defer func() {
|
||||
@@ -249,28 +249,29 @@ func (r managedAssetRuntime) withOwnedWorker(ctx context.Context, endpoints []st
|
||||
// Remove queues before killing the consumer.
|
||||
if table {
|
||||
_, cleanupErr = r.nft(cleanup, "delete table inet "+managedAssetTable+"\n")
|
||||
cleanupErr = qualificationOperation(cleanupErr, stageNFTDelete)
|
||||
}
|
||||
if !joined {
|
||||
select {
|
||||
case e := <-worker.done:
|
||||
joined = true
|
||||
cleanupErr = errors.Join(cleanupErr, e, errors.New("asset worker exited unexpectedly"))
|
||||
cleanupErr = errors.Join(cleanupErr, qualificationOperation(errors.Join(e, errors.New("asset worker exited unexpectedly")), stageStopJoin))
|
||||
default:
|
||||
}
|
||||
}
|
||||
if !joined {
|
||||
cleanupErr = errors.Join(cleanupErr, worker.stop())
|
||||
cleanupErr = errors.Join(cleanupErr, qualificationOperation(worker.stop(), stageStopJoin))
|
||||
select {
|
||||
case e := <-worker.done:
|
||||
joined = true
|
||||
if e != nil {
|
||||
var exit *exec.ExitError
|
||||
if !errors.As(e, &exit) || !exit.ProcessState.Sys().(syscall.WaitStatus).Signaled() || exit.ProcessState.Sys().(syscall.WaitStatus).Signal() != syscall.SIGKILL {
|
||||
cleanupErr = errors.Join(cleanupErr, e)
|
||||
cleanupErr = errors.Join(cleanupErr, qualificationOperation(e, stageStopJoin))
|
||||
}
|
||||
}
|
||||
case <-cleanup.Done():
|
||||
cleanupErr = errors.Join(cleanupErr, errors.New("asset worker join timeout"))
|
||||
cleanupErr = errors.Join(cleanupErr, qualificationOperation(errors.New("asset worker join timeout"), stageStopJoin))
|
||||
}
|
||||
}
|
||||
if cleanupErr != nil {
|
||||
@@ -285,12 +286,12 @@ func (r managedAssetRuntime) withOwnedWorker(ctx context.Context, endpoints []st
|
||||
select {
|
||||
case e := <-worker.done:
|
||||
joined = true
|
||||
return &managedRecoveryFatal{Err: errors.Join(e, errors.New("asset worker exited before READY"))}
|
||||
return &managedRecoveryFatal{Err: qualificationOperation(errors.Join(e, errors.New("asset worker exited before READY")), stageWorkerExit)}
|
||||
default:
|
||||
}
|
||||
ready, e := r.queue(worker.pid)
|
||||
if e != nil {
|
||||
return &managedRecoveryFatal{Err: e}
|
||||
return &managedRecoveryFatal{Err: qualificationOperation(e, stageWorkerQueue)}
|
||||
}
|
||||
if ready {
|
||||
break
|
||||
@@ -300,7 +301,7 @@ func (r managedAssetRuntime) withOwnedWorker(ctx context.Context, endpoints []st
|
||||
if ctx.Err() != nil {
|
||||
return ctx.Err()
|
||||
}
|
||||
return &managedRecoveryFatal{Err: errors.New("asset queue READY timeout")}
|
||||
return &managedRecoveryFatal{Err: qualificationOperation(errors.New("asset queue READY timeout"), stageWorkerQueue)}
|
||||
case <-tick.C:
|
||||
}
|
||||
}
|
||||
@@ -311,7 +312,7 @@ func (r managedAssetRuntime) withOwnedWorker(ctx context.Context, endpoints []st
|
||||
table = true
|
||||
// Recheck ownership after installing the queue and before the first capture.
|
||||
if ready, e := r.queue(worker.pid); e != nil || !ready {
|
||||
return &managedRecoveryFatal{Err: errors.Join(e, errors.New("owned queue lost before capture"))}
|
||||
return &managedRecoveryFatal{Err: qualificationOperation(errors.Join(e, errors.New("owned queue lost before capture")), stageWorkerQueue)}
|
||||
}
|
||||
probeCtx, stopProbe := context.WithCancel(ctx)
|
||||
watchDone := make(chan struct{})
|
||||
@@ -320,7 +321,7 @@ func (r managedAssetRuntime) withOwnedWorker(ctx context.Context, endpoints []st
|
||||
defer close(watchDone)
|
||||
select {
|
||||
case e := <-worker.done:
|
||||
exited <- errors.Join(e, errors.New("asset worker exited during probe"))
|
||||
exited <- qualificationOperation(errors.Join(e, errors.New("asset worker exited during probe")), stageWorkerExit)
|
||||
stopProbe()
|
||||
case <-probeCtx.Done():
|
||||
}
|
||||
|
||||
@@ -71,6 +71,9 @@ type managedServiceInventory struct {
|
||||
func managedServiceResources(cfg healthConfig, player func(context.Context) ([]byte, error), page playerRunner) func(context.Context) (managedServiceInventory, error) {
|
||||
return func(ctx context.Context) (managedServiceInventory, error) {
|
||||
i := managedServiceInventory{images: make(map[string]string), proxy: cfg.SOCKSProxy, runner: runResourceCurl}
|
||||
if _, err := requiredYoutubeAdapter(); err != nil {
|
||||
return i, err
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return i, managedRecoveryContextError(ctx, err)
|
||||
}
|
||||
@@ -105,6 +108,9 @@ func managedServiceResources(cfg healthConfig, player func(context.Context) ([]b
|
||||
}
|
||||
i.media, _, err = playerMediaResource(body)
|
||||
if err != nil {
|
||||
if errors.Is(err, errServiceAdapterUnavailable) {
|
||||
return i, err
|
||||
}
|
||||
i.resourceErr = &managedResourceFailure{"media", "unknown", err}
|
||||
}
|
||||
for attempt := 0; attempt < 2; attempt++ {
|
||||
@@ -122,7 +128,7 @@ func managedServiceResources(cfg healthConfig, player func(context.Context) ([]b
|
||||
var output bytes.Buffer
|
||||
out := &playerBoundedWriter{dst: &output, limit: 5 << 20}
|
||||
// Fixed trusted WWW origin; no redirects, credentials or diagnostics.
|
||||
err = page(bounded, []string{"--disable", "--silent", "--show-error", "--fail", "--noproxy", "", "--proxy", proxy, "--proto", "=https", "--max-time", "30", "--max-filesize", "5242880", origin}, out, io.Discard)
|
||||
err = page(bounded, []string{"--disable", "--silent", "--show-error", "--fail", "--noproxy", "", "--proxy", proxy, "--proto", "=https", "--max-time", "30", "--max-filesize", "5242880", origin}, nil, out, io.Discard)
|
||||
requestErr := bounded.Err()
|
||||
cancel()
|
||||
if ctx.Err() != nil {
|
||||
@@ -152,6 +158,9 @@ func managedServiceResources(cfg healthConfig, player func(context.Context) ([]b
|
||||
|
||||
func managedServiceHealth(proxy string, front func(context.Context) (string, error), inventory func(context.Context) (managedServiceInventory, error), runner subprocessRunner) func(context.Context) (string, error) {
|
||||
return func(ctx context.Context) (string, error) {
|
||||
if _, err := requiredYoutubeAdapter(); err != nil {
|
||||
return "DEGRADED", err
|
||||
}
|
||||
status, err := front(ctx)
|
||||
if err != nil || status != "OK" {
|
||||
return "DEGRADED", &managedResourceFailure{"front", managedFrontendHost, err}
|
||||
|
||||
@@ -66,7 +66,7 @@ func TestManagedAvatarInventoryFallback(t *testing.T) {
|
||||
calls := 0
|
||||
resources := managedServiceResources(cfg, func(context.Context) ([]byte, error) {
|
||||
return []byte(`{"playabilityStatus":{"status":"OK"},"videoDetails":{"thumbnail":{"thumbnails":[{"url":"https://i.ytimg.com/fresh"}]}},"streamingData":{"formats":[{"url":"https://fresh.googlevideo.com/private","mimeType":"video/mp4"}]}}`), nil
|
||||
}, func(ctx context.Context, args []string, out, diagnostic io.Writer) error {
|
||||
}, func(ctx context.Context, args []string, _ io.Reader, out, diagnostic io.Writer) error {
|
||||
calls++
|
||||
if _, ok := ctx.Deadline(); !ok || diagnostic != io.Discard {
|
||||
t.Fatal("unbounded metadata or diagnostics")
|
||||
@@ -148,7 +148,7 @@ func TestManagedAvatarInventoryCancellationNoFallback(t *testing.T) {
|
||||
cancel()
|
||||
}
|
||||
return []byte(`{"playabilityStatus":{"status":"OK"}}`), nil
|
||||
}, func(ctx context.Context, args []string, out, _ io.Writer) error {
|
||||
}, func(ctx context.Context, args []string, _ io.Reader, out, _ io.Writer) error {
|
||||
pages++
|
||||
assertManagedDNSArgs(t, args, "socks5h://127.0.0.1:11080")
|
||||
if args[len(args)-1] != "https://www.youtube.com/" {
|
||||
|
||||
@@ -271,6 +271,21 @@ func managedReport(m *managedTransaction, out io.Writer, openDB func() (*sql.DB,
|
||||
if managedHash(b) != j.Hash && (j.State != "prepared" || managedHash(b) != j.Next) {
|
||||
return errManagedDrift
|
||||
}
|
||||
if j.Version == 4 {
|
||||
if activeOnly {
|
||||
return fmt.Errorf("v4 deployment has no legacy active IDs")
|
||||
}
|
||||
if err := m.validateConfigRef(nil, j.Ref, j.Candidates, j.Hash, j.Scope); err != nil {
|
||||
return err
|
||||
}
|
||||
if j.State == "prepared" {
|
||||
if err := m.validateConfigRef(nil, j.PendingRef, j.PendingCandidates, j.Next, j.PendingScope); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
_, err := fmt.Fprintf(out, "managed-test generation=%d hash=%s state=%s kind=%s\nLive resource health: NOT CHECKED; configured resources only, not production READY.\n", j.Generation, j.Hash, j.State, j.Ref.Kind)
|
||||
return err
|
||||
}
|
||||
groups := map[string]bool{}
|
||||
operatorCoverage := map[string]bool{}
|
||||
assetHosts := map[string]bool{}
|
||||
@@ -420,6 +435,9 @@ func managedTargetSettings(lookup func(string) (string, bool)) (int, error) {
|
||||
}
|
||||
|
||||
func managedCLI(args []string, stdout, stderr io.Writer, d managedDeps) int {
|
||||
if len(args) == 3 && args[0] == "resource-apply" && args[1] == "--resource-config" {
|
||||
return resourceApplyCLI(args[2], stdout, stderr, d)
|
||||
}
|
||||
if len(args) == 1 && args[0] == "resource-status" {
|
||||
return resourceStatusCLI(stdout, d)
|
||||
}
|
||||
@@ -449,7 +467,22 @@ func managedCLI(args []string, stdout, stderr io.Writer, d managedDeps) int {
|
||||
if command == "reconcile" {
|
||||
err = managedPrestartReconcile(m, d)
|
||||
} else if command == "status" || command == "active-ids" {
|
||||
err = managedReport(m, stdout, d.openDB, command == "active-ids", len(args) == 2 && args[1] == "--assets")
|
||||
j, loadErr := m.load()
|
||||
if loadErr != nil {
|
||||
err = loadErr
|
||||
} else if j.Version == 4 {
|
||||
config, _ := d.lookup("ZAPRET_AUTOTUNE_RESOURCE_CONFIG")
|
||||
s, e := openResourceApplySnapshot(config, d)
|
||||
if e != nil {
|
||||
err = errResourcePlan
|
||||
} else {
|
||||
m.resource = s
|
||||
err = managedReport(m, stdout, d.openDB, command == "active-ids")
|
||||
s.close()
|
||||
}
|
||||
} else {
|
||||
err = managedReport(m, stdout, d.openDB, command == "active-ids", len(args) == 2 && args[1] == "--assets")
|
||||
}
|
||||
} else if command == "experiment" {
|
||||
startup := false
|
||||
for i := 1; i < len(args); i += 2 {
|
||||
@@ -504,6 +537,27 @@ func managedCLI(args []string, stdout, stderr io.Writer, d managedDeps) int {
|
||||
// Startup may activate only a known baseline or DB-qualified rendered config.
|
||||
// Reconcile restores bytes before its fixed target-local restart hook runs.
|
||||
func managedPrestartReconcile(m *managedTransaction, d managedDeps) (err error) {
|
||||
j, loadErr := m.load()
|
||||
if loadErr != nil {
|
||||
return loadErr
|
||||
}
|
||||
if j.Version == 4 {
|
||||
config, _ := d.lookup("ZAPRET_AUTOTUNE_RESOURCE_CONFIG")
|
||||
s, e := openResourceApplySnapshot(config, d)
|
||||
if e != nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
defer s.close()
|
||||
m.resource = s
|
||||
defer func() { m.resource = nil }()
|
||||
checks := productionResourceCheckDeps()
|
||||
m.RollbackHealth = func() (string, error) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
defer cancel()
|
||||
return resourceApplyHealth(ctx, s, checks)
|
||||
}
|
||||
return m.Reconcile()
|
||||
}
|
||||
path, err := managedMonitorDatabase(d)
|
||||
if err != nil {
|
||||
return err
|
||||
@@ -543,6 +597,9 @@ func managedPrestartReconcile(m *managedTransaction, d managedDeps) (err error)
|
||||
var errManagedScopeUnresolved = errors.New("managed deployment policy unresolved; explicit reconcile or fresh managed state required")
|
||||
|
||||
func managedActivate(m *managedTransaction, d managedDeps, ids []int64, expected map[int64]string, scopes ...GooglevideoScope) error {
|
||||
if _, err := requiredYoutubeAdapter(); err != nil {
|
||||
return err
|
||||
}
|
||||
info, err := os.Stat(d.script)
|
||||
if err != nil || !info.Mode().IsRegular() || info.Mode().Perm()&0111 == 0 {
|
||||
return fmt.Errorf("target zapret daemon init executable unavailable")
|
||||
@@ -552,6 +609,9 @@ func managedActivate(m *managedTransaction, d managedDeps, ids []int64, expected
|
||||
return err
|
||||
}
|
||||
m.Health = func() (string, error) {
|
||||
if _, err := requiredYoutubeAdapter(); err != nil {
|
||||
return "", err
|
||||
}
|
||||
portText, _ := d.lookup("SOCKS_PORT")
|
||||
port, err := strconv.Atoi(portText)
|
||||
if err != nil {
|
||||
|
||||
@@ -46,7 +46,7 @@ func managedCLIFixture(t *testing.T) (managedDeps, map[string]string, *int) {
|
||||
}
|
||||
return nil
|
||||
},
|
||||
player: func(_ context.Context, args []string, stdout, _ io.Writer) error {
|
||||
player: func(_ context.Context, args []string, _ io.Reader, stdout, _ io.Writer) error {
|
||||
if !strings.Contains(strings.Join(args, "|"), "--noproxy||--proxy|socks5h://127.0.0.1:11080|") {
|
||||
t.Fatalf("unsafe health argv: %q", args)
|
||||
}
|
||||
@@ -212,7 +212,7 @@ func TestManagedCLIApplyAndRollback(t *testing.T) {
|
||||
}
|
||||
switch mode {
|
||||
case "health":
|
||||
d.player = func(_ context.Context, _ []string, stdout, _ io.Writer) error {
|
||||
d.player = func(_ context.Context, _ []string, _ io.Reader, stdout, _ io.Writer) error {
|
||||
_, err := io.WriteString(stdout, `{"playabilityStatus":{"status":"ERROR","secret":"do-not-print"}}`)
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -48,7 +48,7 @@ func TestManagedDNSProductionPlayerAndGuard(t *testing.T) {
|
||||
return nil
|
||||
}
|
||||
requests := 0
|
||||
target.player = func(_ context.Context, args []string, out, _ io.Writer) error {
|
||||
target.player = func(_ context.Context, args []string, _ io.Reader, out, _ io.Writer) error {
|
||||
assertManagedDNSArgs(t, args, "socks5h://127.0.0.1:11080")
|
||||
if args[len(args)-1] != playerURL {
|
||||
t.Fatal("Player hostname changed", args)
|
||||
|
||||
@@ -86,6 +86,9 @@ func managedExperimentalRequest(dir string, consume bool, apply func() error) er
|
||||
// Runs before the PID-1-owned monitor starts; never races a recovery worker.
|
||||
// A real Player-discovered identity is saved WITHOUT cold/adaptive evidence.
|
||||
func managedExperimentalSuffix(m *managedTransaction, d managedDeps) error {
|
||||
if _, err := requiredYoutubeAdapter(); err != nil {
|
||||
return err
|
||||
}
|
||||
singleton, err := acquireManagedMonitorSingleton(d)
|
||||
if err != nil {
|
||||
return err
|
||||
|
||||
@@ -43,7 +43,7 @@ func TestExperimentalProductionProbeCollectorClassification(t *testing.T) {
|
||||
if strings.HasPrefix(mode, "metadata-") {
|
||||
d.resources = managedServiceResources(healthConfig{SOCKSProxy: "socks5://127.0.0.1:11080", VideoID: "dQw4w9WgXcQ"}, func(context.Context) ([]byte, error) {
|
||||
return []byte(fmt.Sprintf(`{"playabilityStatus":{"status":"OK"},"streamingData":{"formats":[{"url":%q,"mimeType":"video/mp4"}]}}`, experimentalResourceFixture().url)), nil
|
||||
}, func(_ context.Context, args []string, out, diagnostic io.Writer) error {
|
||||
}, func(_ context.Context, args []string, _ io.Reader, out, diagnostic io.Writer) error {
|
||||
origin := args[len(args)-1]
|
||||
if (origin != "https://www.youtube.com/" && !(mode == "metadata-malformed" && origin == "https://www.youtube.com/watch?v=dQw4w9WgXcQ")) || diagnostic != io.Discard {
|
||||
t.Fatal("metadata origin/diagnostics changed")
|
||||
|
||||
@@ -132,7 +132,7 @@ func TestExperimentalSuffixCLIExplicit(t *testing.T) {
|
||||
func TestExperimentalSuffixCLIRejectsBypassBeforeWork(t *testing.T) {
|
||||
d, _, _ := managedCLIFixture(t)
|
||||
d.lookup = func(string) (string, bool) { t.Fatal("bypass reached runtime work"); return "", false }
|
||||
d.player = func(context.Context, []string, io.Writer, io.Writer) error {
|
||||
d.player = func(context.Context, []string, io.Reader, io.Writer, io.Writer) error {
|
||||
t.Fatal("bypass requested Player")
|
||||
return nil
|
||||
}
|
||||
@@ -280,7 +280,7 @@ func TestExperimentalSuffixCLIRejectsInheritedPolicy(t *testing.T) {
|
||||
if err := m.Apply(db, []int64{front, asset, id}, GooglevideoScopeExperimental); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
d.player = func(context.Context, []string, io.Writer, io.Writer) error {
|
||||
d.player = func(context.Context, []string, io.Reader, io.Writer, io.Writer) error {
|
||||
t.Fatal("inherited bypass requested Player")
|
||||
return nil
|
||||
}
|
||||
@@ -319,7 +319,7 @@ func TestExperimentalSuffixCLIInterleavedPolicyChange(t *testing.T) {
|
||||
}
|
||||
return db, nil
|
||||
}
|
||||
d.player = func(context.Context, []string, io.Writer, io.Writer) error {
|
||||
d.player = func(context.Context, []string, io.Reader, io.Writer, io.Writer) error {
|
||||
t.Fatal("racing CLI requested Player")
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -179,7 +179,7 @@ func TestExperimentalPromotionRequiresRepresentativeOwnedProof(t *testing.T) {
|
||||
}
|
||||
assertActiveCandidates(t, d.transaction, d.db, []int64{front, thumb, avatar, id})
|
||||
p, err := readGooglevideoProof(d.db, id)
|
||||
if err != nil || p.Version != 1 || len(p.Hosts) != 3 {
|
||||
if err != nil || p.Version != 2 || p.Adapter == nil || len(p.Hosts) != 3 {
|
||||
t.Fatal(p, err)
|
||||
}
|
||||
c, err := readStrategyCandidate(d.db, experiment)
|
||||
|
||||
@@ -5,9 +5,10 @@ import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"zapret-autotune/internal/adapter/youtube"
|
||||
)
|
||||
|
||||
// Ordinary unresolved state, not a target/ownership guard failure.
|
||||
@@ -28,7 +29,7 @@ const managedGooglevideoDiscoveryTimeout = 3 * time.Minute
|
||||
|
||||
// Three hosts each need up to 120s warmup + six 35s transfers. Leave margin
|
||||
// for daemon lifecycle as well; short-lived URLs are refetched, never rewritten.
|
||||
const managedGooglevideoSampleTTL = 20 * time.Minute
|
||||
const managedGooglevideoSampleTTL = youtube.InventoryTTL
|
||||
|
||||
func freshGooglevideoSample(r mediaResource) bool {
|
||||
// The same exact GGC hostname syntax as the representative gate; reject bad
|
||||
@@ -37,13 +38,10 @@ func freshGooglevideoSample(r mediaResource) bool {
|
||||
if !ok || !strings.HasSuffix(r.host, ".googlevideo.com") || !validDomain(r.host) || r.host != strings.ToLower(r.host) || root == "" || len(prefix) < 3 || !strings.HasPrefix(prefix, "rr") || strings.Trim(prefix[2:], "0123456789") != "" {
|
||||
return false
|
||||
}
|
||||
u, err := privateResourceURL(r.url, r.host)
|
||||
if err != nil || u.Path != "/videoplayback" || (r.kind != "video/mp4" && r.kind != "audio/mp4") {
|
||||
if _, err := privateResourceURL(r.url, r.host); err != nil {
|
||||
return false
|
||||
}
|
||||
q := u.Query()
|
||||
expires, err := strconv.ParseInt(q.Get("expire"), 10, 64)
|
||||
return err == nil && expires > time.Now().Add(managedGooglevideoSampleTTL).Unix() && (q.Get("sig") != "" || q.Get("signature") != "" || q.Get("lsig") != "")
|
||||
return youtube.SignedSampleFresh(youtube.Resource{URL: r.url, Host: r.host, Kind: r.kind}, time.Now(), managedGooglevideoSampleTTL)
|
||||
}
|
||||
|
||||
// Choose exactly three actual hosts passing the unchanged representative gate.
|
||||
@@ -70,6 +68,9 @@ func representativeGooglevideoSamples(byHost map[string]mediaResource) []mediaRe
|
||||
func managedGooglevideoSamples(ids []string, request func(context.Context, string) ([]byte, error)) func(context.Context) ([]mediaResource, error) {
|
||||
ids = append([]string(nil), ids...)
|
||||
return func(ctx context.Context) ([]mediaResource, error) {
|
||||
if _, err := requiredYoutubeAdapter(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(ids) == 0 || len(ids) > 5 || request == nil {
|
||||
return nil, &managedRecoveryFatal{Err: errors.New("invalid sample collector configuration")}
|
||||
}
|
||||
@@ -108,6 +109,9 @@ func managedGooglevideoSamples(ids []string, request func(context.Context, strin
|
||||
}
|
||||
r, _, err := playerMediaResource(body)
|
||||
if err != nil {
|
||||
if errors.Is(err, errServiceAdapterUnavailable) {
|
||||
return nil, err
|
||||
}
|
||||
stats.Unavailable++
|
||||
continue
|
||||
} // Missing/SABR is unknown, never fabricated.
|
||||
|
||||
@@ -9,9 +9,10 @@ import (
|
||||
"os"
|
||||
"os/exec"
|
||||
"reflect"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"zapret-autotune/internal/adapter/youtube"
|
||||
)
|
||||
|
||||
// ponytail: one audited composite only. New native variants need an explicit
|
||||
@@ -93,84 +94,55 @@ func newManagedGooglevideoVerifier(target managedDeps) func(context.Context, sto
|
||||
}
|
||||
|
||||
func (r managedAssetRuntime) verifyGooglevideo(ctx context.Context, c storedCandidate, resources []mediaResource) (GooglevideoQualificationProof, error) {
|
||||
p := GooglevideoQualificationProof{Version: 1, Mode: QualificationAdaptive, SourceKind: NativeCircularPostWarmup, Args: c.Args}
|
||||
p := GooglevideoQualificationProof{Version: 2, Mode: QualificationAdaptive, SourceKind: NativeCircularPostWarmup, Args: c.Args}
|
||||
if c.Group != "googlevideo" || c.Mode != QualificationAdaptive {
|
||||
return p, errors.New("adaptive googlevideo definition required")
|
||||
}
|
||||
if err := validateNativeGooglevideoArgs(c.Args); err != nil {
|
||||
return p, err
|
||||
}
|
||||
module, err := requiredYoutubeAdapter()
|
||||
if err != nil {
|
||||
return p, err
|
||||
}
|
||||
binding := module.Binding()
|
||||
p.Adapter = &binding
|
||||
byHost := map[string]mediaResource{}
|
||||
var samples []youtube.Resource
|
||||
for _, resource := range resources {
|
||||
u, err := privateResourceURL(resource.url, resource.host)
|
||||
if err != nil || u.Path != "/videoplayback" || (resource.kind != "video/mp4" && resource.kind != "audio/mp4") {
|
||||
return p, errGooglevideoInsufficientSamples
|
||||
}
|
||||
q := u.Query()
|
||||
expires, err := strconv.ParseInt(q.Get("expire"), 10, 64)
|
||||
if err != nil || expires <= time.Now().Unix()+120 || (q.Get("sig") == "" && q.Get("signature") == "" && q.Get("lsig") == "") {
|
||||
if _, err := privateResourceURL(resource.url, resource.host); err != nil {
|
||||
return p, errGooglevideoInsufficientSamples
|
||||
}
|
||||
byHost[resource.host] = resource
|
||||
samples = append(samples, youtube.Resource{URL: resource.url, Host: resource.host, Kind: resource.kind})
|
||||
}
|
||||
// Validate representative host diversity using the shared gate before capture.
|
||||
for _, host := range c.Endpoints {
|
||||
if _, ok := byHost[host]; !ok {
|
||||
return p, errGooglevideoInsufficientSamples
|
||||
}
|
||||
p.Hosts = append(p.Hosts, GooglevideoHostProof{Host: host})
|
||||
}
|
||||
if err := validateGooglevideoHosts(c.Endpoints); err != nil {
|
||||
if validateGooglevideoHosts(c.Endpoints) != nil {
|
||||
return p, errGooglevideoInsufficientSamples
|
||||
}
|
||||
err := r.withOwnedWorker(ctx, c.Endpoints, c.Args, func(probeCtx context.Context) error {
|
||||
for i := range p.Hosts {
|
||||
h := &p.Hosts[i]
|
||||
h.WarmupLimit = 10
|
||||
start := time.Now()
|
||||
warmCtx, cancel := context.WithTimeout(probeCtx, 120*time.Second)
|
||||
learned := map[string]bool{}
|
||||
for h.WarmupAttempts < h.WarmupLimit && len(learned) < 2 && warmCtx.Err() == nil {
|
||||
protocol := []string{"tls12", "tls13"}[h.WarmupAttempts%2]
|
||||
h.WarmupAttempts++
|
||||
_, pass, err := collectMediaTransfer(warmCtx, r.proxy, byHost[h.Host], protocol, r.curl)
|
||||
if err != nil {
|
||||
h.WarmupFailures++
|
||||
if !managedGooglevideoTransferNonpass(err) && warmCtx.Err() == nil {
|
||||
cancel()
|
||||
return err
|
||||
}
|
||||
} else if pass.Bytes < 1024 {
|
||||
h.WarmupFailures++
|
||||
} else {
|
||||
learned[protocol] = true
|
||||
}
|
||||
}
|
||||
h.WarmupMS = float64(time.Since(start).Microseconds()) / 1000
|
||||
cancel()
|
||||
if probeCtx.Err() != nil {
|
||||
return probeCtx.Err()
|
||||
}
|
||||
if len(learned) != 2 || h.WarmupMS > 120000 {
|
||||
return errCandidateNonpass
|
||||
}
|
||||
for _, protocol := range []string{"tls12", "tls13"} {
|
||||
for n := 0; n < 3; n++ {
|
||||
_, pass, err := collectMediaTransfer(probeCtx, r.proxy, byHost[h.Host], protocol, r.curl)
|
||||
if err != nil {
|
||||
if managedGooglevideoTransferNonpass(err) {
|
||||
return errCandidateNonpass
|
||||
}
|
||||
return err
|
||||
}
|
||||
if pass.Bytes < 1024 {
|
||||
return errCandidateNonpass
|
||||
}
|
||||
h.Passes = append(h.Passes, pass)
|
||||
}
|
||||
}
|
||||
plan, err := module.PlanQualification(c.Endpoints, samples, time.Now())
|
||||
if err != nil {
|
||||
return p, errGooglevideoInsufficientSamples
|
||||
}
|
||||
if !reflect.DeepEqual(plan.Hosts, c.Endpoints) {
|
||||
return p, errAdapterPlan
|
||||
}
|
||||
if err := validateAdapterPlan(binding, plan, byHost); err != nil {
|
||||
return p, err
|
||||
}
|
||||
for _, host := range plan.Hosts {
|
||||
p.Hosts = append(p.Hosts, GooglevideoHostProof{Host: host})
|
||||
}
|
||||
err = r.withOwnedWorker(ctx, c.Endpoints, c.Args, func(probeCtx context.Context) error {
|
||||
hosts, err := executeAdapterPlan(probeCtx, binding, plan, byHost,
|
||||
func(ctx context.Context, resource mediaResource, protocol string) (adapterMeasurement[GooglevideoPass], error) {
|
||||
_, pass, err := collectMediaTransfer(ctx, r.proxy, resource, protocol, r.curl)
|
||||
return adapterMeasurement[GooglevideoPass]{value: pass, bytes: pass.Bytes}, err
|
||||
}, managedGooglevideoTransferNonpass)
|
||||
for i, h := range hosts {
|
||||
p.Hosts[i] = GooglevideoHostProof{Host: h.host, WarmupLimit: h.warmupLimit,
|
||||
WarmupAttempts: h.warmupAttempts, WarmupFailures: h.warmupFailures, WarmupMS: h.warmupMS, Passes: h.samples}
|
||||
}
|
||||
return nil
|
||||
return err
|
||||
})
|
||||
if err != nil {
|
||||
return p, err
|
||||
@@ -239,16 +211,28 @@ func managedResourceTransferNonpass(err error, metadata, image bool) bool {
|
||||
}
|
||||
|
||||
func validateGooglevideoHosts(hosts []string) error {
|
||||
seen, roots, prefixes := map[string]bool{}, map[string]bool{}, map[string]bool{}
|
||||
for _, host := range hosts {
|
||||
prefix, root, ok := strings.Cut(strings.TrimSuffix(host, ".googlevideo.com"), "---sn-")
|
||||
if !ok || !strings.HasSuffix(host, ".googlevideo.com") || !validDomain(host) || host != strings.ToLower(host) || seen[host] || root == "" || len(prefix) < 3 || !strings.HasPrefix(prefix, "rr") || strings.Trim(prefix[2:], "0123456789") != "" {
|
||||
if !validDomain(host) {
|
||||
return errGooglevideoInsufficientSamples
|
||||
}
|
||||
seen[host], roots[strings.Split(root, "-")[0]], prefixes[prefix] = true, true, true
|
||||
}
|
||||
if len(hosts) < 3 || len(roots) < 2 || len(prefixes) < 2 {
|
||||
module, err := requiredYoutubeAdapter()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if module.ValidateHosts(hosts) != nil {
|
||||
return errGooglevideoInsufficientSamples
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Historical v1 reconstruction must not depend on a future registry default.
|
||||
// This pinned contract remains available even when the execution module is absent.
|
||||
func validateLegacyYoutubeV1Hosts(hosts []string) error {
|
||||
for _, host := range hosts {
|
||||
if !validDomain(host) {
|
||||
return errGooglevideoInsufficientSamples
|
||||
}
|
||||
}
|
||||
return (youtube.Module{}).ValidateHosts(hosts)
|
||||
}
|
||||
@@ -352,7 +352,7 @@ func TestManagedMonitorProductionAdapterBinding(t *testing.T) {
|
||||
recovery, _, _ := recoveryFixture(t, false)
|
||||
target, _, _ := managedCLIFixture(t)
|
||||
target.directory = recovery.transaction.Dir
|
||||
target.player = func(ctx context.Context, args []string, out, _ io.Writer) error {
|
||||
target.player = func(ctx context.Context, args []string, _ io.Reader, out, _ io.Writer) error {
|
||||
if ctx.Err() != nil {
|
||||
t.Fatal(ctx.Err())
|
||||
}
|
||||
@@ -580,7 +580,7 @@ func TestManagedMonitorCLIBindingAndCancellation(t *testing.T) {
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
requests := 0
|
||||
d.player = func(ctx context.Context, args []string, out, _ io.Writer) error {
|
||||
d.player = func(ctx context.Context, args []string, _ io.Reader, out, _ io.Writer) error {
|
||||
requests++
|
||||
if !strings.Contains(strings.Join(args, "|"), "--noproxy||--proxy|socks5h://127.0.0.1:11080|") {
|
||||
t.Fatal(args)
|
||||
|
||||
@@ -47,6 +47,9 @@ type managedRecoveryDeps struct {
|
||||
// The injected command runner receives only fixed script/argv from orchestration.
|
||||
func managedRecoveryDependencies(target managedDeps, db *sql.DB, scan cacheScan, check cacheCheck, run func(context.Context, string, ...string) error) (managedRecoveryDeps, error) {
|
||||
d := managedRecoveryDeps{db: db, scan: scan, check: check, run: run}
|
||||
if _, err := requiredYoutubeAdapter(); err != nil {
|
||||
return d, err
|
||||
}
|
||||
if target.lookup == nil || target.player == nil || db == nil || scan == nil || check == nil || run == nil {
|
||||
return d, fmt.Errorf("complete target-local recovery adapter dependencies required")
|
||||
}
|
||||
@@ -59,6 +62,9 @@ func managedRecoveryDependencies(target managedDeps, db *sql.DB, scan cacheScan,
|
||||
}
|
||||
d.transaction = &managedTransaction{Dir: target.directory, Name: "config"}
|
||||
requestVideo := func(ctx context.Context, videoID string) ([]byte, error) {
|
||||
if _, err := requiredYoutubeAdapter(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if err := ctx.Err(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
@@ -67,8 +73,8 @@ func managedRecoveryDependencies(target managedDeps, db *sql.DB, scan cacheScan,
|
||||
}
|
||||
bounded, cancel := context.WithTimeout(ctx, 35*time.Second)
|
||||
defer cancel()
|
||||
body, err := requestPlayer(videoID, cfg.SOCKSProxy, func(_ context.Context, args []string, out, diagnostics io.Writer) error {
|
||||
return target.player(bounded, args, out, diagnostics)
|
||||
body, err := requestPlayer(videoID, cfg.SOCKSProxy, func(_ context.Context, args []string, stdin io.Reader, out, diagnostics io.Writer) error {
|
||||
return target.player(bounded, args, stdin, out, diagnostics)
|
||||
})
|
||||
if bounded.Err() != nil {
|
||||
return nil, managedRecoveryContextError(ctx, errors.Join(err, bounded.Err()))
|
||||
|
||||
@@ -146,7 +146,7 @@ func TestManagedServiceFreshPrivateInventory(t *testing.T) {
|
||||
cfg := healthConfig{SOCKSProxy: "socks5://127.0.0.1:11080"}
|
||||
resources := managedServiceResources(cfg, func(context.Context) ([]byte, error) {
|
||||
return []byte(`{"playabilityStatus":{"status":"OK"},"videoDetails":{"thumbnail":{"thumbnails":[{"url":"https://i.ytimg.com/fresh?private=thumbnail"}]}},"streamingData":{}}`), nil
|
||||
}, func(_ context.Context, args []string, out, _ io.Writer) error {
|
||||
}, func(_ context.Context, args []string, _ io.Reader, out, _ io.Writer) error {
|
||||
if args[len(args)-1] != "https://www.youtube.com/" || strings.Contains(strings.Join(args, " "), "private=") {
|
||||
t.Fatal(args)
|
||||
}
|
||||
|
||||
@@ -30,27 +30,32 @@ func (e *managedRollbackError) Error() string {
|
||||
func (e *managedRollbackError) Unwrap() error { return e.Cause }
|
||||
|
||||
type managedJournal struct {
|
||||
Version int `json:"version"`
|
||||
Generation uint64 `json:"generation"`
|
||||
ID string `json:"transaction_id"`
|
||||
State string `json:"state"`
|
||||
Hash string `json:"committed_hash"`
|
||||
Next string `json:"pending_hash,omitempty"`
|
||||
Candidates []int64 `json:"committed_candidate_ids,omitempty"`
|
||||
PendingCandidates []int64 `json:"pending_candidate_ids,omitempty"`
|
||||
Scope GooglevideoScope `json:"committed_googlevideo_scope,omitempty"`
|
||||
PendingScope GooglevideoScope `json:"pending_googlevideo_scope,omitempty"`
|
||||
Version int `json:"version"`
|
||||
Generation uint64 `json:"generation"`
|
||||
ID string `json:"transaction_id"`
|
||||
State string `json:"state"`
|
||||
Hash string `json:"committed_hash"`
|
||||
Next string `json:"pending_hash,omitempty"`
|
||||
Candidates []int64 `json:"committed_candidate_ids,omitempty"`
|
||||
PendingCandidates []int64 `json:"pending_candidate_ids,omitempty"`
|
||||
Scope GooglevideoScope `json:"committed_googlevideo_scope,omitempty"`
|
||||
PendingScope GooglevideoScope `json:"pending_googlevideo_scope,omitempty"`
|
||||
Ref *managedConfigRef `json:"committed_ref,omitempty"`
|
||||
PendingRef *managedConfigRef `json:"pending_ref,omitempty"`
|
||||
}
|
||||
|
||||
// Hooks take no commands or arguments. The target adapter binds Reload
|
||||
// exclusively to /opt/zapret2/init.d/sysv/zapret2 restart and Health to
|
||||
// health youtube over its configured loopback SOCKS, returning exactly "OK".
|
||||
// exclusively to /opt/zapret2/init.d/sysv/zapret2 restart. Health is the
|
||||
// adapter's owned protocol check, returning exactly "OK" (legacy YouTube or
|
||||
// every required generic resource/TLS pair); optional rollback health is strict.
|
||||
// Reload switches target-local firewall and daemons for apply and rollback.
|
||||
// This engine does not manage SOCKS, health scheduling or DB active state.
|
||||
type managedTransaction struct {
|
||||
Dir, Name string
|
||||
Reload func() error
|
||||
Health func() (string, error)
|
||||
Dir, Name string
|
||||
Reload func() error
|
||||
Health func() (string, error)
|
||||
RollbackHealth func() (string, error)
|
||||
resource *resourceApplySnapshot
|
||||
// Ordinary candidate-ID CLI calls cannot inherit an operator experiment.
|
||||
// Evaluated against the effective policy only while holding the journal lock.
|
||||
forbidExperimentalApply bool
|
||||
@@ -186,7 +191,7 @@ func (m *managedTransaction) load() (managedJournal, error) {
|
||||
if err != nil {
|
||||
return j, err
|
||||
}
|
||||
if err := json.Unmarshal(b, &j); err != nil {
|
||||
if err := strictPlanJSON(b, &j); err != nil {
|
||||
return j, err
|
||||
}
|
||||
validHash := func(s string) bool {
|
||||
@@ -208,9 +213,12 @@ func (m *managedTransaction) load() (managedJournal, error) {
|
||||
}
|
||||
return true
|
||||
}
|
||||
if (j.Version != 1 && j.Version != 2 && j.Version != 3) || !j.Scope.valid() || !j.PendingScope.valid() || (j.Version < 3 && (j.Scope != "" || j.PendingScope != "")) || (j.Version == 3 && (j.Scope == "" || (j.State == "prepared" && j.PendingScope == ""))) || j.ID == "" || !validHash(j.Hash) || (j.State != "committed" && j.State != "prepared") || (j.State == "prepared" && !validHash(j.Next)) || (j.State == "committed" && (j.Next != "" || len(j.PendingCandidates) != 0 || j.PendingScope != "")) || !validIDs(j.Candidates) || !validIDs(j.PendingCandidates) || (j.Version == 1 && (len(j.Candidates) != 0 || len(j.PendingCandidates) != 0)) {
|
||||
if (j.Version != 1 && j.Version != 2 && j.Version != 3 && j.Version != 4) || !j.Scope.valid() || !j.PendingScope.valid() || (j.Version < 3 && (j.Scope != "" || j.PendingScope != "")) || (j.Version == 3 && (j.Scope == "" || (j.State == "prepared" && j.PendingScope == ""))) || j.ID == "" || !validHash(j.Hash) || (j.State != "committed" && j.State != "prepared") || (j.State == "prepared" && !validHash(j.Next)) || (j.State == "committed" && (j.Next != "" || len(j.PendingCandidates) != 0 || j.PendingScope != "" || j.PendingRef != nil)) || !validIDs(j.Candidates) || !validIDs(j.PendingCandidates) || (j.Version == 1 && (len(j.Candidates) != 0 || len(j.PendingCandidates) != 0)) {
|
||||
return j, fmt.Errorf("invalid managed journal")
|
||||
}
|
||||
if j.Version < 4 && (j.Ref != nil || j.PendingRef != nil) || j.Version == 4 && (!validManagedConfigRef(j.Ref, j.Candidates, j.Scope, j.Hash) || j.State == "prepared" && !validManagedConfigRef(j.PendingRef, j.PendingCandidates, j.PendingScope, j.Next)) {
|
||||
return j, fmt.Errorf("invalid managed journal references")
|
||||
}
|
||||
return j, nil
|
||||
}
|
||||
|
||||
@@ -234,7 +242,12 @@ func (m *managedTransaction) Initialize() error {
|
||||
|
||||
func (m *managedTransaction) checkpoint(point string) error {
|
||||
if m.fault != nil {
|
||||
return m.fault(point)
|
||||
if err := m.fault(point); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
if m.resource != nil {
|
||||
return m.resource.guard()
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -265,6 +278,9 @@ func (m *managedTransaction) ActiveCandidates(db *sql.DB) ([]int64, error) {
|
||||
if j.State != "committed" {
|
||||
return fmt.Errorf("incomplete transaction: reconcile first")
|
||||
}
|
||||
if j.Version == 4 && j.Ref.Kind == "generic" {
|
||||
return fmt.Errorf("generic deployment has no legacy candidate IDs")
|
||||
}
|
||||
b, err := m.read(m.Name)
|
||||
if err != nil {
|
||||
return err
|
||||
@@ -299,6 +315,10 @@ func committedManagedJournal(j managedJournal) managedJournal {
|
||||
if j.Version == 3 {
|
||||
j.Scope, j.PendingScope = j.PendingScope, ""
|
||||
}
|
||||
if j.Version == 4 {
|
||||
j.Scope, j.PendingScope = j.PendingScope, ""
|
||||
j.Ref, j.PendingRef = j.PendingRef, nil
|
||||
}
|
||||
return j
|
||||
}
|
||||
|
||||
@@ -342,6 +362,10 @@ func (m *managedTransaction) applyRenderedDB(b []byte, db *sql.DB, ids ...int64)
|
||||
}
|
||||
|
||||
func (m *managedTransaction) applyRenderedPolicy(b []byte, db *sql.DB, requested *GooglevideoScope, ids ...int64) error {
|
||||
return m.applyPolicy(b, db, requested, nil, ids...)
|
||||
}
|
||||
|
||||
func (m *managedTransaction) applyPolicy(b []byte, db *sql.DB, requested *GooglevideoScope, generic *managedConfigRef, ids ...int64) error {
|
||||
return m.locked(func() error {
|
||||
j, err := m.load()
|
||||
if err != nil {
|
||||
@@ -350,9 +374,32 @@ func (m *managedTransaction) applyRenderedPolicy(b []byte, db *sql.DB, requested
|
||||
if j.State != "committed" {
|
||||
return fmt.Errorf("incomplete transaction: reconcile first")
|
||||
}
|
||||
if err := validateManagedRefs(db, j.Candidates, j.Hash, j.Scope); err != nil {
|
||||
if err := m.validateConfigRef(db, j.Ref, j.Candidates, j.Hash, j.Scope); err != nil {
|
||||
return err
|
||||
}
|
||||
if generic == nil && j.Version == 4 {
|
||||
return fmt.Errorf("generic journal cannot authorize legacy apply")
|
||||
}
|
||||
if generic != nil {
|
||||
if m.resource == nil || *generic != m.resource.ref || m.resource.guard() != nil || m.resource.validate(generic, managedHash(b)) != nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
if j.Version == 4 && j.Ref.Kind == "generic" {
|
||||
if reflect.DeepEqual(j.Ref, generic) {
|
||||
current, e := m.read(m.Name)
|
||||
if e != nil || managedHash(current) != j.Hash {
|
||||
return errManagedDrift
|
||||
}
|
||||
return nil
|
||||
}
|
||||
}
|
||||
if !(j.Version == 4 && j.Ref.Kind == "generic") && (len(j.Candidates) != 0 || j.Hash != managedHash([]byte(managedBaseline))) {
|
||||
return fmt.Errorf("generic apply requires disabled baseline")
|
||||
}
|
||||
if m.resource.fresh() != nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
}
|
||||
scope := j.Scope
|
||||
if scope == "" {
|
||||
scope = GooglevideoScopeExact
|
||||
@@ -360,6 +407,9 @@ func (m *managedTransaction) applyRenderedPolicy(b []byte, db *sql.DB, requested
|
||||
if requested != nil {
|
||||
scope = *requested
|
||||
}
|
||||
if generic != nil {
|
||||
scope = ""
|
||||
}
|
||||
if m.forbidExperimentalApply && scope == GooglevideoScopeExperimental {
|
||||
return fmt.Errorf("experimental CLI apply requires explicit guarded operator path")
|
||||
}
|
||||
@@ -391,7 +441,7 @@ func (m *managedTransaction) applyRenderedPolicy(b []byte, db *sql.DB, requested
|
||||
return fmt.Errorf("fresh qualified candidate state required")
|
||||
}
|
||||
}
|
||||
if err := validateManagedRefs(db, ids, managedHash(b), scope); err != nil {
|
||||
if err := m.validateConfigRef(db, generic, ids, managedHash(b), scope); err != nil {
|
||||
return err
|
||||
}
|
||||
prior, err := m.read(m.Name)
|
||||
@@ -433,7 +483,14 @@ func (m *managedTransaction) applyRenderedPolicy(b []byte, db *sql.DB, requested
|
||||
return err
|
||||
}
|
||||
j.Generation++
|
||||
if requested != nil || j.Version == 3 {
|
||||
if generic != nil {
|
||||
if j.Version != 4 {
|
||||
j.Ref = &managedConfigRef{Kind: "baseline"}
|
||||
}
|
||||
j.Version = 4
|
||||
j.Scope, j.PendingScope = "", ""
|
||||
j.PendingRef = generic
|
||||
} else if requested != nil || j.Version == 3 {
|
||||
j.Version = 3
|
||||
if j.Scope == "" {
|
||||
j.Scope = GooglevideoScopeExact
|
||||
@@ -470,7 +527,12 @@ func (m *managedTransaction) applyRenderedPolicy(b []byte, db *sql.DB, requested
|
||||
}
|
||||
}
|
||||
if err == nil {
|
||||
err = m.Reload()
|
||||
if generic != nil {
|
||||
err = m.resource.guard()
|
||||
}
|
||||
if err == nil {
|
||||
err = m.Reload()
|
||||
}
|
||||
}
|
||||
if err == nil {
|
||||
if fault := m.checkpoint("reloaded"); fault != nil {
|
||||
@@ -491,6 +553,15 @@ func (m *managedTransaction) applyRenderedPolicy(b []byte, db *sql.DB, requested
|
||||
}
|
||||
if err == nil {
|
||||
committed := committedManagedJournal(j)
|
||||
if generic != nil {
|
||||
err = m.resource.guard()
|
||||
}
|
||||
if err != nil {
|
||||
if rollback := m.restore(j); rollback != nil {
|
||||
return &managedRollbackError{Cause: err, Rollback: rollback}
|
||||
}
|
||||
return err
|
||||
}
|
||||
current, err = m.read(m.Name)
|
||||
if err == nil && managedHash(current) != j.Next {
|
||||
return errManagedDrift
|
||||
@@ -542,6 +613,15 @@ func (m *managedTransaction) restore(j managedJournal) error {
|
||||
if err := m.Reload(); err != nil {
|
||||
return err
|
||||
}
|
||||
if m.RollbackHealth != nil {
|
||||
status, err := m.RollbackHealth()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if status != "OK" {
|
||||
return &managedHealthRejection{Status: status}
|
||||
}
|
||||
}
|
||||
if err := m.owns(j, false); err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -555,6 +635,7 @@ func (m *managedTransaction) restore(j managedJournal) error {
|
||||
j.State, j.Next = "committed", ""
|
||||
j.PendingCandidates = nil
|
||||
j.PendingScope = ""
|
||||
j.PendingRef = nil
|
||||
return m.journal(j)
|
||||
}
|
||||
|
||||
@@ -570,10 +651,10 @@ func (m *managedTransaction) Reconcile(databases ...*sql.DB) error {
|
||||
if len(databases) != 0 {
|
||||
db = databases[0]
|
||||
}
|
||||
if err := validateManagedRefs(db, j.Candidates, j.Hash, j.Scope); err != nil {
|
||||
if err := m.validateConfigRef(db, j.Ref, j.Candidates, j.Hash, j.Scope); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := validateManagedRefs(db, j.PendingCandidates, j.Next, j.PendingScope); err != nil {
|
||||
if err := m.validateConfigRef(db, j.PendingRef, j.PendingCandidates, j.Next, j.PendingScope); err != nil {
|
||||
return err
|
||||
}
|
||||
if j.State == "prepared" {
|
||||
|
||||
@@ -34,7 +34,7 @@ func (w playerCancelWriter) Write(p []byte) (int, error) {
|
||||
|
||||
func canceledPlayerProcess(t *testing.T, cancel context.CancelFunc, extra error, overflow string) playerRunner {
|
||||
t.Helper()
|
||||
return func(ctx context.Context, _ []string, out, diagnostics io.Writer) error {
|
||||
return func(ctx context.Context, _ []string, _ io.Reader, out, diagnostics io.Writer) error {
|
||||
cmd := exec.CommandContext(ctx, os.Args[0], "-test.run=^TestPlayerCancellationChild$")
|
||||
cmd.Env = append(os.Environ(), "AUTOTUNE_PLAYER_CANCEL_CHILD=1")
|
||||
cmd.Stdout = playerCancelWriter{cancel: cancel}
|
||||
@@ -52,8 +52,8 @@ func TestPlayerRequestCancellationCause(t *testing.T) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
runner := canceledPlayerProcess(t, cancel, nil, "")
|
||||
_, err := requestPlayer("abcdefghijk", "socks5://127.0.0.1:11080", func(_ context.Context, args []string, out, diagnostics io.Writer) error {
|
||||
return runner(ctx, args, out, diagnostics)
|
||||
_, err := requestPlayer("abcdefghijk", "socks5://127.0.0.1:11080", func(_ context.Context, args []string, stdin io.Reader, out, diagnostics io.Writer) error {
|
||||
return runner(ctx, args, stdin, out, diagnostics)
|
||||
})
|
||||
var child *exec.ExitError
|
||||
if !errors.As(err, &child) || !managedRecoveryTimeoutSafe(err) {
|
||||
@@ -67,12 +67,17 @@ func TestPlayerRequestCancellationCause(t *testing.T) {
|
||||
func TestPlayerRequestPublicErrorsOpaque(t *testing.T) {
|
||||
const secret = "https://user:password@example.com/video?signature=private"
|
||||
cause := errors.New(secret)
|
||||
runner := func(_ context.Context, _ []string, _, diagnostics io.Writer) error {
|
||||
runner := func(_ context.Context, args []string, stdin io.Reader, _, diagnostics io.Writer) error {
|
||||
payload, err := io.ReadAll(stdin)
|
||||
if err != nil || !bytes.Contains(payload, []byte("abcdefghijk")) || strings.Contains(strings.Join(args, " "), "abcdefghijk") {
|
||||
t.Fatal("private POST body missing from stdin or exposed in argv")
|
||||
}
|
||||
io.WriteString(diagnostics, secret)
|
||||
return cause
|
||||
diagnostics.Write(payload)
|
||||
return errors.Join(cause, errors.New(string(payload)))
|
||||
}
|
||||
_, err := requestPlayer("abcdefghijk", "socks5://127.0.0.1:11080", runner)
|
||||
if !errors.Is(err, cause) || strings.Contains(err.Error(), secret) || managedRecoveryTimeoutSafe(err) {
|
||||
if !errors.Is(err, cause) || strings.Contains(err.Error(), secret) || strings.Contains(err.Error(), "abcdefghijk") || managedRecoveryTimeoutSafe(err) {
|
||||
t.Fatal("opaque error lost its unsafe cause or exposed it")
|
||||
}
|
||||
t.Setenv("ZAPRET_AUTOTUNE_VIDEO_ID", "abcdefghijk")
|
||||
@@ -84,7 +89,7 @@ func TestPlayerRequestPublicErrorsOpaque(t *testing.T) {
|
||||
} else {
|
||||
code = discoverYoutube([]string{"--video-id", "abcdefghijk", "--proxy", "socks5://127.0.0.1:11080"}, &out, &diagnostics, runner)
|
||||
}
|
||||
if code != 1 || out.Len() != 0 || strings.Contains(diagnostics.String(), secret) || !strings.Contains(diagnostics.String(), "player request failed") {
|
||||
if code != 1 || out.Len() != 0 || strings.Contains(diagnostics.String(), secret) || strings.Contains(diagnostics.String(), "abcdefghijk") || !strings.Contains(diagnostics.String(), "player request failed") {
|
||||
t.Fatalf("%s: code=%d diagnostics=%q", command, code, diagnostics.String())
|
||||
}
|
||||
}
|
||||
@@ -141,7 +146,7 @@ func TestManagedMonitorSIGTERMChild(t *testing.T) {
|
||||
env["ZAPRET_AUTOTUNE_DB"] = managedTargetDirectory + "/candidates.db"
|
||||
ctx, cancel := signal.NotifyContext(context.Background(), syscall.SIGTERM)
|
||||
defer cancel()
|
||||
d.player = func(ctx context.Context, _ []string, _, diagnostics io.Writer) error {
|
||||
d.player = func(ctx context.Context, _ []string, _ io.Reader, _, diagnostics io.Writer) error {
|
||||
cmd := exec.CommandContext(ctx, os.Args[0], "-test.run=^TestPlayerCancellationChild$")
|
||||
cmd.Env = append(os.Environ(), "AUTOTUNE_PLAYER_CANCEL_CHILD=1")
|
||||
cmd.Stdout, cmd.Stderr = os.Stdout, diagnostics
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func playerCurlStub(t *testing.T, script string) string {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
if err := os.WriteFile(filepath.Join(dir, "curl"), []byte("#!/bin/sh\n"+script), 0700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
t.Setenv("PATH", dir+string(os.PathListSeparator)+os.Getenv("PATH"))
|
||||
return dir
|
||||
}
|
||||
|
||||
func TestPlayerPOSTPrivateStdin(t *testing.T) {
|
||||
dir := playerCurlStub(t, `printf '%s\n' "$@" > "$PLAYER_CAPTURE.args"
|
||||
cat > "$PLAYER_CAPTURE.body"
|
||||
printf '%s' "$PLAYER_RESPONSE"
|
||||
`)
|
||||
t.Setenv("PLAYER_CAPTURE", filepath.Join(dir, "request"))
|
||||
t.Setenv("PLAYER_RESPONSE", discoveryFixture)
|
||||
body, err := requestPlayer("SECRET12345", "socks5://127.0.0.1:11080", runPlayerCurl)
|
||||
if err != nil || string(body) != discoveryFixture {
|
||||
t.Fatal("real runner did not preserve the response", err)
|
||||
}
|
||||
args, err := os.ReadFile(filepath.Join(dir, "request.args"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if strings.Contains(string(args), "SECRET12345") || !strings.Contains(string(args), "--json\n@-\n") {
|
||||
t.Fatal("POST body must travel only on stdin")
|
||||
}
|
||||
payload, err := os.ReadFile(filepath.Join(dir, "request.body"))
|
||||
const want = `{"context":{"client":{"androidSdkVersion":30,"clientName":"ANDROID","clientVersion":"21.02.35","osName":"Android","osVersion":"11","userAgent":"com.google.youtube/21.02.35(Linux;U;Android11)gzip"}},"videoId":"SECRET12345"}`
|
||||
if err != nil || string(payload) != want {
|
||||
t.Fatal("POST bytes changed", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPlayerPOSTInputBound(t *testing.T) {
|
||||
for _, id := range []string{"", strings.Repeat("S", playerResponseLimit+1), "SECRET\n1234"} {
|
||||
_, err := requestPlayer(id, "socks5://127.0.0.1:11080", func(context.Context, []string, io.Reader, io.Writer, io.Writer) error {
|
||||
t.Fatal("invalid input invoked curl")
|
||||
return nil
|
||||
})
|
||||
if err == nil || strings.Contains(err.Error(), "SECRET") {
|
||||
t.Fatal("invalid POST input must fail privately before execution")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestPlayerCurlStdinCancellation(t *testing.T) {
|
||||
playerCurlStub(t, "cat > /dev/null\nprintf ready\nexec sleep 60\n")
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
var diagnostics bytes.Buffer
|
||||
err := runPlayerCurl(ctx, []string{"--json", "@-"}, strings.NewReader(`{"videoId":"SECRET12345"}`), playerCancelWriter{cancel: cancel}, &diagnostics)
|
||||
if err == nil || !errors.Is(ctx.Err(), context.Canceled) || !managedRecoveryTimeoutSafe(err) {
|
||||
t.Fatal("real curl runner must join a canceled child", err)
|
||||
}
|
||||
if strings.Contains(diagnostics.String(), "SECRET12345") {
|
||||
t.Fatal("stdin leaked into diagnostics")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,257 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"syscall"
|
||||
)
|
||||
|
||||
// These enums are closed: no caller-controlled strings enter diagnostics.
|
||||
type qualificationStage uint8
|
||||
|
||||
const (
|
||||
stageUnknown qualificationStage = iota
|
||||
stagePreDNSGuard
|
||||
stagePostDNSGuard
|
||||
stageDNS
|
||||
stageTransfer
|
||||
stageAttestation
|
||||
stageWorkerStart
|
||||
stageWorkerQueue
|
||||
stageWorkerExit
|
||||
stageNFTDelete
|
||||
stageStopJoin
|
||||
stageRestore
|
||||
stageFinalStoreGuard
|
||||
stageACKWrite
|
||||
)
|
||||
|
||||
type qualificationReason uint8
|
||||
|
||||
const (
|
||||
reasonUnknown qualificationReason = iota
|
||||
reasonExecution
|
||||
reasonOutputBound
|
||||
reasonMetrics
|
||||
reasonOrigin
|
||||
reasonTLSEvidence
|
||||
reasonFraming
|
||||
reasonBody
|
||||
reasonMIME
|
||||
reasonWorkerIdentity
|
||||
reasonRules
|
||||
reasonFiles
|
||||
reasonDefinition
|
||||
reasonRuntime
|
||||
reasonCollector
|
||||
reasonGuard
|
||||
reasonWorkerScan
|
||||
reasonEngine
|
||||
reasonArgv
|
||||
reasonParent
|
||||
reasonStarttime
|
||||
reasonQueue
|
||||
reasonVersion
|
||||
reasonRulesRead
|
||||
reasonMode
|
||||
reasonPlanStore
|
||||
reasonContext
|
||||
reasonQueuePresentOrUnreadable
|
||||
reasonNFTRead
|
||||
reasonNFTPresent
|
||||
reasonBaseline
|
||||
reasonPriorCleanupVeto
|
||||
reasonWorkerScanRead
|
||||
reasonWorkerScanReadlink
|
||||
reasonWorkerScanForeign
|
||||
)
|
||||
|
||||
func (s qualificationStage) String() string {
|
||||
names := [...]string{"unknown", "pre_dns_guard", "post_dns_guard", "dns", "transfer", "attestation", "worker_start", "worker_queue", "worker_exit", "nft_delete", "stop_join", "restore", "final_store_guard", "ack_write"}
|
||||
if int(s) >= len(names) {
|
||||
return names[0]
|
||||
}
|
||||
return names[s]
|
||||
}
|
||||
|
||||
func (r qualificationReason) String() string {
|
||||
names := [...]string{"unknown", "execution", "output_bound", "metrics", "origin", "tls_evidence", "framing", "body", "mime", "worker_identity", "rules", "files", "definition", "runtime", "collector", "guard", "worker_scan", "engine", "argv", "parent", "starttime", "queue", "version", "rules_read", "mode", "plan_store", "context", "queue_present_or_unreadable", "nft_read", "nft_present", "baseline", "prior_cleanup_veto", "worker_scan_read", "worker_scan_readlink", "worker_scan_foreign"}
|
||||
if int(r) >= len(names) {
|
||||
return names[0]
|
||||
}
|
||||
return names[r]
|
||||
}
|
||||
|
||||
type qualificationDiagnostic struct {
|
||||
Err error
|
||||
Stage qualificationStage
|
||||
Reason qualificationReason
|
||||
Candidate, Resource, Attempt int
|
||||
TLS uint8
|
||||
Errno qualificationErrno
|
||||
ScanPID, OwnedPID int
|
||||
}
|
||||
|
||||
type qualificationErrno uint8
|
||||
|
||||
const (
|
||||
errnoNone qualificationErrno = iota
|
||||
errnoEACCES
|
||||
errnoEPERM
|
||||
errnoESRCH
|
||||
errnoENOENT
|
||||
errnoOther
|
||||
)
|
||||
|
||||
func (e qualificationErrno) String() string {
|
||||
names := [...]string{"none", "eacces", "eperm", "esrch", "enoent", "other"}
|
||||
if int(e) >= len(names) {
|
||||
return "other"
|
||||
}
|
||||
return names[e]
|
||||
}
|
||||
|
||||
func qualificationScanErrno(err error) qualificationErrno {
|
||||
for _, item := range []struct {
|
||||
err error
|
||||
code qualificationErrno
|
||||
}{{syscall.EACCES, errnoEACCES}, {syscall.EPERM, errnoEPERM}, {syscall.ESRCH, errnoESRCH}, {syscall.ENOENT, errnoENOENT}} {
|
||||
if errors.Is(err, item.err) {
|
||||
return item.code
|
||||
}
|
||||
}
|
||||
if err == nil {
|
||||
return errnoNone
|
||||
}
|
||||
return errnoOther
|
||||
}
|
||||
|
||||
func workerScanReason(reason qualificationReason) bool {
|
||||
return reason == reasonWorkerScanRead || reason == reasonWorkerScanReadlink || reason == reasonWorkerScanForeign
|
||||
}
|
||||
|
||||
// Normalize classification without retaining private predicate errors.
|
||||
func qualificationPlanPredicate(err error, fallback qualificationReason) error {
|
||||
d := &qualificationDiagnostic{Err: errResourcePlan, Reason: qualificationErrorReason(err, fallback)}
|
||||
copyQualificationScanMetadata(d, err)
|
||||
return d
|
||||
}
|
||||
|
||||
func copyQualificationScanMetadata(d *qualificationDiagnostic, err error) {
|
||||
var source *qualificationDiagnostic
|
||||
if workerScanReason(d.Reason) && errors.As(err, &source) {
|
||||
d.Errno, d.ScanPID, d.OwnedPID = source.Errno, source.ScanPID, source.OwnedPID
|
||||
}
|
||||
}
|
||||
|
||||
func (e *qualificationDiagnostic) Error() string { return "qualification operation failed" }
|
||||
func (e *qualificationDiagnostic) Unwrap() error { return e.Err }
|
||||
|
||||
func qualificationOperation(err error, stage qualificationStage) error {
|
||||
if err == nil {
|
||||
return nil
|
||||
}
|
||||
d := &qualificationDiagnostic{Err: err, Stage: stage, Reason: qualificationErrorReason(err, reasonUnknown)}
|
||||
copyQualificationScanMetadata(d, err)
|
||||
return d
|
||||
}
|
||||
|
||||
func qualificationPredicate(err error, reason qualificationReason) error {
|
||||
d := &qualificationDiagnostic{Err: err, Reason: reason}
|
||||
copyQualificationScanMetadata(d, err)
|
||||
return d
|
||||
}
|
||||
|
||||
func qualificationErrorReason(err error, fallback qualificationReason) qualificationReason {
|
||||
var diagnostic *qualificationDiagnostic
|
||||
if errors.As(err, &diagnostic) && diagnostic.Reason != reasonUnknown {
|
||||
return diagnostic.Reason
|
||||
}
|
||||
return fallback
|
||||
}
|
||||
|
||||
func attestationMismatch(a, b qualificationAttestation) qualificationReason {
|
||||
switch {
|
||||
case a.PID != b.PID || a.StartTime != b.StartTime || a.WorkerArgsHash != b.WorkerArgsHash || a.Queue != b.Queue:
|
||||
return reasonWorkerIdentity
|
||||
case a.RulesHash != b.RulesHash:
|
||||
return reasonRules
|
||||
case a.LuaHash != b.LuaHash:
|
||||
return reasonFiles
|
||||
case a.Definition != b.Definition:
|
||||
return reasonDefinition
|
||||
case a.Runtime != b.Runtime:
|
||||
return reasonRuntime
|
||||
case a.Collector != b.Collector:
|
||||
return reasonCollector
|
||||
default:
|
||||
return reasonUnknown
|
||||
}
|
||||
}
|
||||
|
||||
// Traverse only for reporting, never for classification. Bound even pathological
|
||||
// error trees; output contains enums and ordinals, never Error() or type names.
|
||||
func writeQualificationDiagnostics(w io.Writer, err error) {
|
||||
budget := 32
|
||||
var visit func(error, qualificationDiagnostic, bool)
|
||||
visit = func(err error, inherited qualificationDiagnostic, cleanup bool) {
|
||||
if err == nil || budget == 0 {
|
||||
return
|
||||
}
|
||||
budget--
|
||||
switch e := err.(type) {
|
||||
case *qualificationDiagnostic:
|
||||
if e.Candidate != 0 {
|
||||
inherited.Candidate = e.Candidate
|
||||
}
|
||||
if e.Resource != 0 {
|
||||
inherited.Resource = e.Resource
|
||||
}
|
||||
if e.Attempt != 0 {
|
||||
inherited.Attempt = e.Attempt
|
||||
}
|
||||
if e.TLS != 0 {
|
||||
inherited.TLS = e.TLS
|
||||
}
|
||||
if e.Stage != stageUnknown && (e.Stage != stageAttestation || inherited.Stage == stageUnknown) {
|
||||
inherited.Stage = e.Stage
|
||||
}
|
||||
if e.Reason != reasonUnknown {
|
||||
inherited.Reason = e.Reason
|
||||
}
|
||||
if workerScanReason(e.Reason) {
|
||||
inherited.Errno, inherited.ScanPID, inherited.OwnedPID = e.Errno, e.ScanPID, e.OwnedPID
|
||||
}
|
||||
cleanup = cleanup || e.Stage == stageNFTDelete || e.Stage == stageStopJoin || e.Stage == stageRestore
|
||||
visit(e.Err, inherited, cleanup)
|
||||
return
|
||||
case *resourceError:
|
||||
inherited.Reason = e.reason
|
||||
case interface{ Unwrap() []error }:
|
||||
for _, child := range e.Unwrap() {
|
||||
visit(child, inherited, cleanup)
|
||||
}
|
||||
return
|
||||
}
|
||||
if e, ok := err.(interface{ Unwrap() error }); ok {
|
||||
visit(e.Unwrap(), inherited, cleanup)
|
||||
return
|
||||
}
|
||||
role, tls := "primary", "none"
|
||||
if cleanup {
|
||||
role = "cleanup"
|
||||
}
|
||||
if inherited.TLS == 12 {
|
||||
tls = "tls12"
|
||||
} else if inherited.TLS == 13 {
|
||||
tls = "tls13"
|
||||
}
|
||||
fmt.Fprintf(w, "qualification diagnostic: role=%s stage=%s reason=%s candidate=%d resource=%d attempt=%d tls=%s", role, inherited.Stage, inherited.Reason, inherited.Candidate, inherited.Resource, inherited.Attempt, tls)
|
||||
if workerScanReason(inherited.Reason) {
|
||||
fmt.Fprintf(w, " errno=%s scan_pid=%d owned_pid=%d", inherited.Errno, inherited.ScanPID, inherited.OwnedPID)
|
||||
}
|
||||
fmt.Fprintln(w)
|
||||
}
|
||||
visit(err, qualificationDiagnostic{}, false)
|
||||
}
|
||||
@@ -0,0 +1,239 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"io"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestQualificationDiagnosticStagesAndPrivacy(t *testing.T) {
|
||||
secret := errors.New("PRIVATE https://secret.example/path Authorization IP env")
|
||||
for _, mode := range []string{"transfer", "next_guard", "post_guard", "attestation", "metrics", "mime", "cleanup"} {
|
||||
t.Run(mode, func(t *testing.T) {
|
||||
q, s, registry, _, calls := qualifierFixture(t)
|
||||
original := q.checks.runner
|
||||
q.checks.runner = func(ctx context.Context, args []string, in io.Reader, out, diag io.Writer) error {
|
||||
if *calls == 3 && (mode == "transfer" || mode == "cleanup") {
|
||||
return secret
|
||||
}
|
||||
var wire bytes.Buffer
|
||||
err := original(ctx, args, in, &wire, diag)
|
||||
if *calls == 4 {
|
||||
if mode == "metrics" {
|
||||
wire.WriteString("PRIVATE")
|
||||
}
|
||||
if mode == "mime" {
|
||||
wire = *bytes.NewBufferString(strings.Replace(wire.String(), "text/plain", "text/plain; PRIVATE", 1))
|
||||
}
|
||||
}
|
||||
_, _ = out.Write(wire.Bytes())
|
||||
return err
|
||||
}
|
||||
guards := 0
|
||||
q.checks.guard = func() error {
|
||||
guards++
|
||||
if (mode == "next_guard" && guards == 9) || (mode == "post_guard" && guards == 8) {
|
||||
return secret
|
||||
}
|
||||
return nil
|
||||
}
|
||||
attest := q.attest
|
||||
q.attest = func(ctx context.Context, pid int, args []string, d strategyDefinition) (qualificationAttestation, error) {
|
||||
a, err := attest(ctx, pid, args, d)
|
||||
if mode == "attestation" && *calls == 4 {
|
||||
a.RulesHash = strings.Repeat("b", 64)
|
||||
}
|
||||
return a, err
|
||||
}
|
||||
if mode == "cleanup" {
|
||||
q.restore = func(context.Context) error { return secret }
|
||||
}
|
||||
_, err := q.complete(context.Background(), s, registry)
|
||||
if err == nil {
|
||||
t.Fatal("failure accepted")
|
||||
}
|
||||
var diagnostic bytes.Buffer
|
||||
writeQualificationDiagnostics(&diagnostic, err)
|
||||
got := diagnostic.String()
|
||||
if strings.Contains(got, "PRIVATE") || strings.Contains(got, "example") || strings.Contains(got, "SECRET") {
|
||||
t.Fatal(got)
|
||||
}
|
||||
want := map[string]string{"transfer": "stage=transfer reason=execution candidate=1 resource=1 attempt=1 tls=tls13", "cleanup": "stage=transfer reason=execution candidate=1 resource=1 attempt=1 tls=tls13", "next_guard": "stage=pre_dns_guard reason=unknown candidate=1 resource=1 attempt=2 tls=tls13", "post_guard": "stage=post_dns_guard reason=unknown candidate=1 resource=1 attempt=1 tls=tls13", "attestation": "stage=pre_dns_guard reason=rules", "metrics": "stage=transfer reason=metrics", "mime": "stage=transfer reason=mime"}[mode]
|
||||
if !strings.Contains(got, want) {
|
||||
t.Fatalf("want %s, got %s", want, got)
|
||||
}
|
||||
if mode == "cleanup" {
|
||||
if !strings.Contains(got, "role=cleanup stage=restore") || qualificationCleanupVerified(err, func(context.Context) error { t.Fatal("cleanup retry"); return nil }) {
|
||||
t.Fatal(got)
|
||||
}
|
||||
}
|
||||
if mode == "transfer" && !errors.Is(err, secret) {
|
||||
t.Fatal("lost original error")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestQualificationDiagnosticsDoNotChangeNegatives(t *testing.T) {
|
||||
for _, mode := range []string{"status500", "mime_mismatch"} {
|
||||
t.Run(mode, func(t *testing.T) {
|
||||
q, s, registry, _, calls := qualifierFixture(t)
|
||||
if mode == "mime_mismatch" {
|
||||
source, err := q.source()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for i := range source.config.Services {
|
||||
for j := range source.config.Services[i].Resources {
|
||||
source.config.Services[i].Resources[j].Expect.ContentType = "image/png"
|
||||
}
|
||||
}
|
||||
q.source = func() (resourceMonitorSource, error) { return source, nil }
|
||||
} else {
|
||||
original := q.checks.runner
|
||||
q.checks.runner = func(ctx context.Context, args []string, in io.Reader, out, diag io.Writer) error {
|
||||
var wire bytes.Buffer
|
||||
err := original(ctx, args, in, &wire, diag)
|
||||
_, _ = io.WriteString(out, strings.ReplaceAll(strings.ReplaceAll(wire.String(), "200 OK", "500 OK"), `"http_code":200`, `"http_code":500`))
|
||||
return err
|
||||
}
|
||||
}
|
||||
proof, err := q.complete(context.Background(), s, registry)
|
||||
if err != nil || *calls != 12 {
|
||||
t.Fatalf("%d %v", *calls, err)
|
||||
}
|
||||
for _, check := range proof.matrix.Runs[0].Checks {
|
||||
if check.Outcome != "expectation_failed" || check.Attempts != 3 {
|
||||
t.Fatal(check)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestQualificationDiagnosticVetoAndFinalStages(t *testing.T) {
|
||||
secret := errors.New("PRIVATE sentinel")
|
||||
for _, stage := range []qualificationStage{stageFinalStoreGuard, stageACKWrite, stageNFTDelete, stageStopJoin, stageWorkerQueue, stageWorkerExit} {
|
||||
err := qualificationOperation(secret, stage)
|
||||
var out bytes.Buffer
|
||||
writeQualificationDiagnostics(&out, err)
|
||||
if !strings.Contains(out.String(), "stage="+stage.String()) || strings.Contains(out.String(), "PRIVATE") || !errors.Is(err, secret) {
|
||||
t.Fatal(out.String())
|
||||
}
|
||||
}
|
||||
cancel := qualificationOperation(context.Canceled, stagePreDNSGuard)
|
||||
if !managedCancellationOnly(cancel, context.Canceled) {
|
||||
t.Fatal("pure cancellation changed")
|
||||
}
|
||||
for _, veto := range []error{secret, &managedRecoveryFatal{Err: context.Canceled}, &managedAssetCleanupFailure{Err: secret}} {
|
||||
if managedCancellationOnly(errors.Join(cancel, veto), context.Canceled) {
|
||||
t.Fatal("mixed veto lost")
|
||||
}
|
||||
}
|
||||
if resourceRejectionOnly(qualificationOperation(&resourcePeerTLSRejection{error: secret}, stageTransfer)) {
|
||||
t.Fatal("arbitrary rejection unwrapping added")
|
||||
}
|
||||
}
|
||||
|
||||
func TestQualificationFinalizeDiagnosticsAndACKGate(t *testing.T) {
|
||||
secret := errors.New("PRIVATE sentinel")
|
||||
for _, mode := range []string{"cleanup", "owned_cleanup", "restore", "guard", "ack", "success"} {
|
||||
t.Run(mode, func(t *testing.T) {
|
||||
var primary error
|
||||
if mode == "cleanup" {
|
||||
primary = &qualificationCleanupFailure{Err: secret}
|
||||
}
|
||||
if mode == "owned_cleanup" {
|
||||
primary = errors.Join(context.Canceled, &managedRecoveryFatal{Err: &managedAssetCleanupFailure{Err: secret}})
|
||||
}
|
||||
restores, guards, acks := 0, 0, 0
|
||||
err := qualificationFinalize(primary, func(context.Context) error {
|
||||
restores++
|
||||
if mode == "restore" {
|
||||
return secret
|
||||
}
|
||||
return nil
|
||||
}, func() error {
|
||||
guards++
|
||||
if mode == "guard" {
|
||||
return secret
|
||||
}
|
||||
return nil
|
||||
}, func() error {
|
||||
acks++
|
||||
if mode == "ack" {
|
||||
return secret
|
||||
}
|
||||
return nil
|
||||
})
|
||||
if mode == "success" {
|
||||
if err != nil || acks != 1 {
|
||||
t.Fatal(err, acks)
|
||||
}
|
||||
return
|
||||
}
|
||||
var out bytes.Buffer
|
||||
writeQualificationDiagnostics(&out, err)
|
||||
stage := stageRestore
|
||||
if mode == "guard" {
|
||||
stage = stageFinalStoreGuard
|
||||
}
|
||||
if mode == "ack" {
|
||||
stage = stageACKWrite
|
||||
}
|
||||
if !strings.Contains(out.String(), "stage="+stage.String()) || strings.Contains(out.String(), "PRIVATE") {
|
||||
t.Fatal(out.String())
|
||||
}
|
||||
if mode != "ack" && acks != 0 {
|
||||
t.Fatal("ACK passed failed gate")
|
||||
}
|
||||
if mode == "cleanup" || mode == "owned_cleanup" {
|
||||
if restores != 0 || guards != 0 || acks != 0 || !strings.Contains(out.String(), "reason=prior_cleanup_veto") {
|
||||
t.Fatal("cleanup failure retried or misdiagnosed", out.String())
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestQualificationOpaqueAttestationReasonUnderPostDNSGuard(t *testing.T) {
|
||||
secret := errors.New("PRIVATE attestation path env argv")
|
||||
q, s, registry, _, transfers := qualifierFixture(t)
|
||||
attest, calls := q.attest, 0
|
||||
q.attest = func(ctx context.Context, pid int, args []string, d strategyDefinition) (qualificationAttestation, error) {
|
||||
calls++
|
||||
if calls == 3 { // Initial attestation, pre-DNS, then post-DNS.
|
||||
return qualificationAttestation{}, qualificationPredicate(secret, reasonWorkerScan)
|
||||
}
|
||||
return attest(ctx, pid, args, d)
|
||||
}
|
||||
proof, err := q.complete(context.Background(), s, registry)
|
||||
if !errors.Is(err, errResourcePlan) || !errors.Is(err, secret) || proof.seal != "" || *transfers != 0 {
|
||||
t.Fatal("ownership veto or execution order changed", err, *transfers)
|
||||
}
|
||||
var out bytes.Buffer
|
||||
writeQualificationDiagnostics(&out, err)
|
||||
if strings.Contains(out.String(), "PRIVATE") || strings.Contains(out.String(), "reason=unknown") ||
|
||||
strings.Count(out.String(), "stage=post_dns_guard reason=worker_scan candidate=1 resource=1 attempt=1 tls=tls12") != 2 {
|
||||
t.Fatal(out.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestQualificationAttestationErrorRetainsOwnershipVeto(t *testing.T) {
|
||||
q, s, registry, _, _ := qualifierFixture(t)
|
||||
attest, calls := q.attest, 0
|
||||
q.attest = func(ctx context.Context, pid int, args []string, d strategyDefinition) (qualificationAttestation, error) {
|
||||
calls++
|
||||
if calls == 2 {
|
||||
return qualificationAttestation{}, context.Canceled
|
||||
}
|
||||
return attest(ctx, pid, args, d)
|
||||
}
|
||||
_, err := q.complete(context.Background(), s, registry)
|
||||
if !errors.Is(err, context.Canceled) || !errors.Is(err, errResourcePlan) || managedCancellationOnly(err, context.Canceled) {
|
||||
t.Fatal("attestation ownership veto changed", err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,198 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"go/ast"
|
||||
"go/format"
|
||||
"go/parser"
|
||||
"go/token"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestQualificationRestorePredicatesAndOrder(t *testing.T) {
|
||||
secret := errors.New("PRIVATE nft process baseline sentinel")
|
||||
for _, reason := range []qualificationReason{reasonContext, reasonGuard, reasonMode, reasonPlanStore, reasonRuntime, reasonWorkerScan, reasonQueuePresentOrUnreadable, reasonNFTRead, reasonNFTPresent, reasonBaseline, reasonUnknown} {
|
||||
t.Run(reason.String(), func(t *testing.T) {
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
if reason == reasonContext {
|
||||
cancel()
|
||||
}
|
||||
var order []string
|
||||
q := resourceQualifier{guard: func() error {
|
||||
order = append(order, "guard")
|
||||
if reason == reasonGuard {
|
||||
return secret
|
||||
}
|
||||
if reason == reasonMode || reason == reasonPlanStore || reason == reasonRuntime {
|
||||
return qualificationPredicate(secret, reason)
|
||||
}
|
||||
return nil
|
||||
}}
|
||||
q.runtime.nft = func(_ context.Context, input string) (string, error) {
|
||||
order = append(order, "nft")
|
||||
if input != "list tables\n" {
|
||||
t.Fatal(input)
|
||||
}
|
||||
if reason == reasonNFTRead {
|
||||
return "", secret
|
||||
}
|
||||
if reason == reasonNFTPresent {
|
||||
return "table inet " + managedAssetTable, nil
|
||||
}
|
||||
return "", nil
|
||||
}
|
||||
err := q.restoreBaseline(ctx, func() error {
|
||||
order = append(order, "worker_queue")
|
||||
if reason == reasonWorkerScan || reason == reasonQueuePresentOrUnreadable {
|
||||
return qualificationPredicate(secret, reason)
|
||||
}
|
||||
return nil
|
||||
}, func() error {
|
||||
order = append(order, "baseline")
|
||||
if reason == reasonBaseline {
|
||||
return errResourcePlan
|
||||
}
|
||||
return nil
|
||||
})
|
||||
want := []string{"guard", "worker_queue", "nft", "baseline"}
|
||||
switch reason {
|
||||
case reasonContext:
|
||||
want = nil
|
||||
case reasonGuard, reasonMode, reasonPlanStore, reasonRuntime:
|
||||
want = want[:1]
|
||||
case reasonWorkerScan, reasonQueuePresentOrUnreadable:
|
||||
want = want[:2]
|
||||
case reasonNFTRead, reasonNFTPresent:
|
||||
want = want[:3]
|
||||
}
|
||||
if !reflect.DeepEqual(order, want) {
|
||||
t.Fatal(order, want)
|
||||
}
|
||||
if reason == reasonUnknown {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return
|
||||
}
|
||||
if !errors.Is(err, errResourcePlan) || errors.Is(err, secret) {
|
||||
t.Fatal("original restore classification changed", err)
|
||||
}
|
||||
var out bytes.Buffer
|
||||
writeQualificationDiagnostics(&out, qualificationOperation(err, stageRestore))
|
||||
if !strings.Contains(out.String(), "role=cleanup stage=restore reason="+reason.String()) || strings.Contains(out.String(), "PRIVATE") {
|
||||
t.Fatal(out.String())
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Check the actual production factory, not just injected diagnostic errors. This
|
||||
// pins opaque predicate order/strictness without starting workers or changing nft.
|
||||
func TestQualificationProductionPredicateMap(t *testing.T) {
|
||||
fset := token.NewFileSet()
|
||||
f, err := parser.ParseFile(fset, "resource_qualify.go", nil, 0)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
text := func(n ast.Node) string {
|
||||
var b bytes.Buffer
|
||||
if err := format.Node(&b, fset, n); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return b.String()
|
||||
}
|
||||
var attest *ast.FuncLit
|
||||
var restore, guard, noWorker, onlyWorker, scanWorkers string
|
||||
ast.Inspect(f, func(n ast.Node) bool {
|
||||
if fn, ok := n.(*ast.FuncDecl); ok {
|
||||
switch fn.Name.Name {
|
||||
case "qualificationNoWorker":
|
||||
noWorker = text(fn.Body)
|
||||
case "qualificationOnlyWorker":
|
||||
onlyWorker = text(fn.Body)
|
||||
case "qualificationScanWorkers":
|
||||
scanWorkers = text(fn.Body)
|
||||
}
|
||||
}
|
||||
a, ok := n.(*ast.AssignStmt)
|
||||
if !ok || len(a.Lhs) != 1 || len(a.Rhs) != 1 {
|
||||
return true
|
||||
}
|
||||
if text(a.Lhs[0]) == "q.attest" {
|
||||
attest, _ = a.Rhs[0].(*ast.FuncLit)
|
||||
}
|
||||
if text(a.Lhs[0]) == "q.restore" {
|
||||
restore = text(a.Rhs[0])
|
||||
}
|
||||
if text(a.Lhs[0]) == "q.guard" && guard == "" {
|
||||
guard = text(a.Rhs[0])
|
||||
}
|
||||
return true
|
||||
})
|
||||
if attest == nil || !strings.Contains(restore, "q.restoreBaseline(ctx, qualificationNoWorker, func() error { return qualificationBaseline(d.directory) })") {
|
||||
t.Fatal("production restore wiring changed", restore)
|
||||
}
|
||||
for _, check := range []string{
|
||||
"if e := qualificationMode(config, strategies, d); e != nil {\n\t\treturn qualificationPredicate(errResourcePlan, qualificationErrorReason(e, reasonMode))",
|
||||
"if e := s.guard(); e != nil {\n\t\treturn qualificationPredicate(errResourcePlan, qualificationErrorReason(e, reasonPlanStore))",
|
||||
"if e := r.guard(); e != nil {\n\t\treturn qualificationPredicate(e, qualificationErrorReason(e, reasonRuntime))",
|
||||
} {
|
||||
if !strings.Contains(guard, check) {
|
||||
t.Fatal("production guard predicate changed", guard)
|
||||
}
|
||||
}
|
||||
if !strings.Contains(noWorker, "qualificationOnlyWorker(0)") || !strings.Contains(noWorker, "qualificationPredicate(e, qualificationErrorReason(e, reasonWorkerScan))") ||
|
||||
!strings.Contains(noWorker, "managedAssetQueueReady(0)") || !strings.Contains(noWorker, "e != nil || occupied") || !strings.Contains(noWorker, "qualificationPredicate(errResourcePlan, reasonQueuePresentOrUnreadable)") {
|
||||
t.Fatal("production worker/queue predicate changed", noWorker)
|
||||
}
|
||||
if !strings.Contains(onlyWorker, "qualificationScanWorkers(owned, os.ReadDir, os.Readlink)") ||
|
||||
!strings.Contains(scanWorkers, "if errors.Is(e, os.ErrNotExist) {\n\t\t\t\tcontinue\n\t\t\t}\n\t\t\treturn &qualificationDiagnostic{Err: errResourcePlan, Reason: reasonWorkerScanReadlink") {
|
||||
t.Fatal("unexpected procfs readlink errors must remain fail-closed", onlyWorker, scanWorkers)
|
||||
}
|
||||
var got []string
|
||||
for _, stmt := range attest.Body.List {
|
||||
branch, ok := stmt.(*ast.IfStmt)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
predicate := text(branch.Cond)
|
||||
if branch.Init != nil {
|
||||
predicate = text(branch.Init) + "; " + predicate
|
||||
}
|
||||
ret, ok := branch.Body.List[0].(*ast.ReturnStmt)
|
||||
if !ok || len(ret.Results) != 2 {
|
||||
t.Fatal("unexpected attestation branch", text(branch))
|
||||
}
|
||||
got = append(got, predicate+" => "+text(ret.Results[1]))
|
||||
}
|
||||
want := []string{
|
||||
"e := q.guard(); e != nil => qualificationPredicate(errResourcePlan, qualificationErrorReason(e, reasonGuard))",
|
||||
"e := qualificationOnlyWorker(pid); e != nil => qualificationPlanPredicate(e, reasonWorkerScan)",
|
||||
`engine == nil || engine.validate() != nil || engine.running(fmt.Sprintf("/proc/%d/exe", pid)) != nil => qualificationPredicate(errResourcePlan, reasonEngine)`,
|
||||
`e != nil || string(cmd) != strings.Join(append([]string{qualificationEngine}, args...), "\x00")+"\x00" => qualificationPredicate(errResourcePlan, reasonArgv)`,
|
||||
"e != nil => qualificationPredicate(e, reasonParent)",
|
||||
"end < 0 => qualificationPredicate(errResourcePlan, reasonParent)",
|
||||
"len(tail) < 2 || tail[1] != strconv.Itoa(os.Getpid()) => qualificationPredicate(errResourcePlan, reasonParent)",
|
||||
"e != nil => qualificationPredicate(e, reasonStarttime)",
|
||||
"e != nil || !ready => qualificationPredicate(errResourcePlan, reasonQueue)",
|
||||
"e != nil => qualificationPredicate(e, reasonFiles)",
|
||||
"version.Run() != nil => qualificationPredicate(errResourcePlan, reasonVersion)",
|
||||
"!valid => qualificationPredicate(errResourcePlan, reasonVersion)",
|
||||
`e != nil || strings.TrimSpace(rules) == "" => qualificationPredicate(errResourcePlan, reasonRulesRead)`,
|
||||
}
|
||||
if !reflect.DeepEqual(got, want) {
|
||||
t.Fatalf("production attestation predicates:\n%v\nwant:\n%v", got, want)
|
||||
}
|
||||
// The factory's strict mode check must precede any store, worker or nft access.
|
||||
q := newResourceQualifier("", "", managedDeps{lookup: func(string) (string, bool) { return "", false }}, nil)
|
||||
defer q.close()
|
||||
_, err = q.attest(context.Background(), 0, nil, strategyDefinition{})
|
||||
if !errors.Is(err, errResourcePlan) || qualificationErrorReason(err, reasonUnknown) != reasonMode {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,168 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"syscall"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestQualificationWorkerScanBranches(t *testing.T) {
|
||||
secret := errors.New("PRIVATE path argv propertyID sentinel")
|
||||
dir := t.TempDir()
|
||||
for _, name := range []string{"0-nonnumeric", "41"} {
|
||||
if err := os.Mkdir(filepath.Join(dir, name), 0700); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
entries, err := os.ReadDir(dir)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
err error
|
||||
code qualificationErrno
|
||||
}{
|
||||
{"eacces", syscall.EACCES, errnoEACCES},
|
||||
{"eperm", syscall.EPERM, errnoEPERM},
|
||||
{"esrch", syscall.ESRCH, errnoESRCH},
|
||||
{"enoent", syscall.ENOENT, errnoENOENT},
|
||||
{"not_exist", os.ErrNotExist, errnoOther},
|
||||
{"other", secret, errnoOther},
|
||||
} {
|
||||
for _, readDirFailure := range []bool{false, true} {
|
||||
t.Run(fmt.Sprintf("%s/readdir=%t", tc.name, readDirFailure), func(t *testing.T) {
|
||||
wrapped := &os.PathError{Op: "PRIVATE", Path: secret.Error(), Err: errors.Join(tc.err, secret)}
|
||||
calls := 0
|
||||
err := qualificationScanWorkers(42, func(path string) ([]os.DirEntry, error) {
|
||||
if path != "/proc" {
|
||||
t.Fatal(path)
|
||||
}
|
||||
if readDirFailure {
|
||||
return nil, wrapped
|
||||
}
|
||||
return entries, nil
|
||||
}, func(path string) (string, error) {
|
||||
calls++
|
||||
if path != "/proc/41/exe" {
|
||||
t.Fatal("nonnumeric entry scanned", path)
|
||||
}
|
||||
return "", wrapped
|
||||
})
|
||||
if !readDirFailure && errors.Is(wrapped, os.ErrNotExist) {
|
||||
if err != nil || calls != 1 {
|
||||
t.Fatal("wrapped ENOENT must skip", err, calls)
|
||||
}
|
||||
return
|
||||
}
|
||||
var d *qualificationDiagnostic
|
||||
if !errors.As(err, &d) || d.Errno != tc.code || d.OwnedPID != 42 {
|
||||
t.Fatal("missing typed metadata", err)
|
||||
}
|
||||
if readDirFailure {
|
||||
if d.Reason != reasonWorkerScanRead || d.ScanPID != 0 || calls != 0 || !errors.Is(err, wrapped) {
|
||||
t.Fatal("readdir classification changed", d, calls)
|
||||
}
|
||||
} else if d.Reason != reasonWorkerScanReadlink || d.ScanPID != 41 || !errors.Is(err, errResourcePlan) || errors.Is(err, wrapped) {
|
||||
t.Fatal("readlink classification changed", d)
|
||||
}
|
||||
assertWorkerScanOutput(t, err, stageUnknown, d.Reason, tc.code, d.ScanPID, 42)
|
||||
})
|
||||
}
|
||||
}
|
||||
for _, tc := range []struct {
|
||||
name, exe string
|
||||
owned int
|
||||
deny bool
|
||||
}{
|
||||
{"owned", "/PRIVATE/nfqws2", 41, false},
|
||||
{"foreign", "/PRIVATE/nfqws2", 42, true},
|
||||
// A foreign --version invocation has the same exe: never inspect argv
|
||||
// or exempt it from the exclusive-worker predicate.
|
||||
{"foreign_version", "/PRIVATE/nfqws2", 0, true},
|
||||
{"unrelated", "/PRIVATE/other", 42, false},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
err := qualificationScanWorkers(tc.owned, func(string) ([]os.DirEntry, error) { return entries, nil }, func(string) (string, error) { return tc.exe, nil })
|
||||
if !tc.deny {
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return
|
||||
}
|
||||
if !errors.Is(err, errResourcePlan) {
|
||||
t.Fatal("foreign worker accepted", err)
|
||||
}
|
||||
assertWorkerScanOutput(t, err, stageUnknown, reasonWorkerScanForeign, errnoNone, 41, tc.owned)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func assertWorkerScanOutput(t *testing.T, err error, stage qualificationStage, reason qualificationReason, errno qualificationErrno, scan, owned int) {
|
||||
t.Helper()
|
||||
var out bytes.Buffer
|
||||
writeQualificationDiagnostics(&out, err)
|
||||
want := fmt.Sprintf(" errno=%s scan_pid=%d owned_pid=%d", errno, scan, owned)
|
||||
if !strings.Contains(out.String(), "stage="+stage.String()+" reason="+reason.String()) || !strings.Contains(out.String(), want) || strings.Contains(out.String(), "PRIVATE") {
|
||||
t.Fatal(out.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestQualificationWorkerScanNormalization(t *testing.T) {
|
||||
secret := errors.New("PRIVATE path argv propertyID sentinel")
|
||||
for _, reason := range []qualificationReason{reasonWorkerScanRead, reasonWorkerScanReadlink, reasonWorkerScanForeign} {
|
||||
for _, errno := range []qualificationErrno{errnoNone, errnoEACCES, errnoEPERM, errnoESRCH, errnoENOENT, errnoOther} {
|
||||
scan := &qualificationDiagnostic{Err: secret, Reason: reason, Errno: errno, ScanPID: 41, OwnedPID: 42}
|
||||
attestation := qualificationPlanPredicate(qualificationOperation(scan, stageAttestation), reasonWorkerScan)
|
||||
if !errors.Is(attestation, errResourcePlan) || errors.Is(attestation, secret) {
|
||||
t.Fatal("attestation classification changed", attestation)
|
||||
}
|
||||
assertWorkerScanOutput(t, qualificationOperation(attestation, stagePostDNSGuard), stagePostDNSGuard, reason, errno, 41, 42)
|
||||
q := resourceQualifier{guard: func() error { return nil }}
|
||||
q.runtime.nft = func(context.Context, string) (string, error) {
|
||||
t.Fatal("restore passed scan veto")
|
||||
return "", nil
|
||||
}
|
||||
restore := q.restoreBaseline(context.Background(), func() error {
|
||||
return qualificationPredicate(scan, qualificationErrorReason(scan, reasonWorkerScan))
|
||||
}, func() error { t.Fatal("unsafe baseline restore"); return nil })
|
||||
if !errors.Is(restore, errResourcePlan) || errors.Is(restore, secret) {
|
||||
t.Fatal("restore classification changed", restore)
|
||||
}
|
||||
assertWorkerScanOutput(t, qualificationOperation(restore, stageRestore), stageRestore, reason, errno, 41, 42)
|
||||
}
|
||||
}
|
||||
var out bytes.Buffer
|
||||
writeQualificationDiagnostics(&out, qualificationPredicate(secret, reasonEngine))
|
||||
if strings.Contains(out.String(), "errno=") || strings.Contains(out.String(), "_pid=") || strings.Contains(out.String(), "PRIVATE") {
|
||||
t.Fatal("scan metadata on non-scan diagnostic", out.String())
|
||||
}
|
||||
}
|
||||
|
||||
func TestQualificationWorkerScanPostDNSVeto(t *testing.T) {
|
||||
q, s, registry, _, transfers := qualifierFixture(t)
|
||||
attest, calls := q.attest, 0
|
||||
q.attest = func(ctx context.Context, pid int, args []string, d strategyDefinition) (qualificationAttestation, error) {
|
||||
calls++
|
||||
if calls == 3 {
|
||||
scan := &qualificationDiagnostic{Err: errors.New("PRIVATE path argv propertyID sentinel"), Reason: reasonWorkerScanReadlink, Errno: errnoEPERM, ScanPID: 41, OwnedPID: pid}
|
||||
return qualificationAttestation{}, qualificationPlanPredicate(scan, reasonWorkerScan)
|
||||
}
|
||||
return attest(ctx, pid, args, d)
|
||||
}
|
||||
proof, err := q.complete(context.Background(), s, registry)
|
||||
if !errors.Is(err, errResourcePlan) || proof.seal != "" || *transfers != 0 {
|
||||
t.Fatal("ownership veto or execution order changed", err, *transfers)
|
||||
}
|
||||
var out bytes.Buffer
|
||||
writeQualificationDiagnostics(&out, err)
|
||||
if strings.Count(out.String(), "stage=post_dns_guard reason=worker_scan_readlink candidate=1 resource=1 attempt=1 tls=tls12 errno=eperm scan_pid=41 owned_pid=") != 2 || strings.Contains(out.String(), "PRIVATE") {
|
||||
t.Fatal(out.String())
|
||||
}
|
||||
}
|
||||
@@ -66,7 +66,7 @@ func TestRecoverGooglevideo(t *testing.T) {
|
||||
var order, checked []string
|
||||
player := cachePlayer(t, &order)
|
||||
if mode == "discovery-error" {
|
||||
player = func(context.Context, []string, io.Writer, io.Writer) error {
|
||||
player = func(context.Context, []string, io.Reader, io.Writer, io.Writer) error {
|
||||
order = append(order, "discover")
|
||||
return fmt.Errorf("SECRET discovery failure")
|
||||
}
|
||||
|
||||
@@ -0,0 +1,468 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"database/sql"
|
||||
"fmt"
|
||||
"io"
|
||||
"net"
|
||||
"os"
|
||||
"os/signal"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"syscall"
|
||||
"time"
|
||||
)
|
||||
|
||||
// v4 tags each side independently; generic references never impersonate SQL IDs.
|
||||
type managedConfigRef struct {
|
||||
Kind string `json:"kind"`
|
||||
Registry string `json:"registry_hash,omitempty"`
|
||||
Matrix string `json:"matrix_hash,omitempty"`
|
||||
Source string `json:"source_fingerprint,omitempty"`
|
||||
Plan string `json:"plan_digest,omitempty"`
|
||||
Config string `json:"config_hash,omitempty"`
|
||||
Policy string `json:"strategy_ids,omitempty"`
|
||||
}
|
||||
|
||||
const resourceStrategyIDsEnv = "ZAPRET_AUTOTUNE_RESOURCE_STRATEGY_IDS"
|
||||
|
||||
// Policy is a canonical bounded allowlist, never qualification evidence.
|
||||
func parseResourceStrategyIDs(value string) (string, error) {
|
||||
if value == "" {
|
||||
return "", nil
|
||||
}
|
||||
if len(value) > 4096 {
|
||||
return "", errResourcePlan
|
||||
}
|
||||
ids := strings.Split(value, ",")
|
||||
if len(ids) > 32 {
|
||||
return "", errResourcePlan
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
for _, id := range ids {
|
||||
if !resourceID.MatchString(id) || seen[id] {
|
||||
return "", errResourcePlan
|
||||
}
|
||||
seen[id] = true
|
||||
}
|
||||
sort.Strings(ids)
|
||||
return strings.Join(ids, ","), nil
|
||||
}
|
||||
|
||||
func validManagedConfigRef(r *managedConfigRef, ids []int64, scope GooglevideoScope, hash string) bool {
|
||||
if r == nil {
|
||||
return false
|
||||
}
|
||||
switch r.Kind {
|
||||
case "generic":
|
||||
policy, e := parseResourceStrategyIDs(r.Policy)
|
||||
return e == nil && policy == r.Policy && len(ids) == 0 && scope == "" && hashOK(r.Registry) && hashOK(r.Matrix) && hashOK(r.Source) && hashOK(r.Plan) && r.Config == hash
|
||||
case "baseline", "legacy":
|
||||
return r.Policy == "" && r.Registry == "" && r.Matrix == "" && r.Source == "" && r.Plan == "" && r.Config == "" && (r.Kind == "legacy" && len(ids) > 0 || r.Kind == "baseline" && len(ids) == 0 && scope == "" && hash == managedHash([]byte(managedBaseline)))
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func (m *managedTransaction) validateConfigRef(db *sql.DB, ref *managedConfigRef, ids []int64, hash string, scope GooglevideoScope) error {
|
||||
if ref == nil {
|
||||
return validateManagedRefs(db, ids, hash, scope)
|
||||
}
|
||||
if !validManagedConfigRef(ref, ids, scope, hash) {
|
||||
return errResourcePlan
|
||||
}
|
||||
if ref.Kind != "generic" {
|
||||
return validateManagedRefs(db, ids, hash, scope)
|
||||
}
|
||||
if m.resource == nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
return m.resource.validate(ref, hash)
|
||||
}
|
||||
|
||||
type resourceApplySnapshot struct {
|
||||
store *resourcePlanStore
|
||||
source resourceMonitorSource
|
||||
registry strategyRegistry
|
||||
matrix qualificationMatrix
|
||||
ref managedConfigRef
|
||||
rendered []byte
|
||||
guard func() error
|
||||
now func() time.Time
|
||||
close func()
|
||||
}
|
||||
|
||||
// Historical reconstruction uses the proof's original completion horizon. Only
|
||||
// new apply checks wall-clock freshness; expiry cannot erase committed identity.
|
||||
func resourceApplyPlan(source resourceMonitorSource, key []byte, registry strategyRegistry, matrix qualificationMatrix, now time.Time, policies ...string) (resourcePlan, []byte, error) {
|
||||
policy := ""
|
||||
if len(policies) != 0 {
|
||||
policy = policies[0]
|
||||
}
|
||||
canonical, e := parseResourceStrategyIDs(policy)
|
||||
if e != nil || canonical != policy {
|
||||
return resourcePlan{}, nil, errResourcePlan
|
||||
}
|
||||
if policy != "" {
|
||||
for _, id := range strings.Split(policy, ",") {
|
||||
found := false
|
||||
for _, d := range registry.Strategies {
|
||||
found = found || d.ID == id
|
||||
}
|
||||
if !found {
|
||||
return resourcePlan{}, nil, errResourcePlan
|
||||
}
|
||||
}
|
||||
}
|
||||
p := deriveResourcePlan(source, key, registry, matrix, now, policy)
|
||||
if p.Evidence != "authenticated" || len(p.Unassigned) != 0 || len(p.Assignments) == 0 {
|
||||
return p, nil, errResourcePlan
|
||||
}
|
||||
for _, s := range source.config.Services {
|
||||
for _, r := range s.Resources {
|
||||
if r.TLS != "both" {
|
||||
return p, nil, errResourcePlan
|
||||
}
|
||||
}
|
||||
}
|
||||
profiles := []string{}
|
||||
seen := map[string]bool{}
|
||||
for _, a := range p.Assignments {
|
||||
d, e := canonicalStrategy(strategyDefinition{ID: a.Strategy, Args: a.Args, Selectors: a.Selectors})
|
||||
if e != nil || d.Selectors.TLS != "both" || len(a.Hosts) == 0 {
|
||||
return p, nil, errResourcePlan
|
||||
}
|
||||
hosts := []string{}
|
||||
for _, host := range a.Hosts {
|
||||
if !validDomain(host) || host != strings.ToLower(host) || net.ParseIP(host) != nil || seen[host] {
|
||||
return p, nil, errResourcePlan
|
||||
}
|
||||
seen[host] = true
|
||||
hosts = append(hosts, "^"+host)
|
||||
}
|
||||
profiles = append(profiles, "--filter-tcp=443 --filter-l7=tls --hostlist-domains="+strings.Join(hosts, ",")+" "+d.Args)
|
||||
}
|
||||
return p, []byte("# Autotuner-owned generic config; runtime acceptance pending.\nNFQWS2_ENABLE=1\n" + managedFirewallConfig + "NFQWS2_OPT='" + strings.Join(profiles, " --new ") + "'\n"), nil
|
||||
}
|
||||
|
||||
func (s *resourceApplySnapshot) fresh() error {
|
||||
p, b, e := resourceApplyPlan(s.source, s.store.files.key, s.registry, s.matrix, s.now(), s.ref.Policy)
|
||||
if e != nil || planDigest(p.Assignments) != s.ref.Plan || managedHash(b) != s.ref.Config {
|
||||
return errResourcePlan
|
||||
}
|
||||
return s.guard()
|
||||
}
|
||||
|
||||
// Each journal side reconstructs independently inside the same immutable boundary.
|
||||
func (s *resourceApplySnapshot) validate(ref *managedConfigRef, hash string) error {
|
||||
if ref == nil || ref.Kind != "generic" || ref.Registry != s.ref.Registry || ref.Matrix != s.ref.Matrix || ref.Source != s.ref.Source || hash != ref.Config {
|
||||
return errResourcePlan
|
||||
}
|
||||
p, b, e := s.historicalPlan(ref.Policy)
|
||||
if e != nil || planDigest(p.Assignments) != ref.Plan || managedHash(b) != ref.Config {
|
||||
return errResourcePlan
|
||||
}
|
||||
return s.guard()
|
||||
}
|
||||
|
||||
func (s *resourceApplySnapshot) historicalPlan(policy string) (resourcePlan, []byte, error) {
|
||||
var horizon int64
|
||||
for _, run := range s.matrix.Runs {
|
||||
if run.Completed > horizon {
|
||||
horizon = run.Completed
|
||||
}
|
||||
}
|
||||
return resourceApplyPlan(s.source, s.store.files.key, s.registry, s.matrix, time.UnixMilli(horizon), policy)
|
||||
}
|
||||
|
||||
func (s *resourceApplySnapshot) selectPolicy(policy string) error {
|
||||
p, b, e := s.historicalPlan(policy)
|
||||
if e != nil {
|
||||
return e
|
||||
}
|
||||
s.ref.Policy, s.ref.Plan, s.ref.Config = policy, planDigest(p.Assignments), managedHash(b)
|
||||
s.rendered = b
|
||||
return nil
|
||||
}
|
||||
|
||||
// No imported proposal or version probe. Keep private proof/source/native files
|
||||
// held across staging, health and commit and bind every selected candidate.
|
||||
func openResourceApplySnapshot(config string, d managedDeps) (*resourceApplySnapshot, error) {
|
||||
return openResourceApplySnapshotAt(config, d, "/opt/zapret2")
|
||||
}
|
||||
|
||||
func openResourceApplySnapshotAt(config string, d managedDeps, root string) (*resourceApplySnapshot, error) {
|
||||
store, e := openResourcePlanStore(d.directory, false)
|
||||
if e != nil {
|
||||
return nil, errResourcePlan
|
||||
}
|
||||
s := &resourceApplySnapshot{store: store, now: time.Now}
|
||||
var engine *qualificationBinary
|
||||
var sourceFile *os.File
|
||||
lua := map[string]*os.File{}
|
||||
s.close = func() {
|
||||
for _, f := range lua {
|
||||
f.Close()
|
||||
}
|
||||
if sourceFile != nil {
|
||||
sourceFile.Close()
|
||||
}
|
||||
if engine != nil {
|
||||
engine.file.Close()
|
||||
}
|
||||
store.close()
|
||||
}
|
||||
fail := func() (*resourceApplySnapshot, error) { s.close(); return nil, errResourcePlan }
|
||||
s.registry, e = store.registry()
|
||||
if e != nil {
|
||||
return fail()
|
||||
}
|
||||
s.matrix, e = store.matrix(s.registry)
|
||||
if e != nil {
|
||||
return fail()
|
||||
}
|
||||
marker, markerBytes, e := store.files.openEntry(managedMarkerName)
|
||||
if e != nil {
|
||||
return fail()
|
||||
}
|
||||
store.held[managedMarkerName], store.snapshots[managedMarkerName] = marker, markerBytes
|
||||
fd, e := syscall.Open(config, syscall.O_RDONLY|syscall.O_NOFOLLOW|syscall.O_NONBLOCK|syscall.O_CLOEXEC, 0)
|
||||
if e != nil {
|
||||
return fail()
|
||||
}
|
||||
sourceFile = os.NewFile(uintptr(fd), "resource source")
|
||||
if resourcePrivateEntry(sourceFile) != nil {
|
||||
return fail()
|
||||
}
|
||||
sourceBytes, e := io.ReadAll(io.LimitReader(sourceFile, 65537))
|
||||
if e != nil || len(sourceBytes) > 65536 {
|
||||
return fail()
|
||||
}
|
||||
s.source, e = readResourceMonitorSource(config, d.lookup, store.files.key)
|
||||
if e != nil {
|
||||
return fail()
|
||||
}
|
||||
engine, e = openQualificationEngine(root)
|
||||
if e != nil {
|
||||
return fail()
|
||||
}
|
||||
native, e := engine.filesHash()
|
||||
if e != nil {
|
||||
return fail()
|
||||
}
|
||||
// Hold Lua inodes as well as the qualification hash; equal-byte replacement
|
||||
// during a transaction is still a trust change.
|
||||
for _, name := range []string{"lua/zapret-lib.lua", "lua/zapret-antidpi.lua"} {
|
||||
path := engine.root + "/" + name
|
||||
f, e := os.Open(path)
|
||||
if e != nil {
|
||||
return fail()
|
||||
}
|
||||
lua[path] = f
|
||||
}
|
||||
p, rendered, e := s.historicalPlan("")
|
||||
if e != nil {
|
||||
return fail()
|
||||
}
|
||||
for _, run := range s.matrix.Runs {
|
||||
if run.Attestation.LuaHash != native {
|
||||
return fail()
|
||||
}
|
||||
}
|
||||
s.rendered = rendered
|
||||
s.ref = managedConfigRef{Kind: "generic", Registry: managedHash(store.snapshots[planRegistryName]), Matrix: managedHash(store.snapshots[planMatrixName]), Source: s.source.fingerprint, Plan: planDigest(p.Assignments), Config: managedHash(rendered)}
|
||||
s.guard = func() error {
|
||||
if store.guard() != nil || managedCheckIdentity(&managedTransaction{Dir: d.directory, Name: "config"}) != nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
current, e := os.Lstat(config)
|
||||
held, he := sourceFile.Stat()
|
||||
b, re := io.ReadAll(io.NewSectionReader(sourceFile, 0, 65537))
|
||||
fresh, se := readResourceMonitorSource(config, d.lookup, store.files.key)
|
||||
hash, ne := engine.filesHash()
|
||||
if e != nil || he != nil || re != nil || se != nil || ne != nil || !os.SameFile(current, held) || resourcePrivateEntry(sourceFile) != nil || !bytes.Equal(b, sourceBytes) || fresh.fingerprint != s.source.fingerprint || hash != native {
|
||||
return errResourcePlan
|
||||
}
|
||||
for path, f := range lua {
|
||||
current, e := os.Lstat(path)
|
||||
held, he := f.Stat()
|
||||
if e != nil || he != nil || !os.SameFile(current, held) {
|
||||
return errResourcePlan
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if s.guard() != nil {
|
||||
return fail()
|
||||
}
|
||||
return s, nil
|
||||
}
|
||||
|
||||
func resourceApplyHealth(ctx context.Context, s *resourceApplySnapshot, checks resourceCheckDeps) (string, error) {
|
||||
guard := checks.guard
|
||||
checks.guard = func() error {
|
||||
if s.guard() != nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
return guard()
|
||||
}
|
||||
passed := true
|
||||
for _, service := range s.source.config.Services {
|
||||
for _, spec := range service.Resources {
|
||||
for _, tls := range []string{"tls12", "tls13"} {
|
||||
result, e := runResourceProtocol(ctx, service.ID, spec, tls, s.source.server, s.source.proxy, checks)
|
||||
if e != nil || ctx.Err() != nil {
|
||||
return "DEGRADED", errResourcePlan
|
||||
}
|
||||
passed = passed && result.Outcome == "pass"
|
||||
}
|
||||
}
|
||||
}
|
||||
if s.guard() != nil {
|
||||
return "BLOCKED", errResourcePlan
|
||||
}
|
||||
if !passed {
|
||||
return "DEGRADED", errResourcePlan
|
||||
}
|
||||
return "OK", nil
|
||||
}
|
||||
|
||||
func resourceApplyMode(config string, d managedDeps) error {
|
||||
if os.Geteuid() != 0 || os.Getppid() != 1 || config == "" {
|
||||
return errResourcePlan
|
||||
}
|
||||
if _, e := managedRuntimeGuard(d); e != nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
cmd, e := os.ReadFile("/proc/1/cmdline")
|
||||
if e != nil || string(cmd) != "/bin/sh\x00/opt/managed-target-entrypoint.sh\x00" {
|
||||
return errResourcePlan
|
||||
}
|
||||
env, e := os.ReadFile("/proc/1/environ")
|
||||
if e != nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
values := map[string]string{}
|
||||
for _, item := range strings.Split(string(env), "\x00") {
|
||||
k, v, _ := strings.Cut(item, "=")
|
||||
values[k] = v
|
||||
}
|
||||
if values["ZAPRET_AUTOTUNE_RESOURCE_APPLY"] != "1" || values["ZAPRET_AUTOTUNE_RESOURCE_CONFIG"] != config || values["ZAPRET_AUTOTUNE_QUALIFY_CONFIG"] != "" || values["ZAPRET_AUTOTUNE_STRATEGY_CONFIG"] != "" {
|
||||
return errResourcePlan
|
||||
}
|
||||
requested, _ := d.lookup(resourceStrategyIDsEnv)
|
||||
if _, e := parseResourceStrategyIDs(requested); e != nil || requested != values[resourceStrategyIDsEnv] {
|
||||
return errResourcePlan
|
||||
}
|
||||
for _, ns := range []string{"pid", "net"} {
|
||||
a, e := os.Readlink("/proc/1/ns/" + ns)
|
||||
b, f := os.Readlink("/proc/self/ns/" + ns)
|
||||
if e != nil || f != nil || a != b {
|
||||
return errResourcePlan
|
||||
}
|
||||
}
|
||||
if !resourceProcessOwned("/proc", os.Getpid(), "/usr/local/bin/zapret-autotune", []string{"/usr/local/bin/zapret-autotune", "managed", "resource-apply", "--resource-config", config}) {
|
||||
return errResourcePlan
|
||||
}
|
||||
dir, e := os.Open("/run/zapret-autotune-ss")
|
||||
if e != nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
defer dir.Close()
|
||||
f, e := resourcePrivateFile(dir, "monitor-ready")
|
||||
if e != nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
defer f.Close()
|
||||
b, e := io.ReadAll(io.LimitReader(f, 32))
|
||||
if e != nil || string(b) != strconv.Itoa(os.Getpid())+"\n" {
|
||||
return errResourcePlan
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func resourceApplyCLI(config string, stdout, stderr io.Writer, d managedDeps) int {
|
||||
fmt.Fprintln(stderr, "controller-startup: resource_apply=entry")
|
||||
blocked := func() int {
|
||||
fmt.Fprintln(stderr, "BLOCKED: generic apply trust/health incomplete; deployment preserved or prepared for recovery")
|
||||
return 78
|
||||
}
|
||||
if resourceApplyMode(config, d) != nil {
|
||||
return blocked()
|
||||
}
|
||||
lock, e := acquireManagedMonitorSingleton(d)
|
||||
if e != nil {
|
||||
return blocked()
|
||||
}
|
||||
defer lock.close()
|
||||
if qualificationNoWorker() != nil {
|
||||
return blocked()
|
||||
}
|
||||
s, e := openResourceApplySnapshot(config, d)
|
||||
if e != nil {
|
||||
return blocked()
|
||||
}
|
||||
defer s.close()
|
||||
signals, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
ctx, cancel := context.WithTimeout(signals, 15*time.Minute)
|
||||
defer cancel()
|
||||
checks := productionResourceCheckDeps()
|
||||
m := &managedTransaction{Dir: d.directory, Name: "config", resource: s, Reload: func() error { return d.run(d.script, "restart") }}
|
||||
m.Health = func() (string, error) { return resourceApplyHealth(ctx, s, checks) }
|
||||
m.RollbackHealth = func() (string, error) {
|
||||
cleanup, cancel := context.WithTimeout(context.WithoutCancel(ctx), 30*time.Second)
|
||||
defer cancel()
|
||||
return resourceApplyHealth(cleanup, s, checks)
|
||||
}
|
||||
if resourceSingletonGuard(lock, d.directory) != nil || m.Reconcile() != nil {
|
||||
return blocked()
|
||||
}
|
||||
requested, _ := d.lookup(resourceStrategyIDsEnv)
|
||||
policy, e := parseResourceStrategyIDs(requested)
|
||||
if e != nil || s.selectPolicy(policy) != nil {
|
||||
return blocked()
|
||||
}
|
||||
originalGuard := s.guard
|
||||
s.guard = func() error {
|
||||
if resourceApplyMode(config, d) != nil || resourceSingletonGuard(lock, d.directory) != nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
return originalGuard()
|
||||
}
|
||||
j, e := m.load()
|
||||
if e != nil {
|
||||
return blocked()
|
||||
}
|
||||
if (j.Version != 4 || j.Ref.Kind == "baseline") && qualificationNoWorker() != nil {
|
||||
return blocked()
|
||||
}
|
||||
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e != nil {
|
||||
return blocked()
|
||||
}
|
||||
fmt.Fprintln(stdout, "generic resource apply: committed; configured resources only, not production READY")
|
||||
return 0
|
||||
}
|
||||
|
||||
func resourceDeploymentIdentity(config string, d managedDeps) error {
|
||||
m := &managedTransaction{Dir: d.directory, Name: "config"}
|
||||
j, e := m.load()
|
||||
if e != nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
if j.Version != 4 {
|
||||
return nil
|
||||
}
|
||||
if j.State != "committed" {
|
||||
return errResourcePlan
|
||||
}
|
||||
s, e := openResourceApplySnapshot(config, d)
|
||||
if e != nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
defer s.close()
|
||||
m.resource = s
|
||||
return m.validateConfigRef(nil, j.Ref, j.Candidates, j.Hash, j.Scope)
|
||||
}
|
||||
@@ -0,0 +1,499 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"net"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func completeApplyFixture(t *testing.T) (resourceMonitorSource, []byte, strategyRegistry, qualificationMatrix, time.Time) {
|
||||
s, key, r, m, now := planFixture(t)
|
||||
// e was intentionally unknown in the proposal fixture; this request covers
|
||||
// only a/b/c/d, including a's cross-service duplicate resource.
|
||||
s.config.Services = append(s.config.Services[:4:4], s.config.Services[5])
|
||||
return s, key, r, m, now
|
||||
}
|
||||
|
||||
func TestResourceApplyRender(t *testing.T) {
|
||||
s, key, r, m, now := completeApplyFixture(t)
|
||||
p, b, e := resourceApplyPlan(s, key, r, m, now)
|
||||
if e != nil || !p.ProposalOnly || len(p.Assignments) != 3 {
|
||||
t.Fatal(p, e)
|
||||
}
|
||||
if !strings.Contains(string(b), "--hostlist-domains=^a.example,^c.example") || strings.Count(string(b), " --new ") != 2 {
|
||||
t.Fatal(string(b))
|
||||
}
|
||||
for i := 0; i < 30; i++ {
|
||||
_, next, e := resourceApplyPlan(s, key, r, m, now)
|
||||
if e != nil || !bytes.Equal(next, b) {
|
||||
t.Fatal("nondeterministic render")
|
||||
}
|
||||
}
|
||||
for _, kind := range []string{"negative", "unknown", "incomplete", "stale", "future", "config", "tls", "unsafe"} {
|
||||
t.Run(kind, func(t *testing.T) {
|
||||
s, key, r, m, now := completeApplyFixture(t)
|
||||
switch kind {
|
||||
case "negative":
|
||||
m.Runs[0].Checks[len(m.Runs[0].Checks)-1].Outcome = "expectation_failed"
|
||||
case "unknown":
|
||||
m.Runs[0].Attestation.Joined = false
|
||||
case "incomplete":
|
||||
m.Runs[0].Checks = m.Runs[0].Checks[:len(m.Runs[0].Checks)-1]
|
||||
case "stale":
|
||||
now = now.Add(25 * time.Hour)
|
||||
case "future":
|
||||
now = now.Add(-time.Hour)
|
||||
case "config":
|
||||
s.fingerprint = strings.Repeat("f", 64)
|
||||
case "tls":
|
||||
s.config.Services[0].Resources[0].TLS = "tls12"
|
||||
case "unsafe":
|
||||
r.Strategies[0].Args = "--lua-desync=execute:cmd=id"
|
||||
}
|
||||
if _, _, e := resourceApplyPlan(s, key, r, m, now); e == nil {
|
||||
t.Fatal("authorized invalid proof")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Uses the production snapshot factory, private HMAC envelopes and shared
|
||||
// transaction engine; native/HTTP fixtures perform no network operations.
|
||||
func nativeApplyFixture(t *testing.T) (*managedTransaction, *resourceApplySnapshot, string, string) {
|
||||
return nativeApplyAlternativesFixture(t, false)
|
||||
}
|
||||
|
||||
func nativeApplyAlternativesFixture(t *testing.T, alternatives bool) (*managedTransaction, *resourceApplySnapshot, string, string) {
|
||||
return nativeApplyProfilesFixture(t, alternatives, false)
|
||||
}
|
||||
|
||||
func nativeApplyProfilesFixture(t *testing.T, alternatives, sameArgs bool) (*managedTransaction, *resourceApplySnapshot, string, string) {
|
||||
t.Helper()
|
||||
root, _, _ := qualificationBinaryFixture(t)
|
||||
dir := t.TempDir()
|
||||
if e := os.Chmod(dir, 0700); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
m := &managedTransaction{Dir: dir, Name: "config", Reload: func() error { return nil }, Health: func() (string, error) { return "OK", nil }}
|
||||
if e := managedInitialize(m); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
store, e := openResourcePlanStore(dir, true)
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
config := resourceCheckFile(t, strings.Replace(resourceCheckExample, `"tls":"tls12"`, `"tls":"both"`, 1))
|
||||
d := managedDeps{directory: dir, lookup: resourceCheckLookup}
|
||||
source, e := readResourceMonitorSource(config, d.lookup, store.files.key)
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
_, _, registry, _, _ := planFixture(t)
|
||||
registry.Strategies = registry.Strategies[:1]
|
||||
if alternatives {
|
||||
other := registry.Strategies[0]
|
||||
other.ID = "zz-alternative"
|
||||
other.Args = "--lua-desync=split:pos=2"
|
||||
if sameArgs {
|
||||
other.Args = registry.Strategies[0].Args
|
||||
}
|
||||
registry.Strategies = append(registry.Strategies, other)
|
||||
}
|
||||
b, _ := json.Marshal(registry)
|
||||
if e := store.importDefinitions(b); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
engine, e := openQualificationEngine(root)
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
hash, e := engine.filesHash()
|
||||
engine.file.Close()
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
now := time.Now()
|
||||
def := registry.Strategies[0]
|
||||
run := qualificationRun{ID: "run-test", Strategy: def.ID, Definition: planDigest(def), Config: source.fingerprint, Started: now.Add(-time.Minute).UnixMilli(), Completed: now.UnixMilli(), Attestation: qualificationAttestation{Collector: "owned-resource-nfq-v1", Runtime: "nfqws2-1.0.2", Definition: planDigest(def), LuaHash: hash, RulesHash: strings.Repeat("b", 64), WorkerArgsHash: strings.Repeat("c", 64), Queue: managedAssetQueue, PID: 123, StartTime: 456, Restored: true, Joined: true}}
|
||||
for _, service := range source.config.Services {
|
||||
for _, spec := range service.Resources {
|
||||
for _, tls := range []string{"tls12", "tls13"} {
|
||||
run.Checks = append(run.Checks, qualificationCheck{Service: service.ID, Resource: spec.ID, TLS: tls, Fingerprint: resourceRowFingerprint(store.files.key, source, spec), Outcome: "pass", Attempts: 3})
|
||||
}
|
||||
}
|
||||
}
|
||||
matrix := qualificationMatrix{Version: 1, Registry: planDigest(registry), Runs: []qualificationRun{run}}
|
||||
if alternatives {
|
||||
other := run
|
||||
other.ID, other.Strategy = "run-alternative", registry.Strategies[1].ID
|
||||
other.Definition = planDigest(registry.Strategies[1])
|
||||
other.Attestation.Definition = other.Definition
|
||||
matrix.Runs = append(matrix.Runs, other)
|
||||
}
|
||||
if e := store.saveCompleted(completedQualification{matrix: matrix, seal: planDigest(matrix)}); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
store.close()
|
||||
s, e := openResourceApplySnapshotAt(config, d, root)
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
t.Cleanup(s.close)
|
||||
m.resource = s
|
||||
return m, s, config, root
|
||||
}
|
||||
|
||||
func TestResourceApplyLifecycle(t *testing.T) {
|
||||
m, s, config, root := nativeApplyFixture(t)
|
||||
reloads := 0
|
||||
m.Reload = func() error { reloads++; return nil }
|
||||
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
j, e := m.load()
|
||||
if e != nil || j.Version != 4 || j.State != "committed" || j.Ref.Kind != "generic" || j.PendingRef != nil || reloads != 1 {
|
||||
t.Fatal(j, e, reloads)
|
||||
}
|
||||
if _, e := m.ActiveCandidates(nil); e == nil {
|
||||
t.Fatal("generic treated as empty baseline")
|
||||
}
|
||||
s.now = func() time.Time { return time.Now().Add(25 * time.Hour) }
|
||||
if s.fresh() == nil {
|
||||
t.Fatal("new expired apply authorized")
|
||||
}
|
||||
if e := m.Reconcile(); e != nil {
|
||||
t.Fatal("aged committed proof invalidated", e)
|
||||
}
|
||||
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e != nil || reloads != 1 {
|
||||
t.Fatal("replay changed runtime", e, reloads)
|
||||
}
|
||||
before, _ := os.ReadFile(filepath.Join(m.Dir, "config"))
|
||||
raw, _ := os.ReadFile(config)
|
||||
if e := os.WriteFile(config, bytes.Replace(raw, []byte("example.com"), []byte("changed.com"), 1), 0600); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if e := m.Reconcile(); e == nil {
|
||||
t.Fatal("source drift reconciled")
|
||||
}
|
||||
after, _ := os.ReadFile(filepath.Join(m.Dir, "config"))
|
||||
if !bytes.Equal(before, after) {
|
||||
t.Fatal("drift erased deployment")
|
||||
}
|
||||
if _, e := openResourceApplySnapshotAt(config, managedDeps{directory: m.Dir, lookup: resourceCheckLookup}, root); e == nil {
|
||||
t.Fatal("changed source accepted after reopen")
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceApplyCrashRecovery(t *testing.T) {
|
||||
for _, point := range []string{"staged", "backup", "prepared", "replaced", "reloaded", "healthy"} {
|
||||
t.Run(point, func(t *testing.T) {
|
||||
m, s, _, _ := nativeApplyFixture(t)
|
||||
m.fault = func(p string) error {
|
||||
if p == point {
|
||||
return errors.New("process death")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e == nil {
|
||||
t.Fatal("fault ignored")
|
||||
}
|
||||
m.fault = nil
|
||||
if e := m.Reconcile(); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
b, _ := m.read("config")
|
||||
j, _ := m.load()
|
||||
if string(b) != managedBaseline || j.State != "committed" || j.Ref != nil && j.Ref.Kind != "baseline" {
|
||||
t.Fatal("pending promoted", j, string(b))
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceApplyRollback(t *testing.T) {
|
||||
for _, failure := range []string{"reload", "health", "rollback-reload", "rollback-health"} {
|
||||
t.Run(failure, func(t *testing.T) {
|
||||
m, s, _, _ := nativeApplyFixture(t)
|
||||
calls := 0
|
||||
m.Reload = func() error {
|
||||
calls++
|
||||
if failure == "reload" && calls == 1 || failure == "rollback-reload" && calls == 2 {
|
||||
return errors.New("reload")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
m.Health = func() (string, error) { return "DEGRADED", nil }
|
||||
m.RollbackHealth = func() (string, error) {
|
||||
if failure == "rollback-health" {
|
||||
return "DEGRADED", nil
|
||||
}
|
||||
return "OK", nil
|
||||
}
|
||||
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e == nil {
|
||||
t.Fatal("failure committed")
|
||||
}
|
||||
j, _ := m.load()
|
||||
b, _ := m.read("config")
|
||||
want := "committed"
|
||||
if strings.HasPrefix(failure, "rollback-") {
|
||||
want = "prepared"
|
||||
}
|
||||
if j.State != want || string(b) != managedBaseline {
|
||||
t.Fatal(j, string(b))
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceApplyHeldMutation(t *testing.T) {
|
||||
for _, kind := range []string{"matrix", "source", "lua", "directory"} {
|
||||
t.Run(kind, func(t *testing.T) {
|
||||
m, s, config, root := nativeApplyFixture(t)
|
||||
m.fault = func(point string) error {
|
||||
if point != "staged" {
|
||||
return nil
|
||||
}
|
||||
path := config
|
||||
switch kind {
|
||||
case "matrix":
|
||||
path = filepath.Join(m.Dir, planMatrixName)
|
||||
case "lua":
|
||||
path = filepath.Join(root, "lua/zapret-lib.lua")
|
||||
case "directory":
|
||||
return os.Rename(m.Dir, m.Dir+"-moved")
|
||||
}
|
||||
b, e := os.ReadFile(path)
|
||||
if e != nil {
|
||||
return e
|
||||
}
|
||||
if e := os.Remove(path); e != nil {
|
||||
return e
|
||||
}
|
||||
return os.WriteFile(path, b, 0600)
|
||||
}
|
||||
reloads := 0
|
||||
m.Reload = func() error { reloads++; return nil }
|
||||
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e == nil || reloads != 0 {
|
||||
t.Fatal("mutation authorized", e, reloads)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceApplyJournalTags(t *testing.T) {
|
||||
m, s, _, _ := nativeApplyFixture(t)
|
||||
for _, kind := range []string{"unknown-version", "mixed", "missing", "unknown-tag", "old-tag"} {
|
||||
t.Run(kind, func(t *testing.T) {
|
||||
j := managedJournal{Version: 4, ID: "fixture", State: "committed", Hash: s.ref.Config, Ref: &s.ref}
|
||||
switch kind {
|
||||
case "unknown-version":
|
||||
j.Version = 5
|
||||
case "mixed":
|
||||
j.Candidates = []int64{1}
|
||||
case "missing":
|
||||
j.Ref = nil
|
||||
case "unknown-tag":
|
||||
r := s.ref
|
||||
r.Kind = "other"
|
||||
j.Ref = &r
|
||||
case "old-tag":
|
||||
j.Version = 3
|
||||
j.Scope = GooglevideoScopeExact
|
||||
}
|
||||
if e := m.journal(j); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if _, e := m.load(); e == nil {
|
||||
t.Fatal("contradictory journal accepted")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceApplyHealthCollector(t *testing.T) {
|
||||
m, s, _, _ := nativeApplyFixture(t)
|
||||
_ = m
|
||||
for _, mode := range []string{"", "verify", "status500"} {
|
||||
calls := 0
|
||||
checks := resourceCheckDeps{guard: func() error { return nil }, resolve: func(context.Context, string, string) ([]net.IP, error) {
|
||||
return []net.IP{net.ParseIP("93.184.216.34")}, nil
|
||||
}, runner: func(_ context.Context, args []string, stdin io.Reader, out, diag io.Writer) error {
|
||||
calls++
|
||||
config, _ := io.ReadAll(stdin)
|
||||
if !bytes.Contains(config, []byte("connect-to")) {
|
||||
t.Fatal("missing pin")
|
||||
}
|
||||
raw, err := strconv.Unquote(strings.TrimPrefix(strings.Split(string(config), "\n")[0], "url = "))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
method, tls := "GET", "tls12"
|
||||
for _, arg := range args {
|
||||
if arg == "--head" {
|
||||
method = "HEAD"
|
||||
}
|
||||
if arg == "--tlsv1.3" {
|
||||
tls = "tls13"
|
||||
}
|
||||
if arg == "--insecure" {
|
||||
t.Fatal("TLS bypass")
|
||||
}
|
||||
}
|
||||
configuredWire(raw, method, tls, mode, out, diag)
|
||||
return nil
|
||||
}}
|
||||
status, e := resourceApplyHealth(context.Background(), s, checks)
|
||||
if calls != 4 || (mode == "" && (e != nil || status != "OK")) || (mode != "" && e == nil) {
|
||||
t.Fatal(mode, calls, status, e)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceApplyUnauthorizedCLI(t *testing.T) {
|
||||
var out bytes.Buffer
|
||||
d := managedDeps{}
|
||||
if code := resourceApplyCLI("/private/SECRET_SENTINEL", &out, &out, d); code != 78 {
|
||||
t.Fatal(code)
|
||||
}
|
||||
if got, want := out.String(), "controller-startup: resource_apply=entry\nBLOCKED: generic apply trust/health incomplete; deployment preserved or prepared for recovery\n"; got != want {
|
||||
t.Fatalf("unexpected diagnostic: %q", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceApplyFreshBeforeStaging(t *testing.T) {
|
||||
for _, kind := range []string{"expired", "forged", "native", "proof", "unsupported"} {
|
||||
t.Run(kind, func(t *testing.T) {
|
||||
m, s, _, root := nativeApplyFixture(t)
|
||||
ref := s.ref
|
||||
switch kind {
|
||||
case "expired":
|
||||
s.now = func() time.Time { return time.Now().Add(25 * time.Hour) }
|
||||
case "forged":
|
||||
ref.Plan = strings.Repeat("a", 64)
|
||||
case "native":
|
||||
if e := os.WriteFile(filepath.Join(root, "lua/zapret-antidpi.lua"), []byte("drift"), 0755); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
case "proof":
|
||||
if e := os.WriteFile(filepath.Join(m.Dir, planMatrixName), []byte(`{"payload":{},"mac":"forged"}`), 0600); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
case "unsupported":
|
||||
s.source.config.Services[0].Resources[0].TLS = "tls13"
|
||||
}
|
||||
reloads, stages := 0, 0
|
||||
m.Reload = func() error { reloads++; return nil }
|
||||
m.fault = func(point string) error {
|
||||
if point == "staged" {
|
||||
stages++
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if e := m.applyPolicy(s.rendered, nil, nil, &ref); e == nil || stages != 0 || reloads != 0 {
|
||||
t.Fatal("unqualified staging", e, stages, reloads)
|
||||
}
|
||||
b, _ := m.read("config")
|
||||
j, _ := m.load()
|
||||
if string(b) != managedBaseline || j.Version != 2 || j.Generation != 0 {
|
||||
t.Fatal("baseline changed", j)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceApplyPreparedExpiredRecoveryHealth(t *testing.T) {
|
||||
m, s, _, _ := nativeApplyFixture(t)
|
||||
m.fault = func(point string) error {
|
||||
if point == "healthy" {
|
||||
return errors.New("crash")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e == nil {
|
||||
t.Fatal("crash missed")
|
||||
}
|
||||
m.fault = nil
|
||||
s.now = func() time.Time { return time.Now().Add(25 * time.Hour) }
|
||||
m.RollbackHealth = func() (string, error) { return "DEGRADED", nil }
|
||||
if e := m.Reconcile(); e == nil {
|
||||
t.Fatal("unhealthy rollback finalized")
|
||||
}
|
||||
j, _ := m.load()
|
||||
b, _ := m.read("config")
|
||||
if j.State != "prepared" || string(b) != managedBaseline {
|
||||
t.Fatal("baseline was not physically restored", j)
|
||||
}
|
||||
m.RollbackHealth = func() (string, error) { return "OK", nil }
|
||||
if e := m.Reconcile(); e != nil {
|
||||
t.Fatal("expiry blocked recovery", e)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceApplyCancellationJoinsBeforeRollback(t *testing.T) {
|
||||
m, s, _, _ := nativeApplyFixture(t)
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
joined := false
|
||||
checks := resourceCheckDeps{guard: func() error { return nil }, resolve: func(context.Context, string, string) ([]net.IP, error) {
|
||||
return []net.IP{net.ParseIP("93.184.216.34")}, nil
|
||||
}, runner: func(ctx context.Context, _ []string, _ io.Reader, _, _ io.Writer) error {
|
||||
cancel()
|
||||
<-ctx.Done()
|
||||
joined = true
|
||||
return ctx.Err()
|
||||
}}
|
||||
m.Health = func() (string, error) { return resourceApplyHealth(ctx, s, checks) }
|
||||
reloads := 0
|
||||
m.Reload = func() error {
|
||||
reloads++
|
||||
if reloads == 2 && !joined {
|
||||
t.Fatal("rollback raced health child")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
m.RollbackHealth = func() (string, error) {
|
||||
if !joined {
|
||||
t.Fatal("rollback health raced child")
|
||||
}
|
||||
return "OK", nil
|
||||
}
|
||||
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e == nil || reloads != 2 || !joined {
|
||||
t.Fatal(e, reloads, joined)
|
||||
}
|
||||
j, _ := m.load()
|
||||
if j.State != "committed" || j.Ref.Kind != "baseline" {
|
||||
t.Fatal(j)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceApplyRefusesActiveLegacy(t *testing.T) {
|
||||
m, s, _, _ := nativeApplyFixture(t)
|
||||
db := testStore(t)
|
||||
id := managedCandidateFixture(t, db, "frontend", "www.youtube.com", "--lua-desync=multisplit", true)
|
||||
m.resource = nil
|
||||
if e := m.Apply(db, []int64{id}, GooglevideoScopeExact); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
before, _ := m.read("config")
|
||||
journal, _ := m.read(managedJournalName)
|
||||
m.resource = s
|
||||
reloads := 0
|
||||
m.Reload = func() error { reloads++; return nil }
|
||||
if e := m.applyPolicy(s.rendered, db, nil, &s.ref); e == nil || reloads != 0 {
|
||||
t.Fatal("active legacy maintenance authorized", e)
|
||||
}
|
||||
after, _ := m.read("config")
|
||||
nextJournal, _ := m.read(managedJournalName)
|
||||
if !bytes.Equal(before, after) || !bytes.Equal(journal, nextJournal) {
|
||||
t.Fatal("legacy state changed")
|
||||
}
|
||||
}
|
||||
@@ -124,9 +124,35 @@ func parseResourceChecks(data []byte, lookup func(string) (string, bool)) (resou
|
||||
}
|
||||
d = json.NewDecoder(bytes.NewReader(data))
|
||||
d.DisallowUnknownFields()
|
||||
if d.Decode(&cfg) != nil || cfg.Version != 1 || len(cfg.Services) == 0 || len(cfg.Services) > 16 {
|
||||
if d.Decode(&cfg) != nil || (cfg.Version != 1 && cfg.Version != 2) || len(cfg.Services) == 0 || len(cfg.Services) > 16 {
|
||||
return fail()
|
||||
}
|
||||
if cfg.Version == 2 {
|
||||
// Presence matters: an omitted minimum must not silently become zero.
|
||||
var fields struct {
|
||||
Services []struct {
|
||||
Resources []struct {
|
||||
Kind *string `json:"kind"`
|
||||
Expect struct {
|
||||
Statuses *[]int `json:"statuses"`
|
||||
ContentType *string `json:"content_type"`
|
||||
MinBytes *int `json:"min_bytes"`
|
||||
MaxBytes *int `json:"max_bytes"`
|
||||
} `json:"expect"`
|
||||
} `json:"resources"`
|
||||
} `json:"services"`
|
||||
}
|
||||
if json.Unmarshal(data, &fields) != nil {
|
||||
return fail()
|
||||
}
|
||||
for _, s := range fields.Services {
|
||||
for _, r := range s.Resources {
|
||||
if r.Kind != nil && *r.Kind == "" || r.Expect.Statuses == nil || r.Expect.ContentType == nil || *r.Expect.ContentType == "" || r.Expect.MinBytes == nil || r.Expect.MaxBytes == nil {
|
||||
return fail()
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
services, total := map[string]bool{}, 0
|
||||
for si := range cfg.Services {
|
||||
s := &cfg.Services[si]
|
||||
@@ -137,8 +163,11 @@ func parseResourceChecks(data []byte, lookup func(string) (string, bool)) (resou
|
||||
ids := map[string]bool{}
|
||||
for ri := range s.Resources {
|
||||
r := &s.Resources[ri]
|
||||
if cfg.Version == 2 && r.Kind == "" {
|
||||
r.Kind = "http"
|
||||
}
|
||||
total++
|
||||
if total > 64 || !resourceID.MatchString(r.ID) || ids[r.ID] || !resourceID.MatchString(r.Group) || r.Kind != "http" {
|
||||
if total > 64 || !resourceID.MatchString(r.ID) || ids[r.ID] || !resourceID.MatchString(r.Group) || (r.Kind != "http" && (cfg.Version != 2 || r.Kind != "image" && r.Kind != "mp4_range")) {
|
||||
return fail()
|
||||
}
|
||||
ids[r.ID] = true
|
||||
@@ -213,6 +242,24 @@ func parseResourceChecks(data []byte, lookup func(string) (string, bool)) (resou
|
||||
return fail()
|
||||
}
|
||||
}
|
||||
if r.Kind != "http" {
|
||||
status := 200
|
||||
if r.Kind == "mp4_range" {
|
||||
status = 206
|
||||
}
|
||||
if r.Method != "GET" || len(r.Expect.Statuses) != 1 || r.Expect.Statuses[0] != status {
|
||||
return fail()
|
||||
}
|
||||
if r.Kind == "image" {
|
||||
// ponytail: public GIF awaits an aggregate frame/pixel-bounded decoder;
|
||||
// legacy image adapters retain their existing GIF contract.
|
||||
if r.Expect.ContentType != "image/png" && r.Expect.ContentType != "image/jpeg" {
|
||||
return fail()
|
||||
}
|
||||
} else if (r.Expect.ContentType != "video/mp4" && r.Expect.ContentType != "audio/mp4") || r.Expect.MinBytes < 16 || r.Expect.MaxBytes > 65536 {
|
||||
return fail()
|
||||
}
|
||||
}
|
||||
if len(r.HeadersEnv) > 16 {
|
||||
return fail()
|
||||
}
|
||||
@@ -326,10 +373,13 @@ func approvedResourceIP(r resourceCheckSpec, ip net.IP) bool {
|
||||
}
|
||||
|
||||
type resourceCheckDeps struct {
|
||||
lookup func(string) (string, bool)
|
||||
guard func() error
|
||||
resolve func(context.Context, string, string) ([]net.IP, error)
|
||||
runner subprocessRunner
|
||||
// Qualification must distinguish infrastructure from verified peer negatives.
|
||||
// Monitoring retains its existing coarse outcome behavior.
|
||||
qualification bool
|
||||
lookup func(string) (string, bool)
|
||||
guard func() error
|
||||
resolve func(context.Context, string, string) ([]net.IP, error)
|
||||
runner subprocessRunner
|
||||
}
|
||||
|
||||
func resolveResourceIPs(ctx context.Context, server, host string) ([]net.IP, error) {
|
||||
@@ -371,11 +421,21 @@ type resourceCheckResult struct {
|
||||
func runResourceProtocol(ctx context.Context, service string, r resourceCheckSpec, protocol, server, proxy string, d resourceCheckDeps) (resourceCheckResult, error) {
|
||||
result := resourceCheckResult{Service: service, Resource: r.ID, Group: r.Group, TLS: protocol, Outcome: "dns_failed"}
|
||||
if err := d.guard(); err != nil {
|
||||
return result, err
|
||||
return result, qualificationOperation(err, stagePreDNSGuard)
|
||||
}
|
||||
probeCtx, cancel := context.WithTimeout(ctx, time.Duration(r.TimeoutMS)*time.Millisecond)
|
||||
defer cancel()
|
||||
ips, err := d.resolve(probeCtx, server, r.host)
|
||||
if d.qualification && err != nil {
|
||||
var dns *net.DNSError
|
||||
knownTimeout := probeCtx.Err() != nil && managedRecoveryTimeoutSafe(err)
|
||||
if !knownTimeout && (!errors.As(err, &dns) || !dns.IsNotFound) {
|
||||
return result, qualificationOperation(err, stageDNS)
|
||||
}
|
||||
}
|
||||
if d.qualification && err == nil && (len(ips) == 0 || len(ips) > 64) {
|
||||
return result, qualificationOperation(errResourceConfig, stageDNS)
|
||||
}
|
||||
if err == nil && len(ips) > 0 && len(ips) <= 64 {
|
||||
approved := true
|
||||
for _, ip := range ips {
|
||||
@@ -386,19 +446,29 @@ func runResourceProtocol(ctx context.Context, service string, r resourceCheckSpe
|
||||
result.Outcome = "destination_denied"
|
||||
if approved {
|
||||
if err := d.guard(); err != nil {
|
||||
return result, err
|
||||
return result, qualificationOperation(err, stagePostDNSGuard)
|
||||
}
|
||||
pin := r.host + ":443:" + net.JoinHostPort(ips[0].String(), "443")
|
||||
options := resourceTransferOptions{generic: true, method: r.Method, limit: r.Expect.MaxBytes, timeout: time.Duration(r.TimeoutMS) * time.Millisecond, config: r.privateConfig + "connect-to = " + privateCurlValue(pin) + "\n"}
|
||||
transfer, _, err := collectResourceTransferOptions(probeCtx, proxy, r.URL, r.host, protocol, false, d.runner, options)
|
||||
options := resourceTransferOptions{method: r.Method, limit: r.Expect.MaxBytes, timeout: time.Duration(r.TimeoutMS) * time.Millisecond, config: r.privateConfig + "connect-to = " + privateCurlValue(pin) + "\n", expect: &r.Expect}
|
||||
switch r.Kind {
|
||||
case "image":
|
||||
options.response = resourceResponseImage
|
||||
case "mp4_range":
|
||||
options.response, options.mediaType = resourceResponseMP4, r.Expect.ContentType
|
||||
}
|
||||
transfer, _, err := collectResourceTransferOptions(probeCtx, proxy, r.URL, r.host, protocol, d.runner, options)
|
||||
if d.qualification && err != nil && err != errResourceExpectation {
|
||||
knownTimeout := probeCtx.Err() != nil && managedRecoveryTimeoutSafe(err)
|
||||
if !knownTimeout && !resourceRejectionOnly(err) && !managedAssetTransferNonpass(err) {
|
||||
return result, qualificationOperation(err, stageTransfer)
|
||||
}
|
||||
}
|
||||
result.Outcome = "transfer_failed"
|
||||
if err == nil {
|
||||
if err == nil || err == errResourceExpectation {
|
||||
result.Bytes, result.TimeMS, result.Status = len(transfer.Body), transfer.LatencyMS, transfer.Status
|
||||
result.Outcome = "expectation_failed"
|
||||
for _, status := range r.Expect.Statuses {
|
||||
if status == transfer.Status && len(transfer.Body) >= r.Expect.MinBytes && (r.Expect.ContentType == "" || transfer.ContentType == r.Expect.ContentType) {
|
||||
result.Outcome = "pass"
|
||||
}
|
||||
if err == nil {
|
||||
result.Outcome = "pass"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -49,6 +49,9 @@ func configuredWire(raw, method, protocol, mode string, out, diag io.Writer) {
|
||||
if mode == "redirect" {
|
||||
status = 302
|
||||
}
|
||||
if mode == "status500" {
|
||||
status = 500
|
||||
}
|
||||
if mode == "redirect-count" {
|
||||
redirects = 1
|
||||
}
|
||||
@@ -91,7 +94,7 @@ func TestResourceCheckSchema(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, data := range []string{
|
||||
resourceCheckExample + ` {}`, strings.Replace(resourceCheckExample, `"version":1`, `"version":2`, 1),
|
||||
resourceCheckExample + ` {}`, strings.Replace(resourceCheckExample, `"version":1`, `"version":3`, 1),
|
||||
strings.Replace(resourceCheckExample, `"version":1`, `"version":1,"version":1`, 1),
|
||||
strings.Replace(resourceCheckExample, `"version":1`, `"unknown":1,"version":1`, 1),
|
||||
strings.Replace(resourceCheckExample, `"id":"first"`, `"id":"First /"`, 1),
|
||||
@@ -362,7 +365,7 @@ func TestResourceCheckTransferContract(t *testing.T) {
|
||||
configuredWire("https://example.com/", method, "tls12", mode, out, diag)
|
||||
return nil
|
||||
}
|
||||
transfer, _, err := collectResourceTransferOptions(context.Background(), "socks5h://127.0.0.1:1080", "https://example.com/", "example.com", "tls12", false, runner, resourceTransferOptions{generic: true, method: method, limit: 100, timeout: time.Second})
|
||||
transfer, _, err := collectResourceTransferOptions(context.Background(), "socks5h://127.0.0.1:1080", "https://example.com/", "example.com", "tls12", runner, resourceTransferOptions{method: method, limit: 100, timeout: time.Second})
|
||||
valid := mode == "pass" || mode == "chunked" || method == "HEAD" && mode == "truncated"
|
||||
if (err == nil) != valid {
|
||||
t.Fatalf("err=%v", err)
|
||||
@@ -481,7 +484,7 @@ func TestResourceCheckCurlSOCKSPinnedDestination(t *testing.T) {
|
||||
}()
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 4*time.Second)
|
||||
defer cancel()
|
||||
_, _, err = collectResourceTransferOptions(ctx, "socks5h://"+listener.Addr().String(), "https://example.com/private?secret=value", "example.com", "tls12", false, runResourceCurl, resourceTransferOptions{generic: true, method: "GET", limit: 100, timeout: 3 * time.Second, config: "connect-to = \"example.com:443:93.184.216.34:443\"\n"})
|
||||
_, _, err = collectResourceTransferOptions(ctx, "socks5h://"+listener.Addr().String(), "https://example.com/private?secret=value", "example.com", "tls12", runResourceCurl, resourceTransferOptions{method: "GET", limit: 100, timeout: 3 * time.Second, config: "connect-to = \"example.com:443:93.184.216.34:443\"\n"})
|
||||
if err == nil {
|
||||
t.Fatal("fixture unexpectedly transferred")
|
||||
}
|
||||
|
||||
@@ -9,6 +9,7 @@ import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"image"
|
||||
"io"
|
||||
"math"
|
||||
"mime"
|
||||
@@ -31,18 +32,28 @@ func runResourceCurl(ctx context.Context, args []string, stdin io.Reader, out, d
|
||||
return cmd.Run()
|
||||
}
|
||||
|
||||
type resourceError struct{ cause error }
|
||||
type resourceError struct {
|
||||
cause error
|
||||
reason qualificationReason
|
||||
}
|
||||
|
||||
// Created only after proxy, URL and TLS inputs have passed collector validation.
|
||||
// Rejection is never qualification, nor evidence that local ownership failed.
|
||||
type resourcePeerTLSRejection struct{ error }
|
||||
|
||||
// Created only after origin, verified TLS and HTTP framing have been decoded.
|
||||
// A configured HTTP peer response rejection is not local ownership failure.
|
||||
type resourcePeerResponseRejection struct{ error }
|
||||
|
||||
func (e *resourcePeerResponseRejection) Error() string { return "resource peer response rejected" }
|
||||
func (e *resourcePeerResponseRejection) Unwrap() error { return e.error }
|
||||
|
||||
func (e *resourcePeerTLSRejection) Error() string { return "resource peer TLS rejected" }
|
||||
func (e *resourcePeerTLSRejection) Unwrap() error { return e.error }
|
||||
|
||||
func resourceRejectionOnly(err error) bool {
|
||||
switch err.(type) {
|
||||
case *resourcePeerTLSRejection, imageBodyRejection:
|
||||
case *resourcePeerTLSRejection, *resourcePeerResponseRejection, imageBodyRejection:
|
||||
return true
|
||||
}
|
||||
if e, ok := err.(*resourceError); ok {
|
||||
@@ -179,39 +190,17 @@ type mediaCapability struct {
|
||||
|
||||
// SABR is deliberately unknown/not supported, never fabricated as readiness.
|
||||
func playerMediaResource(body []byte) (mediaResource, mediaCapability, error) {
|
||||
var p struct {
|
||||
PlayabilityStatus struct {
|
||||
Status string `json:"status"`
|
||||
} `json:"playabilityStatus"`
|
||||
StreamingData struct {
|
||||
Formats []struct {
|
||||
URL string `json:"url"`
|
||||
MimeType string `json:"mimeType"`
|
||||
} `json:"formats"`
|
||||
AdaptiveFormats []struct {
|
||||
URL string `json:"url"`
|
||||
MimeType string `json:"mimeType"`
|
||||
} `json:"adaptiveFormats"`
|
||||
} `json:"streamingData"`
|
||||
module, err := requiredYoutubeAdapter()
|
||||
if err != nil {
|
||||
return mediaResource{}, mediaCapability{}, err
|
||||
}
|
||||
if len(body) > playerResponseLimit || json.Unmarshal(body, &p) != nil || p.PlayabilityStatus.Status != "OK" {
|
||||
return mediaResource{}, mediaCapability{}, errors.New("invalid playable metadata")
|
||||
resources, err := module.DiscoverResponse(body)
|
||||
if err != nil {
|
||||
return mediaResource{}, mediaCapability{}, err
|
||||
}
|
||||
for _, f := range append(p.StreamingData.Formats, p.StreamingData.AdaptiveFormats...) {
|
||||
u, err := url.Parse(f.URL)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
host := u.Hostname()
|
||||
if !strings.HasSuffix(host, ".googlevideo.com") {
|
||||
continue
|
||||
}
|
||||
if _, err := privateResourceURL(f.URL, host); err != nil {
|
||||
continue
|
||||
}
|
||||
kind, _, err := mime.ParseMediaType(f.MimeType)
|
||||
if err == nil && (kind == "video/mp4" || kind == "audio/mp4") {
|
||||
return mediaResource{f.URL, host, kind}, mediaCapability{OrdinaryURL: true}, nil
|
||||
for _, r := range resources {
|
||||
if _, err := privateResourceURL(r.URL, r.Host); err == nil {
|
||||
return mediaResource{r.URL, r.Host, r.Kind}, mediaCapability{OrdinaryURL: true}, nil
|
||||
}
|
||||
}
|
||||
return mediaResource{}, mediaCapability{}, nil
|
||||
@@ -219,22 +208,39 @@ func playerMediaResource(body []byte) (mediaResource, mediaCapability, error) {
|
||||
|
||||
const resourceTrailer = "\nAUTOTUNE_RESOURCE_METRICS:"
|
||||
|
||||
func collectResourceTransfer(ctx context.Context, proxy, raw, host, protocol string, media bool, runner subprocessRunner) (assetTransfer, http.Header, error) {
|
||||
return collectResourceTransferOptions(ctx, proxy, raw, host, protocol, media, runner, resourceTransferOptions{})
|
||||
}
|
||||
type resourceResponseKind uint8
|
||||
|
||||
// A completed generic mismatch retains measured facts, unlike unsafe transfers.
|
||||
var errResourceExpectation = errors.New("resource expectation failed")
|
||||
|
||||
const (
|
||||
resourceResponseGeneric resourceResponseKind = iota
|
||||
resourceResponseImage
|
||||
resourceResponseMP4
|
||||
)
|
||||
|
||||
type resourceTransferOptions struct {
|
||||
method string
|
||||
limit int
|
||||
timeout time.Duration
|
||||
// Private curl config only: never placed in argv or public evidence.
|
||||
config string
|
||||
generic bool
|
||||
config string
|
||||
response resourceResponseKind
|
||||
mediaType string
|
||||
expect *resourceCheckExpect
|
||||
}
|
||||
|
||||
func collectResourceTransferOptions(ctx context.Context, proxy, raw, host, protocol string, media bool, runner subprocessRunner, options resourceTransferOptions) (assetTransfer, http.Header, error) {
|
||||
func collectResourceTransferOptions(ctx context.Context, proxy, raw, host, protocol string, runner subprocessRunner, options resourceTransferOptions) (assetTransfer, http.Header, error) {
|
||||
generic := options.response == resourceResponseGeneric
|
||||
configured := generic || options.expect != nil
|
||||
reason := reasonOrigin
|
||||
fail := func(err error) (assetTransfer, http.Header, error) {
|
||||
return assetTransfer{}, nil, resourceFailure(ctx, err)
|
||||
failure := resourceFailure(ctx, err)
|
||||
failure.(*resourceError).reason = reason
|
||||
return assetTransfer{}, nil, failure
|
||||
}
|
||||
if options.response > resourceResponseMP4 || (!generic && options.method != "GET") || (options.response == resourceResponseMP4 && options.mediaType != "video/mp4" && options.mediaType != "audio/mp4") {
|
||||
return fail(errors.New("invalid response contract"))
|
||||
}
|
||||
proxy, proxyErr := managedProbeProxy(proxy)
|
||||
if proxyErr != nil || runner == nil {
|
||||
@@ -250,22 +256,25 @@ func collectResourceTransferOptions(ctx context.Context, proxy, raw, host, proto
|
||||
return fail(errors.New("invalid TLS protocol"))
|
||||
}
|
||||
limit := 5 << 20
|
||||
if media {
|
||||
if options.response == resourceResponseMP4 {
|
||||
limit = 65536
|
||||
}
|
||||
if options.generic {
|
||||
if configured {
|
||||
limit = options.limit
|
||||
}
|
||||
// Qualification uses HTTP/1.1, not browser HTTP/2. --raw preserves native
|
||||
// transfer framing so net/http, rather than curl and Go twice, decodes it.
|
||||
args := []string{"--disable", "--silent", "--show-error", "--verbose", "--http1.1", "--raw", "--noproxy", "", "--proxy", proxy, "--proto", "=https", tlsFlag, "--tls-max", maxTLS, "--max-time", "30", "--max-filesize", strconv.Itoa(limit), "--suppress-connect-headers", "--include", "--write-out", resourceTrailer + "%{json}", "--config", "-"}
|
||||
if media {
|
||||
if options.response == resourceResponseMP4 {
|
||||
args = append(args, "--range", "0-65535")
|
||||
}
|
||||
if options.generic {
|
||||
if configured {
|
||||
if (options.method != "GET" && options.method != "HEAD") || options.limit < 1 || options.limit > 5<<20 || options.timeout < 100*time.Millisecond || options.timeout > 30*time.Second {
|
||||
return fail(errors.New("invalid transfer options"))
|
||||
}
|
||||
if options.response == resourceResponseMP4 && options.limit > 65536 {
|
||||
return fail(errors.New("invalid transfer options"))
|
||||
}
|
||||
args = append(args, "--max-time", strconv.FormatFloat(options.timeout.Seconds(), 'f', 3, 64))
|
||||
if options.method == "HEAD" {
|
||||
args = append(args, "--head", "--max-filesize", "0")
|
||||
@@ -275,16 +284,18 @@ func collectResourceTransferOptions(ctx context.Context, proxy, raw, host, proto
|
||||
config := "url = \"" + strings.NewReplacer("\\", "\\\\", "\"", "\\\"").Replace(raw) + "\"\n"
|
||||
config += options.config
|
||||
timeout := 35 * time.Second
|
||||
if options.generic {
|
||||
if configured {
|
||||
timeout = options.timeout
|
||||
}
|
||||
childCtx, cancel := context.WithTimeout(ctx, timeout)
|
||||
defer cancel()
|
||||
var output, diagnostic bytes.Buffer
|
||||
reason = reasonExecution
|
||||
out := &playerBoundedWriter{dst: &output, limit: limit + (96 << 10)}
|
||||
diag := &playerBoundedWriter{dst: &diagnostic, limit: 64 << 10}
|
||||
err := runner(childCtx, args, strings.NewReader(config), out, diag)
|
||||
if out.exceeded || diag.exceeded {
|
||||
reason = reasonOutputBound
|
||||
err = errors.Join(err, errors.New("resource output exceeds limit"))
|
||||
}
|
||||
if err != nil || childCtx.Err() != nil {
|
||||
@@ -298,6 +309,7 @@ func collectResourceTransferOptions(ctx context.Context, proxy, raw, host, proto
|
||||
}
|
||||
return fail(errors.Join(err, childCtx.Err()))
|
||||
}
|
||||
reason = reasonMetrics
|
||||
i := bytes.LastIndex(output.Bytes(), []byte(resourceTrailer))
|
||||
if i < 0 {
|
||||
return fail(errors.New("missing transfer metrics"))
|
||||
@@ -314,9 +326,11 @@ func collectResourceTransferOptions(ctx context.Context, proxy, raw, host, proto
|
||||
if json.Unmarshal(output.Bytes()[i+len(resourceTrailer):], &metrics) != nil {
|
||||
return fail(errors.New("invalid transfer metrics"))
|
||||
}
|
||||
reason = reasonOrigin
|
||||
if _, err := privateResourceURL(metrics.URL, host); err != nil || metrics.URL != raw || metrics.Verify == nil || *metrics.Verify != 0 || metrics.Redirects == nil || *metrics.Redirects != 0 {
|
||||
return fail(errors.New("resource origin verification failed"))
|
||||
}
|
||||
reason = reasonTLSEvidence
|
||||
actualTLS := "SSL connection using TLSv1.2"
|
||||
if version == 0x0304 {
|
||||
actualTLS = "SSL connection using TLSv1.3"
|
||||
@@ -328,6 +342,7 @@ func collectResourceTransferOptions(ctx context.Context, proxy, raw, host, proto
|
||||
if !strings.Contains(diagnostic.String(), actualTLS) || strings.Contains(diagnostic.String(), otherTLS) {
|
||||
return fail(errors.New("actual TLS version mismatch"))
|
||||
}
|
||||
reason = reasonFraming
|
||||
wire := output.Bytes()[:i]
|
||||
headerEnd := bytes.Index(wire, []byte("\r\n\r\n"))
|
||||
statusEnd := bytes.Index(wire, []byte("\r\n"))
|
||||
@@ -347,10 +362,7 @@ func collectResourceTransferOptions(ctx context.Context, proxy, raw, host, proto
|
||||
return fail(errors.New("ambiguous response framing"))
|
||||
}
|
||||
r := bufio.NewReader(bytes.NewReader(wire))
|
||||
var request *http.Request
|
||||
if options.generic {
|
||||
request = &http.Request{Method: options.method}
|
||||
}
|
||||
request := &http.Request{Method: options.method}
|
||||
response, err := http.ReadResponse(r, request)
|
||||
if err != nil {
|
||||
return fail(errors.New("invalid response headers"))
|
||||
@@ -358,36 +370,110 @@ func collectResourceTransferOptions(ctx context.Context, proxy, raw, host, proto
|
||||
defer response.Body.Close()
|
||||
// CONNECT headers are suppressed by curl. Interim/duplicate responses and
|
||||
// unrequested versions are outside this contract, never heuristically skipped.
|
||||
if response.ProtoMajor != 1 || response.ProtoMinor != 1 || response.StatusCode < 200 || (!options.generic && response.Header.Get("Content-Type") == "") {
|
||||
if response.ProtoMajor != 1 || response.ProtoMinor != 1 || response.StatusCode < 200 || (!configured && response.Header.Get("Content-Type") == "") {
|
||||
return fail(errors.New("unsupported response contract"))
|
||||
}
|
||||
if options.generic && response.StatusCode >= 300 && response.StatusCode < 400 {
|
||||
return fail(errors.New("redirect rejected"))
|
||||
if generic && response.StatusCode >= 300 && response.StatusCode < 400 {
|
||||
return fail(&resourcePeerResponseRejection{error: errors.New("redirect rejected")})
|
||||
}
|
||||
reason = reasonBody
|
||||
body, err := io.ReadAll(io.LimitReader(response.Body, int64(limit+1)))
|
||||
// With --raw curl counts the wire body, including chunk framing; the image
|
||||
// limit and durable byte identity apply to the fully decoded payload instead.
|
||||
wireSize := len(wire) - headerEnd - 4
|
||||
_, remaining := r.Peek(1)
|
||||
if err != nil || remaining != io.EOF || (!options.generic && len(body) == 0) || len(body) > limit || metrics.Size != float64(wireSize) || response.StatusCode != metrics.Status {
|
||||
if err != nil || remaining != io.EOF || (!configured && len(body) == 0) || len(body) > limit || metrics.Size != float64(wireSize) || response.StatusCode != metrics.Status {
|
||||
return fail(errors.New("incomplete or oversized resource body"))
|
||||
}
|
||||
reason = reasonMetrics
|
||||
for _, n := range []float64{metrics.Time, metrics.Speed} {
|
||||
if n < 0 || (!options.generic && n == 0) || math.IsNaN(n) || math.IsInf(n, 0) {
|
||||
if n < 0 || (!configured && n == 0) || math.IsNaN(n) || math.IsInf(n, 0) {
|
||||
return fail(errors.New("invalid resource metrics"))
|
||||
}
|
||||
}
|
||||
if options.generic && metrics.Time > options.timeout.Seconds()+1 {
|
||||
if configured && metrics.Time > options.timeout.Seconds()+1 {
|
||||
return fail(errors.New("resource timing exceeds budget"))
|
||||
}
|
||||
reason = reasonMIME
|
||||
contentType, _, err := mime.ParseMediaType(response.Header.Get("Content-Type"))
|
||||
if err != nil && (!options.generic || response.Header.Get("Content-Type") != "") {
|
||||
return fail(errors.New("invalid content type"))
|
||||
if err != nil {
|
||||
contentType = ""
|
||||
}
|
||||
t := assetTransfer{EffectiveHost: host, SSLVerifyResult: metrics.Verify, Redirects: *metrics.Redirects, TLSVersion: version, Status: metrics.Status, ContentType: contentType, Body: body, LatencyMS: metrics.Time * 1000, SpeedKBPS: metrics.Speed / 1024}
|
||||
if err != nil && options.expect != nil && (options.response == resourceResponseImage || options.response == resourceResponseMP4) {
|
||||
// A malformed semantic MIME value is a completed typed mismatch only
|
||||
// after the common wire/security checks; never retain the raw value.
|
||||
return t, response.Header, errResourceExpectation
|
||||
}
|
||||
if err != nil && (!configured || response.Header.Get("Content-Type") != "") {
|
||||
return fail(errors.New("invalid content type"))
|
||||
}
|
||||
if err := options.validateResponse(&t, response.Header); err != nil {
|
||||
if err == errResourceExpectation {
|
||||
return t, response.Header, err
|
||||
}
|
||||
return assetTransfer{}, nil, resourceFailure(ctx, err)
|
||||
}
|
||||
return t, response.Header, nil
|
||||
}
|
||||
|
||||
// Adapter contracts run only after the common origin/TLS/framing/bounds checks.
|
||||
// Generic v1 deliberately has no range or decoded-image expectation.
|
||||
func (options resourceTransferOptions) validateResponse(t *assetTransfer, h http.Header) error {
|
||||
if err := options.validateContract(t, h); err != nil {
|
||||
if options.expect != nil {
|
||||
return errResourceExpectation
|
||||
}
|
||||
return err
|
||||
}
|
||||
if e := options.expect; e != nil {
|
||||
for _, status := range e.Statuses {
|
||||
if t.Status == status && len(t.Body) >= e.MinBytes && len(t.Body) <= e.MaxBytes && (e.ContentType == "" || t.ContentType == e.ContentType) {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
return errResourceExpectation
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (options resourceTransferOptions) validateContract(t *assetTransfer, h http.Header) error {
|
||||
switch options.response {
|
||||
case resourceResponseGeneric:
|
||||
return nil
|
||||
case resourceResponseImage:
|
||||
if t.Status != 200 {
|
||||
return errors.New("image HTTP status")
|
||||
}
|
||||
if options.expect != nil {
|
||||
// Reject GIF before the legacy full decoder can allocate its frames.
|
||||
_, format, err := image.DecodeConfig(bytes.NewReader(t.Body))
|
||||
if err != nil || (format != "png" && format != "jpeg") {
|
||||
return errors.New("invalid public image format")
|
||||
}
|
||||
}
|
||||
return normalizeAssetBody(t)
|
||||
case resourceResponseMP4:
|
||||
parts := resourceContentRange.FindStringSubmatch(h.Get("Content-Range"))
|
||||
var end, total int64
|
||||
if len(parts) == 3 {
|
||||
var endErr, totalErr error
|
||||
end, endErr = strconv.ParseInt(parts[1], 10, 64)
|
||||
total, totalErr = strconv.ParseInt(parts[2], 10, 64)
|
||||
if endErr != nil || totalErr != nil {
|
||||
return errors.New("invalid media range or container")
|
||||
}
|
||||
}
|
||||
if t.Status != 206 || t.ContentType != options.mediaType || len(parts) != 3 || end < 15 || end > 65535 || end+1 != int64(len(t.Body)) || total <= end || len(t.Body) < 16 || string(t.Body[4:8]) != "ftyp" || binary.BigEndian.Uint32(t.Body[:4]) < 16 || int(binary.BigEndian.Uint32(t.Body[:4])) > len(t.Body) {
|
||||
return errors.New("invalid media range or container")
|
||||
}
|
||||
t.Size, t.Format, t.BodyHash = len(t.Body), "mp4", fmt.Sprintf("%x", sha256.Sum256(t.Body))
|
||||
return nil
|
||||
default:
|
||||
return errors.New("invalid response contract")
|
||||
}
|
||||
}
|
||||
|
||||
// The ordinary 3/3 sampler is shared by image and media qualification.
|
||||
func sampleResource(ctx context.Context, transfer func(context.Context) (assetTransfer, error)) ([]assetTransfer, error) {
|
||||
var samples []assetTransfer
|
||||
@@ -407,14 +493,7 @@ func collectAssetMeasurement(ctx context.Context, proxy, raw, host, protocol, ar
|
||||
return m, resourceFailure(ctx, errors.New("invalid image endpoint"))
|
||||
}
|
||||
samples, err := sampleResource(ctx, func(ctx context.Context) (assetTransfer, error) {
|
||||
t, _, err := collectResourceTransfer(ctx, proxy, raw, host, protocol, false, runner)
|
||||
if err == nil {
|
||||
if t.Status != 200 {
|
||||
err = errors.New("image HTTP status")
|
||||
} else {
|
||||
err = normalizeAssetBody(&t)
|
||||
}
|
||||
}
|
||||
t, _, err := collectResourceTransferOptions(ctx, proxy, raw, host, protocol, runner, resourceTransferOptions{method: "GET", response: resourceResponseImage})
|
||||
if err != nil {
|
||||
return assetTransfer{}, resourceFailure(ctx, err)
|
||||
}
|
||||
@@ -441,22 +520,18 @@ func collectMediaTransfer(ctx context.Context, proxy string, resource mediaResou
|
||||
if !strings.HasSuffix(resource.host, ".googlevideo.com") || (resource.kind != "video/mp4" && resource.kind != "audio/mp4") {
|
||||
return assetTransfer{}, GooglevideoPass{}, resourceFailure(ctx, errors.New("ordinary media unavailable"))
|
||||
}
|
||||
t, h, err := collectResourceTransfer(ctx, proxy, resource.url, resource.host, protocol, true, runner)
|
||||
t, h, err := collectResourceTransferOptions(ctx, proxy, resource.url, resource.host, protocol, runner, resourceTransferOptions{method: "GET", response: resourceResponseMP4, mediaType: resource.kind})
|
||||
if err != nil {
|
||||
return assetTransfer{}, GooglevideoPass{}, err
|
||||
}
|
||||
parts := resourceContentRange.FindStringSubmatch(h.Get("Content-Range"))
|
||||
var end, total int64
|
||||
if len(parts) == 3 {
|
||||
end, _ = strconv.ParseInt(parts[1], 10, 64)
|
||||
total, _ = strconv.ParseInt(parts[2], 10, 64)
|
||||
}
|
||||
if t.Status != 206 || t.ContentType != resource.kind || len(parts) != 3 || end < 15 || end > 65535 || end+1 != int64(len(t.Body)) || total <= end || len(t.Body) < 16 || string(t.Body[4:8]) != "ftyp" || binary.BigEndian.Uint32(t.Body[:4]) < 16 || int(binary.BigEndian.Uint32(t.Body[:4])) > len(t.Body) {
|
||||
// The shared core has validated and normalized the complete range contract.
|
||||
_, totalText, _ := strings.Cut(h.Get("Content-Range"), "/")
|
||||
total, err := strconv.ParseInt(totalText, 10, 64)
|
||||
if err != nil {
|
||||
return assetTransfer{}, GooglevideoPass{}, resourceFailure(ctx, errors.New("invalid media range or container"))
|
||||
}
|
||||
// Retain only normalized byte identity; no signed URL or payload.
|
||||
t.Size, t.Format, t.BodyHash = len(t.Body), "mp4", fmt.Sprintf("%x", sha256.Sum256(t.Body))
|
||||
t.Body = nil
|
||||
p := GooglevideoPass{Protocol: protocol, VerifiedTLS: true, Status: t.Status, RangeStart: 0, RangeEnd: end, ResourceBytes: total, Bytes: int64(t.Size), FTYP: true, LatencyMS: t.LatencyMS}
|
||||
p := GooglevideoPass{Protocol: protocol, VerifiedTLS: true, Status: t.Status, RangeStart: 0, RangeEnd: int64(t.Size) - 1, ResourceBytes: total, Bytes: int64(t.Size), FTYP: true, LatencyMS: t.LatencyMS}
|
||||
return t, p, nil
|
||||
}
|
||||
@@ -0,0 +1,127 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"fmt"
|
||||
"io"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestResourceCoreMediaRangeIntegers(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name, contentRange string
|
||||
total int64
|
||||
}{
|
||||
{"ordinary", "bytes 0-23/100", 100},
|
||||
{"max-valid", "bytes 0-23/9223372036854775807", 9223372036854775807},
|
||||
{"total-overflow", "bytes 0-23/9223372036854775808", 0},
|
||||
{"total-huge", "bytes 0-23/999999999999999999999999999999999999", 0},
|
||||
{"end-overflow", "bytes 0-9223372036854775808/9223372036854775807", 0},
|
||||
{"nonnumeric", "bytes 0-23/abc", 0},
|
||||
{"unknown", "bytes 0-23/*", 0},
|
||||
{"signed", "bytes 0-23/+100", 0},
|
||||
{"negative", "bytes 0-23/-100", 0},
|
||||
{"signed-end", "bytes 0-+23/100", 0},
|
||||
{"negative-end", "bytes 0--23/100", 0},
|
||||
{"total-equals-end", "bytes 0-23/23", 0},
|
||||
{"total-below-end", "bytes 0-23/22", 0},
|
||||
{"end-body-mismatch", "bytes 0-24/100", 0},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
raw := "https://rr1.unit.googlevideo.com/fresh?sig=private-secret"
|
||||
runner := func(_ context.Context, _ []string, _ io.Reader, out, diag io.Writer) error {
|
||||
var wire bytes.Buffer
|
||||
resourceWire(raw, "media", &wire, diag)
|
||||
fmt.Fprint(out, strings.Replace(wire.String(), "bytes 0-23/100", tc.contentRange, 1))
|
||||
return nil
|
||||
}
|
||||
transfer, _, err := collectResourceTransferOptions(context.Background(), "socks5://127.0.0.1:1080", raw, "rr1.unit.googlevideo.com", "tls12", runner, resourceTransferOptions{method: "GET", response: resourceResponseMP4, mediaType: "video/mp4"})
|
||||
valid := tc.total != 0
|
||||
if (err == nil) != valid || !valid && !reflect.DeepEqual(transfer, assetTransfer{}) {
|
||||
t.Fatalf("core valid=%v transfer=%+v err=%v", valid, transfer, err)
|
||||
}
|
||||
transfer, pass, err := collectMediaTransfer(context.Background(), "socks5://127.0.0.1:1080", mediaResource{raw, "rr1.unit.googlevideo.com", "video/mp4"}, "tls12", runner)
|
||||
if (err == nil) != valid {
|
||||
t.Fatalf("adapter valid=%v err=%v", valid, err)
|
||||
}
|
||||
if valid {
|
||||
if pass.ResourceBytes != tc.total || pass.Bytes != 24 || pass.RangeEnd != 23 || transfer.Size != 24 {
|
||||
t.Fatalf("incorrect normalized facts: %+v %+v", transfer, pass)
|
||||
}
|
||||
} else if !reflect.DeepEqual(transfer, assetTransfer{}) || !reflect.DeepEqual(pass, GooglevideoPass{}) {
|
||||
t.Fatalf("rejected range produced evidence: %+v %+v", transfer, pass)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// Unlike the adapter golden test, these assertions call the shared execution
|
||||
// entry directly: strict validation must not be left in a legacy-only branch.
|
||||
func TestResourceCoreTypedContracts(t *testing.T) {
|
||||
for _, mode := range []string{"pass", "status", "start", "end", "total", "malformed", "ftyp", "box", "decoder"} {
|
||||
for _, kind := range []resourceResponseKind{resourceResponseGeneric, resourceResponseImage, resourceResponseMP4} {
|
||||
t.Run(fmt.Sprintf("%d/%s", kind, mode), func(t *testing.T) {
|
||||
options := resourceTransferOptions{method: "GET", limit: 1024, timeout: time.Second, response: kind, mediaType: "video/mp4"}
|
||||
raw := "https://i.ytimg.com/fresh?sig=private-secret"
|
||||
runner := func(_ context.Context, _ []string, _ io.Reader, out, diag io.Writer) error {
|
||||
var wire bytes.Buffer
|
||||
wireMode := ""
|
||||
if kind == resourceResponseMP4 {
|
||||
wireMode = "media"
|
||||
}
|
||||
if mode == "decoder" && kind != resourceResponseMP4 {
|
||||
wireMode = "type"
|
||||
}
|
||||
resourceWire(raw, wireMode, &wire, diag)
|
||||
data := wire.String()
|
||||
switch mode {
|
||||
case "status":
|
||||
data = strings.ReplaceAll(data, "206", "200")
|
||||
if kind != resourceResponseMP4 {
|
||||
data = strings.ReplaceAll(data, "200", "201")
|
||||
}
|
||||
case "start":
|
||||
data = strings.Replace(data, "bytes 0-23/100", "bytes 1-24/100", 1)
|
||||
case "end":
|
||||
data = strings.Replace(data, "bytes 0-23/100", "bytes 0-24/100", 1)
|
||||
case "total":
|
||||
data = strings.Replace(data, "bytes 0-23/100", "bytes 0-23/23", 1)
|
||||
case "malformed":
|
||||
data = strings.Replace(data, "bytes 0-23/100", "bytes 0-23/*", 1)
|
||||
case "ftyp":
|
||||
data = strings.Replace(data, "ftyp", "xxxx", 1)
|
||||
case "box":
|
||||
data = strings.Replace(data, string([]byte{0, 0, 0, 24}), string([]byte{0, 0, 0, 25}), 1)
|
||||
}
|
||||
fmt.Fprint(out, data)
|
||||
return nil
|
||||
}
|
||||
transfer, _, err := collectResourceTransferOptions(context.Background(), "socks5://127.0.0.1:1080", raw, "i.ytimg.com", "tls12", runner, options)
|
||||
valid := kind == resourceResponseGeneric || mode == "pass" || kind == resourceResponseImage && mode != "decoder" && mode != "status" || kind == resourceResponseMP4 && mode == "decoder"
|
||||
if (err == nil) != valid {
|
||||
t.Fatalf("valid=%v err=%v", valid, err)
|
||||
}
|
||||
if valid && kind != resourceResponseGeneric && (transfer.Size != len(transfer.Body) || transfer.BodyHash == "" || transfer.Format == "") {
|
||||
t.Fatal("strict normalization not performed by core")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceCoreGenericExpectationFacts(t *testing.T) {
|
||||
raw := "https://i.ytimg.com/private"
|
||||
runner := func(_ context.Context, _ []string, _ io.Reader, out, diag io.Writer) error {
|
||||
resourceWire(raw, "", out, diag)
|
||||
return nil
|
||||
}
|
||||
expect := resourceCheckExpect{Statuses: []int{201}, MaxBytes: 1024}
|
||||
transfer, _, err := collectResourceTransferOptions(context.Background(), "socks5://127.0.0.1:1080", raw, "i.ytimg.com", "tls12", runner, resourceTransferOptions{method: "GET", limit: 1024, timeout: time.Second, expect: &expect})
|
||||
if err != errResourceExpectation || transfer.Status != 200 || len(transfer.Body) != len(resourceFixture()) || transfer.LatencyMS != 100 {
|
||||
t.Fatalf("core did not validate generic expectation with intact facts: %+v %v", transfer, err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,193 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/sha256"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
// Exercise the actual legacy adapters and generic producer with the same wire
|
||||
// mutations. This test also runs unchanged against the pre-migration source.
|
||||
func TestResourceCoreAdapterParity(t *testing.T) {
|
||||
for _, adapter := range []string{"image", "media", "GET", "HEAD"} {
|
||||
for _, protocol := range []string{"tls12", "tls13"} {
|
||||
for _, mode := range []string{"pass", "verify", "origin", "redirect", "tls", "framing", "oversize", "range", "ftyp", "decoder"} {
|
||||
t.Run(adapter+"/"+protocol+"/"+mode, func(t *testing.T) {
|
||||
host := "i.ytimg.com"
|
||||
if adapter == "media" {
|
||||
host = "rr1.unit.googlevideo.com"
|
||||
}
|
||||
raw := "https://" + host + "/fresh?sig=private-secret"
|
||||
calls := 0
|
||||
runner := func(_ context.Context, args []string, stdin io.Reader, out, diag io.Writer) error {
|
||||
calls++
|
||||
argv := strings.Join(args, " ")
|
||||
for _, required := range []string{"--disable --silent", "--http1.1 --raw", "--noproxy --proxy socks5h://127.0.0.1:1080", "--proto =https", "--suppress-connect-headers", "--config -"} {
|
||||
if !strings.Contains(argv, required) {
|
||||
t.Fatalf("missing %s", required)
|
||||
}
|
||||
}
|
||||
if strings.Contains(argv, "private-secret") || strings.Contains(argv, host) || strings.Contains(argv, "--insecure") || strings.Contains(argv, "--location") {
|
||||
t.Fatal("unsafe argv")
|
||||
}
|
||||
if strings.Contains(argv, "--range 0-65535") != (adapter == "media") || strings.Contains(argv, "--head") != (adapter == "HEAD") {
|
||||
t.Fatal("method/range changed")
|
||||
}
|
||||
config, _ := io.ReadAll(stdin)
|
||||
if !strings.Contains(string(config), raw) {
|
||||
t.Fatal("URL not in private stdin")
|
||||
}
|
||||
if adapter == "GET" || adapter == "HEAD" {
|
||||
if !strings.Contains(string(config), `connect-to = "i.ytimg.com:443:93.184.216.34:443"`) || !strings.Contains(string(config), "Authorization: private-header") {
|
||||
t.Fatal("private header or CONNECT pin lost")
|
||||
}
|
||||
}
|
||||
var wire, diagnostic bytes.Buffer
|
||||
wireMode := ""
|
||||
if adapter == "media" {
|
||||
wireMode = "media"
|
||||
}
|
||||
if mode == "decoder" && adapter == "image" {
|
||||
wireMode = "type"
|
||||
}
|
||||
resourceWire(raw, wireMode, &wire, &diagnostic)
|
||||
data := wire.String()
|
||||
if adapter == "HEAD" {
|
||||
start := strings.Index(data, "\r\n\r\n") + 4
|
||||
end := strings.LastIndex(data, resourceTrailer)
|
||||
data = data[:start] + data[end:]
|
||||
data = strings.Replace(data, fmt.Sprintf(`"size_download":%d`, len(resourceFixture())), `"size_download":0`, 1)
|
||||
}
|
||||
switch mode {
|
||||
case "verify":
|
||||
data = strings.Replace(data, `"ssl_verify_result":0`, `"ssl_verify_result":60`, 1)
|
||||
case "origin":
|
||||
data = strings.Replace(data, `"url_effective":"https://`, `"url_effective":"https://wrong.`, 1)
|
||||
case "redirect":
|
||||
data = strings.Replace(data, `"num_redirects":0`, `"num_redirects":1`, 1)
|
||||
case "framing":
|
||||
data = strings.Replace(data, "Content-Length:", "Content-Length: 1\r\nContent-Length:", 1)
|
||||
case "oversize":
|
||||
_, err := out.Write(make([]byte, 6<<20))
|
||||
return err
|
||||
case "range":
|
||||
data = strings.Replace(data, "bytes 0-23/100", "bytes 1-24/100", 1)
|
||||
case "ftyp":
|
||||
data = strings.Replace(data, "ftyp", "xxxx", 1)
|
||||
}
|
||||
fmt.Fprint(out, data)
|
||||
actual := "TLSv1.2"
|
||||
if protocol == "tls13" {
|
||||
actual = "TLSv1.3"
|
||||
}
|
||||
if mode == "tls" {
|
||||
actual = "TLSv1.1"
|
||||
}
|
||||
fmt.Fprintln(diag, "SSL connection using "+actual)
|
||||
return nil
|
||||
}
|
||||
valid := mode == "pass" || mode == "range" && adapter != "media" || mode == "ftyp" && adapter != "media" || mode == "decoder" && adapter != "image"
|
||||
var err error
|
||||
var transfer assetTransfer
|
||||
switch adapter {
|
||||
case "image":
|
||||
var m assetMeasurement
|
||||
m, err = collectAssetMeasurement(context.Background(), "socks5://127.0.0.1:1080", raw, host, protocol, "candidate", runner)
|
||||
if err == nil {
|
||||
transfer = m.Transfers[0]
|
||||
if calls != 3 || m.Source != "image-transfer" || m.Endpoint != host || m.Protocol != protocol || m.Args != "candidate" {
|
||||
t.Fatal("legacy measurement changed")
|
||||
}
|
||||
}
|
||||
case "media":
|
||||
var pass GooglevideoPass
|
||||
transfer, pass, err = collectMediaTransfer(context.Background(), "socks5://127.0.0.1:1080", mediaResource{raw, host, "video/mp4"}, protocol, runner)
|
||||
if err == nil && !reflect.DeepEqual(pass, GooglevideoPass{Protocol: protocol, VerifiedTLS: true, Status: 206, RangeStart: 0, RangeEnd: 23, ResourceBytes: 100, Bytes: 24, FTYP: true, LatencyMS: 100}) {
|
||||
t.Fatalf("legacy pass changed: %+v", pass)
|
||||
}
|
||||
default:
|
||||
d := resourceCheckDeps{guard: func() error { return nil }, resolve: func(context.Context, string, string) ([]net.IP, error) {
|
||||
return []net.IP{net.ParseIP("93.184.216.34")}, nil
|
||||
}, runner: runner}
|
||||
r := resourceCheckSpec{ID: "r", Group: "g", URL: raw, host: host, Method: adapter, TimeoutMS: 1000, Expect: resourceCheckExpect{Statuses: []int{200}, ContentType: "image/png", MaxBytes: 1024}, privateConfig: "header = \"Authorization: private-header\"\n"}
|
||||
result, e := runResourceProtocol(context.Background(), "s", r, protocol, "dns", "socks5://127.0.0.1:1080", d)
|
||||
err = e
|
||||
if valid {
|
||||
wantBytes := len(resourceFixture())
|
||||
if adapter == "HEAD" {
|
||||
wantBytes = 0
|
||||
}
|
||||
if result != (resourceCheckResult{Service: "s", Resource: "r", Group: "g", TLS: protocol, Outcome: "pass", Status: 200, Bytes: wantBytes, TimeMS: 100}) {
|
||||
t.Fatalf("generic result changed: %+v", result)
|
||||
}
|
||||
} else if result.Outcome == "pass" {
|
||||
t.Fatal("unsafe generic pass")
|
||||
}
|
||||
if err == nil && !valid {
|
||||
err = fmt.Errorf("rejected transfer")
|
||||
}
|
||||
}
|
||||
if (err == nil) != valid {
|
||||
t.Fatalf("valid=%v err=%v", valid, err)
|
||||
}
|
||||
if valid && (adapter == "image" || adapter == "media") {
|
||||
body, format, status, mime := resourceFixture(), "png", 200, "image/png"
|
||||
if adapter == "media" {
|
||||
body, format, status, mime = []byte{0, 0, 0, 24, 'f', 't', 'y', 'p', 'i', 's', 'o', 'm', 0, 0, 0, 0, 'i', 's', 'o', 'm', 'm', 'p', '4', '2'}, "mp4", 206, "video/mp4"
|
||||
}
|
||||
version := uint16(0x0303)
|
||||
if protocol == "tls13" {
|
||||
version = 0x0304
|
||||
}
|
||||
verify := 0
|
||||
want := assetTransfer{EffectiveHost: host, SSLVerifyResult: &verify, TLSVersion: version, Status: status, ContentType: mime, Size: len(body), Format: format, BodyHash: fmt.Sprintf("%x", sha256.Sum256(body)), LatencyMS: 100, SpeedKBPS: 10000.0 / 1024}
|
||||
if !reflect.DeepEqual(transfer, want) {
|
||||
t.Fatalf("normalized facts changed: got %+v want %+v", transfer, want)
|
||||
}
|
||||
encoded, _ := json.Marshal(transfer)
|
||||
if strings.Contains(string(encoded), "private-secret") {
|
||||
t.Fatal("private evidence")
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceCoreGenericMismatchParity(t *testing.T) {
|
||||
for _, field := range []string{"status", "mime", "bytes"} {
|
||||
for _, qualification := range []bool{false, true} {
|
||||
t.Run(fmt.Sprintf("%s/qualification=%v", field, qualification), func(t *testing.T) {
|
||||
raw := "https://i.ytimg.com/fresh?sig=private-secret"
|
||||
r := resourceCheckSpec{ID: "r", Group: "g", URL: raw, host: "i.ytimg.com", Method: "GET", TimeoutMS: 1000, Expect: resourceCheckExpect{Statuses: []int{200}, ContentType: "image/png", MaxBytes: 1024}}
|
||||
switch field {
|
||||
case "status":
|
||||
r.Expect.Statuses = []int{201}
|
||||
case "mime":
|
||||
r.Expect.ContentType = "image/jpeg"
|
||||
case "bytes":
|
||||
r.Expect.MinBytes = 1000
|
||||
}
|
||||
d := resourceCheckDeps{qualification: qualification, guard: func() error { return nil }, resolve: func(context.Context, string, string) ([]net.IP, error) {
|
||||
return []net.IP{net.ParseIP("93.184.216.34")}, nil
|
||||
}, runner: func(_ context.Context, _ []string, _ io.Reader, out, diag io.Writer) error {
|
||||
resourceWire(raw, "", out, diag)
|
||||
return nil
|
||||
}}
|
||||
result, err := runResourceProtocol(context.Background(), "s", r, "tls12", "dns", "socks5://127.0.0.1:1080", d)
|
||||
want := resourceCheckResult{Service: "s", Resource: "r", Group: "g", TLS: "tls12", Outcome: "expectation_failed", Bytes: len(resourceFixture()), TimeMS: 100, Status: 200}
|
||||
if err != nil || result != want {
|
||||
t.Fatalf("result=%+v err=%v", result, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -74,6 +74,9 @@ func readResourceMonitorSource(path string, lookup func(string) (string, bool),
|
||||
func resourceRowFingerprint(key []byte, source resourceMonitorSource, r resourceCheckSpec) string {
|
||||
b, _ := json.Marshal(r)
|
||||
b, _ = json.Marshal([]string{string(b), r.privateConfig, source.server, source.proxy})
|
||||
if source.config.Version == 2 {
|
||||
b, _ = json.Marshal([]string{"resource-check-v2", string(b)})
|
||||
}
|
||||
return resourceMAC(key, b)
|
||||
}
|
||||
func reconcileResourceRows(key []byte, source resourceMonitorSource, prior []resourceMonitorRow) []resourceMonitorRow {
|
||||
@@ -345,6 +348,9 @@ func resourceMonitorCLIWithCheck(ctx context.Context, path string, stdout, stder
|
||||
if resourceManagedIdentity(target.directory) != nil {
|
||||
return errResourceState
|
||||
}
|
||||
if resourceDeploymentIdentity(path, target) != nil {
|
||||
return errResourceState
|
||||
}
|
||||
return check.guard()
|
||||
},
|
||||
probe: func(ctx context.Context, service string, spec resourceCheckSpec, source resourceMonitorSource, guard func() error) (string, error) {
|
||||
@@ -394,6 +400,9 @@ func resourceStatusCLIAt(stdout io.Writer, target managedDeps, proc string) int
|
||||
if err != nil || check.guard() != nil || resourceManagedIdentity(target.directory) != nil {
|
||||
return blocked()
|
||||
}
|
||||
if resourceDeploymentIdentity(path, target) != nil {
|
||||
return blocked()
|
||||
}
|
||||
rows := reconcileResourceRows(store.key, source, state.Rows)
|
||||
b1, _ := json.Marshal(rows)
|
||||
b2, _ := json.Marshal(state.Rows)
|
||||
|
||||
@@ -0,0 +1,339 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"crypto/sha256"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
var errResourcePlan = errors.New("invalid private qualification data")
|
||||
|
||||
// Input groups are metadata. Only the HTTPS collector's selectors are supported.
|
||||
type strategySelectors struct {
|
||||
Transport string `json:"transport"`
|
||||
Ports string `json:"ports"`
|
||||
L7 string `json:"l7"`
|
||||
TLS string `json:"tls"`
|
||||
}
|
||||
type strategyDefinition struct {
|
||||
ID string `json:"id"`
|
||||
Args string `json:"args"`
|
||||
Selectors strategySelectors `json:"selectors"`
|
||||
}
|
||||
type strategyRegistry struct {
|
||||
Version int `json:"version"`
|
||||
Strategies []strategyDefinition `json:"strategies"`
|
||||
}
|
||||
|
||||
func strictPlanJSON(b []byte, dst interface{}) error {
|
||||
if len(b) > 65536 {
|
||||
return errResourcePlan
|
||||
}
|
||||
d := json.NewDecoder(bytes.NewReader(b))
|
||||
if uniqueResourceJSON(d, 0) != nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
if _, err := d.Token(); err != io.EOF {
|
||||
return errResourcePlan
|
||||
}
|
||||
d = json.NewDecoder(bytes.NewReader(b))
|
||||
d.DisallowUnknownFields()
|
||||
if d.Decode(dst) != nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func planDigest(v interface{}) string {
|
||||
b, _ := json.Marshal(v)
|
||||
h := sha256.Sum256(b)
|
||||
return hex.EncodeToString(h[:])
|
||||
}
|
||||
func canonicalStrategy(d strategyDefinition) (strategyDefinition, error) {
|
||||
d.Args = strings.Join(strings.Fields(d.Args), " ")
|
||||
if !resourceID.MatchString(d.ID) || d.Args == "" || len(d.Args) > 4096 || (&candidate{Args: d.Args}).validateArgs() != nil {
|
||||
return d, errResourcePlan
|
||||
}
|
||||
if d.Selectors.Transport != "tcp" || d.Selectors.Ports != "443" || d.Selectors.L7 != "tls" || (d.Selectors.TLS != "both" && d.Selectors.TLS != "tls12" && d.Selectors.TLS != "tls13") {
|
||||
return d, errResourcePlan
|
||||
}
|
||||
for _, arg := range strings.Fields(d.Args) {
|
||||
option, value, _ := strings.Cut(arg, "=")
|
||||
if option == "--lua-init" {
|
||||
return d, errResourcePlan
|
||||
}
|
||||
if option != "--lua-desync" {
|
||||
continue
|
||||
}
|
||||
parts := strings.Split(value, ":")
|
||||
switch parts[0] {
|
||||
case "fake", "multisplit", "multidisorder", "fakeddisorder", "split", "disorder":
|
||||
default:
|
||||
return d, errResourcePlan
|
||||
}
|
||||
for _, part := range parts[1:] {
|
||||
key, v, ok := strings.Cut(part, "=")
|
||||
if !ok || v == "" || strings.ContainsAny(v, "/@%") || strings.Contains(v, "..") {
|
||||
return d, errResourcePlan
|
||||
}
|
||||
switch key {
|
||||
case "pos", "repeats", "tcp_ts", "tcp_flags_set", "seqovl":
|
||||
case "blob":
|
||||
if v != "fake_default_tls" {
|
||||
return d, errResourcePlan
|
||||
}
|
||||
default:
|
||||
return d, errResourcePlan
|
||||
}
|
||||
}
|
||||
}
|
||||
return d, nil
|
||||
}
|
||||
func parseStrategyRegistry(b []byte) (strategyRegistry, error) {
|
||||
var r strategyRegistry
|
||||
if strictPlanJSON(b, &r) != nil || r.Version != 1 || len(r.Strategies) == 0 || len(r.Strategies) > 32 {
|
||||
return r, errResourcePlan
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
for i, d := range r.Strategies {
|
||||
v, err := canonicalStrategy(d)
|
||||
if err != nil || seen[v.ID] {
|
||||
return r, errResourcePlan
|
||||
}
|
||||
seen[v.ID] = true
|
||||
r.Strategies[i] = v
|
||||
}
|
||||
sort.Slice(r.Strategies, func(i, j int) bool { return r.Strategies[i].ID < r.Strategies[j].ID })
|
||||
return r, nil
|
||||
}
|
||||
|
||||
type qualificationCheck struct {
|
||||
Service string `json:"service"`
|
||||
Resource string `json:"resource"`
|
||||
Fingerprint string `json:"fingerprint"`
|
||||
TLS string `json:"tls"`
|
||||
Outcome string `json:"outcome"`
|
||||
Attempts int `json:"attempts"`
|
||||
}
|
||||
type qualificationRun struct {
|
||||
ID string `json:"id"`
|
||||
Strategy string `json:"strategy"`
|
||||
Definition string `json:"definition"`
|
||||
Config string `json:"config"`
|
||||
Started int64 `json:"started_ms"`
|
||||
Completed int64 `json:"completed_ms"`
|
||||
Attestation qualificationAttestation `json:"attestation"`
|
||||
Checks []qualificationCheck `json:"checks"`
|
||||
}
|
||||
type qualificationAttestation struct {
|
||||
Collector string `json:"collector"`
|
||||
Runtime string `json:"runtime"`
|
||||
Definition string `json:"definition"`
|
||||
LuaHash string `json:"lua_hash"`
|
||||
RulesHash string `json:"rules_hash"`
|
||||
WorkerArgsHash string `json:"worker_args_hash"`
|
||||
Queue int `json:"queue"`
|
||||
PID int `json:"pid"`
|
||||
StartTime int64 `json:"starttime"`
|
||||
Restored bool `json:"restored"`
|
||||
Joined bool `json:"joined"`
|
||||
}
|
||||
type qualificationMatrix struct {
|
||||
Version int `json:"version"`
|
||||
Registry string `json:"registry"`
|
||||
Runs []qualificationRun `json:"runs"`
|
||||
}
|
||||
|
||||
// Only the owned producer (or internal test fixtures) constructs this capability.
|
||||
// JSON imports cannot authorize writes, and post-seal DTO mutation is rejected.
|
||||
type completedQualification struct {
|
||||
matrix qualificationMatrix
|
||||
seal string
|
||||
}
|
||||
|
||||
func hashOK(s string) bool {
|
||||
b, e := hex.DecodeString(s)
|
||||
return e == nil && len(b) == 32 && strings.ToLower(s) == s
|
||||
}
|
||||
func validateMatrix(m qualificationMatrix, registry strategyRegistry) error {
|
||||
if m.Version != 1 || m.Registry != planDigest(registry) || len(m.Runs) > 32 {
|
||||
return errResourcePlan
|
||||
}
|
||||
defs := map[string]strategyDefinition{}
|
||||
for _, d := range registry.Strategies {
|
||||
defs[d.ID] = d
|
||||
}
|
||||
ids, strategies := map[string]bool{}, map[string]bool{}
|
||||
for _, r := range m.Runs {
|
||||
d, ok := defs[r.Strategy]
|
||||
a := r.Attestation
|
||||
if !ok || !resourceID.MatchString(r.ID) || ids[r.ID] || strategies[r.Strategy] || r.Definition != planDigest(d) || !hashOK(r.Config) || r.Started <= 0 || r.Completed < r.Started || r.Completed-r.Started > 86400000 || len(r.Checks) == 0 || len(r.Checks) > 128 {
|
||||
return errResourcePlan
|
||||
}
|
||||
ids[r.ID], strategies[r.Strategy] = true, true
|
||||
if a.Collector != "owned-resource-nfq-v1" || a.Runtime != "nfqws2-1.0.2" || a.Definition != r.Definition || !hashOK(a.LuaHash) || !hashOK(a.RulesHash) || !hashOK(a.WorkerArgsHash) || a.Queue != managedAssetQueue || a.PID <= 0 || a.StartTime <= 0 || !a.Restored || !a.Joined {
|
||||
return errResourcePlan
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
for _, c := range r.Checks {
|
||||
id := c.Service + "/" + c.Resource + "/" + c.TLS
|
||||
if !resourceID.MatchString(c.Service) || !resourceID.MatchString(c.Resource) || !hashOK(c.Fingerprint) || seen[id] || c.Attempts != 3 || (c.TLS != "tls12" && c.TLS != "tls13") {
|
||||
return errResourcePlan
|
||||
}
|
||||
seen[id] = true
|
||||
switch c.Outcome {
|
||||
case "pass", "dns_failed", "destination_denied", "transfer_failed", "expectation_failed", "timeout":
|
||||
default:
|
||||
return errResourcePlan
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
type domainCompatibility struct {
|
||||
Host string `json:"host"`
|
||||
Validated []string `json:"validated"`
|
||||
Negative []string `json:"negative"`
|
||||
Unknown []string `json:"unknown"`
|
||||
}
|
||||
type runtimeAssignment struct {
|
||||
Strategy string `json:"strategy"`
|
||||
Args string `json:"args"`
|
||||
Selectors strategySelectors `json:"selectors"`
|
||||
Hosts []string `json:"hosts"`
|
||||
}
|
||||
type resourcePlan struct {
|
||||
ProposalOnly bool `json:"proposal_only"`
|
||||
Policy string `json:"strategy_ids,omitempty"`
|
||||
Evidence string `json:"evidence"`
|
||||
Domains []domainCompatibility `json:"domains"`
|
||||
Assignments []runtimeAssignment `json:"assignments"`
|
||||
Unassigned []string `json:"unassigned"`
|
||||
}
|
||||
|
||||
func deriveResourcePlan(source resourceMonitorSource, key []byte, registry strategyRegistry, matrix qualificationMatrix, now time.Time, policy ...string) resourcePlan {
|
||||
p := resourcePlan{ProposalOnly: true, Evidence: "unknown", Domains: []domainCompatibility{}, Assignments: []runtimeAssignment{}, Unassigned: []string{}}
|
||||
if len(policy) != 0 {
|
||||
p.Policy = policy[0]
|
||||
}
|
||||
hosts := map[string][]qualificationCheck{}
|
||||
for _, s := range source.config.Services {
|
||||
for _, r := range s.Resources {
|
||||
protocols := []string{r.TLS}
|
||||
if r.TLS == "both" {
|
||||
protocols = []string{"tls12", "tls13"}
|
||||
}
|
||||
for _, tls := range protocols {
|
||||
host := strings.ToLower(r.host)
|
||||
hosts[host] = append(hosts[host], qualificationCheck{Service: s.ID, Resource: r.ID, Fingerprint: resourceRowFingerprint(key, source, r), TLS: tls})
|
||||
}
|
||||
}
|
||||
}
|
||||
runs := map[string]qualificationRun{}
|
||||
if validateMatrix(matrix, registry) == nil {
|
||||
p.Evidence = "authenticated"
|
||||
for _, r := range matrix.Runs {
|
||||
if r.Config == source.fingerprint && r.Completed <= now.UnixMilli() && now.UnixMilli()-r.Completed <= 86400000 {
|
||||
runs[r.Strategy] = r
|
||||
}
|
||||
}
|
||||
}
|
||||
names := []string{}
|
||||
for host := range hosts {
|
||||
names = append(names, host)
|
||||
}
|
||||
sort.Strings(names)
|
||||
for _, host := range names {
|
||||
row := domainCompatibility{Host: host, Validated: []string{}, Negative: []string{}, Unknown: []string{}}
|
||||
for _, d := range registry.Strategies {
|
||||
r, exists := runs[d.ID]
|
||||
checks := map[string]qualificationCheck{}
|
||||
for _, c := range r.Checks {
|
||||
checks[c.Service+"/"+c.Resource+"/"+c.TLS] = c
|
||||
}
|
||||
complete, failed := exists, false
|
||||
for _, expected := range hosts[host] {
|
||||
c, ok := checks[expected.Service+"/"+expected.Resource+"/"+expected.TLS]
|
||||
if !ok || c.Fingerprint != expected.Fingerprint || (d.Selectors.TLS != "both" && d.Selectors.TLS != expected.TLS) {
|
||||
complete = false
|
||||
continue
|
||||
}
|
||||
if c.Outcome != "pass" {
|
||||
failed = true
|
||||
}
|
||||
}
|
||||
if !complete {
|
||||
row.Unknown = append(row.Unknown, d.ID)
|
||||
} else if failed {
|
||||
row.Negative = append(row.Negative, d.ID)
|
||||
} else {
|
||||
row.Validated = append(row.Validated, d.ID)
|
||||
}
|
||||
}
|
||||
p.Domains = append(p.Domains, row)
|
||||
}
|
||||
// Merge only equal canonical args/selectors, retaining per-candidate alternatives.
|
||||
// Greatest uncovered coverage is deterministic greedy, not globally optimal.
|
||||
profiles := map[string]runtimeAssignment{}
|
||||
coverage := map[string]map[string]bool{}
|
||||
for _, d := range registry.Strategies {
|
||||
if len(policy) != 0 && policy[0] != "" && !strings.Contains(","+policy[0]+",", ","+d.ID+",") {
|
||||
continue
|
||||
}
|
||||
profile := planDigest(struct {
|
||||
Args string
|
||||
Selectors strategySelectors
|
||||
}{d.Args, d.Selectors})
|
||||
a, exists := profiles[profile]
|
||||
if !exists || d.ID < a.Strategy {
|
||||
profiles[profile] = runtimeAssignment{Strategy: d.ID, Args: d.Args, Selectors: d.Selectors}
|
||||
}
|
||||
if coverage[profile] == nil {
|
||||
coverage[profile] = map[string]bool{}
|
||||
}
|
||||
for _, row := range p.Domains {
|
||||
for _, id := range row.Validated {
|
||||
if id == d.ID {
|
||||
coverage[profile][row.Host] = true
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
assigned := map[string]bool{}
|
||||
for {
|
||||
best := ""
|
||||
var chosen []string
|
||||
for profile, a := range profiles {
|
||||
uncovered := []string{}
|
||||
for _, host := range names {
|
||||
if coverage[profile][host] && !assigned[host] {
|
||||
uncovered = append(uncovered, host)
|
||||
}
|
||||
}
|
||||
if len(uncovered) > len(chosen) || len(uncovered) > 0 && len(uncovered) == len(chosen) && (best == "" || a.Strategy < profiles[best].Strategy) {
|
||||
best, chosen = profile, uncovered
|
||||
}
|
||||
}
|
||||
if len(chosen) == 0 {
|
||||
break
|
||||
}
|
||||
a := profiles[best]
|
||||
a.Hosts = chosen
|
||||
p.Assignments = append(p.Assignments, a)
|
||||
for _, host := range chosen {
|
||||
assigned[host] = true
|
||||
}
|
||||
}
|
||||
for _, host := range names {
|
||||
if !assigned[host] {
|
||||
p.Unassigned = append(p.Unassigned, host)
|
||||
}
|
||||
}
|
||||
return p
|
||||
}
|
||||
@@ -0,0 +1,300 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"crypto/hmac"
|
||||
"crypto/rand"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
"syscall"
|
||||
"time"
|
||||
)
|
||||
|
||||
const planKeyName = ".resource-plan-key"
|
||||
const planRegistryName = ".resource-plan-strategies.json"
|
||||
const planMatrixName = ".resource-plan-matrix.json"
|
||||
|
||||
// Reuse held-directory/private-entry checks, not monitor publication/lifecycle.
|
||||
// Entries are write-once O_EXCL: no replacement, repair or clobber of foreign files.
|
||||
type resourcePlanStore struct {
|
||||
files *resourceMonitorStore
|
||||
held map[string]*os.File
|
||||
snapshots map[string][]byte
|
||||
}
|
||||
type planEnvelope struct {
|
||||
Payload json.RawMessage `json:"payload"`
|
||||
MAC string `json:"mac"`
|
||||
}
|
||||
|
||||
func openResourcePlanStore(path string, create bool) (*resourcePlanStore, error) {
|
||||
if managedDirectory(path) != nil {
|
||||
return nil, errResourcePlan
|
||||
}
|
||||
fd, err := syscall.Open(path, syscall.O_RDONLY|syscall.O_DIRECTORY|syscall.O_NOFOLLOW|syscall.O_CLOEXEC, 0)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
s := &resourcePlanStore{files: &resourceMonitorStore{path: path, dir: os.NewFile(uintptr(fd), "plan directory")}, held: map[string]*os.File{}, snapshots: map[string][]byte{}}
|
||||
fail := func(err error) (*resourcePlanStore, error) { s.close(); return nil, err }
|
||||
for _, name := range []string{planKeyName, planRegistryName, planMatrixName} {
|
||||
f, b, e := s.files.openEntry(name)
|
||||
if e != nil && !errors.Is(e, os.ErrNotExist) {
|
||||
return fail(errResourcePlan)
|
||||
}
|
||||
if e == nil {
|
||||
s.held[name] = f
|
||||
s.snapshots[name] = b
|
||||
}
|
||||
}
|
||||
key := s.snapshots[planKeyName]
|
||||
if key == nil && create && s.held[planRegistryName] == nil && s.held[planMatrixName] == nil {
|
||||
key = make([]byte, 32)
|
||||
if _, err = rand.Read(key); err != nil {
|
||||
return fail(errResourcePlan)
|
||||
}
|
||||
if s.publish(planKeyName, key) != nil {
|
||||
return fail(errResourcePlan)
|
||||
}
|
||||
}
|
||||
if len(key) != 32 {
|
||||
if !create && key == nil && s.held[planRegistryName] == nil && s.held[planMatrixName] == nil {
|
||||
return fail(os.ErrNotExist)
|
||||
}
|
||||
return fail(errResourcePlan)
|
||||
}
|
||||
s.files.key = append([]byte(nil), key...)
|
||||
if s.guard() != nil {
|
||||
return fail(errResourcePlan)
|
||||
}
|
||||
return s, nil
|
||||
}
|
||||
func (s *resourcePlanStore) close() {
|
||||
for _, f := range s.held {
|
||||
f.Close()
|
||||
}
|
||||
s.files.dir.Close()
|
||||
}
|
||||
func (s *resourcePlanStore) guard() error {
|
||||
if s.files.guard() != nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
names := map[string]bool{planKeyName: true, planRegistryName: true, planMatrixName: true}
|
||||
for name := range s.held {
|
||||
names[name] = true
|
||||
}
|
||||
for name := range names {
|
||||
f := s.held[name]
|
||||
if f == nil {
|
||||
g, _, e := s.files.openEntry(name)
|
||||
if g != nil {
|
||||
g.Close()
|
||||
}
|
||||
if !errors.Is(e, os.ErrNotExist) {
|
||||
return errResourcePlan
|
||||
}
|
||||
continue
|
||||
}
|
||||
b, e := s.files.trustedRead(name, f)
|
||||
if e != nil || !bytes.Equal(b, s.snapshots[name]) {
|
||||
return errResourcePlan
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func (s *resourcePlanStore) publish(name string, b []byte) error {
|
||||
if len(b) > 65536 || s.guard() != nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
fd, e := syscall.Openat(int(s.files.dir.Fd()), name, syscall.O_RDWR|syscall.O_CREAT|syscall.O_EXCL|syscall.O_NOFOLLOW|syscall.O_CLOEXEC, 0600)
|
||||
if e != nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
f := os.NewFile(uintptr(fd), name)
|
||||
// Keep incomplete entries on failure: never delete a possible foreign inode.
|
||||
if _, e = f.Write(b); e != nil {
|
||||
f.Close()
|
||||
return errResourcePlan
|
||||
}
|
||||
if f.Sync() != nil || s.files.entryMatches(name, f) != nil {
|
||||
f.Close()
|
||||
return errResourcePlan
|
||||
}
|
||||
read, e := s.files.trustedRead(name, f)
|
||||
if e != nil || !bytes.Equal(read, b) {
|
||||
f.Close()
|
||||
return errResourcePlan
|
||||
}
|
||||
s.held[name], s.snapshots[name] = f, append([]byte(nil), b...)
|
||||
if s.files.dir.Sync() != nil || s.guard() != nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
return nil
|
||||
}
|
||||
func (s *resourcePlanStore) writeAuthenticated(name string, v interface{}) error {
|
||||
b, e := json.Marshal(v)
|
||||
if e != nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
envelope, e := json.Marshal(planEnvelope{Payload: b, MAC: resourceMAC(s.files.key, b)})
|
||||
if e != nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
return s.publish(name, envelope)
|
||||
}
|
||||
func (s *resourcePlanStore) readAuthenticated(name string, dst interface{}) error {
|
||||
if s.guard() != nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
if s.held[name] == nil {
|
||||
return os.ErrNotExist
|
||||
}
|
||||
var envelope planEnvelope
|
||||
if strictPlanJSON(s.snapshots[name], &envelope) != nil || !hmac.Equal([]byte(envelope.MAC), []byte(resourceMAC(s.files.key, envelope.Payload))) || strictPlanJSON(envelope.Payload, dst) != nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
return s.guard()
|
||||
}
|
||||
func (s *resourcePlanStore) registry() (strategyRegistry, error) {
|
||||
var r strategyRegistry
|
||||
if e := s.readAuthenticated(planRegistryName, &r); e != nil {
|
||||
return r, e
|
||||
}
|
||||
b, _ := json.Marshal(r)
|
||||
canonical, e := parseStrategyRegistry(b)
|
||||
if e != nil || planDigest(canonical) != planDigest(r) {
|
||||
return r, errResourcePlan
|
||||
}
|
||||
return r, nil
|
||||
}
|
||||
func (s *resourcePlanStore) importDefinitions(b []byte) error {
|
||||
r, e := parseStrategyRegistry(b)
|
||||
if e != nil {
|
||||
return e
|
||||
}
|
||||
return s.writeAuthenticated(planRegistryName, r)
|
||||
}
|
||||
func (s *resourcePlanStore) saveCompleted(p completedQualification) error {
|
||||
r, e := s.registry()
|
||||
if e != nil || p.seal == "" || p.seal != planDigest(p.matrix) || validateMatrix(p.matrix, r) != nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
return s.writeAuthenticated(planMatrixName, p.matrix)
|
||||
}
|
||||
func (s *resourcePlanStore) matrix(r strategyRegistry) (qualificationMatrix, error) {
|
||||
var m qualificationMatrix
|
||||
e := s.readAuthenticated(planMatrixName, &m)
|
||||
if e != nil {
|
||||
return m, e
|
||||
}
|
||||
if validateMatrix(m, r) != nil {
|
||||
return m, errResourcePlan
|
||||
}
|
||||
return m, nil
|
||||
}
|
||||
|
||||
func resourcePlanCLI(args []string, stdout, stderr io.Writer, d managedDeps) int {
|
||||
if os.Geteuid() != 0 {
|
||||
fmt.Fprintln(stderr, "blocked: root target role required")
|
||||
return 78
|
||||
}
|
||||
if _, err := managedTargetSettings(d.lookup); err != nil || d.directory != managedTargetDirectory {
|
||||
fmt.Fprintln(stderr, "blocked: managed target required")
|
||||
return 78
|
||||
}
|
||||
if len(args) == 6 && args[0] == "qualify" && args[1] == "--config" && args[3] == "--strategies" && args[5] == "--isolated" {
|
||||
return qualifyResourceCLI(args[2], args[4], stdout, stderr, d)
|
||||
}
|
||||
if len(args) == 3 && args[0] == "qualify" && args[1] == "--config" {
|
||||
fmt.Fprintln(stderr, "unknown: qualification requires --isolated and PID 1 isolated startup; no runtime changes made")
|
||||
return 78
|
||||
}
|
||||
if len(args) == 5 && args[0] == "qualify" && args[1] == "--config" && args[3] == "--strategies" {
|
||||
fmt.Fprintln(stderr, "unknown: qualification requires --isolated and PID 1 isolated startup; no runtime changes made")
|
||||
return 78
|
||||
}
|
||||
if len(args) == 3 && args[0] == "strategy-import" && args[1] == "--strategies" {
|
||||
fd, e := syscall.Open(args[2], syscall.O_RDONLY|syscall.O_NOFOLLOW|syscall.O_NONBLOCK|syscall.O_CLOEXEC, 0)
|
||||
if e != nil {
|
||||
fmt.Fprintln(stderr, "configerror: definitions unavailable")
|
||||
return 64
|
||||
}
|
||||
f := os.NewFile(uintptr(fd), "definitions")
|
||||
defer f.Close()
|
||||
if resourcePrivateEntry(f) != nil {
|
||||
fmt.Fprintln(stderr, "configerror: definitions require owned regular 0600 file")
|
||||
return 64
|
||||
}
|
||||
b, e := io.ReadAll(io.LimitReader(f, 65537))
|
||||
current, statErr := os.Lstat(args[2])
|
||||
held, heldErr := f.Stat()
|
||||
if e != nil || statErr != nil || heldErr != nil || !os.SameFile(current, held) || resourcePrivateEntry(f) != nil {
|
||||
return 64
|
||||
}
|
||||
s, e := openResourcePlanStore(d.directory, true)
|
||||
if e != nil {
|
||||
fmt.Fprintln(stderr, "blocked: private registry unavailable")
|
||||
return 78
|
||||
}
|
||||
defer s.close()
|
||||
if s.importDefinitions(b) != nil {
|
||||
fmt.Fprintln(stderr, "blocked: invalid definitions or immutable registry already exists")
|
||||
return 78
|
||||
}
|
||||
fmt.Fprintln(stdout, "strategy definitions imported; qualification UNKNOWN")
|
||||
return 0
|
||||
}
|
||||
if len(args) != 3 || args[0] != "plan" || args[1] != "--config" {
|
||||
fmt.Fprintln(stderr, "usage: plan --config FILE | strategy-import --strategies FILE | qualify --config FILE --strategies FILE --isolated")
|
||||
return 64
|
||||
}
|
||||
return resourcePlanProposal(args[2], stdout, stderr, d)
|
||||
}
|
||||
|
||||
// Internal reader separated only to test the real CLI trust boundary in a
|
||||
// temporary owned directory, without touching the provisioned target.
|
||||
func resourcePlanProposal(configPath string, stdout, stderr io.Writer, d managedDeps) int {
|
||||
key := make([]byte, 32)
|
||||
registry := strategyRegistry{Version: 1, Strategies: []strategyDefinition{}}
|
||||
var matrix qualificationMatrix
|
||||
s, e := openResourcePlanStore(d.directory, false)
|
||||
blocked := e != nil && !errors.Is(e, os.ErrNotExist)
|
||||
if e == nil {
|
||||
defer s.close()
|
||||
key = s.files.key
|
||||
registry, e = s.registry()
|
||||
if e != nil {
|
||||
blocked = !errors.Is(e, os.ErrNotExist)
|
||||
registry = strategyRegistry{Version: 1, Strategies: []strategyDefinition{}}
|
||||
}
|
||||
if e == nil {
|
||||
matrix, e = s.matrix(registry)
|
||||
if e != nil {
|
||||
blocked = !errors.Is(e, os.ErrNotExist)
|
||||
matrix = qualificationMatrix{}
|
||||
}
|
||||
}
|
||||
}
|
||||
source, e := readResourceMonitorSource(configPath, d.lookup, key)
|
||||
if e != nil {
|
||||
fmt.Fprintln(stderr, "configerror: resource configuration unavailable")
|
||||
return 64
|
||||
}
|
||||
p := deriveResourcePlan(source, key, registry, matrix, time.Now())
|
||||
if blocked {
|
||||
p.Evidence = "blocked"
|
||||
}
|
||||
if s != nil && s.guard() != nil {
|
||||
p = deriveResourcePlan(source, key, registry, qualificationMatrix{}, time.Now())
|
||||
p.Evidence = "blocked"
|
||||
}
|
||||
if json.NewEncoder(stdout).Encode(p) != nil {
|
||||
return 1
|
||||
}
|
||||
if blocked || p.Evidence == "blocked" {
|
||||
return 78
|
||||
}
|
||||
return 0
|
||||
}
|
||||
@@ -0,0 +1,331 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func planFixture(t *testing.T) (resourceMonitorSource, []byte, strategyRegistry, qualificationMatrix, time.Time) {
|
||||
t.Helper()
|
||||
key := bytes.Repeat([]byte{7}, 32)
|
||||
now := time.Unix(1700000000, 0)
|
||||
source := resourceMonitorSource{server: "127.0.0.1:5353", proxy: "socks5h://127.0.0.1:1080", fingerprint: resourceMAC(key, []byte("config-secret")), config: resourceCheckConfig{Version: 1}}
|
||||
for i, host := range []string{"a.example", "b.example", "c.example", "d.example", "e.example", "a.example"} {
|
||||
service := "alpha"
|
||||
group := "group-a"
|
||||
if i == 2 {
|
||||
service = "beta"
|
||||
group = "group-b"
|
||||
}
|
||||
if i == 5 {
|
||||
service = "gamma"
|
||||
group = "group-c"
|
||||
}
|
||||
source.config.Services = append(source.config.Services, resourceCheckService{ID: service + string(rune('a'+i)), Resources: []resourceCheckSpec{{ID: "resource", Group: group, host: host, TLS: "both", URL: "https://" + host + "/SECRET?token=SECRET", privateConfig: "header = SECRET"}}})
|
||||
}
|
||||
registry := strategyRegistry{Version: 1}
|
||||
for i, args := range []string{"--lua-desync=multisplit:pos=1", "--lua-desync=multidisorder:pos=1", "--lua-desync=fake:repeats=2"} {
|
||||
registry.Strategies = append(registry.Strategies, strategyDefinition{ID: "s" + string(rune('1'+i)), Args: args, Selectors: strategySelectors{Transport: "tcp", Ports: "443", L7: "tls", TLS: "both"}})
|
||||
}
|
||||
matrix := qualificationMatrix{Version: 1, Registry: planDigest(registry), Runs: []qualificationRun{}}
|
||||
for _, d := range registry.Strategies {
|
||||
digest := planDigest(d)
|
||||
r := qualificationRun{ID: "run-" + d.ID, Strategy: d.ID, Definition: digest, Config: source.fingerprint, Started: now.Add(-time.Minute).UnixMilli(), Completed: now.UnixMilli(), Attestation: qualificationAttestation{Collector: "owned-resource-nfq-v1", Runtime: "nfqws2-1.0.2", Definition: digest, LuaHash: strings.Repeat("a", 64), RulesHash: strings.Repeat("b", 64), WorkerArgsHash: strings.Repeat("c", 64), Queue: 200, PID: 123, StartTime: 456, Restored: true, Joined: true}}
|
||||
for _, s := range source.config.Services {
|
||||
for _, spec := range s.Resources {
|
||||
if spec.host == "e.example" {
|
||||
continue
|
||||
}
|
||||
pass := d.ID == "s1" && (spec.host == "a.example" || spec.host == "c.example") || d.ID == "s2" && spec.host == "b.example" || d.ID == "s3" && spec.host == "d.example"
|
||||
for _, tls := range []string{"tls12", "tls13"} {
|
||||
outcome := "expectation_failed"
|
||||
if pass {
|
||||
outcome = "pass"
|
||||
}
|
||||
r.Checks = append(r.Checks, qualificationCheck{Service: s.ID, Resource: spec.ID, Fingerprint: resourceRowFingerprint(key, source, spec), TLS: tls, Outcome: outcome, Attempts: 3})
|
||||
}
|
||||
}
|
||||
}
|
||||
matrix.Runs = append(matrix.Runs, r)
|
||||
}
|
||||
return source, key, registry, matrix, now
|
||||
}
|
||||
func cloneMatrix(m qualificationMatrix) qualificationMatrix {
|
||||
b, _ := json.Marshal(m)
|
||||
var v qualificationMatrix
|
||||
json.Unmarshal(b, &v)
|
||||
return v
|
||||
}
|
||||
func TestResourcePlanCoverage(t *testing.T) {
|
||||
source, key, registry, matrix, now := planFixture(t)
|
||||
p := deriveResourcePlan(source, key, registry, matrix, now)
|
||||
want := [][]string{{"a.example", "c.example"}, {"b.example"}, {"d.example"}}
|
||||
if len(p.Assignments) != 3 || !reflect.DeepEqual(p.Unassigned, []string{"e.example"}) {
|
||||
t.Fatalf("%+v", p)
|
||||
}
|
||||
for i, a := range p.Assignments {
|
||||
if !reflect.DeepEqual(a.Hosts, want[i]) {
|
||||
t.Fatalf("%+v", p)
|
||||
}
|
||||
}
|
||||
for i := 0; i < 100; i++ {
|
||||
if !reflect.DeepEqual(p, deriveResourcePlan(source, key, registry, matrix, now)) {
|
||||
t.Fatal("nondeterministic")
|
||||
}
|
||||
}
|
||||
b, _ := json.Marshal(matrix)
|
||||
out, _ := json.Marshal(p)
|
||||
if strings.Contains(string(b)+string(out), "SECRET") || strings.Contains(string(b), "https://") {
|
||||
t.Fatal("secret serialization")
|
||||
}
|
||||
// One bad URL on the same host in another service/group invalidates only a.
|
||||
bad := cloneMatrix(matrix)
|
||||
bad.Runs[0].Checks[len(bad.Runs[0].Checks)-1].Outcome = "expectation_failed"
|
||||
changed := deriveResourcePlan(source, key, registry, bad, now)
|
||||
if len(changed.Domains[0].Validated) != 0 || !reflect.DeepEqual(p.Domains[1:], changed.Domains[1:]) {
|
||||
t.Fatalf("subset failure leaked: %+v", changed)
|
||||
}
|
||||
partial := cloneMatrix(matrix)
|
||||
partial.Runs[0].Checks = partial.Runs[0].Checks[:len(partial.Runs[0].Checks)-1]
|
||||
changed = deriveResourcePlan(source, key, registry, partial, now)
|
||||
if len(changed.Domains[0].Validated) != 0 || len(changed.Domains[0].Unknown) != 1 {
|
||||
t.Fatal("partial TLS qualified")
|
||||
}
|
||||
for _, mutate := range []func(*qualificationMatrix){
|
||||
func(m *qualificationMatrix) { m.Runs[0].Config = strings.Repeat("c", 64) },
|
||||
func(m *qualificationMatrix) {
|
||||
m.Runs[0].Completed = now.Add(-25 * time.Hour).UnixMilli()
|
||||
m.Runs[0].Started = m.Runs[0].Completed - 1
|
||||
},
|
||||
func(m *qualificationMatrix) { m.Runs[0].Attestation.Joined = false },
|
||||
func(m *qualificationMatrix) { m.Runs[0].Checks = append(m.Runs[0].Checks, m.Runs[0].Checks[0]) },
|
||||
func(m *qualificationMatrix) { m.Runs[1].ID = m.Runs[0].ID },
|
||||
func(m *qualificationMatrix) { m.Runs[0].Definition = strings.Repeat("c", 64) },
|
||||
} {
|
||||
m := cloneMatrix(matrix)
|
||||
mutate(&m)
|
||||
if len(deriveResourcePlan(source, key, registry, m, now).Domains[0].Validated) != 0 {
|
||||
t.Fatal("invalid evidence qualified")
|
||||
}
|
||||
}
|
||||
source.fingerprint = strings.Repeat("d", 64)
|
||||
if len(deriveResourcePlan(source, key, registry, matrix, now).Assignments) != 0 {
|
||||
t.Fatal("changed config qualified")
|
||||
}
|
||||
}
|
||||
func TestResourcePlanSelectorsAndAlternatives(t *testing.T) {
|
||||
source, key, registry, matrix, now := planFixture(t)
|
||||
registry.Strategies[1].Args = registry.Strategies[0].Args
|
||||
matrix.Registry = planDigest(registry)
|
||||
matrix.Runs[1].Definition = planDigest(registry.Strategies[1])
|
||||
matrix.Runs[1].Attestation.Definition = matrix.Runs[1].Definition
|
||||
// Add a shared alternative on c without removing original evidence.
|
||||
for i := range matrix.Runs[1].Checks {
|
||||
if matrix.Runs[1].Checks[i].Service == "betac" {
|
||||
matrix.Runs[1].Checks[i].Outcome = "pass"
|
||||
}
|
||||
}
|
||||
p := deriveResourcePlan(source, key, registry, matrix, now)
|
||||
if len(p.Assignments) != 2 || !reflect.DeepEqual(p.Assignments[0].Hosts, []string{"a.example", "b.example", "c.example"}) || len(p.Domains[2].Validated) != 2 {
|
||||
t.Fatalf("merge/alternatives: %+v", p)
|
||||
}
|
||||
registry.Strategies[1].Selectors.TLS = "tls12"
|
||||
matrix.Registry = planDigest(registry)
|
||||
matrix.Runs[1].Definition = planDigest(registry.Strategies[1])
|
||||
matrix.Runs[1].Attestation.Definition = matrix.Runs[1].Definition
|
||||
// Hosts a,c require both; host b only tls12.
|
||||
source.config.Services[1].Resources[0].TLS = "tls12"
|
||||
for i := range matrix.Runs[1].Checks {
|
||||
c := &matrix.Runs[1].Checks[i]
|
||||
if c.Service == "alphab" {
|
||||
c.Fingerprint = resourceRowFingerprint(key, source, source.config.Services[1].Resources[0])
|
||||
}
|
||||
}
|
||||
p = deriveResourcePlan(source, key, registry, matrix, now)
|
||||
if len(p.Assignments) != 3 || p.Assignments[0].Selectors == p.Assignments[1].Selectors {
|
||||
t.Fatalf("selectors merged: %+v", p)
|
||||
}
|
||||
}
|
||||
func TestResourcePlanRegistryValidation(t *testing.T) {
|
||||
_, _, r, _, _ := planFixture(t)
|
||||
d := r.Strategies[0]
|
||||
d.Args = " \t--payload=tls_client_hello\n--lua-desync=multisplit:pos=1 "
|
||||
v, e := canonicalStrategy(d)
|
||||
if e != nil || v.Args != "--payload=tls_client_hello --lua-desync=multisplit:pos=1" {
|
||||
t.Fatalf("canonical: %+v %v", v, e)
|
||||
}
|
||||
for _, args := range []string{"--lua-init=@/opt/zapret2/lua/evil.lua", "--qnum=200", "--user=root", "--fwmark=1", "--filter-mark=1", "--lua-desync=execute:cmd=id", "--lua-desync=fake:blob=/etc/passwd", "--lua-desync=fake:blob=secret.lua", "--lua-desync=fake:repeats=1;id"} {
|
||||
d.Args = args
|
||||
if _, e := canonicalStrategy(d); e == nil {
|
||||
t.Fatalf("accepted %s", args)
|
||||
}
|
||||
}
|
||||
for _, b := range []string{`{"version":1,"version":1,"strategies":[]}`, `{"version":1,"strategies":[],"evidence":true}`} {
|
||||
if _, e := parseStrategyRegistry([]byte(b)); e == nil {
|
||||
t.Fatal("import accepted proof/duplicate")
|
||||
}
|
||||
}
|
||||
}
|
||||
func TestResourcePlanPrivateStore(t *testing.T) {
|
||||
_, _, registry, matrix, _ := planFixture(t)
|
||||
dir := t.TempDir()
|
||||
os.Chmod(dir, 0700)
|
||||
s, e := openResourcePlanStore(dir, true)
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
defer s.close()
|
||||
b, _ := json.Marshal(registry)
|
||||
if s.importDefinitions(b) != nil {
|
||||
t.Fatal("import")
|
||||
}
|
||||
if s.importDefinitions(b) == nil {
|
||||
t.Fatal("clobbered immutable registry")
|
||||
}
|
||||
proof := completedQualification{matrix: cloneMatrix(matrix), seal: planDigest(matrix)}
|
||||
proof.matrix.Runs[0].Checks[0].Outcome = "expectation_failed"
|
||||
if s.saveCompleted(proof) == nil {
|
||||
t.Fatal("post-seal mutation")
|
||||
}
|
||||
if s.saveCompleted(completedQualification{matrix: matrix}) == nil {
|
||||
t.Fatal("unsealed DTO")
|
||||
}
|
||||
if s.saveCompleted(completedQualification{matrix: matrix, seal: planDigest(matrix)}) != nil {
|
||||
t.Fatal("fixture save")
|
||||
}
|
||||
loaded, e := s.matrix(registry)
|
||||
if e != nil || !reflect.DeepEqual(loaded, matrix) {
|
||||
t.Fatalf("load: %v", e)
|
||||
}
|
||||
second, e := openResourcePlanStore(dir, false)
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
second.close()
|
||||
path := filepath.Join(dir, planMatrixName)
|
||||
raw, e := os.ReadFile(path)
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
raw = bytes.Replace(raw, []byte(`"outcome":"pass"`), []byte(`"outcome":"timeout"`), 1)
|
||||
os.WriteFile(path, raw, 0600)
|
||||
if _, e = s.matrix(registry); e == nil {
|
||||
t.Fatal("held mutation accepted")
|
||||
}
|
||||
reopened, e := openResourcePlanStore(dir, false)
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
defer reopened.close()
|
||||
if _, e = reopened.matrix(registry); e == nil {
|
||||
t.Fatal("forged HMAC accepted")
|
||||
}
|
||||
}
|
||||
func TestResourcePlanForeignEntries(t *testing.T) {
|
||||
for _, kind := range []string{"file", "symlink", "replacement"} {
|
||||
t.Run(kind, func(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
os.Chmod(dir, 0700)
|
||||
path := filepath.Join(dir, planRegistryName)
|
||||
if kind == "file" {
|
||||
os.WriteFile(path, []byte("FOREIGN"), 0600)
|
||||
} else if kind == "symlink" {
|
||||
os.Symlink("foreign", path)
|
||||
}
|
||||
s, e := openResourcePlanStore(dir, true)
|
||||
if kind != "replacement" {
|
||||
if e == nil {
|
||||
s.close()
|
||||
t.Fatal("adopted foreign registry without key")
|
||||
}
|
||||
return
|
||||
}
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
defer s.close()
|
||||
_, _, r, _, _ := planFixture(t)
|
||||
b, _ := json.Marshal(r)
|
||||
if s.importDefinitions(b) != nil {
|
||||
t.Fatal("import")
|
||||
}
|
||||
os.Rename(path, path+".old")
|
||||
os.WriteFile(path, s.snapshots[planRegistryName], 0600)
|
||||
if _, e = s.registry(); e == nil {
|
||||
t.Fatal("adopted identical foreign replacement")
|
||||
}
|
||||
if s.importDefinitions(b) == nil {
|
||||
t.Fatal("clobbered replacement")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourcePlanCLIUnknownAndForged(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
os.Chmod(dir, 0700)
|
||||
path := filepath.Join(dir, "resources.json")
|
||||
config := `{"version":1,"services":[{"id":"service","resources":[{"id":"resource","group":"metadata","kind":"http","url":"https://a.example/SECRET","method":"GET","tls":"both","timeout_ms":1000,"expect":{"statuses":[200],"min_bytes":0,"max_bytes":100}}]}]}`
|
||||
if err := os.WriteFile(path, []byte(config), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
env := map[string]string{"ZAPRET_AUTOTUNE_SOCKS_PROXY": "socks5h://127.0.0.1:1080", "ZAPRET_AUTOTUNE_DNS_MODE": "native", "SS_DNS_SERVER": "1.1.1.1"}
|
||||
d := managedDeps{directory: dir, lookup: func(k string) (string, bool) { v, ok := env[k]; return v, ok }}
|
||||
var out, diagnostics bytes.Buffer
|
||||
code := resourcePlanProposal(path, &out, &diagnostics, d)
|
||||
if code != 0 {
|
||||
t.Fatalf("unknown reader: %d %s", code, diagnostics.String())
|
||||
}
|
||||
var p resourcePlan
|
||||
if json.Unmarshal(out.Bytes(), &p) != nil || p.Evidence != "unknown" || len(p.Assignments) != 0 || len(p.Unassigned) != 1 || strings.Contains(out.String(), "SECRET") {
|
||||
t.Fatalf("%s", out.String())
|
||||
}
|
||||
s, err := openResourcePlanStore(dir, true)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_, _, registry, matrix, _ := planFixture(t)
|
||||
definitions, _ := json.Marshal(registry)
|
||||
if s.importDefinitions(definitions) != nil {
|
||||
t.Fatal("import")
|
||||
}
|
||||
source, err := readResourceMonitorSource(path, d.lookup, s.files.key)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for i := range matrix.Runs {
|
||||
r := &matrix.Runs[i]
|
||||
r.Config = source.fingerprint
|
||||
r.Started = time.Now().Add(-time.Minute).UnixMilli()
|
||||
r.Completed = time.Now().UnixMilli()
|
||||
r.Checks = nil
|
||||
for _, tls := range []string{"tls12", "tls13"} {
|
||||
r.Checks = append(r.Checks, qualificationCheck{Service: "service", Resource: "resource", TLS: tls, Outcome: "pass", Fingerprint: resourceRowFingerprint(s.files.key, source, source.config.Services[0].Resources[0]), Attempts: 3})
|
||||
}
|
||||
}
|
||||
if s.saveCompleted(completedQualification{matrix: matrix, seal: planDigest(matrix)}) != nil {
|
||||
t.Fatal("sealed fixture")
|
||||
}
|
||||
s.close()
|
||||
out.Reset()
|
||||
code = resourcePlanProposal(path, &out, &diagnostics, d)
|
||||
if code != 0 || json.Unmarshal(out.Bytes(), &p) != nil || p.Evidence != "authenticated" || len(p.Assignments) != 1 || len(p.Domains[0].Validated) != 3 {
|
||||
t.Fatalf("authenticated CLI %d %s", code, out.String())
|
||||
}
|
||||
// Public JSON PASS and monitor observations cannot become private proof.
|
||||
if err := os.WriteFile(filepath.Join(dir, planMatrixName), []byte(`{"pass":true,"health":"HEALTHY"}`), 0600); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
out.Reset()
|
||||
diagnostics.Reset()
|
||||
code = resourcePlanProposal(path, &out, &diagnostics, d)
|
||||
if code != 78 || !strings.Contains(out.String(), `"evidence":"blocked"`) || strings.Contains(out.String(), `"hosts":[`) {
|
||||
t.Fatalf("forged CLI %d %s %s", code, out.String(), diagnostics.String())
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,684 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"crypto/rand"
|
||||
"encoding/hex"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net"
|
||||
"os"
|
||||
"os/exec"
|
||||
"os/signal"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"sort"
|
||||
"strconv"
|
||||
"strings"
|
||||
"syscall"
|
||||
"time"
|
||||
)
|
||||
|
||||
const qualificationEngine = "/opt/zapret2/nfq2/nfqws2"
|
||||
|
||||
var errQualificationCapacity = errors.New("qualification matrix exceeds private capacity")
|
||||
|
||||
// A fresh disabled baseline needs no daemon stop/config transaction. Deliberately
|
||||
// refuse existing active profiles: maintenance pause/restore is not implemented.
|
||||
// ponytail: isolated startup baseline only; add job-bound profile rollback before
|
||||
// supporting an already active desired set, never just invoke a daemon stop.
|
||||
func qualificationBaseline(dir string) error {
|
||||
if resourceManagedIdentity(dir) != nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
m := &managedTransaction{Dir: dir, Name: "config"}
|
||||
j, e := m.load()
|
||||
if e != nil || j.State != "committed" || len(j.Candidates) != 0 {
|
||||
return errResourcePlan
|
||||
}
|
||||
b, e := m.read("config")
|
||||
if e != nil || string(b) != managedBaseline {
|
||||
return errResourcePlan
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func qualificationMode(config, strategies string, d managedDeps) error {
|
||||
if os.Geteuid() != 0 || os.Getppid() != 1 {
|
||||
return errResourcePlan
|
||||
}
|
||||
if _, e := managedRuntimeGuard(d); e != nil {
|
||||
return e
|
||||
}
|
||||
cmd, e := os.ReadFile("/proc/1/cmdline")
|
||||
if e != nil || string(cmd) != "/bin/sh\x00/opt/managed-target-entrypoint.sh\x00" {
|
||||
return errResourcePlan
|
||||
}
|
||||
env, e := os.ReadFile("/proc/1/environ")
|
||||
if e != nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
values := map[string]string{}
|
||||
for _, s := range strings.Split(string(env), "\x00") {
|
||||
k, v, _ := strings.Cut(s, "=")
|
||||
values[k] = v
|
||||
}
|
||||
if config == "" || strategies == "" || values["ZAPRET_AUTOTUNE_QUALIFY_CONFIG"] != config || values["ZAPRET_AUTOTUNE_STRATEGY_CONFIG"] != strategies {
|
||||
return errResourcePlan
|
||||
}
|
||||
for _, ns := range []string{"pid", "net"} {
|
||||
a, e := os.Readlink("/proc/1/ns/" + ns)
|
||||
b, f := os.Readlink("/proc/self/ns/" + ns)
|
||||
if e != nil || f != nil || a != b {
|
||||
return errResourcePlan
|
||||
}
|
||||
}
|
||||
if !resourceProcessOwned("/proc", os.Getpid(), "/usr/local/bin/zapret-autotune", []string{"/usr/local/bin/zapret-autotune", "qualify", "--config", config, "--strategies", strategies, "--isolated"}) {
|
||||
return errResourcePlan
|
||||
}
|
||||
dir, e := os.Open("/run/zapret-autotune-ss")
|
||||
if e != nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
defer dir.Close()
|
||||
ready, e := resourcePrivateFile(dir, "monitor-ready")
|
||||
if e != nil {
|
||||
return e
|
||||
}
|
||||
defer ready.Close()
|
||||
b, e := io.ReadAll(io.LimitReader(ready, 32))
|
||||
if e != nil || string(b) != strconv.Itoa(os.Getpid())+"\n" {
|
||||
return errResourcePlan
|
||||
}
|
||||
return qualificationBaseline(d.directory)
|
||||
}
|
||||
|
||||
func qualificationFilesHash() (string, error) {
|
||||
engine, e := openQualificationEngine("/opt/zapret2")
|
||||
if e != nil {
|
||||
return "", e
|
||||
}
|
||||
defer engine.file.Close()
|
||||
return engine.filesHash()
|
||||
}
|
||||
|
||||
func (engine *qualificationBinary) filesHash() (string, error) {
|
||||
// Fixed image files only. Bind both executable and Lua inputs before and after
|
||||
// every candidate; never accept a caller-supplied native hash/version label.
|
||||
if engine.validate() != nil {
|
||||
return "", errResourcePlan
|
||||
}
|
||||
hashes := []string{engine.hash}
|
||||
for _, path := range []string{filepath.Join(engine.root, "lua/zapret-lib.lua"), filepath.Join(engine.root, "lua/zapret-antidpi.lua")} {
|
||||
fd, e := syscall.Open(path, syscall.O_RDONLY|syscall.O_NOFOLLOW|syscall.O_NONBLOCK|syscall.O_CLOEXEC, 0)
|
||||
if e != nil {
|
||||
return "", errResourcePlan
|
||||
}
|
||||
f := os.NewFile(uintptr(fd), path)
|
||||
info, e := f.Stat()
|
||||
if e != nil || !info.Mode().IsRegular() || info.Size() > 32<<20 || info.Mode().Perm()&0022 != 0 {
|
||||
f.Close()
|
||||
return "", errResourcePlan
|
||||
}
|
||||
st, ok := info.Sys().(*syscall.Stat_t)
|
||||
if !ok || st.Uid != 0 {
|
||||
f.Close()
|
||||
return "", errResourcePlan
|
||||
}
|
||||
b, e := io.ReadAll(io.LimitReader(f, 32<<20+1))
|
||||
current, se := os.Lstat(path)
|
||||
f.Close()
|
||||
if e != nil || se != nil || !os.SameFile(info, current) || int64(len(b)) != info.Size() {
|
||||
return "", errResourcePlan
|
||||
}
|
||||
hashes = append(hashes, planDigest(b))
|
||||
}
|
||||
return planDigest(hashes), nil
|
||||
}
|
||||
|
||||
func (s *resourcePlanStore) protectBaseline() error {
|
||||
for _, name := range []string{"config", managedMarkerName, managedJournalName, managedBackupName} {
|
||||
f, b, e := s.files.openEntry(name)
|
||||
if e != nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
s.held[name], s.snapshots[name] = f, b
|
||||
}
|
||||
return s.guard()
|
||||
}
|
||||
func qualificationNoWorker() error {
|
||||
if e := qualificationOnlyWorker(0); e != nil {
|
||||
return qualificationPredicate(e, qualificationErrorReason(e, reasonWorkerScan))
|
||||
}
|
||||
occupied, e := managedAssetQueueReady(0)
|
||||
if e != nil || occupied {
|
||||
return qualificationPredicate(errResourcePlan, reasonQueuePresentOrUnreadable)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (q resourceQualifier) restoreBaseline(ctx context.Context, noWorker, baseline func() error) error {
|
||||
if ctx.Err() != nil {
|
||||
return qualificationPredicate(errResourcePlan, reasonContext)
|
||||
}
|
||||
if e := q.guard(); e != nil {
|
||||
return qualificationPredicate(errResourcePlan, qualificationErrorReason(e, reasonGuard))
|
||||
}
|
||||
if e := noWorker(); e != nil {
|
||||
return qualificationPlanPredicate(e, reasonUnknown)
|
||||
}
|
||||
tables, e := q.runtime.nft(ctx, "list tables\n")
|
||||
if e != nil {
|
||||
return qualificationPredicate(errResourcePlan, reasonNFTRead)
|
||||
}
|
||||
if strings.Contains(tables, "table inet "+managedAssetTable) {
|
||||
return qualificationPredicate(errResourcePlan, reasonNFTPresent)
|
||||
}
|
||||
if e := baseline(); e != nil {
|
||||
return qualificationPredicate(e, reasonBaseline)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func qualificationOnlyWorker(owned int) error {
|
||||
return qualificationScanWorkers(owned, os.ReadDir, os.Readlink)
|
||||
}
|
||||
|
||||
func qualificationScanWorkers(owned int, readDir func(string) ([]os.DirEntry, error), readlink func(string) (string, error)) error {
|
||||
entries, e := readDir("/proc")
|
||||
if e != nil {
|
||||
return &qualificationDiagnostic{Err: e, Reason: reasonWorkerScanRead, Errno: qualificationScanErrno(e), OwnedPID: owned}
|
||||
}
|
||||
for _, entry := range entries {
|
||||
pid, e := strconv.Atoi(entry.Name())
|
||||
if e != nil {
|
||||
continue
|
||||
}
|
||||
exe, e := readlink(filepath.Join("/proc", entry.Name(), "exe"))
|
||||
if e != nil {
|
||||
if errors.Is(e, os.ErrNotExist) {
|
||||
continue
|
||||
}
|
||||
return &qualificationDiagnostic{Err: errResourcePlan, Reason: reasonWorkerScanReadlink, Errno: qualificationScanErrno(e), ScanPID: pid, OwnedPID: owned}
|
||||
}
|
||||
if filepath.Base(exe) == "nfqws2" && entry.Name() != strconv.Itoa(owned) {
|
||||
return &qualificationDiagnostic{Err: errResourcePlan, Reason: reasonWorkerScanForeign, ScanPID: pid, OwnedPID: owned}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func qualificationWorkerArgs(endpoints []string, args string) []string {
|
||||
base := managedProbeWorkerArgs(endpoints, "")
|
||||
base = append(base, "--filter-l7=tls") // Supported by pinned 1.0.2; generic caller only.
|
||||
return append(base, strings.Fields(args)...)
|
||||
}
|
||||
|
||||
// All seams are package-private and supplied by the runtime factory, not JSON.
|
||||
type resourceQualifier struct {
|
||||
runtime managedAssetRuntime
|
||||
checks resourceCheckDeps
|
||||
source func() (resourceMonitorSource, error)
|
||||
restore func(context.Context) error
|
||||
attest func(context.Context, int, []string, strategyDefinition) (qualificationAttestation, error)
|
||||
guard func() error
|
||||
now func() time.Time
|
||||
filesHash func() (string, error)
|
||||
close func()
|
||||
}
|
||||
|
||||
func newResourceQualifier(config, strategies string, d managedDeps, s *resourcePlanStore) resourceQualifier {
|
||||
r := newManagedAssetRuntime(d)
|
||||
q := resourceQualifier{runtime: r, checks: productionResourceCheckDeps(), now: time.Now}
|
||||
var engine *qualificationBinary
|
||||
q.close = func() {
|
||||
if engine != nil {
|
||||
engine.file.Close()
|
||||
}
|
||||
}
|
||||
q.filesHash = func() (string, error) {
|
||||
if engine == nil {
|
||||
var e error
|
||||
engine, e = openQualificationEngine("/opt/zapret2")
|
||||
if e != nil {
|
||||
return "", e
|
||||
}
|
||||
}
|
||||
return engine.filesHash()
|
||||
}
|
||||
start, queue := r.start, r.queue
|
||||
r.start = func(args []string) (managedAssetWorker, error) {
|
||||
if engine == nil || engine.validate() != nil {
|
||||
return managedAssetWorker{}, errResourcePlan
|
||||
}
|
||||
return start(args)
|
||||
}
|
||||
r.queue = func(pid int) (bool, error) {
|
||||
if engine == nil || engine.validate() != nil {
|
||||
return false, errResourcePlan
|
||||
}
|
||||
ready, e := queue(pid)
|
||||
// withOwnedWorker checks READY before creating the nft table.
|
||||
if e == nil && ready && pid != 0 && engine.running(fmt.Sprintf("/proc/%d/exe", pid)) != nil {
|
||||
return false, errResourcePlan
|
||||
}
|
||||
return ready, e
|
||||
}
|
||||
q.runtime = r
|
||||
q.source = func() (resourceMonitorSource, error) { return readResourceMonitorSource(config, d.lookup, s.files.key) }
|
||||
q.guard = func() error {
|
||||
if e := qualificationMode(config, strategies, d); e != nil {
|
||||
return qualificationPredicate(errResourcePlan, qualificationErrorReason(e, reasonMode))
|
||||
}
|
||||
if e := s.guard(); e != nil {
|
||||
return qualificationPredicate(errResourcePlan, qualificationErrorReason(e, reasonPlanStore))
|
||||
}
|
||||
if e := r.guard(); e != nil {
|
||||
return qualificationPredicate(e, qualificationErrorReason(e, reasonRuntime))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
q.restore = func(ctx context.Context) error {
|
||||
return q.restoreBaseline(ctx, qualificationNoWorker, func() error { return qualificationBaseline(d.directory) })
|
||||
}
|
||||
q.attest = func(ctx context.Context, pid int, args []string, definition strategyDefinition) (qualificationAttestation, error) {
|
||||
if e := q.guard(); e != nil {
|
||||
return qualificationAttestation{}, qualificationPredicate(errResourcePlan, qualificationErrorReason(e, reasonGuard))
|
||||
}
|
||||
if e := qualificationOnlyWorker(pid); e != nil {
|
||||
return qualificationAttestation{}, qualificationPlanPredicate(e, reasonWorkerScan)
|
||||
}
|
||||
if engine == nil || engine.validate() != nil || engine.running(fmt.Sprintf("/proc/%d/exe", pid)) != nil {
|
||||
return qualificationAttestation{}, qualificationPredicate(errResourcePlan, reasonEngine)
|
||||
}
|
||||
cmd, e := os.ReadFile(fmt.Sprintf("/proc/%d/cmdline", pid))
|
||||
if e != nil || string(cmd) != strings.Join(append([]string{qualificationEngine}, args...), "\x00")+"\x00" {
|
||||
return qualificationAttestation{}, qualificationPredicate(errResourcePlan, reasonArgv)
|
||||
}
|
||||
stat, e := os.ReadFile(fmt.Sprintf("/proc/%d/stat", pid))
|
||||
if e != nil {
|
||||
return qualificationAttestation{}, qualificationPredicate(e, reasonParent)
|
||||
}
|
||||
end := strings.LastIndex(string(stat), ") ")
|
||||
if end < 0 {
|
||||
return qualificationAttestation{}, qualificationPredicate(errResourcePlan, reasonParent)
|
||||
}
|
||||
tail := strings.Fields(string(stat)[end+2:])
|
||||
if len(tail) < 2 || tail[1] != strconv.Itoa(os.Getpid()) {
|
||||
return qualificationAttestation{}, qualificationPredicate(errResourcePlan, reasonParent)
|
||||
}
|
||||
start, e := readMonitorProcessStarttime(pid)
|
||||
if e != nil {
|
||||
return qualificationAttestation{}, qualificationPredicate(e, reasonStarttime)
|
||||
}
|
||||
ready, e := r.queue(pid)
|
||||
if e != nil || !ready {
|
||||
return qualificationAttestation{}, qualificationPredicate(errResourcePlan, reasonQueue)
|
||||
}
|
||||
hash, e := q.filesHash()
|
||||
if e != nil {
|
||||
return qualificationAttestation{}, qualificationPredicate(e, reasonFiles)
|
||||
}
|
||||
versionCtx, cancel := context.WithTimeout(ctx, 5*time.Second)
|
||||
defer cancel()
|
||||
var output bytes.Buffer
|
||||
version := exec.CommandContext(versionCtx, engine.path, "--version")
|
||||
version.Stdout = &playerBoundedWriter{dst: &output, limit: 4096}
|
||||
version.Stderr = io.Discard
|
||||
if version.Run() != nil {
|
||||
return qualificationAttestation{}, qualificationPredicate(errResourcePlan, reasonVersion)
|
||||
}
|
||||
fields := strings.Fields(output.String())
|
||||
valid := false
|
||||
for i, f := range fields {
|
||||
if f == "version" && i+1 < len(fields) && (fields[i+1] == "1.0.2" || fields[i+1] == "v1.0.2") {
|
||||
valid = true
|
||||
}
|
||||
}
|
||||
if !valid {
|
||||
return qualificationAttestation{}, qualificationPredicate(errResourcePlan, reasonVersion)
|
||||
}
|
||||
rules, e := r.nft(ctx, "list table inet "+managedAssetTable+"\n")
|
||||
if e != nil || strings.TrimSpace(rules) == "" {
|
||||
return qualificationAttestation{}, qualificationPredicate(errResourcePlan, reasonRulesRead)
|
||||
}
|
||||
return qualificationAttestation{Collector: "owned-resource-nfq-v1", Runtime: "nfqws2-1.0.2", Definition: planDigest(definition), LuaHash: hash, RulesHash: planDigest(rules), WorkerArgsHash: planDigest(args), Queue: managedAssetQueue, PID: pid, StartTime: start}, nil
|
||||
}
|
||||
return q
|
||||
}
|
||||
|
||||
func (q resourceQualifier) complete(ctx context.Context, s *resourcePlanStore, registry strategyRegistry) (completedQualification, error) {
|
||||
fail := func(e error) (completedQualification, error) { return completedQualification{}, e }
|
||||
if q.guard == nil || q.restore == nil || q.attest == nil || q.source == nil || q.now == nil || q.filesHash == nil || q.guard() != nil {
|
||||
return fail(errResourcePlan)
|
||||
}
|
||||
if s.held[planMatrixName] != nil {
|
||||
return fail(errResourcePlan)
|
||||
}
|
||||
trustedRegistry, registryErr := s.registry()
|
||||
if registryErr != nil || planDigest(trustedRegistry) != planDigest(registry) {
|
||||
return fail(errResourcePlan)
|
||||
}
|
||||
source, e := q.source()
|
||||
if e != nil {
|
||||
return fail(e)
|
||||
}
|
||||
hosts := map[string]bool{}
|
||||
for _, service := range source.config.Services {
|
||||
for _, spec := range service.Resources {
|
||||
if !validDomain(spec.host) || net.ParseIP(spec.host) != nil || spec.host != strings.ToLower(spec.host) {
|
||||
return fail(errResourcePlan)
|
||||
}
|
||||
hosts[spec.host] = true
|
||||
}
|
||||
}
|
||||
endpoints := []string{}
|
||||
for host := range hosts {
|
||||
endpoints = append(endpoints, host)
|
||||
}
|
||||
sort.Strings(endpoints)
|
||||
matrix := qualificationMatrix{Version: 1, Registry: planDigest(registry), Runs: []qualificationRun{}}
|
||||
// Check the worst-case immutable envelope size before launching any worker.
|
||||
estimate := qualificationMatrix{Version: 1, Registry: matrix.Registry}
|
||||
for _, definition := range registry.Strategies {
|
||||
run := qualificationRun{ID: "run-" + strings.Repeat("a", 32), Strategy: definition.ID, Definition: planDigest(definition), Config: source.fingerprint, Started: 9223372036854775807, Completed: 9223372036854775807, Attestation: qualificationAttestation{Collector: "owned-resource-nfq-v1", Runtime: "nfqws2-1.0.2", Definition: planDigest(definition), LuaHash: strings.Repeat("a", 64), RulesHash: strings.Repeat("a", 64), WorkerArgsHash: strings.Repeat("a", 64), Queue: 200, PID: 2147483647, StartTime: 9223372036854775807, Restored: true, Joined: true}}
|
||||
for _, service := range source.config.Services {
|
||||
for _, spec := range service.Resources {
|
||||
protocols := []string{spec.TLS}
|
||||
if spec.TLS == "both" {
|
||||
protocols = []string{"tls12", "tls13"}
|
||||
}
|
||||
for _, protocol := range protocols {
|
||||
run.Checks = append(run.Checks, qualificationCheck{Service: service.ID, Resource: spec.ID, TLS: protocol, Fingerprint: resourceRowFingerprint(s.files.key, source, spec), Outcome: "expectation_failed", Attempts: 3})
|
||||
}
|
||||
}
|
||||
}
|
||||
estimate.Runs = append(estimate.Runs, run)
|
||||
}
|
||||
b, _ := json.Marshal(estimate)
|
||||
// Include the exact authenticated envelope overhead, using maximal timestamp
|
||||
// widths and the longest permitted outcome. Args/selectors are fixed digests.
|
||||
envelope, _ := json.Marshal(planEnvelope{Payload: b, MAC: strings.Repeat("a", 64)})
|
||||
if len(envelope) > 65536 {
|
||||
return fail(errQualificationCapacity)
|
||||
}
|
||||
stable := func() error {
|
||||
if ctx.Err() != nil {
|
||||
return ctx.Err()
|
||||
}
|
||||
if q.guard() != nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
fresh, e := q.source()
|
||||
if e != nil || fresh.fingerprint != source.fingerprint {
|
||||
return errResourcePlan
|
||||
}
|
||||
return nil
|
||||
}
|
||||
for candidateIndex, definition := range registry.Strategies {
|
||||
if _, e := canonicalStrategy(definition); e != nil {
|
||||
return fail(e)
|
||||
}
|
||||
if e := stable(); e != nil {
|
||||
return fail(e)
|
||||
}
|
||||
nativeBefore, nativeErr := q.filesHash()
|
||||
if nativeErr != nil || !hashOK(nativeBefore) {
|
||||
return fail(errResourcePlan)
|
||||
}
|
||||
id := make([]byte, 16)
|
||||
if _, e := rand.Read(id); e != nil {
|
||||
return fail(e)
|
||||
}
|
||||
run := qualificationRun{ID: "run-" + hex.EncodeToString(id), Strategy: definition.ID, Definition: planDigest(definition), Config: source.fingerprint, Started: q.now().UnixMilli()}
|
||||
runtime := q.runtime
|
||||
originalStart := runtime.start
|
||||
pid := 0
|
||||
var args []string
|
||||
runtime.start = func(actual []string) (managedAssetWorker, error) {
|
||||
if !reflect.DeepEqual(actual, managedProbeWorkerArgs(endpoints, definition.Args)) {
|
||||
return managedAssetWorker{}, errResourcePlan
|
||||
}
|
||||
args = qualificationWorkerArgs(endpoints, definition.Args)
|
||||
worker, e := originalStart(args)
|
||||
pid = worker.pid
|
||||
return worker, e
|
||||
}
|
||||
err := runtime.withOwnedWorker(ctx, endpoints, definition.Args, func(captureCtx context.Context) error {
|
||||
if !reflect.DeepEqual(args, qualificationWorkerArgs(endpoints, definition.Args)) {
|
||||
return errResourcePlan
|
||||
}
|
||||
a, e := q.attest(captureCtx, pid, args, definition)
|
||||
if e != nil {
|
||||
return qualificationOperation(e, stageAttestation)
|
||||
}
|
||||
if a.LuaHash != nativeBefore {
|
||||
return &qualificationDiagnostic{Err: errResourcePlan, Stage: stageAttestation, Reason: reasonFiles}
|
||||
}
|
||||
run.Attestation = a
|
||||
checks := q.checks
|
||||
checks.qualification = true
|
||||
originalGuard := checks.guard
|
||||
checks.guard = func() error {
|
||||
if e := stable(); e != nil {
|
||||
return e
|
||||
}
|
||||
if e := originalGuard(); e != nil {
|
||||
return e
|
||||
}
|
||||
current, e := q.attest(captureCtx, pid, args, definition)
|
||||
if e != nil {
|
||||
return qualificationOperation(errors.Join(errResourcePlan, e), stageAttestation)
|
||||
}
|
||||
if current != a {
|
||||
return &qualificationDiagnostic{Err: errResourcePlan, Stage: stageAttestation, Reason: attestationMismatch(a, current)}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
resourceIndex := 0
|
||||
for _, service := range source.config.Services {
|
||||
for _, spec := range service.Resources {
|
||||
resourceIndex++
|
||||
protocols := []string{spec.TLS}
|
||||
if spec.TLS == "both" {
|
||||
protocols = []string{"tls12", "tls13"}
|
||||
}
|
||||
for _, protocol := range protocols {
|
||||
// Unsupported TLS policy is unknown for this candidate, never PASS. Do
|
||||
// still measure the full required set; planner applies selector eligibility.
|
||||
cell := qualificationCheck{Service: service.ID, Resource: spec.ID, Fingerprint: resourceRowFingerprint(s.files.key, source, spec), TLS: protocol, Outcome: "pass"}
|
||||
for repeat := 0; repeat < 3; repeat++ {
|
||||
result, e := runResourceProtocol(captureCtx, service.ID, spec, protocol, source.server, source.proxy, checks)
|
||||
if e == nil && captureCtx.Err() != nil {
|
||||
e = captureCtx.Err()
|
||||
}
|
||||
if e == nil && result.Outcome == "cancelled" {
|
||||
e = context.Canceled
|
||||
}
|
||||
if e != nil {
|
||||
tls := uint8(12)
|
||||
if protocol == "tls13" {
|
||||
tls = 13
|
||||
}
|
||||
return &qualificationDiagnostic{Err: e, Candidate: candidateIndex + 1, Resource: resourceIndex, Attempt: repeat + 1, TLS: tls}
|
||||
}
|
||||
if result.Outcome != "pass" {
|
||||
cell.Outcome = result.Outcome
|
||||
}
|
||||
cell.Attempts++
|
||||
}
|
||||
run.Checks = append(run.Checks, cell)
|
||||
}
|
||||
}
|
||||
}
|
||||
return stable()
|
||||
})
|
||||
cleanup, cancel := context.WithTimeout(context.WithoutCancel(ctx), 30*time.Second)
|
||||
restoreErr := q.restore(cleanup)
|
||||
if restoreErr != nil {
|
||||
restoreErr = &qualificationDiagnostic{Err: restoreErr, Stage: stageRestore, Candidate: candidateIndex + 1}
|
||||
}
|
||||
cancel()
|
||||
if err != nil || restoreErr != nil {
|
||||
err = qualificationOperation(err, stageUnknown)
|
||||
if diagnostic, ok := err.(*qualificationDiagnostic); ok {
|
||||
diagnostic.Candidate = candidateIndex + 1
|
||||
}
|
||||
if restoreErr != nil {
|
||||
return fail(&qualificationCleanupFailure{Err: errors.Join(err, restoreErr)})
|
||||
}
|
||||
return fail(errors.Join(err, restoreErr))
|
||||
}
|
||||
hash, hashErr := q.filesHash()
|
||||
if hashErr != nil || hash != run.Attestation.LuaHash {
|
||||
return fail(errResourcePlan)
|
||||
}
|
||||
if e := stable(); e != nil {
|
||||
return fail(e)
|
||||
}
|
||||
run.Attestation.Joined, run.Attestation.Restored = true, true
|
||||
run.Completed = q.now().UnixMilli()
|
||||
matrix.Runs = append(matrix.Runs, run)
|
||||
}
|
||||
if e := stable(); e != nil {
|
||||
return fail(e)
|
||||
}
|
||||
if validateMatrix(matrix, registry) != nil {
|
||||
return fail(errResourcePlan)
|
||||
}
|
||||
return completedQualification{matrix: matrix, seal: planDigest(matrix)}, nil
|
||||
}
|
||||
|
||||
func qualifyResourceCLI(config, strategies string, stdout, stderr io.Writer, d managedDeps) (code int) {
|
||||
if qualificationMode(config, strategies, d) != nil {
|
||||
fmt.Fprintln(stderr, "blocked: qualification requires PID 1 isolated startup with disabled baseline")
|
||||
return 78
|
||||
}
|
||||
// Refuse existing/foreign proof state before even creating a singleton inode.
|
||||
prior, priorErr := openResourcePlanStore(d.directory, false)
|
||||
if priorErr == nil {
|
||||
exists := prior.held[planMatrixName] != nil
|
||||
prior.close()
|
||||
if exists {
|
||||
fmt.Fprintln(stderr, "blocked: immutable matrix already exists; no runtime changes made")
|
||||
return 78
|
||||
}
|
||||
} else if !errors.Is(priorErr, os.ErrNotExist) {
|
||||
fmt.Fprintln(stderr, "blocked: qualification private state")
|
||||
return 78
|
||||
}
|
||||
lock, e := acquireManagedMonitorSingleton(d)
|
||||
if e != nil {
|
||||
fmt.Fprintln(stderr, "blocked: another controller owns target")
|
||||
return 78
|
||||
}
|
||||
defer lock.close()
|
||||
if qualificationNoWorker() != nil {
|
||||
fmt.Fprintln(stderr, "blocked: conflicting NFQ worker/queue")
|
||||
return 78
|
||||
}
|
||||
s, e := openResourcePlanStore(d.directory, true)
|
||||
if e != nil {
|
||||
fmt.Fprintln(stderr, "blocked: qualification private state")
|
||||
return 78
|
||||
}
|
||||
defer s.close()
|
||||
if s.held[planMatrixName] != nil {
|
||||
fmt.Fprintln(stderr, "blocked: immutable matrix already exists; no runtime changes made")
|
||||
return 78
|
||||
}
|
||||
if s.protectBaseline() != nil {
|
||||
fmt.Fprintln(stderr, "blocked: known baseline snapshot unavailable")
|
||||
return 78
|
||||
}
|
||||
fd, e := syscall.Open(strategies, syscall.O_RDONLY|syscall.O_NOFOLLOW|syscall.O_NONBLOCK|syscall.O_CLOEXEC, 0)
|
||||
if e != nil {
|
||||
return 64
|
||||
}
|
||||
f := os.NewFile(uintptr(fd), "definitions")
|
||||
defer f.Close()
|
||||
if resourcePrivateEntry(f) != nil {
|
||||
return 64
|
||||
}
|
||||
b, e := io.ReadAll(io.LimitReader(f, 65537))
|
||||
info, se := os.Lstat(strategies)
|
||||
held, he := f.Stat()
|
||||
if e != nil || se != nil || he != nil || !os.SameFile(info, held) {
|
||||
return 64
|
||||
}
|
||||
registry, e := parseStrategyRegistry(b)
|
||||
if e != nil {
|
||||
return 64
|
||||
}
|
||||
if s.held[planRegistryName] == nil {
|
||||
if s.importDefinitions(b) != nil {
|
||||
return 78
|
||||
}
|
||||
} else {
|
||||
existing, e := s.registry()
|
||||
if e != nil || planDigest(existing) != planDigest(registry) {
|
||||
return 78
|
||||
}
|
||||
}
|
||||
signals, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||
defer stop()
|
||||
ctx, cancel := context.WithTimeout(signals, 15*time.Minute)
|
||||
defer cancel()
|
||||
q := newResourceQualifier(config, strategies, d, s)
|
||||
defer q.close()
|
||||
var sessionErr error
|
||||
defer func() {
|
||||
if code != 0 && code != 78 {
|
||||
return
|
||||
}
|
||||
if e := qualificationFinalize(sessionErr, q.restore, s.guard, func() error {
|
||||
return writeQualificationCompletion(qualificationCompletionDir, s.files.key, os.Getpid(), code)
|
||||
}); e != nil {
|
||||
writeQualificationDiagnostics(stderr, e)
|
||||
code = 78
|
||||
}
|
||||
}()
|
||||
originalGuard := q.guard
|
||||
q.guard = func() error {
|
||||
current, e := os.Lstat(strategies)
|
||||
heldInfo, he := f.Stat()
|
||||
read, re := io.ReadAll(io.NewSectionReader(f, 0, 65537))
|
||||
if e != nil || he != nil || re != nil || !os.SameFile(current, heldInfo) || resourcePrivateEntry(f) != nil || !bytes.Equal(read, b) {
|
||||
return errResourcePlan
|
||||
}
|
||||
return originalGuard()
|
||||
}
|
||||
proof, e := q.complete(ctx, s, registry)
|
||||
sessionErr = e
|
||||
if errors.Is(e, errQualificationCapacity) {
|
||||
fmt.Fprintln(stderr, "unknown: qualification matrix exceeds 64 KiB capacity; no worker launched")
|
||||
return 64
|
||||
}
|
||||
if e != nil || ctx.Err() != nil {
|
||||
writeQualificationDiagnostics(stderr, e)
|
||||
fmt.Fprintln(stderr, "unknown: qualification incomplete or ownership/cleanup blocked; no completed matrix published")
|
||||
if signals.Err() != nil && managedCancellationOnly(e, signals.Err()) {
|
||||
return 0
|
||||
}
|
||||
return 78
|
||||
}
|
||||
fresh, sourceErr := q.source()
|
||||
if sourceErr != nil || q.guard() != nil || ctx.Err() != nil {
|
||||
fmt.Fprintln(stderr, "unknown: source changed before publication")
|
||||
return 78
|
||||
}
|
||||
for _, run := range proof.matrix.Runs {
|
||||
if run.Config != fresh.fingerprint {
|
||||
fmt.Fprintln(stderr, "unknown: source changed before publication")
|
||||
return 78
|
||||
}
|
||||
}
|
||||
if s.saveCompleted(proof) != nil {
|
||||
fmt.Fprintln(stderr, "blocked: completed matrix publication refused")
|
||||
return 78
|
||||
}
|
||||
return resourcePlanProposal(config, stdout, stderr, d)
|
||||
}
|
||||
@@ -0,0 +1,133 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"io"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"syscall"
|
||||
)
|
||||
|
||||
// Only the packaged nfq2 alias is followed. All target parents and the final
|
||||
// binary remain no-symlink, root-owned and not group/other writable. As with
|
||||
// the runtime contract, a malicious root racing checks is outside the boundary.
|
||||
type qualificationBinary struct {
|
||||
root, path string
|
||||
file *os.File
|
||||
info, alias os.FileInfo
|
||||
hash string
|
||||
}
|
||||
|
||||
func qualificationTrusted(info os.FileInfo) bool {
|
||||
st, ok := info.Sys().(*syscall.Stat_t)
|
||||
return ok && st.Uid == 0 && info.Mode().Perm()&0022 == 0
|
||||
}
|
||||
|
||||
func qualificationParents(path, boundary string) error {
|
||||
for dir := filepath.Dir(path); ; dir = filepath.Dir(dir) {
|
||||
info, e := os.Lstat(dir)
|
||||
if e != nil || !info.IsDir() || !qualificationTrusted(info) {
|
||||
return errResourcePlan
|
||||
}
|
||||
if dir == boundary {
|
||||
return nil
|
||||
}
|
||||
if dir == "/" {
|
||||
return errResourcePlan
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func qualificationEnginePath(root string) (string, os.FileInfo, error) {
|
||||
alias := filepath.Join(root, "nfq2/nfqws2")
|
||||
boundary := root // Private fixture seam; production also checks /opt and /.
|
||||
if root == "/opt/zapret2" {
|
||||
boundary = "/"
|
||||
}
|
||||
if qualificationParents(alias, boundary) != nil {
|
||||
return "", nil, errResourcePlan
|
||||
}
|
||||
info, e := os.Lstat(alias)
|
||||
if e != nil || info.Mode()&os.ModeSymlink == 0 {
|
||||
return "", nil, errResourcePlan
|
||||
}
|
||||
st, ok := info.Sys().(*syscall.Stat_t)
|
||||
if !ok || st.Uid != 0 {
|
||||
return "", nil, errResourcePlan
|
||||
}
|
||||
link, e := os.Readlink(alias)
|
||||
if e != nil {
|
||||
return "", nil, errResourcePlan
|
||||
}
|
||||
// Check the raw spelling before Join can erase symlink/.. traversal. Only
|
||||
// the platform observed in the pinned qualification package is supported.
|
||||
if link != "../binaries/linux-x86_64/nfqws2" {
|
||||
return "", nil, errResourcePlan
|
||||
}
|
||||
path := filepath.Join(root, "binaries/linux-x86_64/nfqws2")
|
||||
if qualificationParents(path, boundary) != nil {
|
||||
return "", nil, errResourcePlan
|
||||
}
|
||||
return path, info, nil
|
||||
}
|
||||
|
||||
func qualificationBinaryHash(f *os.File) (os.FileInfo, string, error) {
|
||||
info, e := f.Stat()
|
||||
if e != nil || !info.Mode().IsRegular() || !qualificationTrusted(info) || info.Mode().Perm()&0111 == 0 || info.Size() > 32<<20 {
|
||||
return nil, "", errResourcePlan
|
||||
}
|
||||
b, e := io.ReadAll(io.NewSectionReader(f, 0, 32<<20+1))
|
||||
if e != nil || int64(len(b)) != info.Size() {
|
||||
return nil, "", errResourcePlan
|
||||
}
|
||||
return info, planDigest(b), nil
|
||||
}
|
||||
|
||||
func openQualificationEngine(root string) (*qualificationBinary, error) {
|
||||
path, alias, e := qualificationEnginePath(root)
|
||||
if e != nil {
|
||||
return nil, e
|
||||
}
|
||||
fd, e := syscall.Open(path, syscall.O_RDONLY|syscall.O_NOFOLLOW|syscall.O_NONBLOCK|syscall.O_CLOEXEC, 0)
|
||||
if e != nil {
|
||||
return nil, errResourcePlan
|
||||
}
|
||||
f := os.NewFile(uintptr(fd), path)
|
||||
info, hash, e := qualificationBinaryHash(f)
|
||||
b := &qualificationBinary{root: root, path: path, alias: alias, file: f, info: info, hash: hash}
|
||||
if e != nil || b.validate() != nil {
|
||||
f.Close()
|
||||
return nil, errResourcePlan
|
||||
}
|
||||
return b, nil
|
||||
}
|
||||
|
||||
func (b *qualificationBinary) validate() error {
|
||||
path, alias, e := qualificationEnginePath(b.root)
|
||||
if e != nil || path != b.path || !os.SameFile(alias, b.alias) {
|
||||
return errResourcePlan
|
||||
}
|
||||
current, e := os.Lstat(path)
|
||||
if e != nil || !current.Mode().IsRegular() || !os.SameFile(current, b.info) || !qualificationTrusted(current) {
|
||||
return errResourcePlan
|
||||
}
|
||||
info, hash, e := qualificationBinaryHash(b.file)
|
||||
if e != nil || !os.SameFile(info, b.info) || hash != b.hash {
|
||||
return errResourcePlan
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (b *qualificationBinary) running(procExe string) error {
|
||||
// /proc/PID/exe is a kernel-owned link, not a catalogue path. Compare the
|
||||
// opened running inode and bytes, never the alias spelling from readlink.
|
||||
f, e := os.Open(procExe)
|
||||
if e != nil {
|
||||
return errResourcePlan
|
||||
}
|
||||
defer f.Close()
|
||||
info, hash, e := qualificationBinaryHash(f)
|
||||
if e != nil || !os.SameFile(info, b.info) || hash != b.hash {
|
||||
return errResourcePlan
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -0,0 +1,265 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"fmt"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func qualificationBinaryFixture(t *testing.T) (string, string, string) {
|
||||
t.Helper()
|
||||
if os.Geteuid() != 0 {
|
||||
t.Skip("root-owned package fixture requires root; run with sudo")
|
||||
}
|
||||
root := t.TempDir()
|
||||
for _, dir := range []string{"nfq2", "binaries/linux-x86_64", "lua"} {
|
||||
if e := os.MkdirAll(filepath.Join(root, dir), 0755); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
}
|
||||
path := filepath.Join(root, "binaries/linux-x86_64/nfqws2")
|
||||
for _, name := range []string{path, filepath.Join(root, "lua/zapret-lib.lua"), filepath.Join(root, "lua/zapret-antidpi.lua")} {
|
||||
if e := os.WriteFile(name, []byte("fixed package bytes"), 0755); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
}
|
||||
alias := filepath.Join(root, "nfq2/nfqws2")
|
||||
if e := os.Symlink("../binaries/linux-x86_64/nfqws2", alias); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
return root, path, alias
|
||||
}
|
||||
|
||||
func TestQualificationPackagedAlias(t *testing.T) {
|
||||
root, path, _ := qualificationBinaryFixture(t)
|
||||
b, e := openQualificationEngine(root)
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
defer b.file.Close()
|
||||
want := planDigest([]string{planDigest([]byte("fixed package bytes")), planDigest([]byte("fixed package bytes")), planDigest([]byte("fixed package bytes"))})
|
||||
if hash, e := b.filesHash(); e != nil || hash != want || b.path != path {
|
||||
t.Fatal("packaged alias hash", hash, e)
|
||||
}
|
||||
proc := filepath.Join(root, "proc-exe")
|
||||
if e := os.Symlink(path, proc); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if b.running(proc) != nil {
|
||||
t.Fatal("canonical running inode rejected")
|
||||
}
|
||||
other := filepath.Join(root, "other")
|
||||
if e := os.WriteFile(other, []byte("fixed package bytes"), 0755); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if b.running(other) == nil {
|
||||
t.Fatal("same hash, different running inode accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func TestQualificationPackagedAliasRejects(t *testing.T) {
|
||||
for _, mode := range []string{"escape", "final-symlink", "writable", "untrusted", "parent-writable", "parent-symlink", "missing"} {
|
||||
t.Run(mode, func(t *testing.T) {
|
||||
root, path, alias := qualificationBinaryFixture(t)
|
||||
var e error
|
||||
switch mode {
|
||||
case "escape":
|
||||
e = os.Remove(alias)
|
||||
if e == nil {
|
||||
e = os.Symlink("/bin/nfqws2", alias)
|
||||
}
|
||||
case "final-symlink":
|
||||
e = os.Remove(path)
|
||||
if e == nil {
|
||||
e = os.Symlink("/bin/true", path)
|
||||
}
|
||||
case "writable":
|
||||
e = os.Chmod(path, 0775)
|
||||
case "untrusted":
|
||||
e = os.Chown(path, 1000, 1000)
|
||||
case "parent-writable":
|
||||
e = os.Chmod(filepath.Dir(path), 0777)
|
||||
case "parent-symlink":
|
||||
dir := filepath.Dir(path)
|
||||
e = os.Rename(dir, dir+"-real")
|
||||
if e == nil {
|
||||
e = os.Symlink(dir+"-real", dir)
|
||||
}
|
||||
case "missing":
|
||||
e = os.Remove(path)
|
||||
}
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if b, e := openQualificationEngine(root); e == nil {
|
||||
b.file.Close()
|
||||
t.Fatal("unsafe package accepted")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestQualificationPackagedAliasRawSpelling(t *testing.T) {
|
||||
for _, link := range []string{
|
||||
"../binaries/linux-x86_64/./nfqws2",
|
||||
"../binaries//linux-x86_64/nfqws2",
|
||||
"../binaries/linux-x86_64/../linux-x86_64/nfqws2",
|
||||
"../binaries/unsupported/nfqws2",
|
||||
"../binaries/linux-x86_64/nested/nfqws2",
|
||||
} {
|
||||
t.Run(link, func(t *testing.T) {
|
||||
root, _, alias := qualificationBinaryFixture(t)
|
||||
if e := os.Remove(alias); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if e := os.Symlink(link, alias); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if b, e := openQualificationEngine(root); e == nil {
|
||||
b.file.Close()
|
||||
t.Fatal("non-package spelling accepted")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestQualificationConcealedTraversalBeforeWorker(t *testing.T) {
|
||||
root, _, alias := qualificationBinaryFixture(t)
|
||||
attacker := t.TempDir()
|
||||
marker := filepath.Join(attacker, "executed")
|
||||
for _, dir := range []string{"subdir", "linux-x86_64"} {
|
||||
if e := os.Mkdir(filepath.Join(attacker, dir), 0755); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
}
|
||||
outside := filepath.Join(attacker, "linux-x86_64/nfqws2")
|
||||
if e := os.WriteFile(outside, []byte(fmt.Sprintf("#!/bin/sh\n: > %q\n", marker)), 0755); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if e := os.Symlink(filepath.Join(attacker, "subdir"), filepath.Join(root, "binaries/redirect")); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if e := os.Remove(alias); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if e := os.Symlink("../binaries/redirect/../linux-x86_64/nfqws2", alias); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
resolved, e := os.Stat(alias)
|
||||
outsideInfo, outsideErr := os.Stat(outside)
|
||||
if e != nil || outsideErr != nil || !os.SameFile(resolved, outsideInfo) {
|
||||
t.Fatal("fixture must resolve outside package", e, outsideErr)
|
||||
}
|
||||
q, s, registry, events, calls := qualifierFixture(t)
|
||||
q.filesHash = func() (string, error) {
|
||||
b, e := openQualificationEngine(root)
|
||||
if e != nil {
|
||||
return "", e
|
||||
}
|
||||
defer b.file.Close()
|
||||
return b.filesHash()
|
||||
}
|
||||
starts := 0
|
||||
q.runtime.start = func([]string) (managedAssetWorker, error) {
|
||||
starts++
|
||||
if e := exec.Command(alias).Run(); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
return managedAssetWorker{}, errResourcePlan
|
||||
}
|
||||
proof, e := q.complete(context.Background(), s, registry)
|
||||
if !errors.Is(e, errResourcePlan) || starts != 0 || *calls != 0 || len(*events) != 0 || proof.seal != "" {
|
||||
t.Fatalf("traversal rejection too late: err=%v starts=%d calls=%d events=%v", e, starts, *calls, *events)
|
||||
}
|
||||
if _, e := os.Stat(marker); !errors.Is(e, os.ErrNotExist) {
|
||||
t.Fatal("outside executable ran", e)
|
||||
}
|
||||
}
|
||||
|
||||
func TestQualificationPackagedAliasProcExe(t *testing.T) {
|
||||
root, path, alias := qualificationBinaryFixture(t)
|
||||
bytes, e := os.ReadFile("/bin/sleep")
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if e := os.WriteFile(path, bytes, 0755); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
b, e := openQualificationEngine(root)
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
defer b.file.Close()
|
||||
cmd := exec.Command(alias, "60")
|
||||
if e := cmd.Start(); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
defer func() { cmd.Process.Kill(); cmd.Wait() }()
|
||||
if b.running(fmt.Sprintf("/proc/%d/exe", cmd.Process.Pid)) != nil {
|
||||
t.Fatal("real packaged worker inode rejected")
|
||||
}
|
||||
if _, e := b.filesHash(); e != nil {
|
||||
t.Fatal("valid pinned alias changed after start", e)
|
||||
}
|
||||
if b.running("/proc/self/exe") == nil {
|
||||
t.Fatal("foreign running executable accepted")
|
||||
}
|
||||
}
|
||||
|
||||
func TestQualificationPackagedAliasMutationBeforeStart(t *testing.T) {
|
||||
for _, mode := range []string{"resolution", "alias-replaced", "binary-replaced", "bytes", "lua-symlink", "lua-writable"} {
|
||||
t.Run(mode, func(t *testing.T) {
|
||||
root, path, alias := qualificationBinaryFixture(t)
|
||||
b, e := openQualificationEngine(root)
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
defer b.file.Close()
|
||||
if _, e := b.filesHash(); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
switch mode {
|
||||
case "resolution", "alias-replaced":
|
||||
link := "../binaries/linux-x86_64/nfqws2"
|
||||
if mode == "resolution" {
|
||||
link = "../binaries/other/nfqws2"
|
||||
if e := os.Mkdir(filepath.Join(root, "binaries/other"), 0755); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if e := os.WriteFile(filepath.Join(root, "binaries/other/nfqws2"), []byte("fixed package bytes"), 0755); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
}
|
||||
e = os.Rename(alias, alias+"-old")
|
||||
if e == nil {
|
||||
e = os.Symlink(link, alias)
|
||||
}
|
||||
case "binary-replaced":
|
||||
e = os.Rename(path, path+"-old")
|
||||
if e == nil {
|
||||
e = os.WriteFile(path, []byte("fixed package bytes"), 0755)
|
||||
}
|
||||
case "bytes":
|
||||
e = os.WriteFile(path, []byte("mutated package"), 0755)
|
||||
case "lua-symlink":
|
||||
lua := filepath.Join(root, "lua/zapret-lib.lua")
|
||||
e = os.Remove(lua)
|
||||
if e == nil {
|
||||
e = os.Symlink(path, lua)
|
||||
}
|
||||
case "lua-writable":
|
||||
e = os.Chmod(filepath.Join(root, "lua/zapret-lib.lua"), 0666)
|
||||
}
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if _, e := b.filesHash(); e == nil {
|
||||
t.Fatal("changed package accepted before start")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,178 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"io"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
"syscall"
|
||||
"time"
|
||||
)
|
||||
|
||||
const qualificationCompletionName = "qualification-completed.json"
|
||||
const qualificationCompletionDir = "/run/zapret-autotune-ss"
|
||||
|
||||
// This acknowledges controller cleanup, NOT candidate qualification. It cannot
|
||||
// authorize a PASS, matrix write or acceptance of ordinary monitor exit 78.
|
||||
type qualificationCompletion struct {
|
||||
Version int `json:"version"`
|
||||
PID int `json:"pid"`
|
||||
Code int `json:"code"`
|
||||
CleanupComplete bool `json:"cleanup_complete"`
|
||||
}
|
||||
|
||||
type qualificationCleanupFailure struct{ Err error }
|
||||
|
||||
func (e *qualificationCleanupFailure) Error() string { return "qualification cleanup unverified" }
|
||||
func (e *qualificationCleanupFailure) Unwrap() error { return e.Err }
|
||||
|
||||
func qualificationCompletionStore(path string, key []byte) (*resourcePlanStore, error) {
|
||||
if managedDirectory(path) != nil || len(key) != 32 {
|
||||
return nil, errResourcePlan
|
||||
}
|
||||
fd, e := syscall.Open(path, syscall.O_RDONLY|syscall.O_DIRECTORY|syscall.O_NOFOLLOW|syscall.O_CLOEXEC, 0)
|
||||
if e != nil {
|
||||
return nil, e
|
||||
}
|
||||
s := &resourcePlanStore{files: &resourceMonitorStore{path: path, dir: os.NewFile(uintptr(fd), "qualification completion directory"), key: append([]byte(nil), key...)}, held: map[string]*os.File{}, snapshots: map[string][]byte{}}
|
||||
f, b, e := s.files.openEntry(qualificationCompletionName)
|
||||
if e == nil {
|
||||
s.held[qualificationCompletionName], s.snapshots[qualificationCompletionName] = f, b
|
||||
} else if !errors.Is(e, os.ErrNotExist) {
|
||||
s.close()
|
||||
return nil, errResourcePlan
|
||||
}
|
||||
return s, nil
|
||||
}
|
||||
|
||||
func writeQualificationCompletion(path string, key []byte, pid, code int) error {
|
||||
if pid <= 1 || (code != 0 && code != 78) {
|
||||
return errResourcePlan
|
||||
}
|
||||
s, e := qualificationCompletionStore(path, key)
|
||||
if e != nil {
|
||||
return e
|
||||
}
|
||||
defer s.close()
|
||||
return s.writeAuthenticated(qualificationCompletionName, qualificationCompletion{Version: 1, PID: pid, Code: code, CleanupComplete: true})
|
||||
}
|
||||
|
||||
func readQualificationCompletion(path string, key []byte, pid, code int) error {
|
||||
s, e := qualificationCompletionStore(path, key)
|
||||
if e != nil {
|
||||
return e
|
||||
}
|
||||
defer s.close()
|
||||
var ack qualificationCompletion
|
||||
if s.readAuthenticated(qualificationCompletionName, &ack) != nil || ack.Version != 1 || ack.PID != pid || pid <= 1 || ack.Code != code || (code != 0 && code != 78) || !ack.CleanupComplete {
|
||||
return errResourcePlan
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func qualificationCleanupVerified(err error, restore func(context.Context) error) bool {
|
||||
var ownedCleanup *managedAssetCleanupFailure
|
||||
var restoration *qualificationCleanupFailure
|
||||
if errors.As(err, &ownedCleanup) || errors.As(err, &restoration) {
|
||||
return false
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
|
||||
defer cancel()
|
||||
return restore(ctx) == nil
|
||||
}
|
||||
|
||||
// Same cleanup gate and ordering as the controller's completion defer. Keeping
|
||||
// operations separate makes ACK refusal diagnosable without exposing errors.
|
||||
func qualificationFinalize(err error, restore func(context.Context) error, guard, ack func() error) error {
|
||||
var restoreErr error
|
||||
if !qualificationCleanupVerified(err, func(ctx context.Context) error {
|
||||
restoreErr = restore(ctx)
|
||||
return restoreErr
|
||||
}) {
|
||||
if restoreErr == nil {
|
||||
return qualificationOperation(qualificationPredicate(errResourcePlan, reasonPriorCleanupVeto), stageRestore)
|
||||
}
|
||||
return qualificationOperation(restoreErr, stageRestore)
|
||||
}
|
||||
if e := guard(); e != nil {
|
||||
return qualificationOperation(e, stageFinalStoreGuard)
|
||||
}
|
||||
return qualificationOperation(ack(), stageACKWrite)
|
||||
}
|
||||
|
||||
func qualificationCompletionCLI(args []string, d managedDeps) int {
|
||||
if len(args) != 2 || os.Geteuid() != 0 || os.Getppid() != 1 {
|
||||
return 78
|
||||
}
|
||||
pid, e := strconv.Atoi(args[0])
|
||||
code, ce := strconv.Atoi(args[1])
|
||||
if e != nil || ce != nil || strconv.Itoa(pid) != args[0] || strconv.Itoa(code) != args[1] {
|
||||
return 78
|
||||
}
|
||||
cmd, e := os.ReadFile("/proc/1/cmdline")
|
||||
if e != nil || !bytes.Equal(cmd, []byte("/bin/sh\x00/opt/managed-target-entrypoint.sh\x00")) {
|
||||
return 78
|
||||
}
|
||||
env, e := os.ReadFile("/proc/1/environ")
|
||||
if e != nil {
|
||||
return 78
|
||||
}
|
||||
values := map[string]string{}
|
||||
for _, field := range strings.Split(string(env), "\x00") {
|
||||
k, v, _ := strings.Cut(field, "=")
|
||||
values[k] = v
|
||||
}
|
||||
if values["ZAPRET_AUTOTUNE_QUALIFY_CONFIG"] == "" || values["ZAPRET_AUTOTUNE_STRATEGY_CONFIG"] == "" {
|
||||
return 78
|
||||
}
|
||||
if _, e := managedRuntimeGuard(d); e != nil {
|
||||
return 78
|
||||
}
|
||||
lock, e := acquireManagedMonitorSingleton(d)
|
||||
if e != nil {
|
||||
return 78
|
||||
}
|
||||
defer lock.close()
|
||||
s, e := openResourcePlanStore(d.directory, false)
|
||||
if e != nil {
|
||||
return 78
|
||||
}
|
||||
defer s.close()
|
||||
if s.protectBaseline() != nil || qualificationBaseline(d.directory) != nil {
|
||||
return 78
|
||||
}
|
||||
dns, e := parseManagedDNSConfig(d.lookup)
|
||||
if e != nil || verifyResourceDNSRuntime(dns) != nil || qualificationNoWorker() != nil {
|
||||
return 78
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
defer cancel()
|
||||
r := newManagedAssetRuntime(d)
|
||||
tables, e := r.nft(ctx, "list tables\n")
|
||||
if e != nil || strings.Contains(tables, "table inet "+managedAssetTable) {
|
||||
return 78
|
||||
}
|
||||
// The private session ACK ties completion to this PID 1 launch, not a stale
|
||||
// target-volume status. The completion itself lives only on this boot's tmpfs.
|
||||
dir, e := os.Open(qualificationCompletionDir)
|
||||
if e != nil {
|
||||
return 78
|
||||
}
|
||||
defer dir.Close()
|
||||
f, e := resourcePrivateFile(dir, "monitor-ready")
|
||||
if e != nil {
|
||||
return 78
|
||||
}
|
||||
defer f.Close()
|
||||
b, e := io.ReadAll(io.LimitReader(f, 32))
|
||||
if e != nil || string(b) != strconv.Itoa(pid)+"\n" {
|
||||
return 78
|
||||
}
|
||||
if readQualificationCompletion(qualificationCompletionDir, s.files.key, pid, code) != nil || s.guard() != nil {
|
||||
return 78
|
||||
}
|
||||
return 0
|
||||
}
|
||||
@@ -0,0 +1,102 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"errors"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestResourceQualifierCompletionOwnership(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
if e := os.Chmod(dir, 0700); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
key := bytes.Repeat([]byte{9}, 32)
|
||||
if e := writeQualificationCompletion(dir, key, 123, 78); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if e := readQualificationCompletion(dir, key, 123, 78); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
for _, mismatch := range []struct {
|
||||
pid, code int
|
||||
key []byte
|
||||
}{{124, 78, key}, {123, 0, key}, {123, 78, bytes.Repeat([]byte{8}, 32)}} {
|
||||
if readQualificationCompletion(dir, mismatch.key, mismatch.pid, mismatch.code) == nil {
|
||||
t.Fatal("accepted foreign launch/status/key")
|
||||
}
|
||||
}
|
||||
if writeQualificationCompletion(dir, key, 123, 78) == nil {
|
||||
t.Fatal("clobbered write-once completion")
|
||||
}
|
||||
path := filepath.Join(dir, qualificationCompletionName)
|
||||
raw, e := os.ReadFile(path)
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if e := os.WriteFile(path, bytes.Replace(raw, []byte(`"cleanup_complete":true`), []byte(`"cleanup_complete":false`), 1), 0600); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if readQualificationCompletion(dir, key, 123, 78) == nil {
|
||||
t.Fatal("accepted forged completion")
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceQualifierCompletionCleanupVeto(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
err error
|
||||
safe bool
|
||||
}{
|
||||
{nil, true}, {errors.New("source changed, proof UNKNOWN"), true}, {context.Canceled, true},
|
||||
{&managedRecoveryFatal{Err: &managedAssetCleanupFailure{Err: errors.New("worker join failed")}}, false},
|
||||
{errors.Join(context.Canceled, &qualificationCleanupFailure{Err: errors.New("baseline restoration failed")}), false},
|
||||
} {
|
||||
called := false
|
||||
got := qualificationCleanupVerified(tc.err, func(context.Context) error { called = true; return nil })
|
||||
if got != tc.safe || called != tc.safe {
|
||||
t.Fatalf("cleanup ACK=%v called=%v for %T", got, called, tc.err)
|
||||
}
|
||||
}
|
||||
if qualificationCleanupVerified(nil, func(context.Context) error { return errors.New("live baseline guard failed") }) {
|
||||
t.Fatal("acknowledged unverified cleanup")
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceQualifierCompletionForeignFile(t *testing.T) {
|
||||
dir := t.TempDir()
|
||||
os.Chmod(dir, 0700)
|
||||
path := filepath.Join(dir, qualificationCompletionName)
|
||||
if e := os.WriteFile(path, []byte("FOREIGN"), 0600); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
key := bytes.Repeat([]byte{9}, 32)
|
||||
if writeQualificationCompletion(dir, key, 123, 78) == nil || readQualificationCompletion(dir, key, 123, 78) == nil {
|
||||
t.Fatal("adopted foreign completion")
|
||||
}
|
||||
b, e := os.ReadFile(path)
|
||||
if e != nil || string(b) != "FOREIGN" {
|
||||
t.Fatal("changed foreign completion")
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceQualifierOwnedCleanupFailureCannotAcknowledge(t *testing.T) {
|
||||
q, s, r, _, _ := qualifierFixture(t)
|
||||
nft := q.runtime.nft
|
||||
q.runtime.nft = func(ctx context.Context, input string) (string, error) {
|
||||
if strings.HasPrefix(input, "delete table") {
|
||||
return "", errors.New("owned nft cleanup failed")
|
||||
}
|
||||
return nft(ctx, input)
|
||||
}
|
||||
proof, err := q.complete(context.Background(), s, r)
|
||||
if err == nil || proof.seal != "" {
|
||||
t.Fatal("unsafe lifecycle sealed proof")
|
||||
}
|
||||
if qualificationCleanupVerified(err, q.restore) {
|
||||
t.Fatal("unsafe owned cleanup authorized completed ACK")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,332 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"net"
|
||||
"os"
|
||||
"os/exec"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func qualifierFixture(t *testing.T, candidates ...int) (resourceQualifier, *resourcePlanStore, strategyRegistry, *[]string, *int) {
|
||||
t.Helper()
|
||||
dir := t.TempDir()
|
||||
os.Chmod(dir, 0700)
|
||||
store, e := openResourcePlanStore(dir, true)
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
t.Cleanup(store.close)
|
||||
raw := `{"version":1,"services":[{"id":"one","resources":[{"id":"good","group":"a","kind":"http","url":"https://a.example/good?SECRET","method":"GET","tls":"both","timeout_ms":1000,"expect":{"statuses":[200],"min_bytes":1,"max_bytes":1000}}]},{"id":"two","resources":[{"id":"other","group":"b","kind":"http","url":"https://a.example/other?SECRET","method":"GET","tls":"both","timeout_ms":1000,"expect":{"statuses":[200],"min_bytes":1,"max_bytes":1000}}]}]}`
|
||||
cfg, e := parseResourceChecks([]byte(raw), func(string) (string, bool) { return "", false })
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
source := resourceMonitorSource{config: cfg, server: "127.0.0.1:5353", proxy: "socks5h://127.0.0.1:1080", fingerprint: resourceMAC(store.files.key, []byte(raw))}
|
||||
_, _, registry, _, _ := planFixture(t)
|
||||
count := 1
|
||||
if len(candidates) > 0 {
|
||||
count = candidates[0]
|
||||
}
|
||||
registry.Strategies = registry.Strategies[:count]
|
||||
b, _ := json.Marshal(registry)
|
||||
if store.importDefinitions(b) != nil {
|
||||
t.Fatal("registry import")
|
||||
}
|
||||
runtime, events, _ := assetRuntimeFixture(t)
|
||||
calls := 0
|
||||
q := resourceQualifier{runtime: runtime, source: func() (resourceMonitorSource, error) { return source, nil }, guard: func() error { return nil }, now: time.Now, filesHash: func() (string, error) { return strings.Repeat("a", 64), nil }}
|
||||
q.restore = func(ctx context.Context) error { *events = append(*events, "restore"); return ctx.Err() }
|
||||
q.attest = func(_ context.Context, pid int, args []string, d strategyDefinition) (qualificationAttestation, error) {
|
||||
if !strings.Contains(strings.Join(args, " "), "--filter-l7=tls") {
|
||||
t.Fatal("generic TLS selector not enforced")
|
||||
}
|
||||
*events = append(*events, "attest")
|
||||
return qualificationAttestation{Collector: "owned-resource-nfq-v1", Runtime: "nfqws2-1.0.2", Definition: planDigest(d), LuaHash: strings.Repeat("a", 64), RulesHash: planDigest(managedAssetRules), WorkerArgsHash: planDigest(args), Queue: 200, PID: pid, StartTime: 456}, nil
|
||||
}
|
||||
q.checks = resourceCheckDeps{guard: func() error { return nil }, resolve: func(context.Context, string, string) ([]net.IP, error) { return []net.IP{net.ParseIP("8.8.8.8")}, nil }, runner: func(_ context.Context, args []string, stdin io.Reader, out, diagnostics io.Writer) error {
|
||||
calls++
|
||||
*events = append(*events, "capture")
|
||||
b, e := io.ReadAll(stdin)
|
||||
if e != nil {
|
||||
return e
|
||||
}
|
||||
if !strings.Contains(string(b), "connect-to =") || strings.Contains(strings.Join(args, " "), "SECRET") {
|
||||
t.Fatal("private pin/argv violated")
|
||||
}
|
||||
raw, e := strconv.Unquote(strings.TrimPrefix(strings.Split(string(b), "\n")[0], "url = "))
|
||||
if e != nil {
|
||||
return e
|
||||
}
|
||||
protocol := "tls12"
|
||||
if strings.Contains(strings.Join(args, " "), "--tlsv1.3") {
|
||||
protocol = "tls13"
|
||||
}
|
||||
configuredWire(raw, "GET", protocol, "pass", out, diagnostics)
|
||||
return nil
|
||||
}}
|
||||
return q, store, registry, events, &calls
|
||||
}
|
||||
|
||||
func TestResourceQualifierCompletedProducer(t *testing.T) {
|
||||
q, s, r, events, calls := qualifierFixture(t)
|
||||
proof, e := q.complete(context.Background(), s, r)
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if *calls != 12 || len(proof.matrix.Runs) != 1 || len(proof.matrix.Runs[0].Checks) != 4 {
|
||||
t.Fatalf("real repeats %d %+v", *calls, proof)
|
||||
}
|
||||
for _, c := range proof.matrix.Runs[0].Checks {
|
||||
if c.Outcome != "pass" || c.Attempts != 3 {
|
||||
t.Fatalf("%+v", c)
|
||||
}
|
||||
}
|
||||
last := -1
|
||||
for _, event := range []string{"start", "ready", "activate", "attest", "capture", "delete", "join", "restore"} {
|
||||
found := -1
|
||||
for i := last + 1; i < len(*events); i++ {
|
||||
if (*events)[i] == event {
|
||||
found = i
|
||||
break
|
||||
}
|
||||
}
|
||||
if found < 0 {
|
||||
t.Fatalf("ordering: %v", *events)
|
||||
}
|
||||
last = found
|
||||
}
|
||||
if !proof.matrix.Runs[0].Attestation.Joined || !proof.matrix.Runs[0].Attestation.Restored {
|
||||
t.Fatal("premature seal")
|
||||
}
|
||||
if s.saveCompleted(proof) != nil {
|
||||
t.Fatal("producer publication")
|
||||
}
|
||||
before := *calls
|
||||
if _, e := q.complete(context.Background(), s, r); e == nil || *calls != before {
|
||||
t.Fatal("retested immutable matrix")
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceQualifierCapacityBeforeWorkers(t *testing.T) {
|
||||
q, _, registry, events, calls := qualifierFixture(t)
|
||||
dir := t.TempDir()
|
||||
if e := os.Chmod(dir, 0700); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
s, e := openResourcePlanStore(dir, true)
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
defer s.close()
|
||||
template := registry.Strategies[0]
|
||||
registry.Strategies = nil
|
||||
for i := 0; i < 32; i++ {
|
||||
definition := template
|
||||
definition.ID = "strategy-" + strconv.Itoa(i)
|
||||
registry.Strategies = append(registry.Strategies, definition)
|
||||
}
|
||||
b, _ := json.Marshal(registry)
|
||||
registry, e = parseStrategyRegistry(b)
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if e = s.importDefinitions(b); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
source, e := q.source()
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
spec := source.config.Services[0].Resources[0]
|
||||
cfg := resourceCheckConfig{Version: 1, Services: []resourceCheckService{{ID: "service"}}}
|
||||
for i := 0; i < 64; i++ {
|
||||
resource := spec
|
||||
resource.ID = "resource-" + strconv.Itoa(i)
|
||||
cfg.Services[0].Resources = append(cfg.Services[0].Resources, resource)
|
||||
}
|
||||
b, _ = json.Marshal(cfg)
|
||||
cfg, e = parseResourceChecks(b, func(string) (string, bool) { return "", false })
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
source.config = cfg
|
||||
source.fingerprint = resourceMAC(s.files.key, b)
|
||||
q.source = func() (resourceMonitorSource, error) { return source, nil }
|
||||
starts := 0
|
||||
start := q.runtime.start
|
||||
q.runtime.start = func(args []string) (managedAssetWorker, error) { starts++; return start(args) }
|
||||
proof, e := q.complete(context.Background(), s, registry)
|
||||
if !errors.Is(e, errQualificationCapacity) || starts != 0 || *calls != 0 || len(*events) != 0 || proof.seal != "" || s.held[planMatrixName] != nil {
|
||||
t.Fatalf("capacity rejection too late: err=%v starts=%d calls=%d events=%v", e, starts, *calls, *events)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceQualifierSequentialCandidates(t *testing.T) {
|
||||
q, s, r, events, calls := qualifierFixture(t, 2)
|
||||
proof, err := q.complete(context.Background(), s, r)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if *calls != 24 || len(proof.matrix.Runs) != 2 {
|
||||
t.Fatal("incomplete sequential batch", *calls)
|
||||
}
|
||||
stages := strings.Join(*events, ",")
|
||||
if strings.Count(stages, "start") != 2 || strings.Count(stages, "restore") != 2 || !strings.Contains(stages, "delete,join,restore,inventory,start") {
|
||||
t.Fatal("next candidate began before previous cleanup", stages)
|
||||
}
|
||||
if s.saveCompleted(proof) != nil {
|
||||
t.Fatal("sequential producer publication")
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceQualifierNegativeAndUnknown(t *testing.T) {
|
||||
for _, mode := range []string{"negative", "status500", "tls-peer", "timeout", "timeout-infrastructure", "infrastructure", "guard", "cancel", "source", "restore", "hash", "mutation"} {
|
||||
t.Run(mode, func(t *testing.T) {
|
||||
q, s, r, _, calls := qualifierFixture(t)
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
original := q.checks.runner
|
||||
if strings.HasPrefix(mode, "timeout") {
|
||||
oldSource := q.source
|
||||
source, e := oldSource()
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
for i := range source.config.Services {
|
||||
for j := range source.config.Services[i].Resources {
|
||||
source.config.Services[i].Resources[j].TimeoutMS = 100
|
||||
}
|
||||
}
|
||||
q.source = func() (resourceMonitorSource, error) { return source, nil }
|
||||
}
|
||||
q.checks.runner = func(ctx context.Context, args []string, stdin io.Reader, out, diag io.Writer) error {
|
||||
if mode == "tls-peer" {
|
||||
(*calls)++
|
||||
return exec.CommandContext(ctx, "/bin/sh", "-c", "exit 35").Run()
|
||||
}
|
||||
if strings.HasPrefix(mode, "timeout") {
|
||||
(*calls)++
|
||||
<-ctx.Done()
|
||||
if mode == "timeout-infrastructure" {
|
||||
return errors.New("local safety failure coincident with timeout")
|
||||
}
|
||||
return ctx.Err()
|
||||
}
|
||||
if mode == "infrastructure" {
|
||||
return errors.New("unknown local collector infrastructure")
|
||||
}
|
||||
if mode == "cancel" {
|
||||
cancel()
|
||||
return context.Canceled
|
||||
}
|
||||
if (mode == "negative" || mode == "status500") && *calls == 6 { // First repeat of other URL; still do all repeats.
|
||||
*calls++
|
||||
b, e := io.ReadAll(stdin)
|
||||
if e != nil {
|
||||
return e
|
||||
}
|
||||
raw, e := strconv.Unquote(strings.TrimPrefix(strings.Split(string(b), "\n")[0], "url = "))
|
||||
if e != nil {
|
||||
return e
|
||||
}
|
||||
wireMode := "redirect"
|
||||
if mode == "status500" {
|
||||
wireMode = "status500"
|
||||
}
|
||||
configuredWire(raw, "GET", "tls12", wireMode, out, diag)
|
||||
return nil
|
||||
}
|
||||
return original(ctx, args, stdin, out, diag)
|
||||
}
|
||||
if mode == "guard" {
|
||||
q.checks.guard = func() error { return errors.New("ownership lost") }
|
||||
}
|
||||
if mode == "source" {
|
||||
source := q.source
|
||||
q.source = func() (resourceMonitorSource, error) {
|
||||
v, e := source()
|
||||
if *calls > 0 {
|
||||
v.fingerprint = strings.Repeat("d", 64)
|
||||
}
|
||||
return v, e
|
||||
}
|
||||
}
|
||||
if mode == "restore" {
|
||||
q.restore = func(context.Context) error { return errors.New("unsafe restore") }
|
||||
}
|
||||
if mode == "hash" {
|
||||
q.filesHash = func() (string, error) { return strings.Repeat("b", 64), nil }
|
||||
}
|
||||
proof, e := q.complete(ctx, s, r)
|
||||
if mode == "negative" || mode == "status500" || mode == "timeout" || mode == "tls-peer" {
|
||||
if e != nil || *calls != 12 {
|
||||
t.Fatalf("negative incomplete: %v %d", e, *calls)
|
||||
}
|
||||
source, _ := q.source()
|
||||
if mode == "status500" && proof.matrix.Runs[0].Checks[2].Outcome != "expectation_failed" {
|
||||
t.Fatal("500 must be a complete configured expectation failure")
|
||||
}
|
||||
if mode == "timeout" {
|
||||
for _, c := range proof.matrix.Runs[0].Checks {
|
||||
if c.Attempts != 3 || c.Outcome != "timeout" {
|
||||
t.Fatal("timeout contract", c)
|
||||
}
|
||||
}
|
||||
}
|
||||
p := deriveResourcePlan(source, s.files.key, r, proof.matrix, time.Now())
|
||||
if len(p.Assignments) != 0 || len(p.Domains[0].Negative) != 1 {
|
||||
t.Fatalf("bad required URL qualified: %+v", p)
|
||||
}
|
||||
return
|
||||
}
|
||||
if mode == "mutation" {
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
proof.matrix.Runs[0].Checks[0].Outcome = "timeout"
|
||||
if s.saveCompleted(proof) == nil {
|
||||
t.Fatal("post-seal mutation")
|
||||
}
|
||||
return
|
||||
}
|
||||
if e == nil || proof.seal != "" || s.held[planMatrixName] != nil {
|
||||
t.Fatalf("published incomplete %s: %v", mode, e)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceQualifierTypedPeerResponseBoundary(t *testing.T) {
|
||||
for _, mode := range []string{"redirect", "extra"} {
|
||||
t.Run(mode, func(t *testing.T) {
|
||||
runner := func(_ context.Context, _ []string, _ io.Reader, out, diag io.Writer) error {
|
||||
configuredWire("https://a.example/", "GET", "tls12", mode, out, diag)
|
||||
return nil
|
||||
}
|
||||
_, _, err := collectResourceTransferOptions(context.Background(), "socks5h://127.0.0.1:1080", "https://a.example/", "a.example", "tls12", runner, resourceTransferOptions{method: "GET", limit: 100, timeout: time.Second})
|
||||
var peer *resourcePeerResponseRejection
|
||||
var fatal *managedRecoveryFatal
|
||||
if mode == "redirect" {
|
||||
if !errors.As(err, &peer) || errors.As(err, &fatal) || !resourceRejectionOnly(err) {
|
||||
t.Fatalf("redirect cause should be typed peer rejection, got %T", err)
|
||||
}
|
||||
} else if err == nil || resourceRejectionOnly(err) || !errors.As(err, &fatal) {
|
||||
t.Fatal("malformed body/evidence became peer negative")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceQualifierRejectsRegularTarget(t *testing.T) {
|
||||
d := managedDependencies(runPlayerCurl)
|
||||
if qualificationMode("resources", "strategies", d) == nil {
|
||||
t.Fatal("regular process authorized isolated qualification")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,278 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"errors"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestResourceStrategyIDs(t *testing.T) {
|
||||
for _, value := range []string{"a,a", "a,", " a", "../a", "a\n", strings.Repeat("a,", 32) + "b", strings.Repeat("a", 4097)} {
|
||||
if _, e := parseResourceStrategyIDs(value); e == nil {
|
||||
t.Fatalf("accepted %q", value)
|
||||
}
|
||||
}
|
||||
if got, e := parseResourceStrategyIDs("b,a"); e != nil || got != "a,b" {
|
||||
t.Fatal(got, e)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceSelectionCoverage(t *testing.T) {
|
||||
s, key, r, m, now := completeApplyFixture(t)
|
||||
// Equal profiles must not inherit coverage from an excluded candidate.
|
||||
r.Strategies[1].Args = r.Strategies[0].Args
|
||||
m.Registry = planDigest(r)
|
||||
m.Runs[1].Definition = planDigest(r.Strategies[1])
|
||||
m.Runs[1].Attestation.Definition = m.Runs[1].Definition
|
||||
defaultPlan, _, e := resourceApplyPlan(s, key, r, m, now)
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
p, _, e := resourceApplyPlan(s, key, r, m, now, "s1,s2,s3")
|
||||
if e != nil || p.Policy != "s1,s2,s3" || len(p.Assignments) != 2 {
|
||||
t.Fatal(p, e)
|
||||
}
|
||||
if !reflect.DeepEqual(p.Domains, defaultPlan.Domains) {
|
||||
t.Fatal("hidden alternatives")
|
||||
}
|
||||
for _, policy := range []string{"s1", "s2,s3", "s1,s3", "unknown", "s2,s1,s3"} {
|
||||
if _, _, e := resourceApplyPlan(s, key, r, m, now, policy); e == nil {
|
||||
t.Fatal("invalid coverage/policy", policy)
|
||||
}
|
||||
}
|
||||
for _, kind := range []string{"m3", "fingerprint", "unknown", "tls"} {
|
||||
bad := cloneMatrix(m)
|
||||
switch kind {
|
||||
case "m3":
|
||||
bad.Runs[0].Checks[0].Attempts = 2
|
||||
case "fingerprint":
|
||||
bad.Runs[0].Checks[0].Fingerprint = strings.Repeat("a", 64)
|
||||
case "unknown":
|
||||
bad.Runs[0].Attestation.Joined = false
|
||||
case "tls":
|
||||
r.Strategies[0].Selectors.TLS = "tls12"
|
||||
bad.Registry = planDigest(r)
|
||||
bad.Runs[0].Definition = planDigest(r.Strategies[0])
|
||||
bad.Runs[0].Attestation.Definition = bad.Runs[0].Definition
|
||||
}
|
||||
if _, _, e := resourceApplyPlan(s, key, r, bad, now, "s1,s2,s3"); e == nil {
|
||||
t.Fatal("invalid selected evidence", kind)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceGenericRotation(t *testing.T) {
|
||||
m, s, config, root := nativeApplyAlternativesFixture(t, true)
|
||||
proofs := map[string][]byte{}
|
||||
for _, name := range []string{planRegistryName, planMatrixName} {
|
||||
b, e := os.ReadFile(filepath.Join(m.Dir, name))
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
proofs[name] = b
|
||||
}
|
||||
reloads := 0
|
||||
m.Reload = func() error { reloads++; return nil }
|
||||
apply := func(policy string) {
|
||||
t.Helper()
|
||||
if e := s.selectPolicy(policy); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
}
|
||||
apply("")
|
||||
a, _ := m.load()
|
||||
old, _ := m.read("config")
|
||||
apply("zz-alternative")
|
||||
b, _ := m.load()
|
||||
if *a.Ref == *b.Ref || a.Hash == b.Hash || b.Ref.Policy != "zz-alternative" {
|
||||
t.Fatal(a, b)
|
||||
}
|
||||
// Reopen with a different startup request: identity is the stored policy.
|
||||
d := managedDeps{directory: m.Dir, lookup: func(k string) (string, bool) {
|
||||
if k == resourceStrategyIDsEnv {
|
||||
return "unknown", true
|
||||
}
|
||||
return resourceCheckLookup(k)
|
||||
}}
|
||||
reopened, e := openResourceApplySnapshotAt(config, d, root)
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
defer reopened.close()
|
||||
m.resource = reopened
|
||||
if e := m.Reconcile(); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
m.resource = s
|
||||
s.now = func() time.Time { return time.Now().Add(25 * time.Hour) }
|
||||
apply("zz-alternative") // historical replay, no reload
|
||||
if reloads != 2 {
|
||||
t.Fatal(reloads)
|
||||
}
|
||||
if e := s.selectPolicy(""); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e == nil {
|
||||
t.Fatal("stale rotation")
|
||||
}
|
||||
if reloads != 2 {
|
||||
t.Fatal(reloads)
|
||||
}
|
||||
s.now = time.Now
|
||||
apply("")
|
||||
got, _ := m.read("config")
|
||||
if !bytes.Equal(old, got) || reloads != 3 {
|
||||
t.Fatal("A not restored", reloads)
|
||||
}
|
||||
for _, policy := range []string{"unknown", "zz-alternative,zz-alternative", "zz-alternative,"} {
|
||||
if s.selectPolicy(policy) == nil {
|
||||
t.Fatal("invalid policy", policy)
|
||||
}
|
||||
}
|
||||
bad := *a.Ref
|
||||
bad.Policy = "zz-alternative"
|
||||
if s.validate(&bad, bad.Config) == nil {
|
||||
t.Fatal("policy forged render")
|
||||
}
|
||||
bad = managedConfigRef{Kind: "baseline", Policy: "zz-alternative"}
|
||||
if validManagedConfigRef(&bad, nil, "", managedHash([]byte(managedBaseline))) {
|
||||
t.Fatal("baseline policy")
|
||||
}
|
||||
for name, before := range proofs {
|
||||
after, e := os.ReadFile(filepath.Join(m.Dir, name))
|
||||
if e != nil || !bytes.Equal(before, after) {
|
||||
t.Fatal("immutable proof changed", name, e)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceSameBytesPolicyRequiresFreshProof(t *testing.T) {
|
||||
m, s, _, _ := nativeApplyProfilesFixture(t, true, true)
|
||||
policyA := s.registry.Strategies[0].ID
|
||||
if e := s.selectPolicy(policyA); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
oldRef := s.ref
|
||||
oldBytes := append([]byte(nil), s.rendered...)
|
||||
before := map[string][]byte{}
|
||||
for _, name := range []string{m.Name, managedJournalName, managedBackupName, planRegistryName, planMatrixName} {
|
||||
b, e := os.ReadFile(filepath.Join(m.Dir, name))
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
before[name] = b
|
||||
}
|
||||
reloads, checkpoints := 0, 0
|
||||
m.Reload = func() error { reloads++; return nil }
|
||||
m.fault = func(string) error { checkpoints++; return nil }
|
||||
assertUnchanged := func() {
|
||||
t.Helper()
|
||||
if reloads != 0 || checkpoints != 0 {
|
||||
t.Fatal("request reached staging/reload", checkpoints, reloads)
|
||||
}
|
||||
for name, want := range before {
|
||||
got, e := os.ReadFile(filepath.Join(m.Dir, name))
|
||||
if e != nil || !bytes.Equal(got, want) {
|
||||
t.Fatal("request mutated deployment/proof", name, e)
|
||||
}
|
||||
}
|
||||
j, e := m.load()
|
||||
if e != nil || j.State != "committed" || j.Ref == nil || *j.Ref != oldRef || j.PendingRef != nil {
|
||||
t.Fatal("committed policy changed", j, e)
|
||||
}
|
||||
}
|
||||
aged := time.Now().Add(25 * time.Hour)
|
||||
s.now = func() time.Time { return aged }
|
||||
if e := s.selectPolicy("zz-alternative"); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if s.ref.Policy == oldRef.Policy || s.ref.Config != oldRef.Config || !bytes.Equal(s.rendered, oldBytes) {
|
||||
t.Fatal("fixture must change policy but preserve rendered bytes", oldRef, s.ref)
|
||||
}
|
||||
if e := s.validate(&s.ref, s.ref.Config); e != nil {
|
||||
t.Fatal("alternative lacks valid historical proof", e)
|
||||
}
|
||||
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); !errors.Is(e, errResourcePlan) {
|
||||
t.Fatal("same-byte rotation bypassed freshness", e)
|
||||
}
|
||||
assertUnchanged()
|
||||
unknown := s.ref
|
||||
unknown.Policy = "unknown"
|
||||
if e := m.validateConfigRef(nil, &unknown, nil, unknown.Config, ""); e == nil {
|
||||
t.Fatal("unknown policy ref authorized")
|
||||
}
|
||||
assertUnchanged()
|
||||
if e := s.selectPolicy(policyA); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if s.fresh() == nil {
|
||||
t.Fatal("fixture proof must be expired")
|
||||
}
|
||||
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e != nil {
|
||||
t.Fatal("identical committed policy replay rejected after expiry", e)
|
||||
}
|
||||
assertUnchanged()
|
||||
}
|
||||
|
||||
func TestResourceRotationRecovery(t *testing.T) {
|
||||
for _, point := range []string{"staged", "backup", "prepared", "replaced", "reloaded", "healthy", "health-failure", "rollback-health"} {
|
||||
t.Run(point, func(t *testing.T) {
|
||||
m, s, _, _ := nativeApplyAlternativesFixture(t, true)
|
||||
if e := s.selectPolicy("zz-alternative"); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
old, _ := m.load()
|
||||
oldBytes, _ := m.read("config")
|
||||
if e := s.selectPolicy(""); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
m.fault = func(p string) error {
|
||||
if p == point {
|
||||
return errors.New("death")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if point == "health-failure" || point == "rollback-health" {
|
||||
m.Health = func() (string, error) { return "DEGRADED", nil }
|
||||
}
|
||||
if point == "rollback-health" {
|
||||
m.RollbackHealth = func() (string, error) { return "DEGRADED", nil }
|
||||
}
|
||||
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e == nil {
|
||||
t.Fatal("fault ignored")
|
||||
}
|
||||
j, _ := m.load()
|
||||
if *j.Ref != *old.Ref {
|
||||
t.Fatal("old policy lost", j)
|
||||
}
|
||||
if point == "rollback-health" && j.State != "prepared" {
|
||||
t.Fatal(j)
|
||||
}
|
||||
m.fault = nil
|
||||
m.RollbackHealth = func() (string, error) { return "OK", nil }
|
||||
s.now = func() time.Time { return time.Now().Add(25 * time.Hour) }
|
||||
if e := m.Reconcile(); e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
j, _ = m.load()
|
||||
got, _ := m.read("config")
|
||||
if j.State != "committed" || *j.Ref != *old.Ref || !bytes.Equal(got, oldBytes) {
|
||||
t.Fatal("pending promoted", j)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,468 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/binary"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"image"
|
||||
"image/gif"
|
||||
"image/jpeg"
|
||||
"io"
|
||||
"net"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
const resourceV2Example = `{"version":2,"services":[{"id":"media","resources":[{"id":"image","group":"images","kind":"image","url":"https://example.com/image?SECRET","method":"GET","tls":"both","timeout_ms":700,"expect":{"statuses":[200],"content_type":"image/png","min_bytes":1,"max_bytes":1024},"headers_env":{"Authorization":"CHECK_TOKEN"}},{"id":"prefix","group":"media","kind":"mp4_range","url":"https://example.com/prefix?SECRET","method":"GET","tls":"both","timeout_ms":700,"expect":{"statuses":[206],"content_type":"video/mp4","min_bytes":16,"max_bytes":65536}}]}]}`
|
||||
|
||||
func resourceV2Source(t *testing.T) resourceMonitorSource {
|
||||
t.Helper()
|
||||
cfg, err := parseResourceChecks([]byte(resourceV2Example), resourceCheckLookup)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return resourceMonitorSource{config: cfg, server: "127.0.0.1:5353", proxy: "socks5h://127.0.0.1:1080", fingerprint: strings.Repeat("a", 64)}
|
||||
}
|
||||
|
||||
func TestResourceV2Schema(t *testing.T) {
|
||||
for _, tc := range []struct{ from, to string }{
|
||||
{`"version":2`, `"version":3`}, {`"version":2`, `"version":1`},
|
||||
{`"kind":"image"`, `"kind":"unknown"`}, {`"method":"GET"`, `"method":"HEAD"`},
|
||||
{`"kind":"image"`, `"kind":""`},
|
||||
{`"statuses":[200]`, `"statuses":[200,201]`}, {`"statuses":[206]`, `"statuses":[200]`},
|
||||
{`"image/png"`, `"image/gif"`}, {`"image/png"`, `"video/mp4"`},
|
||||
{`"video/mp4"`, `"image/png"`}, {`"min_bytes":16`, `"min_bytes":15`},
|
||||
{`"max_bytes":65536`, `"max_bytes":65537`}, {`"min_bytes":1,`, ``},
|
||||
{`"content_type":"image/png",`, ``}, {`"content_type":"image/png"`, `"content_type":null`},
|
||||
{`"Authorization"`, `"rAnGe"`}, {`"Authorization"`, `"Accept-Encoding"`},
|
||||
{`"version":2`, `"version":2,"version":2`}, {`"kind":"image"`, `"module_version":1,"kind":"image"`},
|
||||
} {
|
||||
if _, err := parseResourceChecks([]byte(strings.Replace(resourceV2Example, tc.from, tc.to, 1)), resourceCheckLookup); err == nil {
|
||||
t.Fatalf("accepted %s -> %s", tc.from, tc.to)
|
||||
}
|
||||
}
|
||||
legacyMissing := strings.Replace(resourceCheckExample, `"kind":"http",`, "", 1)
|
||||
if _, err := parseResourceChecks([]byte(legacyMissing), resourceCheckLookup); err == nil {
|
||||
t.Fatal("v1 defaulted kind")
|
||||
}
|
||||
for _, kind := range []string{"", `"kind":"http",`} {
|
||||
raw := strings.ReplaceAll(resourceCheckExample, `"version":1`, `"version":2`)
|
||||
raw = strings.ReplaceAll(raw, `"kind":"http",`, kind)
|
||||
raw = strings.Replace(raw, `"statuses":[200],"min_bytes":0`, `"statuses":[200],"content_type":"text/plain","min_bytes":0`, 1)
|
||||
cfg, err := parseResourceChecks([]byte(raw), resourceCheckLookup)
|
||||
if err != nil || cfg.Services[0].Resources[0].Kind != "http" {
|
||||
t.Fatal("v2 default/explicit http", err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Real HTTP/1.1 framing and curl JSON, but no network and no child secrets.
|
||||
func resourceV2Checks(t *testing.T, mode string, calls *int) resourceCheckDeps {
|
||||
t.Helper()
|
||||
return resourceCheckDeps{qualification: true, guard: func() error { return nil }, resolve: func(context.Context, string, string) ([]net.IP, error) { return []net.IP{net.ParseIP("8.8.8.8")}, nil }, runner: func(ctx context.Context, args []string, stdin io.Reader, out, diag io.Writer) error {
|
||||
(*calls)++
|
||||
config, err := io.ReadAll(stdin)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
argv := strings.Join(args, " ")
|
||||
if strings.Contains(argv, "SECRET") || strings.Contains(argv, "private-token") || !bytes.Contains(config, []byte("connect-to =")) {
|
||||
t.Fatal("secret/pin boundary")
|
||||
}
|
||||
if !strings.Contains(argv, "--max-time 0.700") {
|
||||
t.Fatal("configured curl timeout", argv)
|
||||
}
|
||||
deadline, ok := ctx.Deadline()
|
||||
if !ok || time.Until(deadline) > 700*time.Millisecond {
|
||||
t.Fatal("configured context timeout")
|
||||
}
|
||||
raw, err := strconv.Unquote(strings.TrimPrefix(strings.Split(string(config), "\n")[0], "url = "))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
body, mimeType, status, extra := resourceFixture(), "image/png", 200, ""
|
||||
if strings.Contains(raw, "/prefix") {
|
||||
if !strings.Contains(argv, "--range 0-65535") {
|
||||
t.Fatal("fixed core range missing")
|
||||
}
|
||||
size := 24
|
||||
if mode == "min-range" {
|
||||
size = 16
|
||||
}
|
||||
if mode == "max-range" {
|
||||
size = 65536
|
||||
}
|
||||
body = make([]byte, size)
|
||||
binary.BigEndian.PutUint32(body, 16)
|
||||
copy(body[4:], "ftypisom")
|
||||
mimeType, status = "video/mp4", 206
|
||||
extra = fmt.Sprintf("Content-Range: bytes 0-%d/100000\r\n", size-1)
|
||||
if mode == "bad-range" {
|
||||
extra = "Content-Range: bytes 0-23/9223372036854775808\r\n"
|
||||
}
|
||||
if mode == "bad-body" {
|
||||
copy(body[4:], "xxxx")
|
||||
}
|
||||
} else {
|
||||
if !bytes.Contains(config, []byte("Authorization: Bearer private-token")) {
|
||||
t.Fatal("secret header not on stdin")
|
||||
}
|
||||
if mode == "jpeg" {
|
||||
var b bytes.Buffer
|
||||
if err := jpeg.Encode(&b, image.NewRGBA(image.Rect(0, 0, 1, 1)), nil); err != nil {
|
||||
return err
|
||||
}
|
||||
body, mimeType = b.Bytes(), "image/jpeg"
|
||||
}
|
||||
if mode == "gif" {
|
||||
var b bytes.Buffer
|
||||
if err := gif.Encode(&b, image.NewRGBA(image.Rect(0, 0, 1, 1)), nil); err != nil {
|
||||
return err
|
||||
}
|
||||
body = b.Bytes()
|
||||
}
|
||||
if mode == "bad-body" {
|
||||
body = []byte("not an image")
|
||||
}
|
||||
if mode == "truncated" {
|
||||
body = body[:len(body)/2]
|
||||
}
|
||||
}
|
||||
if mode == "status" {
|
||||
status = 500
|
||||
}
|
||||
if mode == "mime" {
|
||||
mimeType = "text/html"
|
||||
}
|
||||
if mode == "malformed-mime" {
|
||||
mimeType += "; broken"
|
||||
}
|
||||
fmt.Fprintf(out, "HTTP/1.1 %d result\r\nContent-Type: %s\r\nContent-Length: %d\r\n%s\r\n", status, mimeType, len(body), extra)
|
||||
out.Write(body)
|
||||
metrics, _ := json.Marshal(map[string]interface{}{"url_effective": raw, "ssl_verify_result": 0, "num_redirects": 0, "http_code": status, "time_total": 0.1, "speed_download": 10000, "size_download": len(body)})
|
||||
if mode == "metrics" {
|
||||
metrics = []byte("invalid")
|
||||
}
|
||||
fmt.Fprint(out, resourceTrailer, string(metrics))
|
||||
version := "1.2"
|
||||
if strings.Contains(argv, "--tlsv1.3") {
|
||||
version = "1.3"
|
||||
}
|
||||
fmt.Fprintln(diag, "SSL connection using TLSv"+version)
|
||||
if mode == "mixed" {
|
||||
return errors.Join(errors.New("execution failed"), errors.New("invalid media range or container"))
|
||||
}
|
||||
return nil
|
||||
}}
|
||||
}
|
||||
|
||||
func TestResourceV2SharedPublicPaths(t *testing.T) {
|
||||
for _, mode := range []string{"pass", "jpeg", "min-range", "max-range", "bad-body", "bad-range", "status", "mime", "malformed-mime", "gif", "truncated", "metrics", "mixed"} {
|
||||
t.Run(mode, func(t *testing.T) {
|
||||
source := resourceV2Source(t)
|
||||
if mode == "jpeg" {
|
||||
source.config.Services[0].Resources[0].Expect.ContentType = "image/jpeg"
|
||||
}
|
||||
calls := 0
|
||||
checks := resourceV2Checks(t, mode, &calls)
|
||||
negative := mode == "bad-body" || mode == "bad-range" || mode == "status" || mode == "mime" || mode == "malformed-mime" || mode == "gif" || mode == "truncated"
|
||||
unknown := mode == "metrics" || mode == "mixed"
|
||||
for _, spec := range source.config.Services[0].Resources {
|
||||
for _, protocol := range []string{"tls12", "tls13"} {
|
||||
result, err := runResourceProtocol(context.Background(), "media", spec, protocol, source.server, source.proxy, checks)
|
||||
if unknown {
|
||||
if err == nil {
|
||||
t.Fatal("infrastructure became negative")
|
||||
}
|
||||
continue
|
||||
}
|
||||
want := "pass"
|
||||
if negative && (mode != "bad-range" || spec.Kind == "mp4_range") && (mode != "gif" && mode != "truncated" || spec.Kind == "image") {
|
||||
want = "expectation_failed"
|
||||
}
|
||||
if err != nil || result.Outcome != want || result.Bytes == 0 {
|
||||
t.Fatalf("%s %s: %+v %v", spec.Kind, protocol, result, err)
|
||||
}
|
||||
if mode == "malformed-mime" {
|
||||
status, size := 200, len(resourceFixture())
|
||||
if spec.Kind == "mp4_range" {
|
||||
status, size = 206, 24
|
||||
}
|
||||
if result.Status != status || result.Bytes != size || result.TimeMS != 100 {
|
||||
t.Fatalf("lost malformed-MIME facts: %+v", result)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if unknown {
|
||||
return
|
||||
}
|
||||
for _, spec := range source.config.Services[0].Resources {
|
||||
_, err := probeResourceMonitor(context.Background(), "media", spec, source, func() error { return nil }, checks)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
snapshot := &resourceApplySnapshot{source: source, guard: func() error { return nil }}
|
||||
status, err := resourceApplyHealth(context.Background(), snapshot, checks)
|
||||
if (!negative && (err != nil || status != "OK")) || (negative && err == nil) {
|
||||
t.Fatal("apply", status, err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceV2QualifierCompletion(t *testing.T) {
|
||||
for _, mode := range []string{"pass", "bad-body", "bad-range", "malformed-mime", "metrics"} {
|
||||
t.Run(mode, func(t *testing.T) {
|
||||
q, store, registry, _, _ := qualifierFixture(t)
|
||||
source := resourceV2Source(t)
|
||||
q.source = func() (resourceMonitorSource, error) { return source, nil }
|
||||
calls := 0
|
||||
q.checks = resourceV2Checks(t, mode, &calls)
|
||||
proof, err := q.complete(context.Background(), store, registry)
|
||||
if mode == "metrics" {
|
||||
if err == nil {
|
||||
t.Fatal("sealed unknown infrastructure")
|
||||
}
|
||||
return
|
||||
}
|
||||
if err != nil || calls != 12 {
|
||||
t.Fatalf("completion %d %v", calls, err)
|
||||
}
|
||||
for _, cell := range proof.matrix.Runs[0].Checks {
|
||||
want := "pass"
|
||||
if mode == "bad-body" || mode == "malformed-mime" || mode == "bad-range" && cell.Resource == "prefix" {
|
||||
want = "expectation_failed"
|
||||
}
|
||||
if cell.Attempts != 3 || cell.Outcome != want {
|
||||
t.Fatalf("incomplete/wrong cell %+v", cell)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceCoreMalformedMIMEClassification(t *testing.T) {
|
||||
for _, kind := range []resourceResponseKind{resourceResponseGeneric, resourceResponseImage, resourceResponseMP4} {
|
||||
for _, configured := range []bool{false, true} {
|
||||
t.Run(fmt.Sprintf("kind-%d/configured-%t", kind, configured), func(t *testing.T) {
|
||||
raw := "https://i.ytimg.com/private?SECRET"
|
||||
options := resourceTransferOptions{method: "GET", response: kind, mediaType: "video/mp4", limit: 1024, timeout: time.Second}
|
||||
if configured {
|
||||
options.expect = &resourceCheckExpect{Statuses: []int{200}, MaxBytes: 1024}
|
||||
}
|
||||
runner := func(_ context.Context, _ []string, _ io.Reader, out, diag io.Writer) error {
|
||||
var wire bytes.Buffer
|
||||
mode := ""
|
||||
if kind == resourceResponseMP4 {
|
||||
mode = "media"
|
||||
}
|
||||
resourceWire(raw, mode, &wire, diag)
|
||||
data := strings.Replace(wire.String(), "Content-Type: image/png\r\n", "Content-Type: image/png; broken\r\n", 1)
|
||||
data = strings.Replace(data, "Content-Type: video/mp4\r\n", "Content-Type: video/mp4; broken\r\n", 1)
|
||||
_, err := io.WriteString(out, data)
|
||||
return err
|
||||
}
|
||||
transfer, _, err := collectResourceTransferOptions(context.Background(), "socks5h://127.0.0.1:1080", raw, "i.ytimg.com", "tls12", runner, options)
|
||||
if configured && kind != resourceResponseGeneric {
|
||||
status, size := 200, len(resourceFixture())
|
||||
if kind == resourceResponseMP4 {
|
||||
status, size = 206, 24
|
||||
}
|
||||
if err != errResourceExpectation || transfer.Status != status || len(transfer.Body) != size || transfer.LatencyMS != 100 || transfer.ContentType != "" {
|
||||
t.Fatalf("typed facts/classification: %+v %v", transfer, err)
|
||||
}
|
||||
} else {
|
||||
var fatal *managedRecoveryFatal
|
||||
if !errors.As(err, &fatal) || err == errResourceExpectation || resourceRejectionOnly(err) || transfer.Status != 0 || len(transfer.Body) != 0 {
|
||||
t.Fatalf("generic/private malformed MIME relaxed: %+v %v", transfer, err)
|
||||
}
|
||||
var resource *resourceError
|
||||
if !errors.As(err, &resource) || resource.reason != reasonMIME || fatal.Err.Error() != "invalid content type" {
|
||||
t.Fatalf("legacy MIME failure changed: %v", err)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceV2FingerprintCompatibility(t *testing.T) {
|
||||
key := []byte("fingerprint-test-key")
|
||||
cfg, err := parseResourceChecks([]byte(resourceCheckExample), resourceCheckLookup)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
source := resourceMonitorSource{config: cfg, server: "dns", proxy: "proxy"}
|
||||
spec := cfg.Services[0].Resources[0]
|
||||
b, _ := json.Marshal(spec)
|
||||
b, _ = json.Marshal([]string{string(b), spec.privateConfig, source.server, source.proxy})
|
||||
old := resourceMAC(key, b)
|
||||
if resourceRowFingerprint(key, source, spec) != old {
|
||||
t.Fatal("historic v1 serialization changed")
|
||||
}
|
||||
source.config.Version = 2
|
||||
v2 := resourceRowFingerprint(key, source, spec)
|
||||
if v2 == old {
|
||||
t.Fatal("v2 reused v1 proof namespace")
|
||||
}
|
||||
for _, mutate := range []func(*resourceCheckSpec){func(r *resourceCheckSpec) { r.Kind = "image" }, func(r *resourceCheckSpec) { r.Expect.MinBytes++ }, func(r *resourceCheckSpec) { r.Expect.MaxBytes++ }, func(r *resourceCheckSpec) { r.Expect.Statuses = []int{201} }} {
|
||||
r := spec
|
||||
mutate(&r)
|
||||
if resourceRowFingerprint(key, source, r) == v2 {
|
||||
t.Fatal("semantic change reused row proof")
|
||||
}
|
||||
}
|
||||
prior := []resourceMonitorRow{{Service: cfg.Services[0].ID, Resource: spec.ID, Fingerprint: old, Health: "OK", Outcome: "pass"}}
|
||||
rows := reconcileResourceRows(key, source, prior)
|
||||
if rows[0].Health != "UNKNOWN" {
|
||||
t.Fatal("v1 health survived v2 semantics")
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceV2ConfiguredBounds(t *testing.T) {
|
||||
for _, kind := range []string{"image", "mp4_range"} {
|
||||
for _, mode := range []string{"exact", "minimum", "overflow"} {
|
||||
t.Run(kind+"/"+mode, func(t *testing.T) {
|
||||
source := resourceV2Source(t)
|
||||
spec := source.config.Services[0].Resources[0]
|
||||
size := len(resourceFixture())
|
||||
if kind == "mp4_range" {
|
||||
spec = source.config.Services[0].Resources[1]
|
||||
size = 24
|
||||
}
|
||||
spec.Expect.MinBytes, spec.Expect.MaxBytes = size, size
|
||||
if mode == "minimum" {
|
||||
spec.Expect.MinBytes++
|
||||
spec.Expect.MaxBytes++
|
||||
}
|
||||
if mode == "overflow" {
|
||||
spec.Expect.MinBytes--
|
||||
spec.Expect.MaxBytes--
|
||||
}
|
||||
calls := 0
|
||||
checks := resourceV2Checks(t, "pass", &calls)
|
||||
runner := checks.runner
|
||||
checks.runner = func(ctx context.Context, args []string, stdin io.Reader, out, diag io.Writer) error {
|
||||
for i, a := range args {
|
||||
if a == "--max-filesize" && args[i+1] != strconv.Itoa(spec.Expect.MaxBytes) {
|
||||
t.Fatal("ignored configured limit", args)
|
||||
}
|
||||
}
|
||||
return runner(ctx, args, stdin, out, diag)
|
||||
}
|
||||
result, err := runResourceProtocol(context.Background(), "media", spec, "tls12", source.server, source.proxy, checks)
|
||||
switch mode {
|
||||
case "exact":
|
||||
if err != nil || result.Outcome != "pass" {
|
||||
t.Fatal(result, err)
|
||||
}
|
||||
case "minimum":
|
||||
if err != nil || result.Outcome != "expectation_failed" {
|
||||
t.Fatal(result, err)
|
||||
}
|
||||
case "overflow":
|
||||
if err == nil {
|
||||
t.Fatal("body-bound violation became completed expectation", result)
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceV2CancelCannotSeal(t *testing.T) {
|
||||
q, store, registry, _, _ := qualifierFixture(t)
|
||||
source := resourceV2Source(t)
|
||||
q.source = func() (resourceMonitorSource, error) { return source, nil }
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
joined := false
|
||||
calls := 0
|
||||
q.checks = resourceV2Checks(t, "pass", &calls)
|
||||
q.checks.runner = func(child context.Context, _ []string, _ io.Reader, _ io.Writer, _ io.Writer) error {
|
||||
cancel()
|
||||
<-child.Done()
|
||||
joined = true
|
||||
return child.Err()
|
||||
}
|
||||
proof, err := q.complete(ctx, store, registry)
|
||||
if err == nil || !joined || store.saveCompleted(proof) == nil {
|
||||
t.Fatal("cancelled batch acquired proof", err, joined)
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceV2OldMatrixCannotPromote(t *testing.T) {
|
||||
source, key, registry, matrix, now := planFixture(t)
|
||||
if len(deriveResourcePlan(source, key, registry, matrix, now).Assignments) == 0 {
|
||||
t.Fatal("invalid baseline fixture")
|
||||
}
|
||||
source.config.Version = 2
|
||||
for si := range source.config.Services {
|
||||
for ri := range source.config.Services[si].Resources {
|
||||
source.config.Services[si].Resources[ri].Kind = "image"
|
||||
}
|
||||
}
|
||||
// Even retaining the old source MAC cannot bypass per-cell semantic binding.
|
||||
if len(deriveResourcePlan(source, key, registry, matrix, now).Assignments) != 0 {
|
||||
t.Fatal("old HTTP matrix promoted typed images")
|
||||
}
|
||||
}
|
||||
|
||||
func TestResourceV2CLIAndSourceBinding(t *testing.T) {
|
||||
for _, mode := range []string{"pass", "bad-body"} {
|
||||
t.Run(mode, func(t *testing.T) {
|
||||
calls := 0
|
||||
checks := resourceV2Checks(t, mode, &calls)
|
||||
checks.lookup = resourceCheckLookup
|
||||
var out, diag bytes.Buffer
|
||||
code := resourceCheckCLI(context.Background(), resourceCheckFile(t, resourceV2Example), &out, &diag, checks)
|
||||
want := 0
|
||||
if mode != "pass" {
|
||||
want = 1
|
||||
}
|
||||
if code != want || calls != 4 || bytes.Contains(out.Bytes(), []byte("SECRET")) || bytes.Contains(out.Bytes(), []byte("private-token")) || diag.Len() != 0 {
|
||||
t.Fatal(code, calls, out.String(), diag.String())
|
||||
}
|
||||
})
|
||||
}
|
||||
calls := 0
|
||||
checks := resourceV2Checks(t, "pass", &calls)
|
||||
checks.lookup = resourceCheckLookup
|
||||
checks.resolve = func(context.Context, string, string) ([]net.IP, error) {
|
||||
t.Fatal("invalid typed HEAD reached DNS")
|
||||
return nil, nil
|
||||
}
|
||||
var out, diag bytes.Buffer
|
||||
if resourceCheckCLI(context.Background(), resourceCheckFile(t, strings.Replace(resourceV2Example, `"method":"GET"`, `"method":"HEAD"`, 1)), &out, &diag, checks) != 2 || calls != 0 {
|
||||
t.Fatal("invalid typed config executed")
|
||||
}
|
||||
legacy, key := resourceMonitorFixture(t)
|
||||
b, _ := json.Marshal(legacy.config)
|
||||
parts := []string{string(b), legacy.server, legacy.proxy}
|
||||
for _, s := range legacy.config.Services {
|
||||
for _, r := range s.Resources {
|
||||
parts = append(parts, r.privateConfig)
|
||||
}
|
||||
}
|
||||
b, _ = json.Marshal(parts)
|
||||
if legacy.fingerprint != resourceMAC(key, b) {
|
||||
t.Fatal("v1 source serialization changed")
|
||||
}
|
||||
raw := strings.Replace(resourceCheckExample, `"version":1`, `"version":2`, 1)
|
||||
raw = strings.Replace(raw, `"statuses":[200],"min_bytes":0`, `"statuses":[200],"content_type":"text/plain","min_bytes":0`, 1)
|
||||
v2, err := readResourceMonitorSource(resourceCheckFile(t, raw), resourceCheckLookup, key)
|
||||
if err != nil || v2.fingerprint == legacy.fingerprint {
|
||||
t.Fatal("version unbound source", err)
|
||||
}
|
||||
implicit, err := readResourceMonitorSource(resourceCheckFile(t, strings.ReplaceAll(raw, `"kind":"http",`, "")), resourceCheckLookup, key)
|
||||
if err != nil || implicit.fingerprint != v2.fingerprint {
|
||||
t.Fatal("default kind not canonical", err)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"sync"
|
||||
|
||||
"zapret-autotune/internal/adapter"
|
||||
"zapret-autotune/internal/adapter/youtube"
|
||||
)
|
||||
|
||||
// Trusted compiled-in dispatch, deliberately not a public config/plugin API.
|
||||
// An empty registry is a supported fail-closed seam; no HTTP fallback exists.
|
||||
type serviceAdapterKey struct {
|
||||
kind string
|
||||
version int
|
||||
}
|
||||
type serviceAdapterRegistry map[serviceAdapterKey]adapter.Module
|
||||
|
||||
var errServiceAdapterUnavailable = errors.New("required service adapter unavailable")
|
||||
|
||||
// The registry is immutable in production. The lock permits race-free test
|
||||
// replacement while request callbacks/owned lifecycle children are running.
|
||||
var serviceAdapters = struct {
|
||||
sync.RWMutex
|
||||
modules serviceAdapterRegistry
|
||||
}{modules: serviceAdapterRegistry{{youtube.Kind, youtube.Version}: youtube.Module{}}}
|
||||
|
||||
func requiredYoutubeAdapter() (adapter.Module, error) {
|
||||
serviceAdapters.RLock()
|
||||
m, err := serviceAdapters.modules.lookup(youtube.Kind, youtube.Version)
|
||||
serviceAdapters.RUnlock()
|
||||
if err != nil || m.Binding() != (youtube.Module{}).Binding() {
|
||||
return nil, &managedRecoveryFatal{Err: errServiceAdapterUnavailable}
|
||||
}
|
||||
return m, nil
|
||||
}
|
||||
|
||||
func (r serviceAdapterRegistry) lookup(kind string, version int) (adapter.Module, error) {
|
||||
m, ok := r[serviceAdapterKey{kind, version}]
|
||||
if !ok || m == nil {
|
||||
return nil, errServiceAdapterUnavailable
|
||||
}
|
||||
b := m.Binding()
|
||||
if b.Kind != kind || b.Version != version {
|
||||
return nil, errServiceAdapterUnavailable
|
||||
}
|
||||
return m, nil
|
||||
}
|
||||
|
||||
func lookupServiceAdapter(kind string, version int) (adapter.Module, error) {
|
||||
serviceAdapters.RLock()
|
||||
defer serviceAdapters.RUnlock()
|
||||
return serviceAdapters.modules.lookup(kind, version)
|
||||
}
|
||||
@@ -0,0 +1,284 @@
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"zapret-autotune/internal/adapter"
|
||||
"zapret-autotune/internal/adapter/youtube"
|
||||
)
|
||||
|
||||
func replaceServiceAdapters(t *testing.T, modules serviceAdapterRegistry) {
|
||||
t.Helper()
|
||||
serviceAdapters.Lock()
|
||||
previous := serviceAdapters.modules
|
||||
serviceAdapters.modules = modules
|
||||
serviceAdapters.Unlock()
|
||||
t.Cleanup(func() {
|
||||
serviceAdapters.Lock()
|
||||
serviceAdapters.modules = previous
|
||||
serviceAdapters.Unlock()
|
||||
})
|
||||
}
|
||||
|
||||
func TestServiceAdapterDispatchFailClosed(t *testing.T) {
|
||||
for _, key := range []serviceAdapterKey{{"unknown", 1}, {youtube.Kind, 0}, {youtube.Kind, 2}} {
|
||||
if _, err := lookupServiceAdapter(key.kind, key.version); err == nil {
|
||||
t.Fatal("unsupported adapter accepted")
|
||||
}
|
||||
}
|
||||
if _, err := (serviceAdapterRegistry{}).lookup(youtube.Kind, youtube.Version); err == nil {
|
||||
t.Fatal("absent adapter accepted")
|
||||
}
|
||||
replaceServiceAdapters(t, serviceAdapterRegistry{})
|
||||
if _, _, err := playerMediaResource([]byte(`{"playabilityStatus":{"status":"OK"}}`)); err == nil {
|
||||
t.Fatal("discovery bypassed registry")
|
||||
}
|
||||
r, c, resources, events := googlevideoRuntimeFixture(t)
|
||||
if _, err := r.verifyGooglevideo(context.Background(), c, resources); err == nil || len(*events) != 0 {
|
||||
t.Fatal("qualification bypassed registry")
|
||||
}
|
||||
}
|
||||
|
||||
type mismatchedServiceAdapter struct{ adapter.Module }
|
||||
|
||||
func (m mismatchedServiceAdapter) Binding() adapter.Binding {
|
||||
b := m.Module.Binding()
|
||||
b.Version++
|
||||
return b
|
||||
}
|
||||
|
||||
func TestServiceAdapterIdentityBeforeExecution(t *testing.T) {
|
||||
replaceServiceAdapters(t, serviceAdapterRegistry{{youtube.Kind, youtube.Version}: mismatchedServiceAdapter{youtube.Module{}}})
|
||||
r, c, resources, events := googlevideoRuntimeFixture(t)
|
||||
if _, err := r.verifyGooglevideo(context.Background(), c, resources); err == nil || len(*events) != 0 {
|
||||
t.Fatal("identity mismatch reached worker")
|
||||
}
|
||||
}
|
||||
|
||||
func TestServiceAdapterStoredProofRefusals(t *testing.T) {
|
||||
r, c, resources, _ := googlevideoRuntimeFixture(t)
|
||||
p, err := r.verifyGooglevideo(context.Background(), c, resources)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
db := testStore(t)
|
||||
tx, err := db.Begin()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer tx.Rollback()
|
||||
id, err := insertStrategyCandidate(tx, c.Group, c.Endpoints, c.Args, nil, c.Mode)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := saveAdaptiveGooglevideoEvidence(tx, id, p); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := tx.Commit(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
front := managedCandidateFixture(t, db, "frontend", "www.youtube.com", "--lua-desync=multisplit", true)
|
||||
for _, mode := range []string{"kind", "version", "digest", "missing", "unknown-proof", "downgrade", "attestation"} {
|
||||
t.Run(mode, func(t *testing.T) {
|
||||
raw, _ := json.Marshal(p)
|
||||
var proof GooglevideoQualificationProof
|
||||
if err := json.Unmarshal(raw, &proof); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
switch mode {
|
||||
case "kind":
|
||||
proof.Adapter.Kind = "unknown"
|
||||
case "version":
|
||||
proof.Adapter.Version++
|
||||
case "digest":
|
||||
proof.Adapter.PolicySHA256 = "changed"
|
||||
case "missing":
|
||||
proof.Adapter = nil
|
||||
case "unknown-proof":
|
||||
proof.Version = 3
|
||||
case "downgrade":
|
||||
proof.Version = 1
|
||||
case "attestation":
|
||||
proof.Attestation = nil
|
||||
}
|
||||
if proof.validateStored(c) == nil {
|
||||
t.Fatal("malformed proof reconstructed")
|
||||
}
|
||||
raw, _ = json.Marshal(proof)
|
||||
if _, err := db.Exec("UPDATE googlevideo_scope_evidence SET proof=? WHERE candidate_id=?", string(raw), id); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
m, reloads := managedFixture(t)
|
||||
if err := m.Apply(db, []int64{front, id}, GooglevideoScopeSuffix); err == nil || *reloads != 0 {
|
||||
t.Fatal("invalid proof reached apply", err, *reloads)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestServiceAdapterProofBoundary(t *testing.T) {
|
||||
r, c, resources, _ := googlevideoRuntimeFixture(t)
|
||||
p, err := r.verifyGooglevideo(context.Background(), c, resources)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if p.Version != 2 || p.Adapter == nil || *p.Adapter != (youtube.Module{}).Binding() {
|
||||
t.Fatal("missing module identity")
|
||||
}
|
||||
seal := googlevideoProofSeal(p)
|
||||
for _, mutate := range []func(){
|
||||
func() { p.Adapter.Kind = "unknown" },
|
||||
func() { p.Adapter.Version++ },
|
||||
func() { p.Adapter.PolicySHA256 = "changed" },
|
||||
} {
|
||||
original := *p.Adapter
|
||||
mutate()
|
||||
if p.validateStored(c) == nil || googlevideoProofSeal(p) == seal {
|
||||
t.Fatal("module boundary not bound to evidence")
|
||||
}
|
||||
*p.Adapter = original
|
||||
}
|
||||
// Reads retain the original v1 contract, including fresh apply eligibility.
|
||||
p.Version, p.Adapter = 1, nil
|
||||
raw, err := json.Marshal(p)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
legacy, err := decodeGooglevideoProof(string(raw))
|
||||
if err != nil || legacy.validateStored(c) != nil {
|
||||
t.Fatal("legacy reconstruction lost", err)
|
||||
}
|
||||
if legacy.validate(c) != nil {
|
||||
t.Fatal("fresh trusted legacy adaptive proof lost apply eligibility")
|
||||
}
|
||||
legacy.Attestation.CheckedAt = time.Now().Add(-25 * time.Hour)
|
||||
if legacy.validate(c) == nil {
|
||||
t.Fatal("expired legacy adaptive proof authorized new apply")
|
||||
}
|
||||
replaceServiceAdapters(t, serviceAdapterRegistry{})
|
||||
if legacy.validateStored(c) != nil {
|
||||
t.Fatal("historical reconstruction depends on current execution registry")
|
||||
}
|
||||
}
|
||||
|
||||
type changedPolicyServiceAdapter struct{ adapter.Module }
|
||||
|
||||
func (m changedPolicyServiceAdapter) Binding() adapter.Binding {
|
||||
b := m.Module.Binding()
|
||||
b.PolicySHA256 = "changed"
|
||||
return b
|
||||
}
|
||||
|
||||
func TestServiceAdapterDiscoveryPreflight(t *testing.T) {
|
||||
for _, mode := range []string{"missing", "version", "digest"} {
|
||||
t.Run(mode, func(t *testing.T) {
|
||||
modules := serviceAdapterRegistry{}
|
||||
if mode == "version" {
|
||||
modules[serviceAdapterKey{youtube.Kind, youtube.Version}] = mismatchedServiceAdapter{youtube.Module{}}
|
||||
}
|
||||
if mode == "digest" {
|
||||
modules[serviceAdapterKey{youtube.Kind, youtube.Version}] = changedPolicyServiceAdapter{youtube.Module{}}
|
||||
}
|
||||
calls := 0
|
||||
player := func(context.Context, []string, io.Reader, io.Writer, io.Writer) error { calls++; return nil }
|
||||
request := func(context.Context) ([]byte, error) { calls++; return nil, nil }
|
||||
target, _, _ := managedCLIFixture(t)
|
||||
target.player = player
|
||||
target.verifySOCKS = func(int) error { calls++; return nil }
|
||||
r, _, _ := recoveryFixture(t, false)
|
||||
bound, err := managedRecoveryDependencies(target, r.db, r.scan, r.check, r.run)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
calls = 0 // Construction guard is allowed before removing the module.
|
||||
replaceServiceAdapters(t, modules)
|
||||
cfg := healthConfig{VideoID: "dQw4w9WgXcQ", SOCKSProxy: "socks5://127.0.0.1:1080"}
|
||||
assertBlocked := func(err error) {
|
||||
t.Helper()
|
||||
var fatal *managedRecoveryFatal
|
||||
if !errors.Is(err, errServiceAdapterUnavailable) || !errors.As(err, &fatal) || errors.Is(err, errNoQualifiedStrategy) || errors.Is(err, errGooglevideoInsufficientSamples) || managedResourceTransferNonpass(err, true, true) {
|
||||
t.Fatal("adapter configuration became a retryable absence", err)
|
||||
}
|
||||
if calls != 0 {
|
||||
t.Fatal("preflight made network calls", calls)
|
||||
}
|
||||
}
|
||||
_, err = requestPlayer(cfg.VideoID, cfg.SOCKSProxy, player)
|
||||
assertBlocked(err)
|
||||
if managedResourceTransferNonpass(errors.Join(err, context.DeadlineExceeded), true, true) || managedRecoveryCooldown(errors.Join(err, errNoQualifiedStrategy), time.Minute) != time.Minute {
|
||||
t.Fatal("mixed fatal adapter failure became transfer-negative/2h absence")
|
||||
}
|
||||
_, err = bound.discover(context.Background())
|
||||
assertBlocked(err)
|
||||
_, err = bound.health(context.Background())
|
||||
assertBlocked(err)
|
||||
_, err = managedGooglevideoSamples([]string{cfg.VideoID}, func(ctx context.Context, _ string) ([]byte, error) { return request(ctx) })(context.Background())
|
||||
assertBlocked(err)
|
||||
inventory := managedServiceResources(cfg, request, player)
|
||||
_, err = inventory(context.Background())
|
||||
assertBlocked(err)
|
||||
_, err = managedServiceHealth(cfg.SOCKSProxy, func(context.Context) (string, error) { calls++; return "OK", nil }, inventory, func(context.Context, []string, io.Reader, io.Writer, io.Writer) error { calls++; return nil })(context.Background())
|
||||
assertBlocked(err)
|
||||
if code := discoverYoutube([]string{"--video-id", cfg.VideoID, "--proxy", cfg.SOCKSProxy}, io.Discard, io.Discard, player); code != 1 || calls != 0 {
|
||||
t.Fatal("legacy CLI bypassed preflight", code, calls)
|
||||
}
|
||||
_, err = managedRecoveryDependencies(target, r.db, r.scan, r.check, r.run)
|
||||
assertBlocked(err)
|
||||
assertBlocked(managedExperimentalSuffix(nil, managedDeps{}))
|
||||
assertBlocked(managedActivate(nil, managedDeps{}, nil, nil))
|
||||
for _, saved := range []bool{false, true} {
|
||||
code := googlevideoProposal([]string{"--video-id", cfg.VideoID, "--proxy", cfg.SOCKSProxy}, io.Discard, io.Discard, player, func(context.Context, string, string, io.Writer, io.Writer) (string, error) { calls++; return "", nil }, func(context.Context, string, string, string, string, io.Writer, io.Writer) error { calls++; return nil }, saved)
|
||||
if code != 1 || calls != 0 {
|
||||
t.Fatal("proposal pipeline bypassed preflight", code, calls)
|
||||
}
|
||||
}
|
||||
runtime, c, resources, events := googlevideoRuntimeFixture(t)
|
||||
_, err = runtime.verifyGooglevideo(context.Background(), c, resources)
|
||||
assertBlocked(err)
|
||||
if len(*events) != 0 {
|
||||
t.Fatal("preflight launched worker")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestServiceAdapterLegacyNewWriteRefused(t *testing.T) {
|
||||
r, c, resources, _ := googlevideoRuntimeFixture(t)
|
||||
p, err := r.verifyGooglevideo(context.Background(), c, resources)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
p.Version, p.Adapter = 1, nil
|
||||
if p.validate(c) != nil {
|
||||
t.Fatal("trusted legacy read invalid")
|
||||
}
|
||||
// Even an internally constructed seal cannot authorize NEW v1 publication.
|
||||
seal := googlevideoProofSeal(p)
|
||||
p.producerSeal = &seal
|
||||
db := testStore(t)
|
||||
tx, err := db.Begin()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer tx.Rollback()
|
||||
id, err := insertStrategyCandidate(tx, c.Group, c.Endpoints, c.Args, nil, c.Mode)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if saveAdaptiveGooglevideoEvidence(tx, id, p) == nil {
|
||||
t.Fatal("new sealed v1 write accepted")
|
||||
}
|
||||
raw, _ := json.Marshal(p)
|
||||
var dto GooglevideoQualificationProof
|
||||
if err := json.Unmarshal(raw, &dto); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if saveAdaptiveGooglevideoEvidence(tx, id, dto) == nil {
|
||||
t.Fatal("unsealed v1 DTO write accepted")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,168 @@
|
||||
package main
|
||||
|
||||
// Portable trusted-history fixtures: run unchanged on archived d324b41 too.
|
||||
// Direct SQL below models an existing trusted v1 database, not an import API.
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"database/sql"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"reflect"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func legacyYoutubeV1Fixture(t *testing.T, db *sql.DB) (int64, GooglevideoQualificationProof) {
|
||||
t.Helper()
|
||||
r, c, resources, _ := googlevideoRuntimeFixture(t)
|
||||
p, err := r.verifyGooglevideo(context.Background(), c, resources)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
raw, _ := json.Marshal(p)
|
||||
var fields map[string]json.RawMessage
|
||||
if err := json.Unmarshal(raw, &fields); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
delete(fields, "adapter")
|
||||
fields["version"] = json.RawMessage(`1`)
|
||||
raw, _ = json.Marshal(fields)
|
||||
legacy, err := decodeGooglevideoProof(string(raw))
|
||||
if err != nil || legacy.validate(c) != nil {
|
||||
t.Fatal("trusted v1 contract invalid", err)
|
||||
}
|
||||
tx, err := db.Begin()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer tx.Rollback()
|
||||
id, err := insertStrategyCandidate(tx, c.Group, c.Endpoints, c.Args, nil, c.Mode)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := tx.Exec("INSERT INTO googlevideo_scope_evidence(candidate_id,proof) VALUES (?,?)", id, string(raw)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := tx.Commit(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return id, legacy
|
||||
}
|
||||
|
||||
func expireLegacyYoutubeV1(t *testing.T, db *sql.DB, id int64, proof GooglevideoQualificationProof) {
|
||||
t.Helper()
|
||||
proof.Attestation.CheckedAt = time.Now().Add(-25 * time.Hour)
|
||||
raw, _ := json.Marshal(proof)
|
||||
if _, err := db.Exec("UPDATE googlevideo_scope_evidence SET proof=? WHERE candidate_id=?", string(raw), id); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLegacyYoutubeV1FreshApplyEligibility(t *testing.T) {
|
||||
db := testStore(t)
|
||||
id, proof := legacyYoutubeV1Fixture(t, db)
|
||||
c, err := readStrategyCandidate(db, id)
|
||||
if err != nil || !c.Qualified {
|
||||
t.Fatal("fresh stored v1 lost qualification", err)
|
||||
}
|
||||
saved, err := listQualifiedCandidates(db, "googlevideo", 10)
|
||||
if err != nil || len(saved) != 1 || saved[0].ID != id {
|
||||
t.Fatal("fresh stored v1 disappeared from shortlist", err)
|
||||
}
|
||||
front := managedCandidateFixture(t, db, "frontend", "www.youtube.com", "--lua-desync=multisplit", true)
|
||||
ids := []int64{front, id} // Native suffix still does not require assets.
|
||||
m, reloads := managedFixture(t)
|
||||
if err := m.Apply(db, ids, GooglevideoScopeSuffix); err != nil {
|
||||
t.Fatal("fresh v1 apply rejected", err)
|
||||
}
|
||||
config, err := m.read(m.Name)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
expireLegacyYoutubeV1(t, db, id, proof)
|
||||
c, err = readStrategyCandidate(db, id)
|
||||
if err != nil || c.Qualified {
|
||||
t.Fatal("expired v1 retained fresh qualification", err)
|
||||
}
|
||||
before := *reloads
|
||||
if err := m.Apply(db, ids, GooglevideoScopeSuffix); err == nil || *reloads != before {
|
||||
t.Fatal("expired v1 activated")
|
||||
}
|
||||
rendered, err := renderStoredManagedConfigScope(db, ids, GooglevideoScopeSuffix)
|
||||
if err != nil || !bytes.Equal(config, rendered) {
|
||||
t.Fatal("expired historical reconstruction changed", err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLegacyYoutubeV1PreparedRollback(t *testing.T) {
|
||||
for _, point := range []string{"staged", "backup", "prepared", "replaced", "reloaded", "healthy"} {
|
||||
t.Run(point, func(t *testing.T) {
|
||||
db := testStore(t)
|
||||
id, proof := legacyYoutubeV1Fixture(t, db)
|
||||
front := managedCandidateFixture(t, db, "frontend", "www.youtube.com", "--lua-desync=multisplit", true)
|
||||
ids := []int64{front, id}
|
||||
m, reloads := managedFixture(t)
|
||||
if err := m.Apply(db, ids, GooglevideoScopeSuffix); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
before, err := m.load()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
journal, err := m.read(managedJournalName)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
config, err := m.read(m.Name)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
nextFront := managedCandidateFixture(t, db, "frontend", "www.youtube.com", "--lua-desync=fake", true)
|
||||
m.fault = func(stage string) error {
|
||||
if stage == point {
|
||||
return errors.New("fixture crash")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
if err := m.Apply(db, []int64{nextFront, id}, GooglevideoScopeSuffix); err == nil {
|
||||
t.Fatal("fault missed")
|
||||
}
|
||||
if point != "staged" {
|
||||
backup, err := m.read(managedBackupName)
|
||||
if err != nil || !bytes.Equal(backup, config) {
|
||||
t.Fatal("legacy backup changed", err)
|
||||
}
|
||||
}
|
||||
expireLegacyYoutubeV1(t, db, id, proof)
|
||||
m.fault = nil
|
||||
priorReloads := *reloads
|
||||
if err := m.Reconcile(db); err != nil {
|
||||
t.Fatal("expired v1 prepared recovery failed", err)
|
||||
}
|
||||
after, err := m.load()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
wantGeneration, wantReloads := before.Generation+1, priorReloads+1
|
||||
if point == "staged" || point == "backup" {
|
||||
wantGeneration, wantReloads = before.Generation, priorReloads
|
||||
}
|
||||
if after.State != "committed" || after.Hash != before.Hash || after.Scope != before.Scope || !reflect.DeepEqual(after.Candidates, before.Candidates) || len(after.PendingCandidates) != 0 || after.PendingScope != "" || after.Generation != wantGeneration || *reloads != wantReloads {
|
||||
t.Fatal("legacy rollback contract changed", after, *reloads)
|
||||
}
|
||||
restored, err := m.read(m.Name)
|
||||
if err != nil || !bytes.Equal(restored, config) {
|
||||
t.Fatal("rollback config bytes changed", err)
|
||||
}
|
||||
if point == "staged" || point == "backup" {
|
||||
afterJournal, err := m.read(managedJournalName)
|
||||
if err != nil || !bytes.Equal(journal, afterJournal) {
|
||||
t.Fatal("pre-prepare journal changed", err)
|
||||
}
|
||||
}
|
||||
assertActiveCandidates(t, m, db, ids)
|
||||
t.Logf("legacy restored config SHA256=%s", after.Hash)
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,133 @@
|
||||
package main
|
||||
|
||||
// This corpus also runs unmodified against archived d324b41. It uses only
|
||||
// pre-extraction APIs, with explicit expected normalized facts, not wrappers.
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestYoutubeLegacyParserParity(t *testing.T) {
|
||||
good := "https://rr1---sn-alpha.googlevideo.com/videoplayback?expire=1800000000&sig=private-secret"
|
||||
format := func(raw, mime string) map[string]string { return map[string]string{"url": raw, "mimeType": mime} }
|
||||
for _, tc := range []struct {
|
||||
name string
|
||||
formats, adaptive []map[string]string
|
||||
want string
|
||||
}{
|
||||
{"ordinary-first", []map[string]string{format(good, "audio/mp4; codecs=mp4a")}, []map[string]string{format(good+"&second=1", "video/mp4")}, "audio/mp4"},
|
||||
{"nonmp4-fallback", []map[string]string{format(good, "video/webm")}, []map[string]string{format(good, "video/mp4")}, "video/mp4"},
|
||||
{"userinfo", []map[string]string{format(strings.Replace(good, "https://", "https://user@", 1), "video/mp4")}, nil, ""},
|
||||
{"fragment", []map[string]string{format(good+"#secret", "video/mp4")}, nil, ""},
|
||||
{"http", []map[string]string{format(strings.Replace(good, "https:", "http:", 1), "video/mp4")}, nil, ""},
|
||||
{"invalid-dns", []map[string]string{format(strings.Replace(good, "rr1---sn-alpha", "bad_host", 1), "video/mp4")}, nil, ""},
|
||||
{"sabr-only", nil, nil, ""},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
body, _ := json.Marshal(map[string]interface{}{"playabilityStatus": map[string]string{"status": "OK"}, "streamingData": map[string]interface{}{"formats": tc.formats, "adaptiveFormats": tc.adaptive, "serverAbrStreamingUrl": good}})
|
||||
r, capability, err := playerMediaResource(body)
|
||||
if err != nil || r.kind != tc.want || capability.OrdinaryURL != (tc.want != "") || capability.SABRSupported {
|
||||
t.Fatal(r.kind, capability, err)
|
||||
}
|
||||
if tc.want != "" && (r.url != good || r.host != "rr1---sn-alpha.googlevideo.com") {
|
||||
t.Fatal("normalized resource changed")
|
||||
}
|
||||
})
|
||||
}
|
||||
for _, body := range [][]byte{[]byte(`{"secret":`), bytes.Repeat([]byte("x"), playerResponseLimit+1)} {
|
||||
if _, _, err := playerMediaResource(body); err == nil || err.Error() != "invalid playable metadata" {
|
||||
t.Fatal("metadata error contract changed", err)
|
||||
}
|
||||
}
|
||||
for _, delta := range []time.Duration{-time.Hour, 19 * time.Minute, 21 * time.Minute} {
|
||||
r := mediaResource{fmt.Sprintf("https://rr1---sn-alpha.googlevideo.com/videoplayback?expire=%d&lsig=secret", time.Now().Add(delta).Unix()), "rr1---sn-alpha.googlevideo.com", "video/mp4"}
|
||||
if freshGooglevideoSample(r) != (delta > managedGooglevideoSampleTTL) {
|
||||
t.Fatal("inventory freshness changed", delta)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestYoutubeLegacyJournalParity(t *testing.T) {
|
||||
r, c, resources, _ := googlevideoRuntimeFixture(t)
|
||||
p, err := r.verifyGooglevideo(context.Background(), c, resources)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
db := testStore(t)
|
||||
tx, err := db.Begin()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer tx.Rollback()
|
||||
id, err := insertStrategyCandidate(tx, c.Group, c.Endpoints, c.Args, nil, c.Mode)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := saveAdaptiveGooglevideoEvidence(tx, id, p); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := tx.Commit(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
front := managedCandidateFixture(t, db, "frontend", "www.youtube.com", "--lua-desync=multisplit", true)
|
||||
m, _ := managedFixture(t)
|
||||
ids := []int64{front, id} // No new asset requirement for native suffix policy.
|
||||
if err := m.Apply(db, ids, GooglevideoScopeSuffix); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
before, err := m.load()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
journalBytes, err := m.read(managedJournalName)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
config, err := m.read(m.Name)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
// Convert the actual producer facts to the exact historical v1 wire shape.
|
||||
p.Attestation.CheckedAt = time.Now().Add(-25 * time.Hour)
|
||||
raw, _ := json.Marshal(p)
|
||||
var fields map[string]json.RawMessage
|
||||
if err := json.Unmarshal(raw, &fields); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
delete(fields, "adapter")
|
||||
fields["version"] = json.RawMessage(`1`)
|
||||
raw, _ = json.Marshal(fields)
|
||||
if _, err := db.Exec("UPDATE googlevideo_scope_evidence SET proof=? WHERE candidate_id=?", string(raw), id); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
rendered, err := renderStoredManagedConfigScope(db, ids, GooglevideoScopeSuffix)
|
||||
if err != nil || !bytes.Equal(config, rendered) || managedHash(rendered) != before.Hash {
|
||||
t.Fatal("historical render/hash changed", err)
|
||||
}
|
||||
if err := m.Reconcile(db); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
active, err := m.ActiveCandidates(db)
|
||||
if err != nil || len(active) != 2 || active[0] != front || active[1] != id {
|
||||
t.Fatal(active, err)
|
||||
}
|
||||
after, err := m.load()
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
a, _ := json.Marshal(before)
|
||||
b, _ := json.Marshal(after)
|
||||
if !bytes.Equal(a, b) {
|
||||
t.Fatal("historical journal changed")
|
||||
}
|
||||
afterBytes, err := m.read(managedJournalName)
|
||||
if err != nil || !bytes.Equal(journalBytes, afterBytes) {
|
||||
t.Fatal("historical journal bytes changed", err)
|
||||
}
|
||||
t.Logf("historical config SHA256=%s", before.Hash)
|
||||
}
|
||||
@@ -27,7 +27,16 @@ with tempfile.TemporaryDirectory(prefix='target-config-') as tmp:
|
||||
ZAPRET_AUTOTUNE_YOUTUBE_SERVICE='0', ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE='cold',
|
||||
ZAPRET_AUTOTUNE_VIDEO_ID='dQw4w9WgXcQ',
|
||||
ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS='["dQw4w9WgXcQ"]',
|
||||
ZAPRET_AUTOTUNE_TARGET_ROLE='managed-test',
|
||||
ZAPRET_AUTOTUNE_TARGET_ID='zapret-autotune-managed-v1',
|
||||
ZAPRET_AUTOTUNE_TARGET_DIR='/var/lib/zapret-autotune/target',
|
||||
ZAPRET_CONFIG='/var/lib/zapret-autotune/target/config',
|
||||
SOCKS_PORT='1080',
|
||||
ZAPRET_AUTOTUNE_SOCKS_PROXY='socks5h://127.0.0.1:1080')
|
||||
for name in ('ZAPRET_AUTOTUNE_RESOURCE_APPLY', 'ZAPRET_AUTOTUNE_RESOURCE_CONFIG',
|
||||
'ZAPRET_AUTOTUNE_QUALIFY_CONFIG', 'ZAPRET_AUTOTUNE_STRATEGY_CONFIG',
|
||||
'ZAPRET_AUTOTUNE_RESOURCE_STRATEGY_IDS'):
|
||||
base.pop(name, None)
|
||||
for dns, valid in (('', True), ('8.8.8.8', True), ('2001:4860:4860::8888', True),
|
||||
('example.org', False), ('8.8.8.8\n', False), ('-d 8.8.8.8', False)):
|
||||
for path in private.iterdir():
|
||||
@@ -61,6 +70,28 @@ with tempfile.TemporaryDirectory(prefix='target-config-') as tmp:
|
||||
assert base['SS_PASSWORD'].encode() not in result.stdout + result.stderr
|
||||
if not valid:
|
||||
assert not list(private.iterdir())
|
||||
for overrides, valid in (
|
||||
({'ZAPRET_AUTOTUNE_RESOURCE_STRATEGY_IDS': 's1'}, False),
|
||||
({'ZAPRET_AUTOTUNE_RESOURCE_CONFIG': '/private/resources.json',
|
||||
'ZAPRET_AUTOTUNE_RESOURCE_STRATEGY_IDS': 's2,s1'}, True),
|
||||
({'ZAPRET_AUTOTUNE_RESOURCE_CONFIG': '/private/resources.json',
|
||||
'ZAPRET_AUTOTUNE_RESOURCE_STRATEGY_IDS': 's1,s1'}, False),
|
||||
({'ZAPRET_AUTOTUNE_RESOURCE_APPLY': '1'}, False),
|
||||
({'ZAPRET_AUTOTUNE_RESOURCE_APPLY': 'yes'}, False),
|
||||
({'ZAPRET_AUTOTUNE_RESOURCE_APPLY': '1', 'ZAPRET_AUTOTUNE_RESOURCE_CONFIG': '/private/resources.json'}, True),
|
||||
({'ZAPRET_AUTOTUNE_RESOURCE_APPLY': '1', 'ZAPRET_AUTOTUNE_RESOURCE_CONFIG': '/private/resources.json',
|
||||
'ZAPRET_AUTOTUNE_TARGET_ROLE': ''}, False),
|
||||
({'ZAPRET_AUTOTUNE_RESOURCE_APPLY': '1', 'ZAPRET_AUTOTUNE_RESOURCE_CONFIG': '/private/resources.json',
|
||||
'SOCKS_PORT': '1081'}, False),
|
||||
({'ZAPRET_AUTOTUNE_RESOURCE_APPLY': '1', 'ZAPRET_AUTOTUNE_RESOURCE_CONFIG': '/private/resources.json',
|
||||
'ZAPRET_AUTOTUNE_QUALIFY_CONFIG': '/private/resources.json', 'ZAPRET_AUTOTUNE_STRATEGY_CONFIG': '/private/strategies.json'}, False),
|
||||
):
|
||||
for path in private.iterdir():
|
||||
path.unlink()
|
||||
result = subprocess.run(['sh', str(entry)], env=dict(base, **overrides), capture_output=True, timeout=5)
|
||||
assert result.returncode == (0 if valid else 64), result.stderr
|
||||
if not valid:
|
||||
assert not list(private.iterdir()), 'Invalid apply mode created service configs'
|
||||
# Render privately, never print the config/password; config does not deploy.
|
||||
compose_env = {k: v for k, v in base.items() if not k.startswith('ZAPRET_AUTOTUNE_')}
|
||||
factory_cases = []
|
||||
@@ -100,6 +131,18 @@ with tempfile.TemporaryDirectory(prefix='target-config-') as tmp:
|
||||
factory_cases.append({'environment': {k: v for k, v in settings.items()
|
||||
if k.startswith('ZAPRET_AUTOTUNE_')}, 'ids': ids})
|
||||
# Validate the actual rendered DNS environment, not an invented equivalent.
|
||||
result = subprocess.run(['docker', 'compose', '--env-file', '/dev/null', '-f',
|
||||
str(ROOT / 'deploy/managed-target/compose.yaml'), 'config', '--format', 'json'],
|
||||
env=dict(compose_env, ZAPRET_AUTOTUNE_RESOURCE_APPLY='1',
|
||||
ZAPRET_AUTOTUNE_RESOURCE_STRATEGY_IDS='s2,s1',
|
||||
ZAPRET_AUTOTUNE_RESOURCE_CONFIG='/private/resources.json',
|
||||
ZAPRET_AUTOTUNE_STOP_GRACE_PERIOD='90s'), capture_output=True, timeout=10)
|
||||
assert result.returncode == 0, 'Generic Compose config failed (output suppressed)'
|
||||
target = json.loads(result.stdout)['services']['target']
|
||||
assert target['environment']['ZAPRET_AUTOTUNE_RESOURCE_APPLY'] == '1'
|
||||
assert target['environment']['ZAPRET_AUTOTUNE_RESOURCE_STRATEGY_IDS'] == 's2,s1'
|
||||
assert target['environment']['ZAPRET_AUTOTUNE_RESOURCE_CONFIG'] == '/private/resources.json'
|
||||
assert target['stop_grace_period'] == '1m30s'
|
||||
# Compose must not inject Google bootstrap into a custom-provider request.
|
||||
for service in ('0', '1'):
|
||||
for provider in ({}, {'ZAPRET_AUTOTUNE_DOH_URL': 'https://secret.invalid/token'}):
|
||||
|
||||
@@ -0,0 +1,177 @@
|
||||
"""Offline deterministic startup guards: python3 deploy/managed-target/check_startup_diagnostics.py."""
|
||||
from pathlib import Path
|
||||
import os
|
||||
import stat
|
||||
import subprocess
|
||||
import tempfile
|
||||
import time
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[2]
|
||||
entry = (ROOT / 'managed-target-entrypoint.sh').read_text()
|
||||
capture = entry[entry.index('capture_monitor_session() {'):entry.index('# Wait with a watchdog:')]
|
||||
launch = entry[entry.index('launch_controller() {'):entry.index('run_cycle() {')]
|
||||
fixture_launch = launch.replace(' 8<&- 9<&- &', ' 8<&- 9<&-').replace(
|
||||
'monitor_ack_deadline=$(($(date +%s) + 5))', 'monitor_ack_deadline=100')
|
||||
secret = 'SECRET_SENTINEL'
|
||||
roles = ('managed resource-apply --resource-config', 'managed monitor', 'qualify --isolated')
|
||||
|
||||
# Injected empty/malformed ACKs must still fail closed, not retry the read.
|
||||
for role in roles:
|
||||
for scenario, diagnostic in (
|
||||
('empty', 'ack=read_failed'), ('malformed', 'ack=invalid_pid'),
|
||||
('wrong', 'ack=identity_mismatch'), ('session', 'launch=session_mismatch'),
|
||||
('gone', 'launch=child_gone'), ('timeout', 'launch=ack_timeout'),
|
||||
):
|
||||
with tempfile.TemporaryDirectory(prefix='startup-diagnostics-') as tmp:
|
||||
script = f'''set -eu
|
||||
config_dir='{tmp}'
|
||||
monitor_sid=
|
||||
scenario={scenario}
|
||||
date() {{ echo 100; }}
|
||||
setsid() {{
|
||||
case "$scenario" in
|
||||
empty) : > "$config_dir/monitor-ready" ;;
|
||||
malformed) echo '{secret}' > "$config_dir/monitor-ready" ;;
|
||||
wrong) echo 1 > "$config_dir/monitor-ready" ;;
|
||||
session) echo "$!" > "$config_dir/monitor-ready" ;;
|
||||
esac
|
||||
}}
|
||||
kill() {{ [ "$scenario" != gone ]; }}
|
||||
signal_tree() {{ :; }}
|
||||
proc_identity() {{ proc_sid=1; }}
|
||||
{capture}
|
||||
sleep 30 & seed=$!
|
||||
{fixture_launch}
|
||||
code=0
|
||||
launch_controller {role} '{secret}' || code=$?
|
||||
[ "$code" -eq 78 ]
|
||||
[ -z "$monitor_sid" ]
|
||||
command kill "$seed"
|
||||
wait "$seed" 2>/dev/null || true
|
||||
'''
|
||||
result = subprocess.run(['sh', '-c', script], capture_output=True, timeout=3)
|
||||
assert result.returncode == 0, (role, scenario, result.stderr)
|
||||
assert result.stderr == f'controller-startup: {diagnostic}\n'.encode(), result.stderr
|
||||
assert secret.encode() not in result.stdout + result.stderr
|
||||
|
||||
helper = (ROOT / 'managed-target-supervision.sh').read_text()
|
||||
for role in roles:
|
||||
with tempfile.TemporaryDirectory(prefix='delayed-ack-') as tmp:
|
||||
root = Path(tmp)
|
||||
wrapper_path = root / 'wrapper.sh'
|
||||
write = ' printf \'%s\\n\' "$$" > "$ack_temp" || return 1'
|
||||
assert helper.count(write) == 1
|
||||
wrapper_path.write_text(helper.replace(write, ''' : > "$config_dir/barrier"
|
||||
while [ ! -e "$config_dir/release" ]; do sleep 0.01; done
|
||||
''' + write))
|
||||
real_launch = launch.replace('/opt/managed-target-supervision.sh', str(wrapper_path)).replace(
|
||||
'/usr/local/bin/zapret-autotune', '/bin/true')
|
||||
process = subprocess.Popen(['sh', '-c', f'''set -eu
|
||||
config_dir='{tmp}'
|
||||
monitor_sid=
|
||||
. '{ROOT / 'managed-target-supervision.sh'}'
|
||||
{capture}
|
||||
{real_launch}
|
||||
launch_controller {role} '{secret}'
|
||||
[ "$monitor_sid" = "$monitor_pid" ]
|
||||
wait "$monitor_pid"
|
||||
# An owned completed launch can be replaced for the next controller.
|
||||
monitor_sid=
|
||||
launch_controller {role} '{secret}'
|
||||
[ "$monitor_sid" = "$monitor_pid" ]
|
||||
wait "$monitor_pid"
|
||||
'''], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
|
||||
try:
|
||||
deadline = time.monotonic() + 2
|
||||
while not (root / 'barrier').exists():
|
||||
assert process.poll() is None
|
||||
assert time.monotonic() < deadline
|
||||
time.sleep(0.01)
|
||||
assert not (root / 'monitor-ready').exists()
|
||||
pending, = root.glob('.monitor-ready.*')
|
||||
assert pending.read_bytes() == b''
|
||||
assert stat.S_IMODE(pending.stat().st_mode) == 0o600
|
||||
(root / 'release').touch()
|
||||
stdout, stderr = process.communicate(timeout=3)
|
||||
assert process.returncode == 0, stderr
|
||||
assert stderr == b'controller-startup: wrapper=exec_pending\n' * 2, stderr
|
||||
ready = root / 'monitor-ready'
|
||||
assert ready.read_text().strip().isdigit()
|
||||
assert ready.read_bytes().endswith(b'\n')
|
||||
assert ready.stat().st_nlink == 1
|
||||
assert ready.stat().st_uid == os.getuid()
|
||||
assert stat.S_IMODE(ready.stat().st_mode) == 0o600
|
||||
assert not list(root.glob('.monitor-ready.*'))
|
||||
assert secret.encode() not in stdout + stderr
|
||||
finally:
|
||||
(root / 'release').touch()
|
||||
if process.poll() is None:
|
||||
process.kill()
|
||||
process.communicate(timeout=3)
|
||||
|
||||
wrapper = helper[helper.index('if [ "${1-}" = --monitor ]; then'):helper.index('elif [ "${1-}" = --watchdog ]; then')] + '\nfi\n'
|
||||
for identity, diagnostic, code in (
|
||||
('return 1', 'identity_unavailable', 78),
|
||||
('proc_sid=1', 'not_session_leader', 78),
|
||||
('proc_sid=$$', 'exec_pending', 0),
|
||||
):
|
||||
with tempfile.TemporaryDirectory(prefix='wrapper-diagnostics-') as tmp:
|
||||
result = subprocess.run(['sh', '-c', f'proc_identity() {{ {identity}; }}\n{wrapper}',
|
||||
'fixture', '--monitor', tmp, '/bin/true', secret], capture_output=True, timeout=3)
|
||||
assert result.returncode == code, result.stderr
|
||||
assert result.stderr == f'controller-startup: wrapper={diagnostic}\n'.encode(), result.stderr
|
||||
ready = Path(tmp) / 'monitor-ready'
|
||||
assert ready.exists() == (code == 0)
|
||||
assert secret.encode() not in result.stdout + result.stderr
|
||||
|
||||
# Both publication and parent cleanup preserve unknown artifacts, including links.
|
||||
for artifact in ('file', 'symlink', 'directory'):
|
||||
with tempfile.TemporaryDirectory(prefix='foreign-ack-') as tmp:
|
||||
root = Path(tmp)
|
||||
ready = root / 'monitor-ready'
|
||||
if artifact == 'file':
|
||||
ready.write_text(secret)
|
||||
elif artifact == 'symlink':
|
||||
ready.symlink_to(root / 'absent')
|
||||
else:
|
||||
ready.mkdir()
|
||||
before = ready.lstat()
|
||||
result = subprocess.run(['sh', '-c', f'proc_identity() {{ proc_sid=$$; }}\n{wrapper}',
|
||||
'fixture', '--monitor', tmp, '/bin/true', secret], capture_output=True, timeout=3)
|
||||
assert result.returncode == 78, result.stderr
|
||||
assert result.stderr == b'controller-startup: wrapper=ack_publish_failed\n', result.stderr
|
||||
result = subprocess.run(['sh', '-c', f'''set -eu
|
||||
config_dir='{tmp}'
|
||||
{launch}
|
||||
launch_controller managed monitor
|
||||
'''], capture_output=True, timeout=3)
|
||||
assert result.returncode == 78, result.stderr
|
||||
assert result.stderr == b'controller-startup: launch=ack_conflict\n', result.stderr
|
||||
assert ready.lstat() == before
|
||||
if artifact == 'file':
|
||||
assert ready.read_text() == secret
|
||||
assert not list(root.glob('.monitor-ready.*'))
|
||||
|
||||
with tempfile.TemporaryDirectory(prefix='replaced-ack-') as tmp:
|
||||
result = subprocess.run(['sh', '-c', f'''set -eu
|
||||
config_dir='{tmp}'
|
||||
printf '123\n' > "$config_dir/monitor-ready"
|
||||
monitor_ready_identity=$(stat -c '%d:%i:%u:%a:%h' "$config_dir/monitor-ready")
|
||||
monitor_ready_pid=123
|
||||
printf '{secret}' > "$config_dir/replacement"
|
||||
mv "$config_dir/replacement" "$config_dir/monitor-ready"
|
||||
{launch}
|
||||
launch_controller managed monitor
|
||||
'''], capture_output=True, timeout=3)
|
||||
assert result.returncode == 78, result.stderr
|
||||
assert result.stderr == b'controller-startup: launch=ack_conflict\n'
|
||||
assert (Path(tmp) / 'monitor-ready').read_text() == secret
|
||||
|
||||
with tempfile.TemporaryDirectory(prefix='unsafe-ack-dir-') as tmp:
|
||||
os.chmod(tmp, 0o755)
|
||||
result = subprocess.run(['sh', '-c', f'proc_identity() {{ proc_sid=$$; }}\n{wrapper}',
|
||||
'fixture', '--monitor', tmp, '/bin/true'], capture_output=True, timeout=3)
|
||||
assert result.returncode == 78, result.stderr
|
||||
assert result.stderr == b'controller-startup: wrapper=ack_publish_failed\n'
|
||||
assert not list(Path(tmp).iterdir())
|
||||
print('startup diagnostic fixtures passed (shared apply/monitor/qualify launch and atomic wrapper)')
|
||||
@@ -4,6 +4,19 @@ from pathlib import Path
|
||||
|
||||
|
||||
directory = Path(__file__).resolve().parent
|
||||
root = directory.parents[1]
|
||||
# ponytail: guard the current literal COPY and deny-all/allowlist ignore shape;
|
||||
# use a Dockerfile/ignore parser if these become general patterns or multi-module.
|
||||
ignore = [line for line in (root / ".dockerignore").read_text().splitlines()
|
||||
if line and not line.startswith("#")]
|
||||
assert ignore[0] == "*" and all(line.startswith("!") for line in ignore[1:])
|
||||
package_roots = {path.relative_to(root).parts[0] for path in root.rglob("*.go")}
|
||||
for name in ("Dockerfile", "Dockerfile.target"):
|
||||
builder = (root / name).read_text().split("\nFROM ", 1)[0]
|
||||
before_build = builder[:builder.index("go build")]
|
||||
for package in package_roots:
|
||||
assert f"COPY {package}/ ./{package}/\n" in before_build, (name, package)
|
||||
assert f"!{package}/" in ignore and f"!{package}/**" in ignore, package
|
||||
compose = (directory / "compose.yaml").read_text()
|
||||
entrypoint = (directory.parents[1] / "managed-target-entrypoint.sh").read_text()
|
||||
readme = (directory / "README.md").read_text()
|
||||
@@ -55,7 +68,8 @@ assert "Docker administrators/root can still read container" in readme
|
||||
assert "removed\nfrom the entrypoint environment" in readme
|
||||
assert "No host networking" in readme
|
||||
assert ' restart: "no"\n' in compose
|
||||
assert " stop_grace_period: 240s\n" in compose
|
||||
assert " stop_grace_period: ${ZAPRET_AUTOTUNE_STOP_GRACE_PERIOD-240s}\n" in compose
|
||||
assert 'ZAPRET_AUTOTUNE_RESOURCE_APPLY: ${ZAPRET_AUTOTUNE_RESOURCE_APPLY-0}' in compose
|
||||
assert 'bounded_wait "$monitor_pid" 45' in entrypoint
|
||||
assert '/bin/sh /opt/managed-target-supervision.sh --watchdog' in entrypoint
|
||||
assert 'setsid /bin/sh /opt/managed-target-supervision.sh --monitor' in entrypoint
|
||||
@@ -77,6 +91,9 @@ for command, seconds in (("/usr/local/bin/zapret-autotune managed init", 45),
|
||||
assert entrypoint.index("managed reconcile || return 78") < entrypoint.index("zapret2 start || return 78")
|
||||
experimental = 'managed apply --googlevideo-experiment native-circular --googlevideo-scope experimental_googlevideo_suffix --googlevideo-request startup || exit 78'
|
||||
assert entrypoint.index('zapret2 start || return 78') < entrypoint.index(experimental)
|
||||
assert entrypoint.index(experimental) < entrypoint.index('setsid /bin/sh /opt/managed-target-supervision.sh --monitor')
|
||||
assert entrypoint.index(experimental) < entrypoint.index('launch_controller "$@" || return 78')
|
||||
generic_apply = 'launch_controller managed resource-apply --resource-config "$ZAPRET_AUTOTUNE_RESOURCE_CONFIG"'
|
||||
assert entrypoint.index('managed reconcile || return 78') < entrypoint.index(generic_apply) < entrypoint.index('zapret2 start || return 78')
|
||||
assert entrypoint.index(generic_apply) < entrypoint.index('set -- managed monitor')
|
||||
assert entrypoint.index('managed experimental-request-check || return 78') < entrypoint.index('managed init || return 78')
|
||||
assert 'rm "${ZAPRET_CONFIG%/*}/.autotune-experimental-suffix-request"' not in entrypoint
|
||||
@@ -24,7 +24,7 @@ DNS_CLI = Path(CLI_TMP.name) / 'autotune'
|
||||
subprocess.run(['go', 'build', '-o', str(DNS_CLI), './cmd/zapret-autotune'],
|
||||
cwd=ROOT, env=dict(os.environ, GOPROXY='off', GOTOOLCHAIN='local'), check=True)
|
||||
FAKE = r'''#!/usr/bin/env python3
|
||||
import ctypes, os, pathlib, signal, subprocess, sys, time
|
||||
import ctypes, json, os, pathlib, signal, subprocess, sys, time
|
||||
root = pathlib.Path(os.environ['FIXTURE'])
|
||||
name = pathlib.Path(sys.argv[0]).name
|
||||
scenario = os.environ['SCENARIO']
|
||||
@@ -38,7 +38,7 @@ def increment(key):
|
||||
return n
|
||||
def linger(code=0):
|
||||
def stop(*_):
|
||||
if os.environ.get('ZAPRET_AUTOTUNE_DNS_MODE', 'doh') == 'doh' and scenario not in ('dns-death', 'dns-killed', 'dns-replace', 'dns-replace-symlink', 'dns-prestart'):
|
||||
if os.environ.get('ZAPRET_AUTOTUNE_DNS_MODE', 'doh') == 'doh' and scenario not in ('dns-death', 'dns-killed', 'dns-replace', 'dns-replace-symlink', 'dns-prestart', 'generic-apply-dns-loss', 'generic-apply-dns-restart'):
|
||||
os.kill(int((root / 'dns-pid').read_text()), 0)
|
||||
sys.exit(code)
|
||||
signal.signal(signal.SIGTERM, stop)
|
||||
@@ -67,10 +67,26 @@ if name == 'awk':
|
||||
sys.exit(0)
|
||||
if name == 'date':
|
||||
n = increment('clock')
|
||||
print(100000 + (n * 21601 if scenario == 'window' else n + (21601 if (root / 'budget-pause').exists() else 0)))
|
||||
elapsed = int((root / 'elapsed').read_text()) if (root / 'elapsed').exists() else 0
|
||||
print(100000 + elapsed + (n * 21601 if scenario == 'window' else n + (21601 if (root / 'budget-pause').exists() else 0)))
|
||||
sys.exit(0)
|
||||
if name == 'sleep':
|
||||
seconds = int(sys.argv[1])
|
||||
if seconds == 60 and scenario.startswith('generic-apply-'):
|
||||
# Three former 60s retry windows; the failed producer becomes healthy
|
||||
# after its first call, so any unintended retry would reach native READY.
|
||||
n = increment('blocked_windows')
|
||||
(root / 'elapsed').write_text(str(n * 60))
|
||||
log('blocked-window ' + str(n))
|
||||
if n == 3:
|
||||
os.kill(os.getppid(), signal.SIGTERM)
|
||||
time.sleep(.01)
|
||||
sys.exit(0)
|
||||
if scenario.startswith('qualify-') and (root / 'qualified').exists():
|
||||
log('safe-idle')
|
||||
os.kill(os.getppid(), signal.SIGTERM)
|
||||
time.sleep(.02)
|
||||
sys.exit(0)
|
||||
if scenario.startswith('generic-budget-') and seconds != 60:
|
||||
os.execv(os.environ['REAL_SLEEP'], [os.environ['REAL_SLEEP'], *sys.argv[1:]])
|
||||
if seconds == 60 and scenario in ('generic-ss', 'generic-unsafe', 'generic-budget-monitor', 'generic-budget-reconcile', 'generic-budget-ack'):
|
||||
@@ -105,6 +121,8 @@ if name in ('ss-server', 'ss-local'):
|
||||
os.kill(int((root / 'dns-pid').read_text()), 0)
|
||||
assert (pathlib.Path(sys.argv[2]).stat().st_mode & 0o777) == 0o600
|
||||
log(name)
|
||||
if name == 'ss-local':
|
||||
(root / 'ss-pid').write_text(str(os.getpid()))
|
||||
if scenario.startswith('generic-budget-'):
|
||||
signal.signal(signal.SIGTERM, signal.SIG_IGN)
|
||||
while True:
|
||||
@@ -125,8 +143,32 @@ if name == 'zapret2':
|
||||
sys.exit(1)
|
||||
sys.exit(0)
|
||||
assert name == 'autotune'
|
||||
if sys.argv[1] == 'qualification-race-barrier':
|
||||
# Deterministic interleaving: child has exited, shell has not reached wait.
|
||||
pid = int(sys.argv[2])
|
||||
deadline = time.monotonic() + 5
|
||||
while True:
|
||||
stat = pathlib.Path('/proc/' + str(pid) + '/stat')
|
||||
if not stat.exists() or stat.read_text().rsplit(') ', 1)[1].split()[0] == 'Z':
|
||||
break
|
||||
assert time.monotonic() < deadline, 'qualifier did not exit at barrier'
|
||||
time.sleep(.001)
|
||||
log('term-before-parent-reap')
|
||||
os.kill(os.getppid(), signal.SIGTERM)
|
||||
sys.exit(0)
|
||||
if sys.argv[1] == 'qualification-completion-check':
|
||||
ack = root / 'completion-ack'
|
||||
if not ack.exists() or ack.is_symlink() or ack.stat().st_mode & 0o777 != 0o600:
|
||||
sys.exit(78)
|
||||
payload = json.loads(ack.read_text())
|
||||
assert payload == {'pid': int(sys.argv[2]), 'code': int(sys.argv[3]), 'cleanup_complete': True}
|
||||
log('completion-checked')
|
||||
sys.exit(0)
|
||||
if sys.argv[1:] == ['target-config-check']:
|
||||
sys.exit(0)
|
||||
if sys.argv[1:] == ['apply-launch']:
|
||||
log('apply-launch')
|
||||
sys.exit(0)
|
||||
if sys.argv[1] == 'dns-forwarder':
|
||||
if sys.argv[2] in ('readiness-check', 'readiness-cleanup'):
|
||||
# exec preserves PID1's actual inherited anchors; no simulated unlink.
|
||||
@@ -194,6 +236,40 @@ if sys.argv[1] == 'dns-forwarder':
|
||||
os.kill(os.getpid(), signal.SIGKILL)
|
||||
while True:
|
||||
time.sleep(.01)
|
||||
if sys.argv[1] == 'qualify':
|
||||
assert sys.argv[2:] == ['--config', '/operator/resources.json', '--strategies', '/operator/strategies.json', '--isolated']
|
||||
assert os.getsid(0) == os.getpid(), 'qualifier lacks owned session'
|
||||
assert 'SS_PASSWORD' not in os.environ
|
||||
events = (root / 'events').read_text().splitlines()
|
||||
assert 'start' not in events and 'stop' not in events
|
||||
assert events[-1] == 'reconcile', events
|
||||
log('qualify')
|
||||
assert increment('qualification_runs') == 1, 'qualification restarted'
|
||||
def acknowledge(code):
|
||||
ack = root / 'completion-ack'
|
||||
ack.write_text(json.dumps({'pid': os.getpid(), 'code': code, 'cleanup_complete': True}))
|
||||
ack.chmod(0o600)
|
||||
if not scenario.endswith('-race'):
|
||||
(root / 'qualified').touch()
|
||||
if scenario in ('qualify-unsafe', 'qualify-unsafe-race'):
|
||||
if not scenario.endswith('-race'):
|
||||
(root / 'qualified').touch()
|
||||
sys.exit(78)
|
||||
if scenario in ('qualify-blocked', 'qualify-blocked-race'):
|
||||
acknowledge(78)
|
||||
sys.exit(78)
|
||||
if scenario == 'qualify-cancel':
|
||||
def restored(*_):
|
||||
log('qualification-restored')
|
||||
acknowledge(0)
|
||||
sys.exit(0)
|
||||
signal.signal(signal.SIGTERM, restored)
|
||||
os.kill(os.getppid(), signal.SIGTERM)
|
||||
while True:
|
||||
time.sleep(.01)
|
||||
log('qualification-restored')
|
||||
acknowledge(0)
|
||||
sys.exit(0)
|
||||
command = sys.argv[2]
|
||||
if command == 'init':
|
||||
log('init')
|
||||
@@ -224,6 +300,14 @@ elif command == 'monitor':
|
||||
f.write(str(os.getpid()) + '\n')
|
||||
n = increment('monitor_runs')
|
||||
log('monitor ' + str(n))
|
||||
if n == 1 and scenario == 'generic-apply-monitor-restart':
|
||||
sys.exit(75)
|
||||
if n == 1 and scenario == 'generic-apply-ss-restart':
|
||||
os.kill(int((root / 'ss-pid').read_text()), signal.SIGKILL)
|
||||
linger()
|
||||
if n == 1 and scenario == 'generic-apply-dns-restart':
|
||||
os.kill(int((root / 'dns-pid').read_text()), signal.SIGKILL)
|
||||
linger()
|
||||
assert os.getsid(0) == os.getpid(), 'monitor not in its own verified session'
|
||||
if scenario.startswith('generic-budget-'):
|
||||
time.sleep(.03)
|
||||
@@ -265,6 +349,40 @@ elif command == 'monitor':
|
||||
time.sleep(.03)
|
||||
os.kill(os.getppid(), signal.SIGTERM)
|
||||
linger()
|
||||
elif command == 'resource-apply':
|
||||
assert sys.argv[3:] == ['--resource-config', '/operator/resources.json']
|
||||
assert os.environ['ZAPRET_AUTOTUNE_RESOURCE_APPLY'] == '1'
|
||||
assert os.environ['ZAPRET_AUTOTUNE_RESOURCE_STRATEGY_IDS'] == 's2,s1'
|
||||
assert os.getsid(0) == os.getpid(), 'apply lacks owned session'
|
||||
assert (root / 'private' / 'monitor-ready').read_text() == str(os.getpid()) + '\n'
|
||||
events = (root / 'events').read_text().splitlines()
|
||||
assert events[-2:] == ['reconcile', 'apply-launch'] and 'start' not in events
|
||||
log('resource-apply')
|
||||
n = increment('apply_runs')
|
||||
if scenario == 'generic-apply-blocked' and n == 1:
|
||||
sys.exit(78)
|
||||
if scenario == 'generic-apply-abrupt' and n == 1:
|
||||
sys.exit(23)
|
||||
if scenario == 'generic-apply-cleanup' and n == 1:
|
||||
log('resource-returned-zero')
|
||||
sys.exit(0)
|
||||
if scenario == 'generic-apply-signal' and n == 1:
|
||||
os.kill(os.getpid(), signal.SIGKILL)
|
||||
if scenario in ('generic-apply-dns-loss', 'generic-apply-ss-loss') and n == 1:
|
||||
os.kill(int((root / ('dns-pid' if scenario.endswith('dns-loss') else 'ss-pid')).read_text()), signal.SIGKILL)
|
||||
signal.signal(signal.SIGTERM, lambda *_: sys.exit(0))
|
||||
while True:
|
||||
time.sleep(.01)
|
||||
if scenario == 'generic-apply-cancel':
|
||||
def restored(*_):
|
||||
log('resource-restored')
|
||||
sys.exit(0)
|
||||
signal.signal(signal.SIGTERM, restored)
|
||||
os.kill(os.getppid(), signal.SIGTERM)
|
||||
while True:
|
||||
time.sleep(.01)
|
||||
log('resource-applied')
|
||||
sys.exit(0)
|
||||
'''
|
||||
|
||||
|
||||
@@ -284,6 +402,8 @@ def run(scenario, expected, dns='', mode='native'):
|
||||
(binary / name).unlink()
|
||||
configs = root / 'private'
|
||||
configs.mkdir()
|
||||
if scenario == 'generic-apply-disabled':
|
||||
(root / 'prepared').touch()
|
||||
if scenario.startswith('dns-stale'):
|
||||
if scenario == 'dns-stale-symlink':
|
||||
(configs / 'dns-ready').symlink_to(configs / 'missing')
|
||||
@@ -304,6 +424,25 @@ def run(scenario, expected, dns='', mode='native'):
|
||||
text = text.replace('config_dir=/run/zapret-autotune-ss', f'config_dir={configs}')
|
||||
text = text.replace('/usr/local/bin/zapret-autotune', str(binary / 'autotune'))
|
||||
text = text.replace('/opt/zapret2/init.d/sysv/zapret2', str(binary / 'zapret2'))
|
||||
if scenario.startswith('generic-apply-'):
|
||||
launch = ' launch_controller managed resource-apply'
|
||||
assert launch in text
|
||||
text = text.replace(launch, f' {binary / "autotune"} apply-launch\n' + launch)
|
||||
if scenario == 'generic-apply-launch-fail':
|
||||
text = text.replace(' launch_controller managed resource-apply', ' false || return 78\n launch_controller managed resource-apply')
|
||||
if scenario == 'generic-apply-ack-fail':
|
||||
broken_helper = root / 'ack-helper.sh'
|
||||
broken_helper.write_text(helper.read_text().replace(' ack_publish ||', ' false && ack_publish ||'))
|
||||
text = text.replace(str(helper), str(broken_helper))
|
||||
if scenario == 'generic-apply-cleanup':
|
||||
text = text.replace('cleanup_session || { echo', "cleanup_session() { return 78; }\n cleanup_session || { echo", 1)
|
||||
if scenario.endswith('-race'):
|
||||
barrier = f'{binary / "autotune"} qualification-race-barrier "$monitor_pid"\n'
|
||||
loop = 'while dns_alive && kill -0 "$server_pid" && kill -0 "$local_pid" && kill -0 "$monitor_pid"; do'
|
||||
assert loop in text
|
||||
# The apply child has its own wait loop before this monitor loop.
|
||||
before, after = text.rsplit(loop, 1)
|
||||
text = before + barrier + loop + after
|
||||
if realtime:
|
||||
# Private fixture scaling only: production has no configurable grace.
|
||||
text = text.replace('$(date +%s) + 80', '$(date +%s) + 12')
|
||||
@@ -317,6 +456,17 @@ def run(scenario, expected, dns='', mode='native'):
|
||||
env['ZAPRET_AUTOTUNE_RESOURCE_CONFIG'] = '/operator/resources.json'
|
||||
else:
|
||||
env.pop('ZAPRET_AUTOTUNE_RESOURCE_CONFIG', None)
|
||||
env.pop('ZAPRET_AUTOTUNE_QUALIFY_CONFIG', None)
|
||||
env.pop('ZAPRET_AUTOTUNE_STRATEGY_CONFIG', None)
|
||||
env.pop('ZAPRET_AUTOTUNE_RESOURCE_APPLY', None)
|
||||
env.pop('ZAPRET_AUTOTUNE_RESOURCE_STRATEGY_IDS', None)
|
||||
if scenario.startswith('generic-apply-') and scenario != 'generic-apply-disabled':
|
||||
env['ZAPRET_AUTOTUNE_RESOURCE_APPLY'] = '1'
|
||||
env['ZAPRET_AUTOTUNE_RESOURCE_STRATEGY_IDS'] = 's2,s1'
|
||||
if scenario.startswith('qualify-'):
|
||||
env['ZAPRET_AUTOTUNE_QUALIFY_CONFIG'] = '/operator/resources.json'
|
||||
if scenario != 'qualify-missing':
|
||||
env['ZAPRET_AUTOTUNE_STRATEGY_CONFIG'] = '/operator/strategies.json'
|
||||
if mode is None:
|
||||
env.pop('ZAPRET_AUTOTUNE_DNS_MODE', None)
|
||||
else:
|
||||
@@ -341,7 +491,12 @@ def run(scenario, expected, dns='', mode='native'):
|
||||
peer.wait(timeout=2)
|
||||
events = (root / 'events').read_text().splitlines() if (root / 'events').exists() else []
|
||||
assert result.returncode == expected, (scenario, result.returncode, result.stderr, events)
|
||||
assert b'Illegal number' not in result.stderr, (scenario, result.stderr)
|
||||
assert b'ab' * 32 not in result.stdout + result.stderr
|
||||
if scenario in ('generic-apply-blocked', 'generic-apply-abrupt', 'generic-apply-cleanup'):
|
||||
code = {'generic-apply-blocked': 78, 'generic-apply-abrupt': 23, 'generic-apply-cleanup': 0}[scenario]
|
||||
assert f'controller-startup: apply_wait_exit={code}\n'.encode() in result.stderr, result.stderr
|
||||
assert (b'controller-startup: apply_cleanup=failed\n' in result.stderr) == (scenario == 'generic-apply-cleanup'), result.stderr
|
||||
for pid in ((root / 'children').read_text().splitlines() if (root / 'children').exists() else []):
|
||||
assert not Path(f'/proc/{pid}').exists(), (scenario, 'child not joined', pid)
|
||||
if scenario == 'orphans':
|
||||
@@ -357,7 +512,7 @@ def run(scenario, expected, dns='', mode='native'):
|
||||
# Every activation and stop must immediately follow successful reconcile.
|
||||
for i, event in enumerate(events):
|
||||
if event in ('start', 'stop'):
|
||||
assert events[i - 1] == 'reconcile', (scenario, events)
|
||||
assert events[i - 1] == 'reconcile' or event == 'start' and events[i - 1] == 'resource-applied', (scenario, events)
|
||||
foreign = scenario in ('dns-symlink', 'dns-preack-foreign', 'dns-replace', 'dns-replace-symlink', 'dns-stale', 'dns-stale-symlink')
|
||||
if foreign:
|
||||
ready = configs / 'dns-ready'
|
||||
@@ -371,6 +526,15 @@ def run(scenario, expected, dns='', mode='native'):
|
||||
assert not (configs / 'dns-ready').exists(), (scenario, 'stale DNS readiness')
|
||||
if mode == 'native':
|
||||
assert 'dns-start' not in events
|
||||
if scenario.startswith('generic-apply-') and 'blocked-window 1' in events:
|
||||
assert events.count('apply-launch') == 1, (scenario, events)
|
||||
assert int((root / 'elapsed').read_text()) == 180, (scenario, events)
|
||||
assert events.count('dns-start') == events.count('ss-server') == events.count('ss-local') == 1, (scenario, events)
|
||||
assert 'start' not in events and not any(e.startswith('monitor ') for e in events), (scenario, events)
|
||||
assert 'cooldown 15' not in events and 'cooldown 30' not in events, (scenario, events)
|
||||
assert events[events.index('blocked-window 1'):] == ['blocked-window 1', 'blocked-window 2', 'blocked-window 3'], (scenario, events)
|
||||
if scenario != 'generic-apply-dns-loss':
|
||||
assert events.index('dns-stop') < events.index('blocked-window 1'), (scenario, events)
|
||||
return events
|
||||
|
||||
|
||||
@@ -401,6 +565,32 @@ assert events.count('generic-pause') == 1, events
|
||||
assert events.count('dns-start') == events.count('dns-stop') == 4, events
|
||||
events = run('generic-unsafe', 0)
|
||||
assert 'start' not in events and events.count('generic-pause') == 1, events
|
||||
events = run('generic-apply-pass', 0, mode='doh')
|
||||
assert events.count('resource-apply') == 1 and events.count('start') == 1, events
|
||||
assert events.index('resource-applied') < events.index('start') < events.index('monitor 1'), events
|
||||
events = run('generic-apply-cancel', 0, mode='doh')
|
||||
assert 'resource-restored' in events and 'start' not in events and 'monitor 1' not in events, events
|
||||
assert events.index('resource-restored') < events.index('stop') < events.index('dns-stop'), events
|
||||
events = run('generic-apply-blocked', 0, mode='doh')
|
||||
assert events.count('resource-apply') == 1 and 'blocked-window 3' in events, events
|
||||
for scenario, code in (('generic-apply-abrupt', 0), ('generic-apply-cleanup', 78)):
|
||||
events = run(scenario, code, mode='doh')
|
||||
assert 'start' not in events and 'monitor 1' not in events and 'resource-applied' not in events, events
|
||||
for scenario in ('signal', 'dns-loss', 'ss-loss', 'launch-fail', 'ack-fail'):
|
||||
events = run('generic-apply-' + scenario, 78 if scenario == 'ack-fail' else 0, mode='doh')
|
||||
assert 'blocked-window 3' in events and 'resource-applied' not in events, events
|
||||
assert events.count('resource-apply') == (0 if scenario in ('launch-fail', 'ack-fail') else 1), events
|
||||
for scenario in ('monitor-restart', 'ss-restart', 'dns-restart'):
|
||||
events = run('generic-apply-' + scenario, 0, mode='doh')
|
||||
assert events.count('resource-apply') == events.count('apply-launch') == 1, events
|
||||
assert events.count('start') == events.count('stop') == 2 and 'monitor 2' in events, events
|
||||
assert events.count('dns-start') == 2 and events.count('dns-stop') == (1 if scenario == 'dns-restart' else 2), events
|
||||
# Independent fresh PID 1 invocations reset the RAM latch; flag=0 still repairs
|
||||
# prepared bytes by reconcile, without launching apply.
|
||||
events = run('generic-apply-pass', 0, mode='doh')
|
||||
assert events.count('resource-apply') == 1, events
|
||||
events = run('generic-apply-disabled', 0, mode='doh')
|
||||
assert 'resource-apply' not in events and 'start' in events, events
|
||||
for scenario in ('generic-budget-monitor', 'generic-budget-reconcile', 'generic-budget-ack'):
|
||||
events = run(scenario, 78, mode='doh')
|
||||
assert events.count('start') == 1 and 'stop' not in events, (scenario, events)
|
||||
@@ -432,6 +622,25 @@ for scenario in ('dns-term', 'dns-int'):
|
||||
events = run(scenario, 0, mode='doh')
|
||||
assert events == ['dns-start', 'dns-stop'], events
|
||||
|
||||
for scenario in ('qualify-pass', 'qualify-negative', 'qualify-blocked', 'qualify-cancel'):
|
||||
events = run(scenario, 0, mode='doh')
|
||||
assert events.count('qualify') == 1 and not any(e.startswith('monitor ') for e in events), events
|
||||
assert 'start' not in events and 'stop' not in events, events
|
||||
assert events.index('dns-ready') < events.index('qualify') < events.index('dns-stop'), events
|
||||
if scenario != 'qualify-blocked':
|
||||
assert events.index('qualification-restored') < events.index('dns-stop'), events
|
||||
events = run('qualify-missing', 64, mode='doh')
|
||||
assert not events, events
|
||||
for scenario, code in (('qualify-blocked-race', 0), ('qualify-unsafe-race', 78), ('qualify-unsafe', 78)):
|
||||
events = run(scenario, code, mode='doh')
|
||||
assert events.count('qualify') == 1 and 'start' not in events and 'stop' not in events, events
|
||||
if scenario.endswith('-race'):
|
||||
assert events.count('term-before-parent-reap') == 1, events
|
||||
if code == 0:
|
||||
assert events.index('term-before-parent-reap') < events.index('completion-checked') < events.index('dns-stop'), events
|
||||
else:
|
||||
assert 'completion-checked' not in events, events
|
||||
|
||||
|
||||
def check_watchdog_traps(shell):
|
||||
"""Same bounded_wait from normal flow, plain EXIT, and TERM -> EXIT traps."""
|
||||
@@ -476,10 +685,47 @@ esac
|
||||
assert not Path(f'/proc/{pid}').exists(), (shell, mode, 'orphan sleeper', pid)
|
||||
|
||||
|
||||
def check_session_deadline(shell):
|
||||
"""Startup sweep has no aggregate deadline; termination must keep its cap."""
|
||||
entrypoint = (ROOT / 'managed-target-entrypoint.sh').read_text()
|
||||
cleanup_session = entrypoint[entrypoint.index('cleanup_session() {'):entrypoint.index('capture_monitor_session() {')]
|
||||
for deadline, live_until, expected, elapsed in (('', 102, 0, 2), ('180', 102, 0, 2), ('101', 110, 78, 1)):
|
||||
with tempfile.TemporaryDirectory(prefix='session-deadline-') as tmp:
|
||||
script = Path(tmp) / 'check.sh'
|
||||
script.write_text(f'''set -eu
|
||||
generic_mode=1
|
||||
generic_cleanup_deadline='{deadline}'
|
||||
monitor_sid=42
|
||||
clock=100
|
||||
date() {{ printf '%s\\n' "$clock"; }}
|
||||
pause() {{ clock=$((clock + $1)); }}
|
||||
signal_session() {{
|
||||
[ "$1" = 42 ] || return 78
|
||||
case "$2" in
|
||||
TERM|KILL) ;;
|
||||
CHECK) session_live=1; [ "$clock" -lt {live_until} ] || session_live=0 ;;
|
||||
*) return 78 ;;
|
||||
esac
|
||||
}}
|
||||
{cleanup_session}
|
||||
code=0
|
||||
cleanup_session || code=$?
|
||||
[ "$code" -eq {expected} ]
|
||||
[ "$clock" -eq {100 + elapsed} ]
|
||||
[ "$generic_cleanup_deadline" = '{deadline}' ]
|
||||
''')
|
||||
result = subprocess.run([*shell, str(script)], capture_output=True, timeout=2)
|
||||
assert result.returncode == 0, (shell, deadline, result.stderr)
|
||||
expected_stderr = b'error: owned monitor session survived cleanup\n' if expected else b''
|
||||
assert result.stderr == expected_stderr, (shell, deadline, result.stderr)
|
||||
|
||||
|
||||
check_watchdog_traps(['/bin/sh'])
|
||||
check_session_deadline(['/bin/sh'])
|
||||
busybox = shutil.which('busybox')
|
||||
if busybox:
|
||||
check_watchdog_traps([busybox, 'ash'])
|
||||
check_session_deadline([busybox, 'ash'])
|
||||
for pid, seconds in (('1', '5'), ('0', '5'), ('-1', '5'), ('42', '0'), ('42', 'bad')):
|
||||
result = subprocess.run(['/bin/sh', str(ROOT / 'managed-target-supervision.sh'), '--watchdog', pid, seconds, '/'], capture_output=True, timeout=2)
|
||||
assert result.returncode == 64
|
||||
|
||||
@@ -19,6 +19,11 @@ services:
|
||||
ZAPRET_AUTOTUNE_DOH_TIMEOUT_SEC: ${ZAPRET_AUTOTUNE_DOH_TIMEOUT_SEC-5}
|
||||
ZAPRET_AUTOTUNE_YOUTUBE_SERVICE: ${ZAPRET_AUTOTUNE_YOUTUBE_SERVICE-0}
|
||||
ZAPRET_AUTOTUNE_RESOURCE_CONFIG: ${ZAPRET_AUTOTUNE_RESOURCE_CONFIG-}
|
||||
ZAPRET_AUTOTUNE_RESOURCE_APPLY: ${ZAPRET_AUTOTUNE_RESOURCE_APPLY-0}
|
||||
ZAPRET_AUTOTUNE_RESOURCE_STRATEGY_IDS: ${ZAPRET_AUTOTUNE_RESOURCE_STRATEGY_IDS-}
|
||||
# INTRUSIVE isolated startup batch. Never route working traffic here.
|
||||
ZAPRET_AUTOTUNE_QUALIFY_CONFIG: ${ZAPRET_AUTOTUNE_QUALIFY_CONFIG-}
|
||||
ZAPRET_AUTOTUNE_STRATEGY_CONFIG: ${ZAPRET_AUTOTUNE_STRATEGY_CONFIG-}
|
||||
ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE: ${ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE-cold}
|
||||
ZAPRET_AUTOTUNE_VIDEO_ID: ${ZAPRET_AUTOTUNE_VIDEO_ID-dQw4w9WgXcQ}
|
||||
ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS: '${ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS:-}'
|
||||
@@ -42,7 +47,8 @@ services:
|
||||
- /run/zapret-autotune-ss:rw,noexec,nosuid,nodev,size=1m,mode=0700,uid=0,gid=0
|
||||
networks: [target]
|
||||
restart: "no"
|
||||
stop_grace_period: 240s
|
||||
# Generic mode uses 90s; retain the legacy adapter's longer default.
|
||||
stop_grace_period: ${ZAPRET_AUTOTUNE_STOP_GRACE_PERIOD-240s}
|
||||
sysctls:
|
||||
net.ipv4.ip_unprivileged_port_start: "0"
|
||||
networks:
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
// Package adapter defines the closed internal contract used by compiled-in
|
||||
// service adapters. It is not the public resource-check configuration API.
|
||||
package adapter
|
||||
|
||||
import "time"
|
||||
|
||||
type Binding struct {
|
||||
Kind string `json:"kind"`
|
||||
Version int `json:"version"`
|
||||
PolicySHA256 string `json:"policy_sha256"`
|
||||
}
|
||||
|
||||
// Resource and Plan are ephemeral; neither private inputs nor policy caps belong
|
||||
// in evidence. Only core-validated, measured facts can become a sealed proof.
|
||||
type Resource struct {
|
||||
URL string `json:"-"`
|
||||
Host string `json:"-"`
|
||||
Kind string `json:"-"`
|
||||
}
|
||||
|
||||
type Plan struct {
|
||||
Hosts []string `json:"-"`
|
||||
Protocols [2]string `json:"-"`
|
||||
WarmupLimit int `json:"-"`
|
||||
WarmupTimeout time.Duration `json:"-"`
|
||||
Samples int `json:"-"`
|
||||
MinBytes int64 `json:"-"`
|
||||
}
|
||||
|
||||
type Module interface {
|
||||
Binding() Binding
|
||||
DiscoverResponse([]byte) ([]Resource, error)
|
||||
ValidateHosts([]string) error
|
||||
PlanQualification([]string, []Resource, time.Time) (Plan, error)
|
||||
}
|
||||
@@ -0,0 +1,29 @@
|
||||
# Internal YouTube adapter
|
||||
|
||||
The compiled-in `youtube/1` module parses bounded Player JSON and supplies the
|
||||
ordinary signed MP4 qualification policy and ordered host/TLS sample plan.
|
||||
The existing YouTube execution paths select it explicitly through the core's
|
||||
kind/version registry. Unsupported or absent modules fail closed.
|
||||
|
||||
The core retains origin validation, HTTPS/SNI/certificate checks, transport
|
||||
framing/body bounds, retries/cancellation, worker ownership, cleanup/join and
|
||||
proof sealing. Module output is not evidence or permission to claim PASS.
|
||||
Signed URLs remain ephemeral and cannot serialize through the internal contract.
|
||||
|
||||
New adaptive proof v2 binds the module kind, version and immutable policy digest
|
||||
inside its producer seal. Unbound proof v1 explicitly means the historical
|
||||
YouTube v1 policy: fresh trusted stored proofs retain their original apply
|
||||
eligibility; expired proofs reconstruct history but cannot authorize fresh apply.
|
||||
Only new producer/persistence writes require sealed module-bound v2 evidence.
|
||||
Cold historical evidence is unchanged. Discovery entrypoints preflight the module
|
||||
before requests; absence/mismatched identity or policy is a fatal configuration
|
||||
failure, not unavailable inventory or a no-strategy cooldown.
|
||||
|
||||
Legacy routing remains SOCKS5h with encrypted-resolver ownership guards and
|
||||
origin/SNI/certificate/TLS checks, not generic all-answer approval/IP pinning.
|
||||
Full request-policy capabilities and parity are prerequisites to generic cutover.
|
||||
|
||||
This is a real internal legacy-execution integration, not a fully generic
|
||||
controller, public v3 configuration, native addon, runtime plugin or readiness
|
||||
claim. Another compiled-in module can implement the closed internal contract;
|
||||
public GET/HEAD v1/v2 configuration remains unchanged.
|
||||
@@ -0,0 +1,120 @@
|
||||
// Package youtube contains pure service semantics, not transport or worker ownership.
|
||||
package youtube
|
||||
|
||||
import (
|
||||
"crypto/sha256"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"mime"
|
||||
"net/url"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"zapret-autotune/internal/adapter"
|
||||
)
|
||||
|
||||
const Kind = "youtube"
|
||||
const Version = 1
|
||||
const InventoryTTL = 20 * time.Minute
|
||||
|
||||
// Immutable v1 policy. Changes require another module version, not a new meaning
|
||||
// for existing evidence. Legacy proof v1 uses this same historical policy.
|
||||
const policyV1 = "youtube/1;ggc>=3;roots>=2;prefixes>=2;ordinary-signed-mp4;fresh>120s;inventory>20m;tls12,tls13;warmup<=10,120s;steady=3/protocol;min=1024"
|
||||
|
||||
type Binding = adapter.Binding
|
||||
|
||||
type Module struct{}
|
||||
|
||||
func (Module) Binding() Binding {
|
||||
return Binding{Kind: Kind, Version: Version, PolicySHA256: fmt.Sprintf("%x", sha256.Sum256([]byte(policyV1)))}
|
||||
}
|
||||
|
||||
// Resource is ephemeral: never serialize signed URLs or inventory metadata.
|
||||
type Resource = adapter.Resource
|
||||
|
||||
// DiscoverResponse preserves player format order. The core must validate every
|
||||
// returned origin before selecting or fetching it. SABR is not supported.
|
||||
func (Module) DiscoverResponse(body []byte) ([]Resource, error) {
|
||||
type format struct {
|
||||
URL string `json:"url"`
|
||||
MimeType string `json:"mimeType"`
|
||||
}
|
||||
var p struct {
|
||||
PlayabilityStatus struct {
|
||||
Status string `json:"status"`
|
||||
} `json:"playabilityStatus"`
|
||||
StreamingData struct {
|
||||
Formats []format `json:"formats"`
|
||||
AdaptiveFormats []format `json:"adaptiveFormats"`
|
||||
} `json:"streamingData"`
|
||||
}
|
||||
if len(body) > 1<<20 || json.Unmarshal(body, &p) != nil || p.PlayabilityStatus.Status != "OK" {
|
||||
return nil, errors.New("invalid playable metadata")
|
||||
}
|
||||
var resources []Resource
|
||||
for _, f := range append(p.StreamingData.Formats, p.StreamingData.AdaptiveFormats...) {
|
||||
u, err := url.Parse(f.URL)
|
||||
if err != nil || !strings.HasSuffix(u.Hostname(), ".googlevideo.com") {
|
||||
continue
|
||||
}
|
||||
kind, _, err := mime.ParseMediaType(f.MimeType)
|
||||
if err == nil && (kind == "video/mp4" || kind == "audio/mp4") {
|
||||
resources = append(resources, Resource{URL: f.URL, Host: u.Hostname(), Kind: kind})
|
||||
}
|
||||
}
|
||||
return resources, nil
|
||||
}
|
||||
|
||||
// ValidateHosts checks service diversity; DNS/origin policy remains in the core.
|
||||
func (Module) ValidateHosts(hosts []string) error {
|
||||
seen, roots, prefixes := map[string]bool{}, map[string]bool{}, map[string]bool{}
|
||||
for _, host := range hosts {
|
||||
prefix, root, ok := strings.Cut(strings.TrimSuffix(host, ".googlevideo.com"), "---sn-")
|
||||
if !ok || !strings.HasSuffix(host, ".googlevideo.com") || host != strings.ToLower(host) || seen[host] || root == "" || len(prefix) < 3 || !strings.HasPrefix(prefix, "rr") || strings.Trim(prefix[2:], "0123456789") != "" {
|
||||
return errors.New("insufficient representative GGC hosts")
|
||||
}
|
||||
seen[host], roots[strings.Split(root, "-")[0]], prefixes[prefix] = true, true, true
|
||||
}
|
||||
if len(hosts) < 3 || len(roots) < 2 || len(prefixes) < 2 {
|
||||
return errors.New("insufficient representative GGC hosts")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
type Plan = adapter.Plan
|
||||
|
||||
// PlanQualification grants no PASS capability. Core validates origins first,
|
||||
// executes the ordered schedule, validates measured facts, and seals after join.
|
||||
func (m Module) PlanQualification(hosts []string, resources []Resource, now time.Time) (Plan, error) {
|
||||
p := Plan{Hosts: append([]string(nil), hosts...), Protocols: [2]string{"tls12", "tls13"}, WarmupLimit: 10, WarmupTimeout: 120 * time.Second, Samples: 3, MinBytes: 1024}
|
||||
if err := m.ValidateHosts(hosts); err != nil {
|
||||
return Plan{}, err
|
||||
}
|
||||
seen := map[string]bool{}
|
||||
for _, r := range resources {
|
||||
if !SignedSampleFresh(r, now, 120*time.Second) {
|
||||
return Plan{}, errors.New("insufficient fresh signed media samples")
|
||||
}
|
||||
seen[r.Host] = true
|
||||
}
|
||||
for _, host := range hosts {
|
||||
if !seen[host] {
|
||||
return Plan{}, errors.New("missing representative sample")
|
||||
}
|
||||
}
|
||||
return p, nil
|
||||
}
|
||||
|
||||
// SignedSampleFresh checks service semantics only, not permission to fetch.
|
||||
// The core independently validates HTTPS origins before calling this helper.
|
||||
func SignedSampleFresh(r Resource, now time.Time, ttl time.Duration) bool {
|
||||
u, err := url.Parse(r.URL)
|
||||
if err != nil || u.Path != "/videoplayback" || (r.Kind != "video/mp4" && r.Kind != "audio/mp4") {
|
||||
return false
|
||||
}
|
||||
q := u.Query()
|
||||
expire, err := strconv.ParseInt(q.Get("expire"), 10, 64)
|
||||
return err == nil && expire > now.Add(ttl).Unix() && (q.Get("sig") != "" || q.Get("signature") != "" || q.Get("lsig") != "")
|
||||
}
|
||||
@@ -0,0 +1,90 @@
|
||||
package youtube
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestDiscoverResponse(t *testing.T) {
|
||||
m := Module{}
|
||||
resources, err := m.DiscoverResponse([]byte(`{"playabilityStatus":{"status":"OK"},"streamingData":{"formats":[{"url":"https://rr1.unit.googlevideo.com/videoplayback?sig=secret","mimeType":"audio/mp4"}],"adaptiveFormats":[{"url":"https://rr2.unit.googlevideo.com/videoplayback?sig=secret","mimeType":"video/mp4; codecs=avc1"}]}}`))
|
||||
if err != nil || len(resources) != 2 || resources[0].Kind != "audio/mp4" || resources[1].Kind != "video/mp4" {
|
||||
t.Fatal(resources, err)
|
||||
}
|
||||
resources, err = m.DiscoverResponse([]byte(`{"playabilityStatus":{"status":"OK"},"streamingData":{"serverAbrStreamingUrl":"https://rr1.unit.googlevideo.com/sabr?sig=secret"}}`))
|
||||
if err != nil || len(resources) != 0 {
|
||||
t.Fatal("SABR fabricated ordinary readiness")
|
||||
}
|
||||
for _, body := range []string{"{", strings.Repeat(" ", (1<<20)+1), `{"playabilityStatus":{"status":"ERROR"}}`} {
|
||||
if _, err := m.DiscoverResponse([]byte(body)); err == nil || strings.Contains(err.Error(), "secret") {
|
||||
t.Fatal("invalid metadata accepted or disclosed")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestQualificationPlan(t *testing.T) {
|
||||
m, now := Module{}, time.Unix(1800000000, 0)
|
||||
if m.Binding().PolicySHA256 != "2cfc521a9784c574ceee16430aa485b9c0e9e64795486a23df5117b6265d89d5" {
|
||||
t.Fatal("immutable youtube/1 policy changed: introduce a new version")
|
||||
}
|
||||
hosts := []string{"rr1---sn-alpha.googlevideo.com", "rr2---sn-beta.googlevideo.com", "rr5---sn-gamma.googlevideo.com"}
|
||||
var resources []Resource
|
||||
for _, host := range hosts {
|
||||
resources = append(resources, Resource{URL: fmt.Sprintf("https://%s/videoplayback?expire=%d&sig=secret", host, now.Unix()+121), Host: host, Kind: "video/mp4"})
|
||||
}
|
||||
p, err := m.PlanQualification(hosts, resources, now)
|
||||
if err != nil || strings.Join(p.Hosts, ",") != strings.Join(hosts, ",") || p.Protocols != [2]string{"tls12", "tls13"} || p.Samples != 3 || p.MinBytes != 1024 || p.WarmupLimit != 10 || p.WarmupTimeout != 120*time.Second {
|
||||
t.Fatal(p, err)
|
||||
}
|
||||
for _, mode := range []string{"expired", "unsigned", "sabr", "missing", "diversity"} {
|
||||
t.Run(mode, func(t *testing.T) {
|
||||
r, h := append([]Resource(nil), resources...), append([]string(nil), hosts...)
|
||||
switch mode {
|
||||
case "expired":
|
||||
r[0].URL = strings.Replace(r[0].URL, "1800000121", "1800000120", 1)
|
||||
case "unsigned":
|
||||
r[0].URL = strings.Split(r[0].URL, "&sig=")[0]
|
||||
case "sabr":
|
||||
r[0].URL = strings.Replace(r[0].URL, "videoplayback", "sabr", 1)
|
||||
case "missing":
|
||||
r = r[:2]
|
||||
case "diversity":
|
||||
h[1] = h[0]
|
||||
}
|
||||
if _, err := m.PlanQualification(h, r, now); err == nil {
|
||||
t.Fatal("invalid sample accepted")
|
||||
}
|
||||
})
|
||||
}
|
||||
encoded, err := json.Marshal(struct {
|
||||
Resources []Resource
|
||||
Plan Plan
|
||||
}{resources, p})
|
||||
if err != nil || strings.Contains(string(encoded), "secret") || strings.Contains(string(encoded), hosts[0]) {
|
||||
t.Fatal("ephemeral data serialized", string(encoded), err)
|
||||
}
|
||||
}
|
||||
|
||||
func TestSignedSampleFresh(t *testing.T) {
|
||||
now := time.Unix(1800000000, 900000000)
|
||||
for _, tc := range []struct {
|
||||
query string
|
||||
want bool
|
||||
}{
|
||||
{"expire=1800000121&sig=secret", true},
|
||||
{"expire=1800000120&sig=secret", false},
|
||||
{"expire=1800000121&signature=secret", true},
|
||||
{"expire=1800000121&lsig=secret", true},
|
||||
{"expire=1800000121", false},
|
||||
{"expire=invalid&sig=secret", false},
|
||||
{"expire=9223372036854775808&sig=secret", false},
|
||||
} {
|
||||
r := Resource{URL: "https://rr1---sn-alpha.googlevideo.com/videoplayback?" + tc.query, Kind: "video/mp4"}
|
||||
if SignedSampleFresh(r, now, 120*time.Second) != tc.want {
|
||||
t.Fatal("signed expiry boundary changed")
|
||||
}
|
||||
}
|
||||
}
|
||||
+113
-20
@@ -18,6 +18,25 @@ dns_mode=$(/usr/local/bin/zapret-autotune dns-forwarder mode) || exit 64
|
||||
dns_address=$(/usr/local/bin/zapret-autotune dns-forwarder address) || exit 64
|
||||
generic_mode=0
|
||||
[ -z "${ZAPRET_AUTOTUNE_RESOURCE_CONFIG-}" ] || generic_mode=1
|
||||
qualification_mode=0
|
||||
qualification_attempted=0
|
||||
# RAM-only authorization: a new PID 1 may attempt again, never a runtime retry.
|
||||
resource_apply_state=unattempted
|
||||
if [ -n "${ZAPRET_AUTOTUNE_QUALIFY_CONFIG-}${ZAPRET_AUTOTUNE_STRATEGY_CONFIG-}" ]; then
|
||||
[ -n "${ZAPRET_AUTOTUNE_QUALIFY_CONFIG-}" ] && [ -n "${ZAPRET_AUTOTUNE_STRATEGY_CONFIG-}" ] || {
|
||||
echo 'error: isolated qualification requires both config and strategy paths' >&2; exit 64;
|
||||
}
|
||||
qualification_mode=1
|
||||
generic_mode=1
|
||||
echo 'INTRUSIVE ISOLATED qualification target: do not route working traffic here' >&2
|
||||
fi
|
||||
case "${ZAPRET_AUTOTUNE_RESOURCE_APPLY-0}" in
|
||||
0|'') ;;
|
||||
1) [ "$generic_mode" -eq 1 ] && [ "$qualification_mode" -eq 0 ] && [ -n "${ZAPRET_AUTOTUNE_RESOURCE_CONFIG-}" ] || {
|
||||
echo 'error: resource apply requires generic config and excludes qualification' >&2; exit 64;
|
||||
} ;;
|
||||
*) echo 'error: invalid resource apply flag' >&2; exit 64 ;;
|
||||
esac
|
||||
case "$dns_mode" in doh|native) ;; *) exit 64 ;; esac
|
||||
config_dir=/run/zapret-autotune-ss
|
||||
awk '$2 == "/run/zapret-autotune-ss" && $3 == "tmpfs" { found=1 } END { exit !found }' /proc/mounts || {
|
||||
@@ -94,7 +113,7 @@ cleanup_session() {
|
||||
for session_signal in TERM KILL; do
|
||||
signal_session "$monitor_sid" "$session_signal" || return 78
|
||||
session_deadline=$(($(date +%s) + 3))
|
||||
if [ "$generic_mode" -eq 1 ] && [ "$session_deadline" -gt "$generic_cleanup_deadline" ]; then
|
||||
if [ "$generic_mode" -eq 1 ] && [ -n "$generic_cleanup_deadline" ] && [ "$session_deadline" -gt "$generic_cleanup_deadline" ]; then
|
||||
session_deadline=$generic_cleanup_deadline
|
||||
fi
|
||||
while :; do
|
||||
@@ -113,9 +132,13 @@ cleanup_session() {
|
||||
capture_monitor_session() {
|
||||
[ -z "$monitor_sid" ] || return 0
|
||||
if [ -e "$config_dir/monitor-ready" ]; then
|
||||
read -r candidate_sid < "$config_dir/monitor-ready" || return 78
|
||||
case "$candidate_sid" in ''|*[!0-9]*) return 78 ;; esac
|
||||
[ "$candidate_sid" = "$monitor_pid" ] && [ "$candidate_sid" -gt 1 ] && [ "$candidate_sid" != "$$" ] || return 78
|
||||
{ read -r candidate_sid < "$config_dir/monitor-ready"; } 2>/dev/null || {
|
||||
echo 'controller-startup: ack=read_failed' >&2; return 78;
|
||||
}
|
||||
case "$candidate_sid" in ''|*[!0-9]*) echo 'controller-startup: ack=invalid_pid' >&2; return 78 ;; esac
|
||||
[ "$candidate_sid" = "$monitor_pid" ] && [ "$candidate_sid" -gt 1 ] && [ "$candidate_sid" != "$$" ] || {
|
||||
echo 'controller-startup: ack=identity_mismatch' >&2; return 78;
|
||||
}
|
||||
monitor_sid=$candidate_sid
|
||||
elif proc_identity "$monitor_pid" && [ "$proc_sid" = "$monitor_pid" ] && [ "$proc_sid" -gt 1 ]; then
|
||||
monitor_sid=$proc_sid
|
||||
@@ -196,9 +219,15 @@ cleanup_cycle() {
|
||||
if [ -n "$monitor_pid" ]; then
|
||||
capture_monitor_session || cleanup_code=78
|
||||
kill -TERM "$monitor_pid" 2>/dev/null || true
|
||||
bounded_wait "$monitor_pid" 45
|
||||
if [ "$qualification_mode" -eq 1 ]; then bounded_wait "$monitor_pid" 75; else bounded_wait "$monitor_pid" 45; fi
|
||||
# Cancellation must restore safely even when SS failure requested retry.
|
||||
case "$waited_code" in 0|143) ;; *) echo "error: monitor cleanup status $waited_code" >&2; cleanup_code=78 ;; esac
|
||||
if [ "$qualification_mode" -eq 1 ]; then
|
||||
# Exit 78 is an outcome only with a producer-owned completed-cleanup
|
||||
# ACK and a fresh baseline/worker/nft check, while SS+DNS still live.
|
||||
cleanup_command 10 /usr/local/bin/zapret-autotune qualification-completion-check "$monitor_pid" "$waited_code" || cleanup_code=78
|
||||
else
|
||||
case "$waited_code" in 0|143) ;; *) echo "error: monitor cleanup status $waited_code" >&2; cleanup_code=78 ;; esac
|
||||
fi
|
||||
[ "$join_timed_out" -eq 0 ] || { echo 'error: monitor join timeout' >&2; cleanup_code=78; }
|
||||
# SIGTERM may have arrived before the launch acknowledgement was read.
|
||||
capture_monitor_session || cleanup_code=78
|
||||
@@ -309,7 +338,47 @@ start_dns() {
|
||||
pause 1
|
||||
done
|
||||
}
|
||||
launch_controller() {
|
||||
# Only an ACK retained from our previous launch is owned stale state.
|
||||
# Unknown files (including dangling links) are evidence, not debris.
|
||||
if [ -e "$config_dir/monitor-ready" ] || [ -L "$config_dir/monitor-ready" ]; then
|
||||
if [ -z "${monitor_ready_identity-}" ] || [ -L "$config_dir/monitor-ready" ] ||
|
||||
[ ! -f "$config_dir/monitor-ready" ] ||
|
||||
[ "$(stat -c '%d:%i:%u:%a:%h' "$config_dir/monitor-ready")" != "$monitor_ready_identity" ] ||
|
||||
! { read -r stale_pid < "$config_dir/monitor-ready"; } 2>/dev/null ||
|
||||
[ "$stale_pid" != "${monitor_ready_pid-}" ]; then
|
||||
echo 'controller-startup: launch=ack_conflict' >&2
|
||||
return 78
|
||||
fi
|
||||
rm -f "$config_dir/monitor-ready" || return 78
|
||||
fi
|
||||
monitor_ready_identity=
|
||||
setsid /bin/sh /opt/managed-target-supervision.sh --monitor "$config_dir" /usr/local/bin/zapret-autotune "$@" 8<&- 9<&- &
|
||||
monitor_pid=$!
|
||||
monitor_ack_deadline=$(($(date +%s) + 5))
|
||||
while [ ! -e "$config_dir/monitor-ready" ]; do
|
||||
if ! kill -0 "$monitor_pid" 2>/dev/null || [ "$(date +%s)" -ge "$monitor_ack_deadline" ]; then
|
||||
if ! kill -0 "$monitor_pid" 2>/dev/null; then
|
||||
echo 'controller-startup: launch=child_gone' >&2
|
||||
else
|
||||
echo 'controller-startup: launch=ack_timeout' >&2
|
||||
fi
|
||||
signal_tree "$monitor_pid" KILL
|
||||
return 78
|
||||
fi
|
||||
done
|
||||
capture_monitor_session || return 78
|
||||
monitor_ready_identity=$(stat -c '%d:%i:%u:%a:%h' "$config_dir/monitor-ready") || return 78
|
||||
monitor_ready_pid=$monitor_pid
|
||||
if proc_identity "$monitor_pid" && [ "$proc_sid" != "$monitor_sid" ]; then
|
||||
echo 'controller-startup: launch=session_mismatch' >&2
|
||||
monitor_sid=
|
||||
return 78
|
||||
fi
|
||||
}
|
||||
run_cycle() {
|
||||
[ "$resource_apply_state" != failed ] || return 78
|
||||
[ "$qualification_attempted" -eq 0 ] || return 78
|
||||
start_dns || return $?
|
||||
set -- ss-server -c "$config_dir/server.json"
|
||||
# DNS is not a secret. Native -d avoids assuming support for a JSON DNS key.
|
||||
@@ -338,8 +407,24 @@ fi
|
||||
# Never activate pending or unknown bytes. Any reconciliation failure is fatal,
|
||||
# not a transient (75) restart request. Reload restores known bytes first.
|
||||
timeout -k 5 45 /usr/local/bin/zapret-autotune managed reconcile || return 78
|
||||
if [ "$qualification_mode" -eq 0 ]; then
|
||||
zapret_started=1
|
||||
if [ "${ZAPRET_AUTOTUNE_RESOURCE_APPLY-0}" = 1 ] && [ "$resource_apply_state" = unattempted ]; then
|
||||
resource_apply_state=failed
|
||||
launch_controller managed resource-apply --resource-config "$ZAPRET_AUTOTUNE_RESOURCE_CONFIG" || return 78
|
||||
while dns_alive && kill -0 "$server_pid" && kill -0 "$local_pid" && kill -0 "$monitor_pid"; do pause 1; done
|
||||
{ dns_alive && kill -0 "$server_pid" && kill -0 "$local_pid"; } || return 75
|
||||
apply_code=0
|
||||
wait "$monitor_pid" || apply_code=$?
|
||||
echo "controller-startup: apply_wait_exit=$apply_code" >&2
|
||||
monitor_pid=
|
||||
cleanup_session || { echo 'controller-startup: apply_cleanup=failed' >&2; return 78; }
|
||||
[ "$apply_code" -eq 0 ] || return 78
|
||||
resource_apply_state=succeeded
|
||||
fi
|
||||
# Idempotent apply may have only verified the already committed config.
|
||||
timeout -k 5 30 /opt/zapret2/init.d/sysv/zapret2 start || return 78
|
||||
fi
|
||||
|
||||
# Private operator request, consumed only after a successful guarded transaction.
|
||||
# Never suppress qualification gates globally or race the owned monitor.
|
||||
@@ -351,27 +436,31 @@ fi
|
||||
# fork a monitor we already need to cancel: an early TERM can race interpreter
|
||||
# startup before its cancellation handler exists. Clean this cycle directly.
|
||||
{ dns_alive && kill -0 "$server_pid" 2>/dev/null && kill -0 "$local_pid" 2>/dev/null; } || return 75
|
||||
rm -f "$config_dir/monitor-ready"
|
||||
setsid /bin/sh /opt/managed-target-supervision.sh --monitor "$config_dir" /usr/local/bin/zapret-autotune managed monitor 8<&- 9<&- &
|
||||
monitor_pid=$!
|
||||
monitor_ack_deadline=$(($(date +%s) + 5))
|
||||
while [ ! -e "$config_dir/monitor-ready" ]; do
|
||||
if ! kill -0 "$monitor_pid" 2>/dev/null || [ "$(date +%s)" -ge "$monitor_ack_deadline" ]; then
|
||||
signal_tree "$monitor_pid" KILL
|
||||
return 78
|
||||
fi
|
||||
done
|
||||
capture_monitor_session || return 78
|
||||
if proc_identity "$monitor_pid" && [ "$proc_sid" != "$monitor_sid" ]; then
|
||||
monitor_sid=
|
||||
return 78
|
||||
if [ "$qualification_mode" -eq 1 ]; then
|
||||
qualification_attempted=1
|
||||
set -- qualify --config "$ZAPRET_AUTOTUNE_QUALIFY_CONFIG" --strategies "$ZAPRET_AUTOTUNE_STRATEGY_CONFIG" --isolated
|
||||
else
|
||||
set -- managed monitor
|
||||
fi
|
||||
launch_controller "$@" || return 78
|
||||
while dns_alive && kill -0 "$server_pid" && kill -0 "$local_pid" && kill -0 "$monitor_pid"; do
|
||||
pause 1
|
||||
done
|
||||
if ! kill -0 "$monitor_pid" 2>/dev/null; then
|
||||
code=0
|
||||
wait "$monitor_pid" || code=$?
|
||||
if [ "$qualification_mode" -eq 1 ]; then
|
||||
if ! timeout -k 1 10 /usr/local/bin/zapret-autotune qualification-completion-check "$monitor_pid" "$code"; then
|
||||
generic_cleanup_failed=1
|
||||
fi
|
||||
monitor_pid=
|
||||
# One finite batch only, including negative/blocked completion. No live
|
||||
# daemon and no automatic retest. Keep owned SS/DoH alive for inspection.
|
||||
echo "isolated qualification finished (status $code); target safe-idle" >&2
|
||||
cleanup_session || return 78
|
||||
while dns_alive && kill -0 "$server_pid" && kill -0 "$local_pid"; do pause 1; done
|
||||
return 78
|
||||
fi
|
||||
monitor_pid=
|
||||
echo 'error: target managed monitor exited' >&2
|
||||
case "$code" in
|
||||
@@ -417,6 +506,10 @@ while :; do
|
||||
fi
|
||||
code=78
|
||||
fi
|
||||
if [ "$resource_apply_state" = failed ]; then
|
||||
echo 'BLOCKED: resource apply failed/incomplete; operator repair/restart required' >&2
|
||||
while :; do pause 60; done
|
||||
fi
|
||||
if [ "$generic_mode" -eq 1 ] && [ "$code" -eq 78 ]; then
|
||||
echo 'BLOCKED: generic target trust/startup failure; checks paused' >&2
|
||||
pause 60
|
||||
|
||||
@@ -61,11 +61,30 @@ if [ "${1-}" = --monitor ]; then
|
||||
[ "$#" -ge 3 ] || exit 64
|
||||
config_dir=$2
|
||||
shift 2
|
||||
proc_identity "$$" && [ "$proc_sid" = "$$" ] && [ "$$" -gt 1 ] || exit 78
|
||||
proc_identity "$$" || { echo 'controller-startup: wrapper=identity_unavailable' >&2; exit 78; }
|
||||
[ "$proc_sid" = "$$" ] && [ "$$" -gt 1 ] || {
|
||||
echo 'controller-startup: wrapper=not_session_leader' >&2; exit 78;
|
||||
}
|
||||
# Acknowledge the private session before exec, so even an immediate monitor
|
||||
# crash leaves its ownership recorded. No worker can precede this handshake.
|
||||
trap 'exit 143' INT TERM
|
||||
printf '%s\n' "$$" > "$config_dir/monitor-ready"
|
||||
ack_temp=
|
||||
trap '[ -z "$ack_temp" ] || rm -f "$ack_temp"' EXIT
|
||||
ack_publish() {
|
||||
# Private same-UID directory: no untrusted writer can race the absent
|
||||
# destination check. Rename exposes complete bytes with a single link.
|
||||
[ -d "$config_dir" ] && [ ! -L "$config_dir" ] &&
|
||||
[ "$(stat -c '%u:%a' "$config_dir")" = "$(id -u):700" ] || return 1
|
||||
umask 077
|
||||
ack_temp=$(mktemp "$config_dir/.monitor-ready.XXXXXX") || return 1
|
||||
printf '%s\n' "$$" > "$ack_temp" || return 1
|
||||
[ ! -e "$config_dir/monitor-ready" ] && [ ! -L "$config_dir/monitor-ready" ] || return 1
|
||||
mv -T "$ack_temp" "$config_dir/monitor-ready" || return 1
|
||||
ack_temp=
|
||||
}
|
||||
ack_publish || { echo 'controller-startup: wrapper=ack_publish_failed' >&2; exit 78; }
|
||||
trap - EXIT
|
||||
echo 'controller-startup: wrapper=exec_pending' >&2
|
||||
exec "$@"
|
||||
elif [ "${1-}" = --watchdog ]; then
|
||||
set -eu
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
{
|
||||
"version": 2,
|
||||
"services": [
|
||||
{
|
||||
"id": "example",
|
||||
"resources": [
|
||||
{
|
||||
"id": "image",
|
||||
"group": "images",
|
||||
"kind": "image",
|
||||
"url_env": "RESOURCE_IMAGE_URL",
|
||||
"method": "GET",
|
||||
"tls": "both",
|
||||
"timeout_ms": 10000,
|
||||
"expect": {"statuses": [200], "content_type": "image/png", "min_bytes": 1, "max_bytes": 5242880}
|
||||
},
|
||||
{
|
||||
"id": "prefix",
|
||||
"group": "media",
|
||||
"kind": "mp4_range",
|
||||
"url_env": "RESOURCE_MP4_URL",
|
||||
"method": "GET",
|
||||
"tls": "both",
|
||||
"timeout_ms": 10000,
|
||||
"expect": {"statuses": [206], "content_type": "video/mp4", "min_bytes": 16, "max_bytes": 65536}
|
||||
}
|
||||
]
|
||||
}
|
||||
]
|
||||
}
|
||||
Reference in new issue
Block a user