Files
zapret-autotune/RESOURCE_PLAN.md
T
ZeusinaandGPT-6 Bot 8d43dd9363 feat: rotate generic profiles across verified alternatives
Select bounded canonical strategy allowlists at explicit PID1 startup within the same immutable authenticated registry, matrix and source. Reconstruct stored policies independently for historical replay, reconciliation and rollback; preserve freshness for new policies even when rendered bytes match. Retain legacy refusal and one-shot startup guards.

Local fixtures and independent local review passed; live native rotation acceptance remains pending. No uninterrupted rotation, new qualification or proof-store overwrite.

Co-authored-by: GPT-6 <gpt-6@openai.com>
2026-10-15 10:18:14 +03:00

27 KiB
Raw Blame History

Resource compatibility planner and isolated qualifier

plan remains proposal-only and never changes runtime or config. Explicit generic startup apply/replay and the per-PID-1 failure latch have passed bounded remote acceptance with independent privileged verification (see below). This is not automatic recovery, full concurrency acceptance or production READY. Explicit isolated qualification temporarily installs an owned NFQ worker and queue rules, but never changes legacy SQL groups, renderer or apply journals. Monitor HEALTHY, old SQL measurements and public check JSON are not proof. Without a completed owned qualification batch, compatibility is UNKNOWN and hosts are unassigned. No PASS/evidence import is exposed.

Commands require root and the existing managed-test target settings:

zapret-autotune strategy-import --strategies /private/strategies.json
zapret-autotune plan --config /private/resources.json
zapret-autotune qualify --config /private/resources.json --strategies /private/strategies.json --isolated

strategy-import requires an owned, regular, single-link 0600 file. Only definitions are accepted; duplicate keys, unknown fields and evidence fields are rejected. Example:

{
  "version": 1,
  "strategies": [
    {
      "id": "split-one",
      "args": "--payload=tls_client_hello --lua-desync=multisplit:pos=1",
      "selectors": {"transport":"tcp","ports":"443","l7":"tls","tls":"both"}
    }
  ]
}

The registry is one immutable batch (1–32 unique IDs). This conservative slice accepts only TCP/443/TLS selectors and tls12, tls13, or both policy. Argument whitespace is canonicalized; option/Lua stage order is preserved. Existing payload/range validation is reused. Lua stages are restricted to fake, multisplit, multidisorder, fakeddisorder, split and disorder; parameters to pos, repeats, tcp_ts, tcp_flags_set, seqovl and blob=fake_default_tls. File/inline Lua, queue/user/fwmark/filter options and arbitrary stages are rejected. Acceptance of a definition is not runtime verification.

The private target directory holds .resource-plan-key (32 random bytes), .resource-plan-strategies.json and, when an owned producer exists, .resource-plan-matrix.json. JSON envelopes have payload and an HMAC-SHA256 mac. Each file is write-once, owned, 0600, no-follow, single-link and bounded to 64 KiB. Held directory/file identities and exact snapshots are checked; foreign entries, replacement inodes and modified bytes are never clobbered or silently adopted. Interrupted writes remain blocked for explicit operator repair. There is no automatic rotation/update/repair in this slice. The private key and same-UID/root boundary must be protected: HMAC is not protection from an actor who can read the key and replace the whole target state between processes.

Matrix v1 binds the registry digest and at most one completed run per candidate. Each run binds ID, definition digest (canonical args and selectors), keyed config fingerprint, start/completion milliseconds, resource/TLS outcomes and keyed resource fingerprints. Its attestation records collector/runtime, definition digest, composite executable/Lua input hash, installed rules hash, actual worker-argv hash, queue and worker PID/start identity, plus restored/joined completion. No URLs, headers, resolved IPs, bodies or raw errors are persisted. A nonserialized producer seal protects the completed in-memory snapshot against mutation before publication. Only the owned producer and test-internal fixtures create seals; JSON DTOs cannot authorize publication. Every resource/protocol cell records exactly three actual attempts; any nonpass in those attempts makes the cell negative (never an average or partial PASS).

plan loads only authenticated private state and the existing resource-config snapshot (including resolved environment secrets, DNS/proxy and private headers in keyed fingerprints). Every resource on an exact host, across all services and input group labels, must have matching complete results for every required TLS protocol. Complete failed checks are negative; missing, expired (24 hours), future, mismatched or partial checks are unknown. Changed config invalidates old run evidence. Malformed/unauthenticated state yields blocked, unassigned output and exit 78; valid missing evidence yields unknown output and exit 0.

Output contains proposal_only, evidence (authentication status, not health), per-host validated/negative/unknown candidate alternatives, assignments and unassigned. Compatible equal args/selectors merge coverage across input labels. Greedy greatest-uncovered coverage, then lexical candidate ID, produces deterministic disjoint exact-host assignments. It does not claim global optimality or suffix coverage. Explicit legacy YouTube suffix policy is unchanged.

INTRUSIVE / ISOLATED startup only

Set both ZAPRET_AUTOTUNE_QUALIFY_CONFIG and ZAPRET_AUTOTUNE_STRATEGY_CONFIG to container paths on a dedicated target. Do not send working user traffic to this target's published SOCKS port. This is not noninterrupting background search. Provision the private resource and 0600 strategy files separately; compose forwards paths but does not mount files for you. Start with the managed disabled baseline, not an existing active profile. Qualification v1 accepts lowercase DNS hostnames, not literal IP targets; the ordinary resource-check command's broader literal-IP policy is unchanged.

PID 1 starts its owned SS+DoH services, initializes/reconciles known state, then launches one finite qualify ... --isolated session instead of starting the live zapret daemon or monitor. Qualifier checks root/target namespace, PID 1's actual entrypoint/environment, direct-child executable/arguments/session ACK, owned SS and DNS readiness anchors. A flag or docker-exec environment alone does not authorize qualification. Missing either startup path fails closed. Without --isolated, qualify remains UNKNOWN/exit 78 without runtime changes.

The shared monitor singleton excludes legacy and generic controllers. Any existing NFQ worker/queue, active profile, unknown config/journal or existing immutable matrix is refused. Known baseline config/journal/backup/identity inodes and bytes remain held unchanged throughout the job. This first producer does not pause/stop/restart existing active profiles; job-bound maintenance rollback is deferred rather than pretending a path-only daemon stop is safe.

Each candidate uses the existing worker-before-nft/queue-ready lifecycle on queue 200, fixed Lua imports and fwmark 0x10000000, plus native --filter-l7=tls. The actual child executable, argv, parent, start identity, queue socket ownership, executable/Lua hashes, native --version output 1.0.2 and installed nft rule bytes are attested before capture and checked again at each collector guard. DoH's existing 0x50000000 socket mark bypass remains. TLS policy is collector/eligibility policy, not a nonexistent native TLS-version filter. All configured protocols are measured; incompatible selector policies remain UNKNOWN in the proposal.

The existing GET/HEAD generic collector preserves TLS origin verification, configured body/status/content-type expectations, private stdin and IP pinning. Completed expectation failures and typed peer response/TLS/transport negatives do not stop the remaining repeats/cells. Guard/source/unknown infrastructure or batch cancellation veto the entire batch; no partial positive matrix is saved. Config (including resolved environment/header secrets) and strategy-source snapshots are checked after DNS/before transfer and before sealing/publication. Workers are joined and temporary nft queues removed before verifying unchanged disabled baseline/runtime and sealing. The batch has a fixed 15-minute deadline with separate bounded cleanup; PID 1 keeps SS+DoH alive through cleanup. The bounded 64 KiB matrix size (including its authenticated envelope and worst-case field widths) is checked before any worker/nft operation; excessive catalogue/resource combinations return UNKNOWN/exit 64 even within individual count caps (32 strategies, 64 resources).

Successful batches, including complete negatives, return the assignment proposal. Incomplete/unsafe jobs return UNKNOWN/blocked exit 78 without proof. PID 1 then stays safe-idle with SS+DoH for inspection, for either success or blocked completion: no qualification retry loop, duplicate controller or automatic generic monitor start. Retests require future explicit immutable snapshot rotation, not overwriting prior evidence.

Completed controller cleanup is acknowledged separately from qualification in an exclusive, 0600 HMAC-authenticated qualification-completed.json on the current boot's SS tmpfs. It binds the launched PID, exit outcome (0 or 78) and verified cleanup; it is not strategy proof. PID 1 checks that ACK against its private session launch and revalidates the disabled baseline, singleton, SS/DoH ownership and absence of NFQ workers/temporary nft queues before shutting down dependencies. Thus TERM after a clean blocked qualifier has exited but before its parent reaps it can exit cleanly. Worker/nft/join/restoration failure, missing/foreign/mutated ACK or unverified early failure still veto clean shutdown with exit 78. Ordinary legacy/generic monitor exit 78 remains a safety veto.

No unsupported --filter-mark or blockcheck embedded/stream/progress/deadline flags are used. Background noninterrupting search, active-profile maintenance and immutable snapshot rotation for repeated runs remain deferred.

Explicit generic transactional startup apply (local implementation)

On the target with its already completed private qualification batch, unset both qualification startup paths and set:

ZAPRET_AUTOTUNE_RESOURCE_CONFIG=/private/resources.json
ZAPRET_AUTOTUNE_RESOURCE_APPLY=1
ZAPRET_AUTOTUNE_STOP_GRACE_PERIOD=90s

Compose forwards the apply flag and configurable stop grace (legacy default remains 240s). Generic PID 1 cleanup has one shared 80-second budget. No external manager, new proof importer, SQL frontend surrogate or parallel journal exists. PID 1 starts owned SS/DoH, reconciles known bytes, launches the acknowledged managed resource-apply --resource-config FILE child before the checks-only monitor, and starts the native daemon only with verified committed config. This is at most one apply launch per PID 1 lifetime. A RAM latch is marked failed before launch and succeeds only after child wait=0 and owned-session cleanup. Launch/ACK failure, nonzero/signal exit, SS/DoH loss, or incomplete cleanup leaves it failed: after authoritative cleanup PID 1 remains cancellably BLOCKED, without restarting SS/DoH, native zapret, apply or monitoring, even across former 60s retry windows. Cleanup still stops SS/DoH (DNS last); BLOCKED does not keep them alive. Successful apply is skipped on later monitor/SS restarts, but every runtime cycle still reconciles committed refs before native start. Qualification and legacy retry behavior are unchanged.

Ceiling: this is not a durable manual-intent latch. A fresh PID 1 (including an external Docker restart policy) resets authorization and, with flag=1, permits one new attempt. Operators must disable unattended external restart/recreation for this workflow. Set ZAPRET_AUTOTUNE_RESOURCE_APPLY=0 for explicit repair-only startup: prepared state is reconciled, never promoted or reapplied. Identical committed requests still resume by authenticated refs without a new transaction. The command refuses standalone/docker-exec callers: actual PID 1 environment, entrypoint, namespace, direct-child executable/argv and held launch ACK are required, plus the existing controller singleton and no pre-existing NFQ worker.

First apply supports disabled baseline → generic. Explicit PID 1 startup selection also permits generic → generic rotation among already verified alternatives in the same immutable authenticated registry/matrix/source. Active legacy profiles, changed source/catalogue/proofs and uninterrupted live rotation remain unsupported. Every configured resource must require both TLS protocols and every assigned exact host must have complete, candidate-bound passing proof for all its resources across all input labels. New apply requires completion within 24 hours, never in the future. Negative/unknown/unassigned hosts cannot authorize apply. Registry definitions may still describe single-TLS proposals, but native apply rejects them instead of widening unsupported TLS-version selectors.

Canonical ordered args/selectors define profiles separated by --new; each contains --filter-tcp=443 --filter-l7=tls --hostlist-domains=^a,^b. Exact-host semantics use the pinned native source https://raw.githubusercontent.com/bol-van/zapret2/4df94497869e18ed4fdf1272141427b2fbfc6604/nfq2/hostlist.c: inline AppendHostlistItem uses addpool, leading ^ sets HOSTLIST_POOL_FLAG_STRICT_MATCH, and SearchHostList rejects suffix fallback when that flag is set and !bHostFull. Existing YouTube suffix policy is unchanged. The packaged executable/Lua hash must match private attestation; held native, Lua, source, marker and proof identities/bytes are guarded through the transaction. No active queue version probe or mark/filter change is made (DoH500/probe100/live400).

The existing journal gains v4 independently tagged committed/pending refs (baseline, legacy, generic). Generic refs exclude legacy IDs/scopes and bind exact authenticated registry/matrix envelope hashes, source fingerprint, internally derived canonical assignment digest and complete config hash. Optional scalar strategy_ids retains the canonical selection policy on each generic ref; missing/empty means the original default greedy policy. No policy is permitted on baseline/legacy refs. This remains v4: older strict-decoding engines reject the new field, so downgrade is blocked, not silently reinterpreted. Unknown versions/tags, mixed refs and unresolved historical reconstruction block without adopting or erasing foreign state; v1–v3 legacy behavior remains. Staging/backup/prepared/replace/fixed restart/health/commit reuse the same engine. All required resources/TLS are checked through the existing synchronous owned SS/DoH collector (all-answer SSRF validation, CONNECT pin, SNI/cert validation, private stdin). No Player API health or public check JSON authorizes commit.

Committed identical requests are idempotent: historical proof expiry alone does not trigger reload, reapply or rejection on restart. Authenticated integrity, source and native drift still block. Prepared recovery never promotes pending bytes, including after successful health: it restores prior bytes and reloads. Generic rollback additionally checks all resources before finalizing the journal. Strict first-slice ceiling: a DPI-broken disabled baseline can legitimately fail these checks; restored bytes then remain prepared/BLOCKED until operator repair. Physical restoration is not falsely reported as healthy rollback. Legacy rollback has no new health requirement. Cancellation joins synchronous checks before rollback; PID 1 bounds the owned session and retains prepared state on unsafe/unfinished cleanup. ActiveCandidates refuses generic deployments; managed/resource status validates generic refs without inventing legacy groups.

Explicit startup selection (local implementation, remote acceptance pending)

Set ZAPRET_AUTOTUNE_RESOURCE_STRATEGY_IDS=s1,s2 with ZAPRET_AUTOTUNE_RESOURCE_CONFIG and ZAPRET_AUTOTUNE_RESOURCE_APPLY=1 on a fresh container PID 1. Empty retains default greedy selection. The bounded list accepts at most 32 unique registry IDs, canonicalized in lexical order; malformed, duplicate, unknown or incomplete-coverage requests cannot authorize a transaction. Eligibility is filtered before equal-args/selectors coverage merging; excluded candidates cannot lend proof to a selected profile. Compatibility evidence and all alternatives remain visible; the policy is not proof or a new qualification. No registry/matrix files are written by rotation. Exact hosts, both TLS versions, all required cells/M=3, native/source integrity and the 24-hour freshness gate remain required for every new selection. Identical committed selection replays without reload even after expiry. Distinct redundant allowlists are distinct policies and may create a transaction even if rendered bytes coincide.

PID 1 forwards the setting in environment only; the apply child authenticates it against PID 1's original environment, not CLI arguments. Syntax is validated even with APPLY=0; nonempty policy requires generic resource config. With APPLY=0 selection is ignored for reconciliation/monitoring (registry membership is not resolved): committed and pending refs reconstruct independently from their own stored policies and historical completion horizon, never the requested setting. Prepared recovery restores the old generic bytes and policy, never promotes the pending side; failed rollback health retains prepared state for repair. Startup still allows just one attempt per PID 1; external restart policies reset that authorization. There is no monitor autotuning, new qualification, proof import, snapshot overwrite or uninterrupted maintenance in this slice.

Local root fixture tests cover the production snapshot factory and shared engine; independent local review and deterministic 180-second retry-window fixtures passed. The bounded remote gates below now cover guarded apply/replay, strict rollback, prepared repair and the per-PID-1 failure latch; they do not authorize live rollout.

Bounded generic apply/replay and failure-latch acceptance

DONE within the stated bounds, not full generic autotuning or service READY. Fault acceptance used source 643d0c6; the earlier healthy committed replay used 36aace0 after the atomic launch-ACK correction. Independent privileged audit returned PASS_BOUNDED_GENERIC_APPLY_FAULTS for the observed A/B/C safety gates. Root-private evidence references (no private payloads reproduced here):

  • /home/agent/generic-latch-643d0c6/field-private/field-report.md
  • /home/agent/generic-latch-643d0c6/field-private/independent-audit.safe.json
  • /home/agent/ack-recreate-36aace0/field-private/field-report.md

Observed acceptance

  • Real apply health reused authentic, still-fresh qualification: 16 M=3 cells, 48 prior TLS transfers, two strategies and four resources. No new qualification, timestamp rewrite, fake ACK or imported proof was used. Pinned native 1.0.2/Lua stayed unchanged. The earlier healthy start/recreate each made eight TLS checks with four authenticated HEALTHY resources; v4 committed generation 1, refs, config and proof bytes remained unchanged. Both official stops exited 0.
  • A — one HTTP 500: eight apply plus eight rollback requests, one 500 total. Rollback committed prior baseline generation 1 and cleared pending. Request count, journal and config stayed frozen at 45.096/90.084/135.124 seconds; one apply only, no automatic retry. PID 1 stayed alive/BLOCKED with SS/DNS and other owned runtime stopped. Official stop exited 0.
  • B — persistent HTTP 500: 24 requests and three 500s: eight apply, eight rollback and eight cleanup-reconcile checks, not a second apply. Prior baseline bytes were physically restored, but failed health retained PREPARED committed baseline/pending generic refs. State stayed unchanged for 80.097 seconds with no reapply; official stop exited 78. After repairing only fixture behavior and changing startup APPLY=1 to APPLY=0, the same state/source/environment completed eight reconcile and eight monitor checks. Baseline became committed, pending cleared, and generic was never promoted; official stop exited 0.
  • C — TERM during a real slow request: one canceled request plus eight passing rollback checks. The canceled curl was joined before the first rollback request; baseline generation 1 committed and pending cleared, with no false generic commit. Actual PID 1 exit was 78, with monitor cleanup status 78, not 0. Apply health cancellation is collapsed into errPlan/CLI 78; outside the qualification-specific completion path, the supervisor accepts only 0/143 and retains a sticky failure even after verified rollback. This is conservative outcome conflation, not an observed rollback-health failure or cleanup defect. Do not blanket-allow 78 or manufacture a completion ACK. A normal-0 path would require authenticated completion plus fresh guards; that optional future refinement is not a blocker for this bounded P0 safety slice.

Bounds and remaining work

  • A/B/repair/C totalled 65 actual HTTPS requests. All 171 observed owned PID/start identities disappeared, owned netns were empty, and owned targets/network were removed. Foreign SS/container full-inspect hashes stayed unchanged. Passive 100ms inventories do not cover every shorter-lived helper; no exhaustive argv or packet-level privacy audit is claimed. Same-root HMAC-key mutation is outside the trust boundary.
  • The latch is one RAM authorization per PID 1, not durable manual intent. Fresh startup/external restart policy reauthorizes ENV=1; keep unattended external restart/recreation disabled. ENV=0 repairs/reconciles only. Historical committed replay/repair allows proof expiry; a new apply still requires fresh, complete source-bound qualification within 24 hours. Natural remote 24-hour expiry remains unit-covered only; remote qualifier singleton conflict is still untested.
  • Use an execution-free remote harness: stage inputs before startup; no target exec, cp or nsenter during active guards. Observe host metadata/private files and Docker logs/inspect; official stop is the cancellation operation. Earlier scan EACCES nonrecurrence does not establish its cause.
  • No production READY, causal DPI benefit, active-profile maintenance, plan or snapshot rotation, automatic recovery or monitor autotuning is accepted here. Next roadmap work is dynamic profile/plan rotation and compatible selectors, generic resource tuning and bounded strategy-DB collections; bounded-ID Prometheus remains a separate planned function. No new framework, probes or native/live upgrade was introduced. Legacy YouTube exit 78 and unfinished images/avatars remain NOT READY.

Bounded remote acceptance — source 5cdc0cd

The isolated producer/registry/matrix/planner slice is bounded-qualified. Independent privileged evidence audits passed the observed positive, fault and active-state refusal gates. Full remote concurrency acceptance is PARTIAL; this is not generic apply, active-profile maintenance or service READY.

Root-private proof references (retained reports, not logs copied into this doc):

  • /home/agent/zapret2-autotune-matrix-5cdc0cd/execution-free-private/execution-free-report.md
  • /home/agent/zapret2-autotune-matrix-5cdc0cd/fault-final-private/qualification-fault-report.md
  • /home/agent/zapret2-autotune-matrix-5cdc0cd/refusal-gap-private/remaining-refusal-reports.md

Observed positive and fault gates

  • The execution-free private batch made 60 actual transfers: two candidates, five resources, both TLS protocols, three attempts per cell. Its 20 aggregated cells contain 16 PASS and four expectation_failed: mandatory resource d's HTTP 500 fails for both candidates/protocols, leaving d unassigned. Exact hosts a/b/c receive the greedy multidisorder proposal with authentic multisplit alternatives. Input groups remain metadata, not assignment boundaries; there is no generic suffix coverage or automatic apply.
  • Independent verification authenticated the exact stored payload HMAC, not a reserialized JSON equivalent. Distinct native arguments/worker-argv hashes, executable/Lua/rules evidence, queue 200 and NFQ 1.0.2 attestations bind the two joined/restored runs. Disabled baseline bytes stayed unchanged; sampled workers, queue and temporary NFT rules were gone. Official PID 1 shutdown exited 0 in 1.31s; foreign inspect hashes stayed unchanged.
  • Mid-batch cancellation and effective semantic config drift each stopped after two requests, published no matrix and preserved the native baseline. Official PID 1 exits were 0 in 2.46s and 0.44s respectively; the deliberately changed config source was retained, not silently rolled back.
  • Unsafe Lua/queue/fwmark/user/file definitions and evidence/schema imports were rejected without transfers. Oversized capacity was rejected before worker launch; unauthorized isolated startup was refused. Corrupt MAC blocked planning while preserving payload bytes; changed effective config produced unknown/unassigned output without changing the original authenticated matrix. Immutable catalogue replacement and completed-matrix retest were refused without transfers or overwriting original state.
  • A genuine archived v3 committed journal with two candidates passed official reconcile/ActiveCandidates validation before qualification refused active profile state. Profile/config, journal, last-good and target identity remained byte-identical; zero transfers/workers and no matrix. Actual PID 1 exit 78 was an expected early preflight refusal, not an exit-0 cleanup ACK. This did not activate or maintain a running active-profile NFQ daemon; that remains unsupported. Whole-DB byte preservation is not claimed.

Explicit limits and testing lesson

  • Remote qualifier singleton conflict is NOT RUN. The earlier planted-lock fixture failed initialization before that branch and does not count. The shared-lock unit second-acquire EWOULDBLOCK check passed, but is not remote qualifier concurrency acceptance. No deterministic official lifecycle seam exists for this bounded test; no test framework or racy injection was added.
  • Natural 24-hour remote expiry is NOT RUN. The existing unit check rejects a structurally valid 25-hour-old proof; this is not observed remote expiry.
  • Privacy sentinel absence was checked in tested logs/state and sampled fault/ refusal argv. Positive active argv and exhaustive DNS packet-level privacy were not proved. Configured/observed encrypted DNS is not exhaustive proof against transient fallback.
  • Compatibility on this baseline is not causal DPI-desync benefit. Historical worker-scan EACCES origin remains unknown; nonrecurrence in the execution-free batch does not identify its cause. Earlier harness failures remain recorded in the referenced reports, not erased or relabeled as product acceptance.
  • Stage all inputs before target startup. From startup through qualification completion, prohibit target exec, cp and nsenter (even passive exec starts a process). Observe host /proc metadata and Docker logs/inspect only; inspect runtime and run plan after finalization/cleanup. The explicit official stop used for cancellation is the documented exception, not an observer exec.
  • Owned test containers/network were removed and private evidence retained. Foreign SS1080 stayed running and legacy11082 stayed stopped with exit 78. No live upgrade or push occurred; unresolved legacy exit 78, YouTube/image readiness, transactional generic apply/profile-journal migration, immutable rotation and noninterrupting search remain outside this producer acceptance; the later bounded apply/replay gates are recorded separately above.