Files
zapret-autotune/RESOURCE_PLAN.md
T
ZeusinaandGPT-6 Bot 643d0c6220 fix: prevent automatic retries of failed startup apply
Latch explicit resource apply once per PID 1 lifetime. Fail closed before launch and permanently pause after authoritative cleanup on failed or incomplete apply; retain reconciliation on successful monitor and service restarts.

Cover deterministic 180-second retry windows, launch/ACK/child/session/service failures, cancellation, successful restarts and fresh-PID1 flags. Document RAM-only reset limits and repair-only startup; no remote health acceptance is claimed.

Co-authored-by: GPT-6 <gpt-6@openai.com>
2026-10-15 08:05:27 +03:00

324 lines
20 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Resource compatibility planner and isolated qualifier
`plan` remains **proposal-only** and never changes runtime or config.
Explicit generic startup apply is implemented and independently reviewed locally;
remote acceptance is pending. This is not automatic recovery or production READY.
The per-PID-1 one-shot latch correction passed independent local review only.
Explicit isolated qualification temporarily installs an owned NFQ worker and
queue rules, but never changes legacy SQL groups, renderer or apply journals.
Monitor HEALTHY, old SQL measurements and public check JSON are not proof.
Without a completed owned qualification batch, compatibility is UNKNOWN and
hosts are unassigned. No PASS/evidence import is exposed.
Commands require root and the existing `managed-test` target settings:
```
zapret-autotune strategy-import --strategies /private/strategies.json
zapret-autotune plan --config /private/resources.json
zapret-autotune qualify --config /private/resources.json --strategies /private/strategies.json --isolated
```
`strategy-import` requires an owned, regular, single-link 0600 file. Only
definitions are accepted; duplicate keys, unknown fields and evidence fields
are rejected. Example:
```json
{
"version": 1,
"strategies": [
{
"id": "split-one",
"args": "--payload=tls_client_hello --lua-desync=multisplit:pos=1",
"selectors": {"transport":"tcp","ports":"443","l7":"tls","tls":"both"}
}
]
}
```
The registry is one immutable batch (1–32 unique IDs). This conservative slice
accepts only TCP/443/TLS selectors and `tls12`, `tls13`, or `both` policy.
Argument whitespace is canonicalized; option/Lua stage order is preserved.
Existing payload/range validation is reused. Lua stages are restricted to
fake, multisplit, multidisorder, fakeddisorder, split and disorder; parameters
to pos, repeats, tcp_ts, tcp_flags_set, seqovl and blob=fake_default_tls.
File/inline Lua, queue/user/fwmark/filter options and arbitrary stages are
rejected. Acceptance of a definition is **not** runtime verification.
The private target directory holds `.resource-plan-key` (32 random bytes),
`.resource-plan-strategies.json` and, when an owned producer exists,
`.resource-plan-matrix.json`. JSON envelopes have `payload` and an HMAC-SHA256
`mac`. Each file is write-once, owned, 0600, no-follow, single-link and bounded
to 64 KiB. Held directory/file identities and exact snapshots are checked;
foreign entries, replacement inodes and modified bytes are never clobbered or
silently adopted. Interrupted writes remain blocked for explicit operator
repair. There is no automatic rotation/update/repair in this slice. The private
key and same-UID/root boundary must be protected: HMAC is not protection from an
actor who can read the key and replace the whole target state between processes.
Matrix v1 binds the registry digest and at most one completed run per candidate.
Each run binds ID, definition digest (canonical args **and selectors**), keyed
config fingerprint, start/completion milliseconds, resource/TLS outcomes and
keyed resource fingerprints. Its attestation records collector/runtime,
definition digest, composite executable/Lua input hash, installed rules hash,
actual worker-argv hash, queue and worker PID/start identity, plus
restored/joined completion. No URLs, headers, resolved IPs, bodies or raw errors
are persisted. A nonserialized producer seal protects the completed in-memory
snapshot against mutation before publication. Only the owned producer and
test-internal fixtures create seals; JSON DTOs cannot authorize publication.
Every resource/protocol cell records exactly three actual attempts; any nonpass
in those attempts makes the cell negative (never an average or partial PASS).
`plan` loads only authenticated private state and the existing resource-config
snapshot (including resolved environment secrets, DNS/proxy and private headers
in keyed fingerprints). Every resource on an exact host, across all services
and input group labels, must have matching complete results for every required
TLS protocol. Complete failed checks are negative; missing, expired (24 hours),
future, mismatched or partial checks are unknown. Changed config invalidates old
run evidence. Malformed/unauthenticated state yields blocked, unassigned output
and exit 78; valid missing evidence yields unknown output and exit 0.
Output contains `proposal_only`, `evidence` (authentication status, **not**
health), per-host `validated`/`negative`/`unknown` candidate alternatives,
`assignments` and `unassigned`. Compatible equal args/selectors merge coverage
across input labels. Greedy greatest-uncovered coverage, then lexical candidate
ID, produces deterministic disjoint exact-host assignments. It does not claim
global optimality or suffix coverage. Explicit legacy YouTube suffix policy is
unchanged.
## INTRUSIVE / ISOLATED startup only
Set **both** `ZAPRET_AUTOTUNE_QUALIFY_CONFIG` and
`ZAPRET_AUTOTUNE_STRATEGY_CONFIG` to container paths on a dedicated target.
Do not send working user traffic to this target's published SOCKS port. This is
not noninterrupting background search. Provision the private resource and 0600
strategy files separately; compose forwards paths but does not mount files for
you. Start with the managed disabled baseline, not an existing active profile.
Qualification v1 accepts lowercase DNS hostnames, not literal IP targets; the
ordinary resource-check command's broader literal-IP policy is unchanged.
PID 1 starts its owned SS+DoH services, initializes/reconciles known state, then
launches one finite `qualify ... --isolated` session **instead of** starting the
live zapret daemon or monitor. Qualifier checks root/target namespace, PID 1's
actual entrypoint/environment, direct-child executable/arguments/session ACK,
owned SS and DNS readiness anchors. A flag or docker-exec environment alone
does not authorize qualification. Missing either startup path fails closed.
Without `--isolated`, `qualify` remains UNKNOWN/exit 78 without runtime changes.
The shared monitor singleton excludes legacy and generic controllers. Any
existing NFQ worker/queue, active profile, unknown config/journal or existing
immutable matrix is refused. Known baseline config/journal/backup/identity
inodes and bytes remain held unchanged throughout the job. This first producer
does **not** pause/stop/restart existing active profiles; job-bound maintenance
rollback is deferred rather than pretending a path-only daemon stop is safe.
Each candidate uses the existing worker-before-nft/queue-ready lifecycle on
queue 200, fixed Lua imports and fwmark 0x10000000, plus native
`--filter-l7=tls`. The actual child executable, argv, parent, start identity,
queue socket ownership, executable/Lua hashes, native `--version` output 1.0.2
and installed nft rule bytes are attested before capture and checked again at
each collector guard. DoH's existing 0x50000000 socket mark bypass remains.
TLS policy is collector/eligibility policy, not a nonexistent native TLS-version
filter. All configured protocols are measured; incompatible selector policies
remain UNKNOWN in the proposal.
The existing GET/HEAD generic collector preserves TLS origin verification,
configured body/status/content-type expectations, private stdin and IP pinning.
Completed expectation failures and typed peer response/TLS/transport negatives
do not stop the remaining repeats/cells. Guard/source/unknown infrastructure or
batch cancellation veto the entire batch; no partial positive matrix is saved.
Config (including resolved environment/header secrets) and strategy-source
snapshots are checked after DNS/before transfer and before sealing/publication.
Workers are joined and temporary nft queues removed before verifying unchanged
disabled baseline/runtime and sealing. The batch has a fixed 15-minute deadline
with separate bounded cleanup; PID 1 keeps SS+DoH alive through cleanup.
The bounded 64 KiB matrix size (including its authenticated envelope and
worst-case field widths) is checked before any worker/nft operation; excessive
catalogue/resource combinations return UNKNOWN/exit 64 even within individual
count caps (32 strategies, 64 resources).
Successful batches, including complete negatives, return the assignment proposal.
Incomplete/unsafe jobs return UNKNOWN/blocked exit 78 without proof. PID 1 then
stays safe-idle with SS+DoH for inspection, for either success or blocked
completion: no qualification retry loop, duplicate controller or automatic
generic monitor start. Retests require future explicit immutable snapshot
rotation, not overwriting prior evidence.
Completed controller cleanup is acknowledged separately from qualification in
an exclusive, 0600 HMAC-authenticated `qualification-completed.json` on the
current boot's SS tmpfs. It binds the launched PID, exit outcome (0 or 78) and
verified cleanup; it is **not** strategy proof. PID 1 checks that ACK against its
private session launch and revalidates the disabled baseline, singleton,
SS/DoH ownership and absence of NFQ workers/temporary nft queues before shutting
down dependencies. Thus TERM after a clean blocked qualifier has exited but
before its parent reaps it can exit cleanly. Worker/nft/join/restoration failure,
missing/foreign/mutated ACK or unverified early failure still veto clean shutdown
with exit 78. Ordinary legacy/generic monitor exit 78 remains a safety veto.
No unsupported `--filter-mark` or blockcheck embedded/stream/progress/deadline
flags are used. Background noninterrupting search, active-profile maintenance
and immutable snapshot rotation for repeated runs remain deferred.
## Explicit generic transactional startup apply (local implementation)
On the target with its already completed private qualification batch, unset both
qualification startup paths and set:
```
ZAPRET_AUTOTUNE_RESOURCE_CONFIG=/private/resources.json
ZAPRET_AUTOTUNE_RESOURCE_APPLY=1
ZAPRET_AUTOTUNE_STOP_GRACE_PERIOD=90s
```
Compose forwards the apply flag and configurable stop grace (legacy default
remains 240s). Generic PID 1 cleanup has one shared 80-second budget. No external
manager, new proof importer, SQL frontend surrogate or parallel journal exists.
PID 1 starts owned SS/DoH, reconciles known bytes, launches the acknowledged
`managed resource-apply --resource-config FILE` child before the checks-only
monitor, and starts the native daemon only with verified committed config.
This is at most one apply launch per PID 1 lifetime. A RAM latch is marked failed
before launch and succeeds only after child wait=0 and owned-session cleanup.
Launch/ACK failure, nonzero/signal exit, SS/DoH loss, or incomplete cleanup leaves
it failed: after authoritative cleanup PID 1 remains cancellably BLOCKED, without
restarting SS/DoH, native zapret, apply or monitoring, even across former 60s retry
windows. Cleanup still stops SS/DoH (DNS last); BLOCKED does not keep them alive.
Successful apply is skipped on later monitor/SS restarts, but every runtime cycle
still reconciles committed refs before native start. Qualification and legacy
retry behavior are unchanged.
**Ceiling:** this is not a durable manual-intent latch. A fresh PID 1 (including
an external Docker restart policy) resets authorization and, with flag=1, permits
one new attempt. Operators must disable unattended external restart/recreation
for this workflow. Set `ZAPRET_AUTOTUNE_RESOURCE_APPLY=0` for explicit repair-only
startup: prepared state is reconciled, never promoted or reapplied. Identical
committed requests still resume by authenticated refs without a new transaction.
The command refuses standalone/docker-exec callers: actual PID 1 environment,
entrypoint, namespace, direct-child executable/argv and held launch ACK are
required, plus the existing controller singleton and no pre-existing NFQ worker.
First apply supports **disabled baseline → generic only**. Active legacy profiles
and different generic plan rotation are refused; no maintenance interruption is
implemented. Every configured resource must require `both` TLS protocols and
every assigned exact host must have complete, candidate-bound passing proof for
all its resources across all input labels. New apply requires completion within
24 hours, never in the future. Negative/unknown/unassigned hosts cannot authorize
apply. Registry definitions may still describe single-TLS proposals, but native
apply rejects them instead of widening unsupported TLS-version selectors.
Canonical ordered args/selectors define profiles separated by `--new`; each
contains `--filter-tcp=443 --filter-l7=tls --hostlist-domains=^a,^b`. Exact-host
semantics use the pinned native source
`https://raw.githubusercontent.com/bol-van/zapret2/4df94497869e18ed4fdf1272141427b2fbfc6604/nfq2/hostlist.c`:
inline `AppendHostlistItem` uses `addpool`, leading `^` sets
`HOSTLIST_POOL_FLAG_STRICT_MATCH`, and `SearchHostList` rejects suffix fallback
when that flag is set and `!bHostFull`. Existing YouTube suffix policy is unchanged.
The packaged executable/Lua hash must match private attestation; held native,
Lua, source, marker and proof identities/bytes are guarded through the transaction.
No active queue version probe or mark/filter change is made (DoH500/probe100/live400).
The **existing** journal gains v4 independently tagged committed/pending refs
(`baseline`, `legacy`, `generic`). Generic refs exclude legacy IDs/scopes and bind
exact authenticated registry/matrix envelope hashes, source fingerprint,
internally derived canonical assignment digest and complete config hash.
Unknown versions/tags, mixed refs and unresolved historical reconstruction block
without adopting or erasing foreign state; v1–v3 legacy behavior remains.
Staging/backup/prepared/replace/fixed restart/health/commit reuse the same engine.
All required resources/TLS are checked through the existing synchronous owned
SS/DoH collector (all-answer SSRF validation, CONNECT pin, SNI/cert validation,
private stdin). No Player API health or public check JSON authorizes commit.
Committed identical requests are idempotent: historical proof expiry alone does
not trigger reload, reapply or rejection on restart. Authenticated integrity,
source and native drift still block. Prepared recovery **never** promotes pending
bytes, including after successful health: it restores prior bytes and reloads.
Generic rollback additionally checks all resources before finalizing the journal.
**Strict first-slice ceiling:** a DPI-broken disabled baseline can legitimately
fail these checks; restored bytes then remain `prepared`/BLOCKED until operator
repair. Physical restoration is not falsely reported as healthy rollback.
Legacy rollback has no new health requirement. Cancellation joins synchronous
checks before rollback; PID 1 bounds the owned session and retains prepared state
on unsafe/unfinished cleanup. `ActiveCandidates` refuses generic deployments;
managed/resource status validates generic refs without inventing legacy groups.
Local root fixture tests cover the production snapshot factory and shared engine;
independent local review passed. Actual native/network/PID 1 apply acceptance is
**not run**. The next gate is isolated bounded remote acceptance, not live rollout.
The per-PID-1 latch correction passed independent local review and deterministic
180-second simulated retry-window fixtures; no remote PASS is claimed for it.
Roadmap: authorized isolated acceptance; no remote deployment is authorized by
local fixtures.
## Bounded remote acceptance — source `5cdc0cd`
The isolated producer/registry/matrix/planner slice is **bounded-qualified**.
Independent privileged evidence audits passed the observed positive, fault and
active-state refusal gates. **Full remote concurrency acceptance is PARTIAL**;
this is not generic apply, active-profile maintenance or service READY.
Root-private proof references (retained reports, not logs copied into this doc):
- `/home/agent/zapret2-autotune-matrix-5cdc0cd/execution-free-private/execution-free-report.md`
- `/home/agent/zapret2-autotune-matrix-5cdc0cd/fault-final-private/qualification-fault-report.md`
- `/home/agent/zapret2-autotune-matrix-5cdc0cd/refusal-gap-private/remaining-refusal-reports.md`
### Observed positive and fault gates
- The execution-free private batch made 60 actual transfers: two candidates,
five resources, both TLS protocols, three attempts per cell. Its 20 aggregated
cells contain 16 PASS and four `expectation_failed`: mandatory resource d's
HTTP 500 fails for both candidates/protocols, leaving d unassigned. Exact
hosts a/b/c receive the greedy multidisorder proposal with authentic multisplit
alternatives. Input groups remain metadata, not assignment boundaries;
there is no generic suffix coverage or automatic apply.
- Independent verification authenticated the exact stored payload HMAC, not a
reserialized JSON equivalent. Distinct native arguments/worker-argv hashes,
executable/Lua/rules evidence, queue 200 and NFQ 1.0.2 attestations bind the
two joined/restored runs. Disabled baseline bytes stayed unchanged; sampled
workers, queue and temporary NFT rules were gone. Official PID 1 shutdown
exited 0 in 1.31s; foreign inspect hashes stayed unchanged.
- Mid-batch cancellation and effective semantic config drift each stopped
after two requests, published no matrix and preserved the native baseline.
Official PID 1 exits were 0 in 2.46s and 0.44s respectively; the deliberately
changed config source was retained, not silently rolled back.
- Unsafe Lua/queue/fwmark/user/file definitions and evidence/schema imports
were rejected without transfers. Oversized capacity was rejected before
worker launch; unauthorized isolated startup was refused. Corrupt MAC blocked
planning while preserving payload bytes; changed effective config produced
unknown/unassigned output without changing the original authenticated matrix.
Immutable catalogue replacement and completed-matrix retest were refused
without transfers or overwriting original state.
- A genuine archived v3 committed journal with two candidates passed official
`reconcile`/`ActiveCandidates` validation before qualification refused active
profile state. Profile/config, journal, last-good and target identity remained
byte-identical; zero transfers/workers and no matrix. Actual PID 1 exit 78 was
an expected early preflight refusal, **not** an exit-0 cleanup ACK. This did
not activate or maintain a running active-profile NFQ daemon; that remains
unsupported. Whole-DB byte preservation is not claimed.
### Explicit limits and testing lesson
- Remote qualifier singleton conflict is **NOT RUN**. The earlier planted-lock
fixture failed initialization before that branch and does not count. The
shared-lock unit second-acquire `EWOULDBLOCK` check passed, but is not remote
qualifier concurrency acceptance. No deterministic official lifecycle seam
exists for this bounded test; no test framework or racy injection was added.
- Natural 24-hour remote expiry is **NOT RUN**. The existing unit check rejects
a structurally valid 25-hour-old proof; this is not observed remote expiry.
- Privacy sentinel absence was checked in tested logs/state and sampled fault/
refusal argv. Positive active argv and exhaustive DNS packet-level privacy
were not proved. Configured/observed encrypted DNS is not exhaustive proof
against transient fallback.
- Compatibility on this baseline is not causal DPI-desync benefit. Historical
worker-scan EACCES origin remains unknown; nonrecurrence in the execution-free
batch does not identify its cause. Earlier harness failures remain recorded
in the referenced reports, not erased or relabeled as product acceptance.
- Stage all inputs before target startup. From startup through qualification
completion, prohibit target `exec`, `cp` and `nsenter` (even passive exec starts
a process). Observe host `/proc` metadata and Docker logs/inspect only; inspect
runtime and run `plan` after finalization/cleanup. The explicit official stop
used for cancellation is the documented exception, not an observer exec.
- Owned test containers/network were removed and private evidence retained.
Foreign SS1080 stayed running and legacy11082 stayed stopped with exit 78.
No live upgrade or push occurred; unresolved legacy exit 78, YouTube/image
readiness, transactional generic apply/profile-journal migration, immutable
rotation and noninterrupting search remain outside this acceptance.