feat: add guarded generic startup apply

Co-authored-by: GPT-6 <gpt-6@openai.com>
This commit is contained in:
ZeusinaandGPT-6 Bot committed 2026-10-14 22:02:00 +03:00
1 parent 3743cdbd3e
commit de453ed2c8
11 files changed
+1255 -49

No files matched your search

+77 -3
View File
@@ -1,6 +1,8 @@
# Resource compatibility planner and isolated qualifier
This is a **proposal-only** P0 slice. `plan` never changes runtime or config.
`plan` remains **proposal-only** and never changes runtime or config.
Explicit generic startup apply is implemented and independently reviewed locally;
remote acceptance is pending. This is not automatic recovery or production READY.
Explicit isolated qualification temporarily installs an owned NFQ worker and
queue rules, but never changes legacy SQL groups, renderer or apply journals.
Monitor HEALTHY, old SQL measurements and public check JSON are not proof.
@@ -152,8 +154,80 @@ missing/foreign/mutated ACK or unverified early failure still veto clean shutdow
with exit 78. Ordinary legacy/generic monitor exit 78 remains a safety veto.
No unsupported `--filter-mark` or blockcheck embedded/stream/progress/deadline
flags are used. Background noninterrupting search and generic apply migration
remain deferred, as does immutable snapshot rotation for repeated runs.
flags are used. Background noninterrupting search, active-profile maintenance
and immutable snapshot rotation for repeated runs remain deferred.
## Explicit generic transactional startup apply (local implementation)
On the target with its already completed private qualification batch, unset both
qualification startup paths and set:
```
ZAPRET_AUTOTUNE_RESOURCE_CONFIG=/private/resources.json
ZAPRET_AUTOTUNE_RESOURCE_APPLY=1
ZAPRET_AUTOTUNE_STOP_GRACE_PERIOD=90s
```
Compose forwards the apply flag and configurable stop grace (legacy default
remains 240s). Generic PID 1 cleanup has one shared 80-second budget. No external
manager, new proof importer, SQL frontend surrogate or parallel journal exists.
PID 1 starts owned SS/DoH, reconciles known bytes, launches the acknowledged
`managed resource-apply --resource-config FILE` child before the checks-only
monitor, and starts the native daemon only with verified committed config.
This is one finite apply invocation per startup cycle, not exactly once per
container: PID 1 can retry cycles under its existing restart budget. Identical
committed requests resume by authenticated refs without reapplying the transaction.
The command refuses standalone/docker-exec callers: actual PID 1 environment,
entrypoint, namespace, direct-child executable/argv and held launch ACK are
required, plus the existing controller singleton and no pre-existing NFQ worker.
First apply supports **disabled baseline → generic only**. Active legacy profiles
and different generic plan rotation are refused; no maintenance interruption is
implemented. Every configured resource must require `both` TLS protocols and
every assigned exact host must have complete, candidate-bound passing proof for
all its resources across all input labels. New apply requires completion within
24 hours, never in the future. Negative/unknown/unassigned hosts cannot authorize
apply. Registry definitions may still describe single-TLS proposals, but native
apply rejects them instead of widening unsupported TLS-version selectors.
Canonical ordered args/selectors define profiles separated by `--new`; each
contains `--filter-tcp=443 --filter-l7=tls --hostlist-domains=^a,^b`. Exact-host
semantics use the pinned native source
`https://raw.githubusercontent.com/bol-van/zapret2/4df94497869e18ed4fdf1272141427b2fbfc6604/nfq2/hostlist.c`:
inline `AppendHostlistItem` uses `addpool`, leading `^` sets
`HOSTLIST_POOL_FLAG_STRICT_MATCH`, and `SearchHostList` rejects suffix fallback
when that flag is set and `!bHostFull`. Existing YouTube suffix policy is unchanged.
The packaged executable/Lua hash must match private attestation; held native,
Lua, source, marker and proof identities/bytes are guarded through the transaction.
No active queue version probe or mark/filter change is made (DoH500/probe100/live400).
The **existing** journal gains v4 independently tagged committed/pending refs
(`baseline`, `legacy`, `generic`). Generic refs exclude legacy IDs/scopes and bind
exact authenticated registry/matrix envelope hashes, source fingerprint,
internally derived canonical assignment digest and complete config hash.
Unknown versions/tags, mixed refs and unresolved historical reconstruction block
without adopting or erasing foreign state; v1–v3 legacy behavior remains.
Staging/backup/prepared/replace/fixed restart/health/commit reuse the same engine.
All required resources/TLS are checked through the existing synchronous owned
SS/DoH collector (all-answer SSRF validation, CONNECT pin, SNI/cert validation,
private stdin). No Player API health or public check JSON authorizes commit.
Committed identical requests are idempotent: historical proof expiry alone does
not trigger reload, reapply or rejection on restart. Authenticated integrity,
source and native drift still block. Prepared recovery **never** promotes pending
bytes, including after successful health: it restores prior bytes and reloads.
Generic rollback additionally checks all resources before finalizing the journal.
**Strict first-slice ceiling:** a DPI-broken disabled baseline can legitimately
fail these checks; restored bytes then remain `prepared`/BLOCKED until operator
repair. Physical restoration is not falsely reported as healthy rollback.
Legacy rollback has no new health requirement. Cancellation joins synchronous
checks before rollback; PID 1 bounds the owned session and retains prepared state
on unsafe/unfinished cleanup. `ActiveCandidates` refuses generic deployments;
managed/resource status validates generic refs without inventing legacy groups.
Local root fixture tests cover the production snapshot factory and shared engine;
independent local review passed. Actual native/network/PID 1 apply acceptance is
**not run**. The next gate is isolated bounded remote acceptance, not live rollout.
## Bounded remote acceptance — source `5cdc0cd`
+55 -1
View File
@@ -271,6 +271,21 @@ func managedReport(m *managedTransaction, out io.Writer, openDB func() (*sql.DB,
if managedHash(b) != j.Hash && (j.State != "prepared" || managedHash(b) != j.Next) {
return errManagedDrift
}
if j.Version == 4 {
if activeOnly {
return fmt.Errorf("v4 deployment has no legacy active IDs")
}
if err := m.validateConfigRef(nil, j.Ref, j.Candidates, j.Hash, j.Scope); err != nil {
return err
}
if j.State == "prepared" {
if err := m.validateConfigRef(nil, j.PendingRef, j.PendingCandidates, j.Next, j.PendingScope); err != nil {
return err
}
}
_, err := fmt.Fprintf(out, "managed-test generation=%d hash=%s state=%s kind=%s\nLive resource health: NOT CHECKED; configured resources only, not production READY.\n", j.Generation, j.Hash, j.State, j.Ref.Kind)
return err
}
groups := map[string]bool{}
operatorCoverage := map[string]bool{}
assetHosts := map[string]bool{}
@@ -420,6 +435,9 @@ func managedTargetSettings(lookup func(string) (string, bool)) (int, error) {
}
func managedCLI(args []string, stdout, stderr io.Writer, d managedDeps) int {
if len(args) == 3 && args[0] == "resource-apply" && args[1] == "--resource-config" {
return resourceApplyCLI(args[2], stdout, stderr, d)
}
if len(args) == 1 && args[0] == "resource-status" {
return resourceStatusCLI(stdout, d)
}
@@ -449,7 +467,22 @@ func managedCLI(args []string, stdout, stderr io.Writer, d managedDeps) int {
if command == "reconcile" {
err = managedPrestartReconcile(m, d)
} else if command == "status" || command == "active-ids" {
err = managedReport(m, stdout, d.openDB, command == "active-ids", len(args) == 2 && args[1] == "--assets")
j, loadErr := m.load()
if loadErr != nil {
err = loadErr
} else if j.Version == 4 {
config, _ := d.lookup("ZAPRET_AUTOTUNE_RESOURCE_CONFIG")
s, e := openResourceApplySnapshot(config, d)
if e != nil {
err = errResourcePlan
} else {
m.resource = s
err = managedReport(m, stdout, d.openDB, command == "active-ids")
s.close()
}
} else {
err = managedReport(m, stdout, d.openDB, command == "active-ids", len(args) == 2 && args[1] == "--assets")
}
} else if command == "experiment" {
startup := false
for i := 1; i < len(args); i += 2 {
@@ -504,6 +537,27 @@ func managedCLI(args []string, stdout, stderr io.Writer, d managedDeps) int {
// Startup may activate only a known baseline or DB-qualified rendered config.
// Reconcile restores bytes before its fixed target-local restart hook runs.
func managedPrestartReconcile(m *managedTransaction, d managedDeps) (err error) {
j, loadErr := m.load()
if loadErr != nil {
return loadErr
}
if j.Version == 4 {
config, _ := d.lookup("ZAPRET_AUTOTUNE_RESOURCE_CONFIG")
s, e := openResourceApplySnapshot(config, d)
if e != nil {
return errResourcePlan
}
defer s.close()
m.resource = s
defer func() { m.resource = nil }()
checks := productionResourceCheckDeps()
m.RollbackHealth = func() (string, error) {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
return resourceApplyHealth(ctx, s, checks)
}
return m.Reconcile()
}
path, err := managedMonitorDatabase(d)
if err != nil {
return err
+104 -24
View File
@@ -30,27 +30,32 @@ func (e *managedRollbackError) Error() string {
func (e *managedRollbackError) Unwrap() error { return e.Cause }
type managedJournal struct {
Version int `json:"version"`
Generation uint64 `json:"generation"`
ID string `json:"transaction_id"`
State string `json:"state"`
Hash string `json:"committed_hash"`
Next string `json:"pending_hash,omitempty"`
Candidates []int64 `json:"committed_candidate_ids,omitempty"`
PendingCandidates []int64 `json:"pending_candidate_ids,omitempty"`
Scope GooglevideoScope `json:"committed_googlevideo_scope,omitempty"`
PendingScope GooglevideoScope `json:"pending_googlevideo_scope,omitempty"`
Version int `json:"version"`
Generation uint64 `json:"generation"`
ID string `json:"transaction_id"`
State string `json:"state"`
Hash string `json:"committed_hash"`
Next string `json:"pending_hash,omitempty"`
Candidates []int64 `json:"committed_candidate_ids,omitempty"`
PendingCandidates []int64 `json:"pending_candidate_ids,omitempty"`
Scope GooglevideoScope `json:"committed_googlevideo_scope,omitempty"`
PendingScope GooglevideoScope `json:"pending_googlevideo_scope,omitempty"`
Ref *managedConfigRef `json:"committed_ref,omitempty"`
PendingRef *managedConfigRef `json:"pending_ref,omitempty"`
}
// Hooks take no commands or arguments. The target adapter binds Reload
// exclusively to /opt/zapret2/init.d/sysv/zapret2 restart and Health to
// health youtube over its configured loopback SOCKS, returning exactly "OK".
// exclusively to /opt/zapret2/init.d/sysv/zapret2 restart. Health is the
// adapter's owned protocol check, returning exactly "OK" (legacy YouTube or
// every required generic resource/TLS pair); optional rollback health is strict.
// Reload switches target-local firewall and daemons for apply and rollback.
// This engine does not manage SOCKS, health scheduling or DB active state.
type managedTransaction struct {
Dir, Name string
Reload func() error
Health func() (string, error)
Dir, Name string
Reload func() error
Health func() (string, error)
RollbackHealth func() (string, error)
resource *resourceApplySnapshot
// Ordinary candidate-ID CLI calls cannot inherit an operator experiment.
// Evaluated against the effective policy only while holding the journal lock.
forbidExperimentalApply bool
@@ -186,7 +191,7 @@ func (m *managedTransaction) load() (managedJournal, error) {
if err != nil {
return j, err
}
if err := json.Unmarshal(b, &j); err != nil {
if err := strictPlanJSON(b, &j); err != nil {
return j, err
}
validHash := func(s string) bool {
@@ -208,9 +213,12 @@ func (m *managedTransaction) load() (managedJournal, error) {
}
return true
}
if (j.Version != 1 && j.Version != 2 && j.Version != 3) || !j.Scope.valid() || !j.PendingScope.valid() || (j.Version < 3 && (j.Scope != "" || j.PendingScope != "")) || (j.Version == 3 && (j.Scope == "" || (j.State == "prepared" && j.PendingScope == ""))) || j.ID == "" || !validHash(j.Hash) || (j.State != "committed" && j.State != "prepared") || (j.State == "prepared" && !validHash(j.Next)) || (j.State == "committed" && (j.Next != "" || len(j.PendingCandidates) != 0 || j.PendingScope != "")) || !validIDs(j.Candidates) || !validIDs(j.PendingCandidates) || (j.Version == 1 && (len(j.Candidates) != 0 || len(j.PendingCandidates) != 0)) {
if (j.Version != 1 && j.Version != 2 && j.Version != 3 && j.Version != 4) || !j.Scope.valid() || !j.PendingScope.valid() || (j.Version < 3 && (j.Scope != "" || j.PendingScope != "")) || (j.Version == 3 && (j.Scope == "" || (j.State == "prepared" && j.PendingScope == ""))) || j.ID == "" || !validHash(j.Hash) || (j.State != "committed" && j.State != "prepared") || (j.State == "prepared" && !validHash(j.Next)) || (j.State == "committed" && (j.Next != "" || len(j.PendingCandidates) != 0 || j.PendingScope != "" || j.PendingRef != nil)) || !validIDs(j.Candidates) || !validIDs(j.PendingCandidates) || (j.Version == 1 && (len(j.Candidates) != 0 || len(j.PendingCandidates) != 0)) {
return j, fmt.Errorf("invalid managed journal")
}
if j.Version < 4 && (j.Ref != nil || j.PendingRef != nil) || j.Version == 4 && (!validManagedConfigRef(j.Ref, j.Candidates, j.Scope, j.Hash) || j.State == "prepared" && !validManagedConfigRef(j.PendingRef, j.PendingCandidates, j.PendingScope, j.Next)) {
return j, fmt.Errorf("invalid managed journal references")
}
return j, nil
}
@@ -234,7 +242,12 @@ func (m *managedTransaction) Initialize() error {
func (m *managedTransaction) checkpoint(point string) error {
if m.fault != nil {
return m.fault(point)
if err := m.fault(point); err != nil {
return err
}
}
if m.resource != nil {
return m.resource.guard()
}
return nil
}
@@ -265,6 +278,9 @@ func (m *managedTransaction) ActiveCandidates(db *sql.DB) ([]int64, error) {
if j.State != "committed" {
return fmt.Errorf("incomplete transaction: reconcile first")
}
if j.Version == 4 && j.Ref.Kind == "generic" {
return fmt.Errorf("generic deployment has no legacy candidate IDs")
}
b, err := m.read(m.Name)
if err != nil {
return err
@@ -299,6 +315,10 @@ func committedManagedJournal(j managedJournal) managedJournal {
if j.Version == 3 {
j.Scope, j.PendingScope = j.PendingScope, ""
}
if j.Version == 4 {
j.Scope, j.PendingScope = j.PendingScope, ""
j.Ref, j.PendingRef = j.PendingRef, nil
}
return j
}
@@ -342,6 +362,10 @@ func (m *managedTransaction) applyRenderedDB(b []byte, db *sql.DB, ids ...int64)
}
func (m *managedTransaction) applyRenderedPolicy(b []byte, db *sql.DB, requested *GooglevideoScope, ids ...int64) error {
return m.applyPolicy(b, db, requested, nil, ids...)
}
func (m *managedTransaction) applyPolicy(b []byte, db *sql.DB, requested *GooglevideoScope, generic *managedConfigRef, ids ...int64) error {
return m.locked(func() error {
j, err := m.load()
if err != nil {
@@ -350,9 +374,33 @@ func (m *managedTransaction) applyRenderedPolicy(b []byte, db *sql.DB, requested
if j.State != "committed" {
return fmt.Errorf("incomplete transaction: reconcile first")
}
if err := validateManagedRefs(db, j.Candidates, j.Hash, j.Scope); err != nil {
if err := m.validateConfigRef(db, j.Ref, j.Candidates, j.Hash, j.Scope); err != nil {
return err
}
if generic == nil && j.Version == 4 {
return fmt.Errorf("generic journal cannot authorize legacy apply")
}
if generic != nil {
if m.resource == nil || m.resource.guard() != nil || m.resource.validate(generic, managedHash(b)) != nil {
return errResourcePlan
}
if j.Version == 4 && j.Ref.Kind == "generic" {
if reflect.DeepEqual(j.Ref, generic) {
current, e := m.read(m.Name)
if e != nil || managedHash(current) != j.Hash {
return errManagedDrift
}
return nil
}
return fmt.Errorf("generic profile rotation unsupported")
}
if len(j.Candidates) != 0 || j.Hash != managedHash([]byte(managedBaseline)) {
return fmt.Errorf("generic apply requires disabled baseline")
}
if m.resource.fresh() != nil {
return errResourcePlan
}
}
scope := j.Scope
if scope == "" {
scope = GooglevideoScopeExact
@@ -360,6 +408,9 @@ func (m *managedTransaction) applyRenderedPolicy(b []byte, db *sql.DB, requested
if requested != nil {
scope = *requested
}
if generic != nil {
scope = ""
}
if m.forbidExperimentalApply && scope == GooglevideoScopeExperimental {
return fmt.Errorf("experimental CLI apply requires explicit guarded operator path")
}
@@ -391,7 +442,7 @@ func (m *managedTransaction) applyRenderedPolicy(b []byte, db *sql.DB, requested
return fmt.Errorf("fresh qualified candidate state required")
}
}
if err := validateManagedRefs(db, ids, managedHash(b), scope); err != nil {
if err := m.validateConfigRef(db, generic, ids, managedHash(b), scope); err != nil {
return err
}
prior, err := m.read(m.Name)
@@ -433,7 +484,12 @@ func (m *managedTransaction) applyRenderedPolicy(b []byte, db *sql.DB, requested
return err
}
j.Generation++
if requested != nil || j.Version == 3 {
if generic != nil {
j.Version = 4
j.Ref = &managedConfigRef{Kind: "baseline"}
j.Scope, j.PendingScope = "", ""
j.PendingRef = generic
} else if requested != nil || j.Version == 3 {
j.Version = 3
if j.Scope == "" {
j.Scope = GooglevideoScopeExact
@@ -470,7 +526,12 @@ func (m *managedTransaction) applyRenderedPolicy(b []byte, db *sql.DB, requested
}
}
if err == nil {
err = m.Reload()
if generic != nil {
err = m.resource.guard()
}
if err == nil {
err = m.Reload()
}
}
if err == nil {
if fault := m.checkpoint("reloaded"); fault != nil {
@@ -491,6 +552,15 @@ func (m *managedTransaction) applyRenderedPolicy(b []byte, db *sql.DB, requested
}
if err == nil {
committed := committedManagedJournal(j)
if generic != nil {
err = m.resource.guard()
}
if err != nil {
if rollback := m.restore(j); rollback != nil {
return &managedRollbackError{Cause: err, Rollback: rollback}
}
return err
}
current, err = m.read(m.Name)
if err == nil && managedHash(current) != j.Next {
return errManagedDrift
@@ -542,6 +612,15 @@ func (m *managedTransaction) restore(j managedJournal) error {
if err := m.Reload(); err != nil {
return err
}
if m.RollbackHealth != nil {
status, err := m.RollbackHealth()
if err != nil {
return err
}
if status != "OK" {
return &managedHealthRejection{Status: status}
}
}
if err := m.owns(j, false); err != nil {
return err
}
@@ -555,6 +634,7 @@ func (m *managedTransaction) restore(j managedJournal) error {
j.State, j.Next = "committed", ""
j.PendingCandidates = nil
j.PendingScope = ""
j.PendingRef = nil
return m.journal(j)
}
@@ -570,10 +650,10 @@ func (m *managedTransaction) Reconcile(databases ...*sql.DB) error {
if len(databases) != 0 {
db = databases[0]
}
if err := validateManagedRefs(db, j.Candidates, j.Hash, j.Scope); err != nil {
if err := m.validateConfigRef(db, j.Ref, j.Candidates, j.Hash, j.Scope); err != nil {
return err
}
if err := validateManagedRefs(db, j.PendingCandidates, j.Next, j.PendingScope); err != nil {
if err := m.validateConfigRef(db, j.PendingRef, j.PendingCandidates, j.Next, j.PendingScope); err != nil {
return err
}
if j.State == "prepared" {
+395
View File
@@ -0,0 +1,395 @@
package main
import (
"bytes"
"context"
"database/sql"
"fmt"
"io"
"net"
"os"
"os/signal"
"strconv"
"strings"
"syscall"
"time"
)
// v4 tags each side independently; generic references never impersonate SQL IDs.
type managedConfigRef struct {
Kind string `json:"kind"`
Registry string `json:"registry_hash,omitempty"`
Matrix string `json:"matrix_hash,omitempty"`
Source string `json:"source_fingerprint,omitempty"`
Plan string `json:"plan_digest,omitempty"`
Config string `json:"config_hash,omitempty"`
}
func validManagedConfigRef(r *managedConfigRef, ids []int64, scope GooglevideoScope, hash string) bool {
if r == nil {
return false
}
switch r.Kind {
case "generic":
return len(ids) == 0 && scope == "" && hashOK(r.Registry) && hashOK(r.Matrix) && hashOK(r.Source) && hashOK(r.Plan) && r.Config == hash
case "baseline", "legacy":
return r.Registry == "" && r.Matrix == "" && r.Source == "" && r.Plan == "" && r.Config == "" && (r.Kind == "legacy" && len(ids) > 0 || r.Kind == "baseline" && len(ids) == 0 && scope == "" && hash == managedHash([]byte(managedBaseline)))
}
return false
}
func (m *managedTransaction) validateConfigRef(db *sql.DB, ref *managedConfigRef, ids []int64, hash string, scope GooglevideoScope) error {
if ref == nil {
return validateManagedRefs(db, ids, hash, scope)
}
if !validManagedConfigRef(ref, ids, scope, hash) {
return errResourcePlan
}
if ref.Kind != "generic" {
return validateManagedRefs(db, ids, hash, scope)
}
if m.resource == nil {
return errResourcePlan
}
return m.resource.validate(ref, hash)
}
type resourceApplySnapshot struct {
store *resourcePlanStore
source resourceMonitorSource
registry strategyRegistry
matrix qualificationMatrix
ref managedConfigRef
rendered []byte
guard func() error
now func() time.Time
close func()
}
// Historical reconstruction uses the proof's original completion horizon. Only
// new apply checks wall-clock freshness; expiry cannot erase committed identity.
func resourceApplyPlan(source resourceMonitorSource, key []byte, registry strategyRegistry, matrix qualificationMatrix, now time.Time) (resourcePlan, []byte, error) {
p := deriveResourcePlan(source, key, registry, matrix, now)
if p.Evidence != "authenticated" || len(p.Unassigned) != 0 || len(p.Assignments) == 0 {
return p, nil, errResourcePlan
}
for _, s := range source.config.Services {
for _, r := range s.Resources {
if r.TLS != "both" {
return p, nil, errResourcePlan
}
}
}
profiles := []string{}
seen := map[string]bool{}
for _, a := range p.Assignments {
d, e := canonicalStrategy(strategyDefinition{ID: a.Strategy, Args: a.Args, Selectors: a.Selectors})
if e != nil || d.Selectors.TLS != "both" || len(a.Hosts) == 0 {
return p, nil, errResourcePlan
}
hosts := []string{}
for _, host := range a.Hosts {
if !validDomain(host) || host != strings.ToLower(host) || net.ParseIP(host) != nil || seen[host] {
return p, nil, errResourcePlan
}
seen[host] = true
hosts = append(hosts, "^"+host)
}
profiles = append(profiles, "--filter-tcp=443 --filter-l7=tls --hostlist-domains="+strings.Join(hosts, ",")+" "+d.Args)
}
return p, []byte("# Autotuner-owned generic config; runtime acceptance pending.\nNFQWS2_ENABLE=1\n" + managedFirewallConfig + "NFQWS2_OPT='" + strings.Join(profiles, " --new ") + "'\n"), nil
}
func (s *resourceApplySnapshot) fresh() error {
p, b, e := resourceApplyPlan(s.source, s.store.files.key, s.registry, s.matrix, s.now())
if e != nil || planDigest(p.Assignments) != s.ref.Plan || managedHash(b) != s.ref.Config {
return errResourcePlan
}
return s.guard()
}
// ponytail: one immutable qualification batch, baseline-only activation and
// identical-request resume. Rotation/active maintenance need versioned snapshots
// and job-bound rollback, not overwriting the held registry or matrix.
func (s *resourceApplySnapshot) validate(ref *managedConfigRef, hash string) error {
if ref == nil || *ref != s.ref || hash != s.ref.Config {
return errResourcePlan
}
return s.guard()
}
// No imported proposal or version probe. Keep private proof/source/native files
// held across staging, health and commit and bind every selected candidate.
func openResourceApplySnapshot(config string, d managedDeps) (*resourceApplySnapshot, error) {
return openResourceApplySnapshotAt(config, d, "/opt/zapret2")
}
func openResourceApplySnapshotAt(config string, d managedDeps, root string) (*resourceApplySnapshot, error) {
store, e := openResourcePlanStore(d.directory, false)
if e != nil {
return nil, errResourcePlan
}
s := &resourceApplySnapshot{store: store, now: time.Now}
var engine *qualificationBinary
var sourceFile *os.File
lua := map[string]*os.File{}
s.close = func() {
for _, f := range lua {
f.Close()
}
if sourceFile != nil {
sourceFile.Close()
}
if engine != nil {
engine.file.Close()
}
store.close()
}
fail := func() (*resourceApplySnapshot, error) { s.close(); return nil, errResourcePlan }
s.registry, e = store.registry()
if e != nil {
return fail()
}
s.matrix, e = store.matrix(s.registry)
if e != nil {
return fail()
}
marker, markerBytes, e := store.files.openEntry(managedMarkerName)
if e != nil {
return fail()
}
store.held[managedMarkerName], store.snapshots[managedMarkerName] = marker, markerBytes
fd, e := syscall.Open(config, syscall.O_RDONLY|syscall.O_NOFOLLOW|syscall.O_NONBLOCK|syscall.O_CLOEXEC, 0)
if e != nil {
return fail()
}
sourceFile = os.NewFile(uintptr(fd), "resource source")
if resourcePrivateEntry(sourceFile) != nil {
return fail()
}
sourceBytes, e := io.ReadAll(io.LimitReader(sourceFile, 65537))
if e != nil || len(sourceBytes) > 65536 {
return fail()
}
s.source, e = readResourceMonitorSource(config, d.lookup, store.files.key)
if e != nil {
return fail()
}
engine, e = openQualificationEngine(root)
if e != nil {
return fail()
}
native, e := engine.filesHash()
if e != nil {
return fail()
}
// Hold Lua inodes as well as the qualification hash; equal-byte replacement
// during a transaction is still a trust change.
for _, name := range []string{"lua/zapret-lib.lua", "lua/zapret-antidpi.lua"} {
path := engine.root + "/" + name
f, e := os.Open(path)
if e != nil {
return fail()
}
lua[path] = f
}
var horizon int64
for _, run := range s.matrix.Runs {
if run.Completed > horizon {
horizon = run.Completed
}
}
p, rendered, e := resourceApplyPlan(s.source, store.files.key, s.registry, s.matrix, time.UnixMilli(horizon))
if e != nil {
return fail()
}
for _, run := range s.matrix.Runs {
if run.Attestation.LuaHash != native {
return fail()
}
}
s.rendered = rendered
s.ref = managedConfigRef{Kind: "generic", Registry: managedHash(store.snapshots[planRegistryName]), Matrix: managedHash(store.snapshots[planMatrixName]), Source: s.source.fingerprint, Plan: planDigest(p.Assignments), Config: managedHash(rendered)}
s.guard = func() error {
if store.guard() != nil || managedCheckIdentity(&managedTransaction{Dir: d.directory, Name: "config"}) != nil {
return errResourcePlan
}
current, e := os.Lstat(config)
held, he := sourceFile.Stat()
b, re := io.ReadAll(io.NewSectionReader(sourceFile, 0, 65537))
fresh, se := readResourceMonitorSource(config, d.lookup, store.files.key)
hash, ne := engine.filesHash()
if e != nil || he != nil || re != nil || se != nil || ne != nil || !os.SameFile(current, held) || resourcePrivateEntry(sourceFile) != nil || !bytes.Equal(b, sourceBytes) || fresh.fingerprint != s.source.fingerprint || hash != native {
return errResourcePlan
}
for path, f := range lua {
current, e := os.Lstat(path)
held, he := f.Stat()
if e != nil || he != nil || !os.SameFile(current, held) {
return errResourcePlan
}
}
return nil
}
if s.guard() != nil {
return fail()
}
return s, nil
}
func resourceApplyHealth(ctx context.Context, s *resourceApplySnapshot, checks resourceCheckDeps) (string, error) {
guard := checks.guard
checks.guard = func() error {
if s.guard() != nil {
return errResourcePlan
}
return guard()
}
passed := true
for _, service := range s.source.config.Services {
for _, spec := range service.Resources {
for _, tls := range []string{"tls12", "tls13"} {
result, e := runResourceProtocol(ctx, service.ID, spec, tls, s.source.server, s.source.proxy, checks)
if e != nil || ctx.Err() != nil {
return "DEGRADED", errResourcePlan
}
passed = passed && result.Outcome == "pass"
}
}
}
if s.guard() != nil {
return "BLOCKED", errResourcePlan
}
if !passed {
return "DEGRADED", errResourcePlan
}
return "OK", nil
}
func resourceApplyMode(config string, d managedDeps) error {
if os.Geteuid() != 0 || os.Getppid() != 1 || config == "" {
return errResourcePlan
}
if _, e := managedRuntimeGuard(d); e != nil {
return errResourcePlan
}
cmd, e := os.ReadFile("/proc/1/cmdline")
if e != nil || string(cmd) != "/bin/sh\x00/opt/managed-target-entrypoint.sh\x00" {
return errResourcePlan
}
env, e := os.ReadFile("/proc/1/environ")
if e != nil {
return errResourcePlan
}
values := map[string]string{}
for _, item := range strings.Split(string(env), "\x00") {
k, v, _ := strings.Cut(item, "=")
values[k] = v
}
if values["ZAPRET_AUTOTUNE_RESOURCE_APPLY"] != "1" || values["ZAPRET_AUTOTUNE_RESOURCE_CONFIG"] != config || values["ZAPRET_AUTOTUNE_QUALIFY_CONFIG"] != "" || values["ZAPRET_AUTOTUNE_STRATEGY_CONFIG"] != "" {
return errResourcePlan
}
for _, ns := range []string{"pid", "net"} {
a, e := os.Readlink("/proc/1/ns/" + ns)
b, f := os.Readlink("/proc/self/ns/" + ns)
if e != nil || f != nil || a != b {
return errResourcePlan
}
}
if !resourceProcessOwned("/proc", os.Getpid(), "/usr/local/bin/zapret-autotune", []string{"/usr/local/bin/zapret-autotune", "managed", "resource-apply", "--resource-config", config}) {
return errResourcePlan
}
dir, e := os.Open("/run/zapret-autotune-ss")
if e != nil {
return errResourcePlan
}
defer dir.Close()
f, e := resourcePrivateFile(dir, "monitor-ready")
if e != nil {
return errResourcePlan
}
defer f.Close()
b, e := io.ReadAll(io.LimitReader(f, 32))
if e != nil || string(b) != strconv.Itoa(os.Getpid())+"\n" {
return errResourcePlan
}
return nil
}
func resourceApplyCLI(config string, stdout, stderr io.Writer, d managedDeps) int {
blocked := func() int {
fmt.Fprintln(stderr, "BLOCKED: generic apply trust/health incomplete; deployment preserved or prepared for recovery")
return 78
}
if resourceApplyMode(config, d) != nil {
return blocked()
}
lock, e := acquireManagedMonitorSingleton(d)
if e != nil {
return blocked()
}
defer lock.close()
if qualificationNoWorker() != nil {
return blocked()
}
s, e := openResourceApplySnapshot(config, d)
if e != nil {
return blocked()
}
defer s.close()
signals, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
ctx, cancel := context.WithTimeout(signals, 15*time.Minute)
defer cancel()
checks := productionResourceCheckDeps()
m := &managedTransaction{Dir: d.directory, Name: "config", resource: s, Reload: func() error { return d.run(d.script, "restart") }}
m.Health = func() (string, error) { return resourceApplyHealth(ctx, s, checks) }
m.RollbackHealth = func() (string, error) {
cleanup, cancel := context.WithTimeout(context.WithoutCancel(ctx), 30*time.Second)
defer cancel()
return resourceApplyHealth(cleanup, s, checks)
}
if resourceSingletonGuard(lock, d.directory) != nil || m.Reconcile() != nil {
return blocked()
}
originalGuard := s.guard
s.guard = func() error {
if resourceApplyMode(config, d) != nil || resourceSingletonGuard(lock, d.directory) != nil {
return errResourcePlan
}
return originalGuard()
}
j, e := m.load()
if e != nil {
return blocked()
}
if (j.Version != 4 || j.Ref.Kind == "baseline") && qualificationNoWorker() != nil {
return blocked()
}
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e != nil {
return blocked()
}
fmt.Fprintln(stdout, "generic resource apply: committed; configured resources only, not production READY")
return 0
}
func resourceDeploymentIdentity(config string, d managedDeps) error {
m := &managedTransaction{Dir: d.directory, Name: "config"}
j, e := m.load()
if e != nil {
return errResourcePlan
}
if j.Version != 4 {
return nil
}
if j.State != "committed" {
return errResourcePlan
}
s, e := openResourceApplySnapshot(config, d)
if e != nil {
return errResourcePlan
}
defer s.close()
m.resource = s
return m.validateConfigRef(nil, j.Ref, j.Candidates, j.Hash, j.Scope)
}
+463
View File
@@ -0,0 +1,463 @@
package main
import (
"bytes"
"context"
"encoding/json"
"errors"
"io"
"net"
"os"
"path/filepath"
"strconv"
"strings"
"testing"
"time"
)
func completeApplyFixture(t *testing.T) (resourceMonitorSource, []byte, strategyRegistry, qualificationMatrix, time.Time) {
s, key, r, m, now := planFixture(t)
// e was intentionally unknown in the proposal fixture; this request covers
// only a/b/c/d, including a's cross-service duplicate resource.
s.config.Services = append(s.config.Services[:4:4], s.config.Services[5])
return s, key, r, m, now
}
func TestResourceApplyRender(t *testing.T) {
s, key, r, m, now := completeApplyFixture(t)
p, b, e := resourceApplyPlan(s, key, r, m, now)
if e != nil || !p.ProposalOnly || len(p.Assignments) != 3 {
t.Fatal(p, e)
}
if !strings.Contains(string(b), "--hostlist-domains=^a.example,^c.example") || strings.Count(string(b), " --new ") != 2 {
t.Fatal(string(b))
}
for i := 0; i < 30; i++ {
_, next, e := resourceApplyPlan(s, key, r, m, now)
if e != nil || !bytes.Equal(next, b) {
t.Fatal("nondeterministic render")
}
}
for _, kind := range []string{"negative", "unknown", "incomplete", "stale", "future", "config", "tls", "unsafe"} {
t.Run(kind, func(t *testing.T) {
s, key, r, m, now := completeApplyFixture(t)
switch kind {
case "negative":
m.Runs[0].Checks[len(m.Runs[0].Checks)-1].Outcome = "expectation_failed"
case "unknown":
m.Runs[0].Attestation.Joined = false
case "incomplete":
m.Runs[0].Checks = m.Runs[0].Checks[:len(m.Runs[0].Checks)-1]
case "stale":
now = now.Add(25 * time.Hour)
case "future":
now = now.Add(-time.Hour)
case "config":
s.fingerprint = strings.Repeat("f", 64)
case "tls":
s.config.Services[0].Resources[0].TLS = "tls12"
case "unsafe":
r.Strategies[0].Args = "--lua-desync=execute:cmd=id"
}
if _, _, e := resourceApplyPlan(s, key, r, m, now); e == nil {
t.Fatal("authorized invalid proof")
}
})
}
}
// Uses the production snapshot factory, private HMAC envelopes and shared
// transaction engine; native/HTTP fixtures perform no network operations.
func nativeApplyFixture(t *testing.T) (*managedTransaction, *resourceApplySnapshot, string, string) {
t.Helper()
root, _, _ := qualificationBinaryFixture(t)
dir := t.TempDir()
if e := os.Chmod(dir, 0700); e != nil {
t.Fatal(e)
}
m := &managedTransaction{Dir: dir, Name: "config", Reload: func() error { return nil }, Health: func() (string, error) { return "OK", nil }}
if e := managedInitialize(m); e != nil {
t.Fatal(e)
}
store, e := openResourcePlanStore(dir, true)
if e != nil {
t.Fatal(e)
}
config := resourceCheckFile(t, strings.Replace(resourceCheckExample, `"tls":"tls12"`, `"tls":"both"`, 1))
d := managedDeps{directory: dir, lookup: resourceCheckLookup}
source, e := readResourceMonitorSource(config, d.lookup, store.files.key)
if e != nil {
t.Fatal(e)
}
_, _, registry, _, _ := planFixture(t)
registry.Strategies = registry.Strategies[:1]
b, _ := json.Marshal(registry)
if e := store.importDefinitions(b); e != nil {
t.Fatal(e)
}
engine, e := openQualificationEngine(root)
if e != nil {
t.Fatal(e)
}
hash, e := engine.filesHash()
engine.file.Close()
if e != nil {
t.Fatal(e)
}
now := time.Now()
def := registry.Strategies[0]
run := qualificationRun{ID: "run-test", Strategy: def.ID, Definition: planDigest(def), Config: source.fingerprint, Started: now.Add(-time.Minute).UnixMilli(), Completed: now.UnixMilli(), Attestation: qualificationAttestation{Collector: "owned-resource-nfq-v1", Runtime: "nfqws2-1.0.2", Definition: planDigest(def), LuaHash: hash, RulesHash: strings.Repeat("b", 64), WorkerArgsHash: strings.Repeat("c", 64), Queue: managedAssetQueue, PID: 123, StartTime: 456, Restored: true, Joined: true}}
for _, service := range source.config.Services {
for _, spec := range service.Resources {
for _, tls := range []string{"tls12", "tls13"} {
run.Checks = append(run.Checks, qualificationCheck{Service: service.ID, Resource: spec.ID, TLS: tls, Fingerprint: resourceRowFingerprint(store.files.key, source, spec), Outcome: "pass", Attempts: 3})
}
}
}
matrix := qualificationMatrix{Version: 1, Registry: planDigest(registry), Runs: []qualificationRun{run}}
if e := store.saveCompleted(completedQualification{matrix: matrix, seal: planDigest(matrix)}); e != nil {
t.Fatal(e)
}
store.close()
s, e := openResourceApplySnapshotAt(config, d, root)
if e != nil {
t.Fatal(e)
}
t.Cleanup(s.close)
m.resource = s
return m, s, config, root
}
func TestResourceApplyLifecycle(t *testing.T) {
m, s, config, root := nativeApplyFixture(t)
reloads := 0
m.Reload = func() error { reloads++; return nil }
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e != nil {
t.Fatal(e)
}
j, e := m.load()
if e != nil || j.Version != 4 || j.State != "committed" || j.Ref.Kind != "generic" || j.PendingRef != nil || reloads != 1 {
t.Fatal(j, e, reloads)
}
if _, e := m.ActiveCandidates(nil); e == nil {
t.Fatal("generic treated as empty baseline")
}
s.now = func() time.Time { return time.Now().Add(25 * time.Hour) }
if s.fresh() == nil {
t.Fatal("new expired apply authorized")
}
if e := m.Reconcile(); e != nil {
t.Fatal("aged committed proof invalidated", e)
}
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e != nil || reloads != 1 {
t.Fatal("replay changed runtime", e, reloads)
}
before, _ := os.ReadFile(filepath.Join(m.Dir, "config"))
raw, _ := os.ReadFile(config)
if e := os.WriteFile(config, bytes.Replace(raw, []byte("example.com"), []byte("changed.com"), 1), 0600); e != nil {
t.Fatal(e)
}
if e := m.Reconcile(); e == nil {
t.Fatal("source drift reconciled")
}
after, _ := os.ReadFile(filepath.Join(m.Dir, "config"))
if !bytes.Equal(before, after) {
t.Fatal("drift erased deployment")
}
if _, e := openResourceApplySnapshotAt(config, managedDeps{directory: m.Dir, lookup: resourceCheckLookup}, root); e == nil {
t.Fatal("changed source accepted after reopen")
}
}
func TestResourceApplyCrashRecovery(t *testing.T) {
for _, point := range []string{"staged", "backup", "prepared", "replaced", "reloaded", "healthy"} {
t.Run(point, func(t *testing.T) {
m, s, _, _ := nativeApplyFixture(t)
m.fault = func(p string) error {
if p == point {
return errors.New("process death")
}
return nil
}
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e == nil {
t.Fatal("fault ignored")
}
m.fault = nil
if e := m.Reconcile(); e != nil {
t.Fatal(e)
}
b, _ := m.read("config")
j, _ := m.load()
if string(b) != managedBaseline || j.State != "committed" || j.Ref != nil && j.Ref.Kind != "baseline" {
t.Fatal("pending promoted", j, string(b))
}
})
}
}
func TestResourceApplyRollback(t *testing.T) {
for _, failure := range []string{"reload", "health", "rollback-reload", "rollback-health"} {
t.Run(failure, func(t *testing.T) {
m, s, _, _ := nativeApplyFixture(t)
calls := 0
m.Reload = func() error {
calls++
if failure == "reload" && calls == 1 || failure == "rollback-reload" && calls == 2 {
return errors.New("reload")
}
return nil
}
m.Health = func() (string, error) { return "DEGRADED", nil }
m.RollbackHealth = func() (string, error) {
if failure == "rollback-health" {
return "DEGRADED", nil
}
return "OK", nil
}
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e == nil {
t.Fatal("failure committed")
}
j, _ := m.load()
b, _ := m.read("config")
want := "committed"
if strings.HasPrefix(failure, "rollback-") {
want = "prepared"
}
if j.State != want || string(b) != managedBaseline {
t.Fatal(j, string(b))
}
})
}
}
func TestResourceApplyHeldMutation(t *testing.T) {
for _, kind := range []string{"matrix", "source", "lua", "directory"} {
t.Run(kind, func(t *testing.T) {
m, s, config, root := nativeApplyFixture(t)
m.fault = func(point string) error {
if point != "staged" {
return nil
}
path := config
switch kind {
case "matrix":
path = filepath.Join(m.Dir, planMatrixName)
case "lua":
path = filepath.Join(root, "lua/zapret-lib.lua")
case "directory":
return os.Rename(m.Dir, m.Dir+"-moved")
}
b, e := os.ReadFile(path)
if e != nil {
return e
}
if e := os.Remove(path); e != nil {
return e
}
return os.WriteFile(path, b, 0600)
}
reloads := 0
m.Reload = func() error { reloads++; return nil }
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e == nil || reloads != 0 {
t.Fatal("mutation authorized", e, reloads)
}
})
}
}
func TestResourceApplyJournalTags(t *testing.T) {
m, s, _, _ := nativeApplyFixture(t)
for _, kind := range []string{"unknown-version", "mixed", "missing", "unknown-tag", "old-tag"} {
t.Run(kind, func(t *testing.T) {
j := managedJournal{Version: 4, ID: "fixture", State: "committed", Hash: s.ref.Config, Ref: &s.ref}
switch kind {
case "unknown-version":
j.Version = 5
case "mixed":
j.Candidates = []int64{1}
case "missing":
j.Ref = nil
case "unknown-tag":
r := s.ref
r.Kind = "other"
j.Ref = &r
case "old-tag":
j.Version = 3
j.Scope = GooglevideoScopeExact
}
if e := m.journal(j); e != nil {
t.Fatal(e)
}
if _, e := m.load(); e == nil {
t.Fatal("contradictory journal accepted")
}
})
}
}
func TestResourceApplyHealthCollector(t *testing.T) {
m, s, _, _ := nativeApplyFixture(t)
_ = m
for _, mode := range []string{"", "verify", "status500"} {
calls := 0
checks := resourceCheckDeps{guard: func() error { return nil }, resolve: func(context.Context, string, string) ([]net.IP, error) {
return []net.IP{net.ParseIP("93.184.216.34")}, nil
}, runner: func(_ context.Context, args []string, stdin io.Reader, out, diag io.Writer) error {
calls++
config, _ := io.ReadAll(stdin)
if !bytes.Contains(config, []byte("connect-to")) {
t.Fatal("missing pin")
}
raw, err := strconv.Unquote(strings.TrimPrefix(strings.Split(string(config), "\n")[0], "url = "))
if err != nil {
t.Fatal(err)
}
method, tls := "GET", "tls12"
for _, arg := range args {
if arg == "--head" {
method = "HEAD"
}
if arg == "--tlsv1.3" {
tls = "tls13"
}
if arg == "--insecure" {
t.Fatal("TLS bypass")
}
}
configuredWire(raw, method, tls, mode, out, diag)
return nil
}}
status, e := resourceApplyHealth(context.Background(), s, checks)
if calls != 4 || (mode == "" && (e != nil || status != "OK")) || (mode != "" && e == nil) {
t.Fatal(mode, calls, status, e)
}
}
}
func TestResourceApplyUnauthorizedCLI(t *testing.T) {
var out bytes.Buffer
d := managedDeps{}
if code := resourceApplyCLI("/private/config", &out, &out, d); code != 78 {
t.Fatal(code)
}
}
func TestResourceApplyFreshBeforeStaging(t *testing.T) {
for _, kind := range []string{"expired", "forged", "native", "proof", "unsupported"} {
t.Run(kind, func(t *testing.T) {
m, s, _, root := nativeApplyFixture(t)
ref := s.ref
switch kind {
case "expired":
s.now = func() time.Time { return time.Now().Add(25 * time.Hour) }
case "forged":
ref.Plan = strings.Repeat("a", 64)
case "native":
if e := os.WriteFile(filepath.Join(root, "lua/zapret-antidpi.lua"), []byte("drift"), 0755); e != nil {
t.Fatal(e)
}
case "proof":
if e := os.WriteFile(filepath.Join(m.Dir, planMatrixName), []byte(`{"payload":{},"mac":"forged"}`), 0600); e != nil {
t.Fatal(e)
}
case "unsupported":
s.source.config.Services[0].Resources[0].TLS = "tls13"
}
reloads, stages := 0, 0
m.Reload = func() error { reloads++; return nil }
m.fault = func(point string) error {
if point == "staged" {
stages++
}
return nil
}
if e := m.applyPolicy(s.rendered, nil, nil, &ref); e == nil || stages != 0 || reloads != 0 {
t.Fatal("unqualified staging", e, stages, reloads)
}
b, _ := m.read("config")
j, _ := m.load()
if string(b) != managedBaseline || j.Version != 2 || j.Generation != 0 {
t.Fatal("baseline changed", j)
}
})
}
}
func TestResourceApplyPreparedExpiredRecoveryHealth(t *testing.T) {
m, s, _, _ := nativeApplyFixture(t)
m.fault = func(point string) error {
if point == "healthy" {
return errors.New("crash")
}
return nil
}
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e == nil {
t.Fatal("crash missed")
}
m.fault = nil
s.now = func() time.Time { return time.Now().Add(25 * time.Hour) }
m.RollbackHealth = func() (string, error) { return "DEGRADED", nil }
if e := m.Reconcile(); e == nil {
t.Fatal("unhealthy rollback finalized")
}
j, _ := m.load()
b, _ := m.read("config")
if j.State != "prepared" || string(b) != managedBaseline {
t.Fatal("baseline was not physically restored", j)
}
m.RollbackHealth = func() (string, error) { return "OK", nil }
if e := m.Reconcile(); e != nil {
t.Fatal("expiry blocked recovery", e)
}
}
func TestResourceApplyCancellationJoinsBeforeRollback(t *testing.T) {
m, s, _, _ := nativeApplyFixture(t)
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
joined := false
checks := resourceCheckDeps{guard: func() error { return nil }, resolve: func(context.Context, string, string) ([]net.IP, error) {
return []net.IP{net.ParseIP("93.184.216.34")}, nil
}, runner: func(ctx context.Context, _ []string, _ io.Reader, _, _ io.Writer) error {
cancel()
<-ctx.Done()
joined = true
return ctx.Err()
}}
m.Health = func() (string, error) { return resourceApplyHealth(ctx, s, checks) }
reloads := 0
m.Reload = func() error {
reloads++
if reloads == 2 && !joined {
t.Fatal("rollback raced health child")
}
return nil
}
m.RollbackHealth = func() (string, error) {
if !joined {
t.Fatal("rollback health raced child")
}
return "OK", nil
}
if e := m.applyPolicy(s.rendered, nil, nil, &s.ref); e == nil || reloads != 2 || !joined {
t.Fatal(e, reloads, joined)
}
j, _ := m.load()
if j.State != "committed" || j.Ref.Kind != "baseline" {
t.Fatal(j)
}
}
func TestResourceApplyRefusesActiveLegacy(t *testing.T) {
m, s, _, _ := nativeApplyFixture(t)
db := testStore(t)
id := managedCandidateFixture(t, db, "frontend", "www.youtube.com", "--lua-desync=multisplit", true)
m.resource = nil
if e := m.Apply(db, []int64{id}, GooglevideoScopeExact); e != nil { t.Fatal(e) }
before, _ := m.read("config"); journal, _ := m.read(managedJournalName)
m.resource = s
reloads := 0; m.Reload = func() error { reloads++; return nil }
if e := m.applyPolicy(s.rendered, db, nil, &s.ref); e == nil || reloads != 0 { t.Fatal("active legacy maintenance authorized", e) }
after, _ := m.read("config"); nextJournal, _ := m.read(managedJournalName)
if !bytes.Equal(before, after) || !bytes.Equal(journal, nextJournal) { t.Fatal("legacy state changed") }
}
+6
View File
@@ -345,6 +345,9 @@ func resourceMonitorCLIWithCheck(ctx context.Context, path string, stdout, stder
if resourceManagedIdentity(target.directory) != nil {
return errResourceState
}
if resourceDeploymentIdentity(path, target) != nil {
return errResourceState
}
return check.guard()
},
probe: func(ctx context.Context, service string, spec resourceCheckSpec, source resourceMonitorSource, guard func() error) (string, error) {
@@ -394,6 +397,9 @@ func resourceStatusCLIAt(stdout io.Writer, target managedDeps, proc string) int
if err != nil || check.guard() != nil || resourceManagedIdentity(target.directory) != nil {
return blocked()
}
if resourceDeploymentIdentity(path, target) != nil {
return blocked()
}
rows := reconcileResourceRows(store.key, source, state.Rows)
b1, _ := json.Marshal(rows)
b2, _ := json.Marshal(state.Rows)
+35
View File
@@ -27,7 +27,15 @@ with tempfile.TemporaryDirectory(prefix='target-config-') as tmp:
ZAPRET_AUTOTUNE_YOUTUBE_SERVICE='0', ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE='cold',
ZAPRET_AUTOTUNE_VIDEO_ID='dQw4w9WgXcQ',
ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS='["dQw4w9WgXcQ"]',
ZAPRET_AUTOTUNE_TARGET_ROLE='managed-test',
ZAPRET_AUTOTUNE_TARGET_ID='zapret-autotune-managed-v1',
ZAPRET_AUTOTUNE_TARGET_DIR='/var/lib/zapret-autotune/target',
ZAPRET_CONFIG='/var/lib/zapret-autotune/target/config',
SOCKS_PORT='1080',
ZAPRET_AUTOTUNE_SOCKS_PROXY='socks5h://127.0.0.1:1080')
for name in ('ZAPRET_AUTOTUNE_RESOURCE_APPLY', 'ZAPRET_AUTOTUNE_RESOURCE_CONFIG',
'ZAPRET_AUTOTUNE_QUALIFY_CONFIG', 'ZAPRET_AUTOTUNE_STRATEGY_CONFIG'):
base.pop(name, None)
for dns, valid in (('', True), ('8.8.8.8', True), ('2001:4860:4860::8888', True),
('example.org', False), ('8.8.8.8\n', False), ('-d 8.8.8.8', False)):
for path in private.iterdir():
@@ -61,6 +69,23 @@ with tempfile.TemporaryDirectory(prefix='target-config-') as tmp:
assert base['SS_PASSWORD'].encode() not in result.stdout + result.stderr
if not valid:
assert not list(private.iterdir())
for overrides, valid in (
({'ZAPRET_AUTOTUNE_RESOURCE_APPLY': '1'}, False),
({'ZAPRET_AUTOTUNE_RESOURCE_APPLY': 'yes'}, False),
({'ZAPRET_AUTOTUNE_RESOURCE_APPLY': '1', 'ZAPRET_AUTOTUNE_RESOURCE_CONFIG': '/private/resources.json'}, True),
({'ZAPRET_AUTOTUNE_RESOURCE_APPLY': '1', 'ZAPRET_AUTOTUNE_RESOURCE_CONFIG': '/private/resources.json',
'ZAPRET_AUTOTUNE_TARGET_ROLE': ''}, False),
({'ZAPRET_AUTOTUNE_RESOURCE_APPLY': '1', 'ZAPRET_AUTOTUNE_RESOURCE_CONFIG': '/private/resources.json',
'SOCKS_PORT': '1081'}, False),
({'ZAPRET_AUTOTUNE_RESOURCE_APPLY': '1', 'ZAPRET_AUTOTUNE_RESOURCE_CONFIG': '/private/resources.json',
'ZAPRET_AUTOTUNE_QUALIFY_CONFIG': '/private/resources.json', 'ZAPRET_AUTOTUNE_STRATEGY_CONFIG': '/private/strategies.json'}, False),
):
for path in private.iterdir():
path.unlink()
result = subprocess.run(['sh', str(entry)], env=dict(base, **overrides), capture_output=True, timeout=5)
assert result.returncode == (0 if valid else 64), result.stderr
if not valid:
assert not list(private.iterdir()), 'Invalid apply mode created service configs'
# Render privately, never print the config/password; config does not deploy.
compose_env = {k: v for k, v in base.items() if not k.startswith('ZAPRET_AUTOTUNE_')}
factory_cases = []
@@ -100,6 +125,16 @@ with tempfile.TemporaryDirectory(prefix='target-config-') as tmp:
factory_cases.append({'environment': {k: v for k, v in settings.items()
if k.startswith('ZAPRET_AUTOTUNE_')}, 'ids': ids})
# Validate the actual rendered DNS environment, not an invented equivalent.
result = subprocess.run(['docker', 'compose', '--env-file', '/dev/null', '-f',
str(ROOT / 'deploy/managed-target/compose.yaml'), 'config', '--format', 'json'],
env=dict(compose_env, ZAPRET_AUTOTUNE_RESOURCE_APPLY='1',
ZAPRET_AUTOTUNE_RESOURCE_CONFIG='/private/resources.json',
ZAPRET_AUTOTUNE_STOP_GRACE_PERIOD='90s'), capture_output=True, timeout=10)
assert result.returncode == 0, 'Generic Compose config failed (output suppressed)'
target = json.loads(result.stdout)['services']['target']
assert target['environment']['ZAPRET_AUTOTUNE_RESOURCE_APPLY'] == '1'
assert target['environment']['ZAPRET_AUTOTUNE_RESOURCE_CONFIG'] == '/private/resources.json'
assert target['stop_grace_period'] == '1m30s'
# Compose must not inject Google bootstrap into a custom-provider request.
for service in ('0', '1'):
for provider in ({}, {'ZAPRET_AUTOTUNE_DOH_URL': 'https://secret.invalid/token'}):
+6 -2
View File
@@ -55,7 +55,8 @@ assert "Docker administrators/root can still read container" in readme
assert "removed\nfrom the entrypoint environment" in readme
assert "No host networking" in readme
assert ' restart: "no"\n' in compose
assert " stop_grace_period: 240s\n" in compose
assert " stop_grace_period: ${ZAPRET_AUTOTUNE_STOP_GRACE_PERIOD-240s}\n" in compose
assert 'ZAPRET_AUTOTUNE_RESOURCE_APPLY: ${ZAPRET_AUTOTUNE_RESOURCE_APPLY-0}' in compose
assert 'bounded_wait "$monitor_pid" 45' in entrypoint
assert '/bin/sh /opt/managed-target-supervision.sh --watchdog' in entrypoint
assert 'setsid /bin/sh /opt/managed-target-supervision.sh --monitor' in entrypoint
@@ -77,6 +78,9 @@ for command, seconds in (("/usr/local/bin/zapret-autotune managed init", 45),
assert entrypoint.index("managed reconcile || return 78") < entrypoint.index("zapret2 start || return 78")
experimental = 'managed apply --googlevideo-experiment native-circular --googlevideo-scope experimental_googlevideo_suffix --googlevideo-request startup || exit 78'
assert entrypoint.index('zapret2 start || return 78') < entrypoint.index(experimental)
assert entrypoint.index(experimental) < entrypoint.index('setsid /bin/sh /opt/managed-target-supervision.sh --monitor')
assert entrypoint.index(experimental) < entrypoint.index('launch_controller "$@" || return 78')
generic_apply = 'launch_controller managed resource-apply --resource-config "$ZAPRET_AUTOTUNE_RESOURCE_CONFIG"'
assert entrypoint.index('managed reconcile || return 78') < entrypoint.index(generic_apply) < entrypoint.index('zapret2 start || return 78')
assert entrypoint.index(generic_apply) < entrypoint.index('set -- managed monitor')
assert entrypoint.index('managed experimental-request-check || return 78') < entrypoint.index('managed init || return 78')
assert 'rm "${ZAPRET_CONFIG%/*}/.autotune-experimental-suffix-request"' not in entrypoint
+74 -3
View File
@@ -78,7 +78,7 @@ if name == 'sleep':
sys.exit(0)
if scenario.startswith('generic-budget-') and seconds != 60:
os.execv(os.environ['REAL_SLEEP'], [os.environ['REAL_SLEEP'], *sys.argv[1:]])
if seconds == 60 and scenario in ('generic-ss', 'generic-unsafe', 'generic-budget-monitor', 'generic-budget-reconcile', 'generic-budget-ack'):
if seconds == 60 and scenario in ('generic-ss', 'generic-unsafe', 'generic-budget-monitor', 'generic-budget-reconcile', 'generic-budget-ack', 'generic-apply-blocked'):
log('generic-pause')
(root / 'budget-pause').touch()
if scenario != 'generic-ss':
@@ -325,6 +325,26 @@ elif command == 'monitor':
time.sleep(.03)
os.kill(os.getppid(), signal.SIGTERM)
linger()
elif command == 'resource-apply':
assert sys.argv[3:] == ['--resource-config', '/operator/resources.json']
assert os.environ['ZAPRET_AUTOTUNE_RESOURCE_APPLY'] == '1'
assert os.getsid(0) == os.getpid(), 'apply lacks owned session'
assert (root / 'private' / 'monitor-ready').read_text() == str(os.getpid()) + '\n'
events = (root / 'events').read_text().splitlines()
assert events[-1] == 'reconcile' and 'start' not in events
log('resource-apply')
if scenario == 'generic-apply-blocked':
sys.exit(78)
if scenario == 'generic-apply-cancel':
def restored(*_):
log('resource-restored')
sys.exit(0)
signal.signal(signal.SIGTERM, restored)
os.kill(os.getppid(), signal.SIGTERM)
while True:
time.sleep(.01)
log('resource-applied')
sys.exit(0)
'''
@@ -368,7 +388,9 @@ def run(scenario, expected, dns='', mode='native'):
barrier = f'{binary / "autotune"} qualification-race-barrier "$monitor_pid"\n'
loop = 'while dns_alive && kill -0 "$server_pid" && kill -0 "$local_pid" && kill -0 "$monitor_pid"; do'
assert loop in text
text = text.replace(loop, barrier + loop, 1)
# The apply child has its own wait loop before this monitor loop.
before, after = text.rsplit(loop, 1)
text = before + barrier + loop + after
if realtime:
# Private fixture scaling only: production has no configurable grace.
text = text.replace('$(date +%s) + 80', '$(date +%s) + 12')
@@ -384,6 +406,9 @@ def run(scenario, expected, dns='', mode='native'):
env.pop('ZAPRET_AUTOTUNE_RESOURCE_CONFIG', None)
env.pop('ZAPRET_AUTOTUNE_QUALIFY_CONFIG', None)
env.pop('ZAPRET_AUTOTUNE_STRATEGY_CONFIG', None)
env.pop('ZAPRET_AUTOTUNE_RESOURCE_APPLY', None)
if scenario.startswith('generic-apply-'):
env['ZAPRET_AUTOTUNE_RESOURCE_APPLY'] = '1'
if scenario.startswith('qualify-'):
env['ZAPRET_AUTOTUNE_QUALIFY_CONFIG'] = '/operator/resources.json'
if scenario != 'qualify-missing':
@@ -412,6 +437,7 @@ def run(scenario, expected, dns='', mode='native'):
peer.wait(timeout=2)
events = (root / 'events').read_text().splitlines() if (root / 'events').exists() else []
assert result.returncode == expected, (scenario, result.returncode, result.stderr, events)
assert b'Illegal number' not in result.stderr, (scenario, result.stderr)
assert b'ab' * 32 not in result.stdout + result.stderr
for pid in ((root / 'children').read_text().splitlines() if (root / 'children').exists() else []):
assert not Path(f'/proc/{pid}').exists(), (scenario, 'child not joined', pid)
@@ -428,7 +454,7 @@ def run(scenario, expected, dns='', mode='native'):
# Every activation and stop must immediately follow successful reconcile.
for i, event in enumerate(events):
if event in ('start', 'stop'):
assert events[i - 1] == 'reconcile', (scenario, events)
assert events[i - 1] == 'reconcile' or event == 'start' and events[i - 1] == 'resource-applied', (scenario, events)
foreign = scenario in ('dns-symlink', 'dns-preack-foreign', 'dns-replace', 'dns-replace-symlink', 'dns-stale', 'dns-stale-symlink')
if foreign:
ready = configs / 'dns-ready'
@@ -472,6 +498,14 @@ assert events.count('generic-pause') == 1, events
assert events.count('dns-start') == events.count('dns-stop') == 4, events
events = run('generic-unsafe', 0)
assert 'start' not in events and events.count('generic-pause') == 1, events
events = run('generic-apply-pass', 0, mode='doh')
assert events.count('resource-apply') == 1 and events.count('start') == 1, events
assert events.index('resource-applied') < events.index('start') < events.index('monitor 1'), events
events = run('generic-apply-cancel', 0, mode='doh')
assert 'resource-restored' in events and 'start' not in events and 'monitor 1' not in events, events
assert events.index('resource-restored') < events.index('stop') < events.index('dns-stop'), events
events = run('generic-apply-blocked', 0, mode='doh')
assert 'start' not in events and 'monitor 1' not in events and 'generic-pause' in events, events
for scenario in ('generic-budget-monitor', 'generic-budget-reconcile', 'generic-budget-ack'):
events = run(scenario, 78, mode='doh')
assert events.count('start') == 1 and 'stop' not in events, (scenario, events)
@@ -566,10 +600,47 @@ esac
assert not Path(f'/proc/{pid}').exists(), (shell, mode, 'orphan sleeper', pid)
def check_session_deadline(shell):
"""Startup sweep has no aggregate deadline; termination must keep its cap."""
entrypoint = (ROOT / 'managed-target-entrypoint.sh').read_text()
cleanup_session = entrypoint[entrypoint.index('cleanup_session() {'):entrypoint.index('capture_monitor_session() {')]
for deadline, live_until, expected, elapsed in (('', 102, 0, 2), ('180', 102, 0, 2), ('101', 110, 78, 1)):
with tempfile.TemporaryDirectory(prefix='session-deadline-') as tmp:
script = Path(tmp) / 'check.sh'
script.write_text(f'''set -eu
generic_mode=1
generic_cleanup_deadline='{deadline}'
monitor_sid=42
clock=100
date() {{ printf '%s\\n' "$clock"; }}
pause() {{ clock=$((clock + $1)); }}
signal_session() {{
[ "$1" = 42 ] || return 78
case "$2" in
TERM|KILL) ;;
CHECK) session_live=1; [ "$clock" -lt {live_until} ] || session_live=0 ;;
*) return 78 ;;
esac
}}
{cleanup_session}
code=0
cleanup_session || code=$?
[ "$code" -eq {expected} ]
[ "$clock" -eq {100 + elapsed} ]
[ "$generic_cleanup_deadline" = '{deadline}' ]
''')
result = subprocess.run([*shell, str(script)], capture_output=True, timeout=2)
assert result.returncode == 0, (shell, deadline, result.stderr)
expected_stderr = b'error: owned monitor session survived cleanup\n' if expected else b''
assert result.stderr == expected_stderr, (shell, deadline, result.stderr)
check_watchdog_traps(['/bin/sh'])
check_session_deadline(['/bin/sh'])
busybox = shutil.which('busybox')
if busybox:
check_watchdog_traps([busybox, 'ash'])
check_session_deadline([busybox, 'ash'])
for pid, seconds in (('1', '5'), ('0', '5'), ('-1', '5'), ('42', '0'), ('42', 'bad')):
result = subprocess.run(['/bin/sh', str(ROOT / 'managed-target-supervision.sh'), '--watchdog', pid, seconds, '/'], capture_output=True, timeout=2)
assert result.returncode == 64
+3 -1
View File
@@ -19,6 +19,7 @@ services:
ZAPRET_AUTOTUNE_DOH_TIMEOUT_SEC: ${ZAPRET_AUTOTUNE_DOH_TIMEOUT_SEC-5}
ZAPRET_AUTOTUNE_YOUTUBE_SERVICE: ${ZAPRET_AUTOTUNE_YOUTUBE_SERVICE-0}
ZAPRET_AUTOTUNE_RESOURCE_CONFIG: ${ZAPRET_AUTOTUNE_RESOURCE_CONFIG-}
ZAPRET_AUTOTUNE_RESOURCE_APPLY: ${ZAPRET_AUTOTUNE_RESOURCE_APPLY-0}
# INTRUSIVE isolated startup batch. Never route working traffic here.
ZAPRET_AUTOTUNE_QUALIFY_CONFIG: ${ZAPRET_AUTOTUNE_QUALIFY_CONFIG-}
ZAPRET_AUTOTUNE_STRATEGY_CONFIG: ${ZAPRET_AUTOTUNE_STRATEGY_CONFIG-}
@@ -45,7 +46,8 @@ services:
- /run/zapret-autotune-ss:rw,noexec,nosuid,nodev,size=1m,mode=0700,uid=0,gid=0
networks: [target]
restart: "no"
stop_grace_period: 240s
# Generic mode uses 90s; retain the legacy adapter's longer default.
stop_grace_period: ${ZAPRET_AUTOTUNE_STOP_GRACE_PERIOD-240s}
sysctls:
net.ipv4.ip_unprivileged_port_start: "0"
networks:
+37 -15
View File
@@ -28,6 +28,13 @@ if [ -n "${ZAPRET_AUTOTUNE_QUALIFY_CONFIG-}${ZAPRET_AUTOTUNE_STRATEGY_CONFIG-}"
generic_mode=1
echo 'INTRUSIVE ISOLATED qualification target: do not route working traffic here' >&2
fi
case "${ZAPRET_AUTOTUNE_RESOURCE_APPLY-0}" in
0|'') ;;
1) [ "$generic_mode" -eq 1 ] && [ "$qualification_mode" -eq 0 ] && [ -n "${ZAPRET_AUTOTUNE_RESOURCE_CONFIG-}" ] || {
echo 'error: resource apply requires generic config and excludes qualification' >&2; exit 64;
} ;;
*) echo 'error: invalid resource apply flag' >&2; exit 64 ;;
esac
case "$dns_mode" in doh|native) ;; *) exit 64 ;; esac
config_dir=/run/zapret-autotune-ss
awk '$2 == "/run/zapret-autotune-ss" && $3 == "tmpfs" { found=1 } END { exit !found }' /proc/mounts || {
@@ -104,7 +111,7 @@ cleanup_session() {
for session_signal in TERM KILL; do
signal_session "$monitor_sid" "$session_signal" || return 78
session_deadline=$(($(date +%s) + 3))
if [ "$generic_mode" -eq 1 ] && [ "$session_deadline" -gt "$generic_cleanup_deadline" ]; then
if [ "$generic_mode" -eq 1 ] && [ -n "$generic_cleanup_deadline" ] && [ "$session_deadline" -gt "$generic_cleanup_deadline" ]; then
session_deadline=$generic_cleanup_deadline
fi
while :; do
@@ -325,6 +332,23 @@ start_dns() {
pause 1
done
}
launch_controller() {
rm -f "$config_dir/monitor-ready"
setsid /bin/sh /opt/managed-target-supervision.sh --monitor "$config_dir" /usr/local/bin/zapret-autotune "$@" 8<&- 9<&- &
monitor_pid=$!
monitor_ack_deadline=$(($(date +%s) + 5))
while [ ! -e "$config_dir/monitor-ready" ]; do
if ! kill -0 "$monitor_pid" 2>/dev/null || [ "$(date +%s)" -ge "$monitor_ack_deadline" ]; then
signal_tree "$monitor_pid" KILL
return 78
fi
done
capture_monitor_session || return 78
if proc_identity "$monitor_pid" && [ "$proc_sid" != "$monitor_sid" ]; then
monitor_sid=
return 78
fi
}
run_cycle() {
[ "$qualification_attempted" -eq 0 ] || return 78
start_dns || return $?
@@ -357,6 +381,17 @@ fi
timeout -k 5 45 /usr/local/bin/zapret-autotune managed reconcile || return 78
if [ "$qualification_mode" -eq 0 ]; then
zapret_started=1
if [ "${ZAPRET_AUTOTUNE_RESOURCE_APPLY-0}" = 1 ]; then
launch_controller managed resource-apply --resource-config "$ZAPRET_AUTOTUNE_RESOURCE_CONFIG" || return 78
while dns_alive && kill -0 "$server_pid" && kill -0 "$local_pid" && kill -0 "$monitor_pid"; do pause 1; done
{ dns_alive && kill -0 "$server_pid" && kill -0 "$local_pid"; } || return 75
apply_code=0
wait "$monitor_pid" || apply_code=$?
monitor_pid=
cleanup_session || return 78
[ "$apply_code" -eq 0 ] || return 78
fi
# Idempotent apply may have only verified the already committed config.
timeout -k 5 30 /opt/zapret2/init.d/sysv/zapret2 start || return 78
fi
@@ -377,20 +412,7 @@ if [ "$qualification_mode" -eq 1 ]; then
else
set -- managed monitor
fi
setsid /bin/sh /opt/managed-target-supervision.sh --monitor "$config_dir" /usr/local/bin/zapret-autotune "$@" 8<&- 9<&- &
monitor_pid=$!
monitor_ack_deadline=$(($(date +%s) + 5))
while [ ! -e "$config_dir/monitor-ready" ]; do
if ! kill -0 "$monitor_pid" 2>/dev/null || [ "$(date +%s)" -ge "$monitor_ack_deadline" ]; then
signal_tree "$monitor_pid" KILL
return 78
fi
done
capture_monitor_session || return 78
if proc_identity "$monitor_pid" && [ "$proc_sid" != "$monitor_sid" ]; then
monitor_sid=
return 78
fi
launch_controller "$@" || return 78
while dns_alive && kill -0 "$server_pid" && kill -0 "$local_pid" && kill -0 "$monitor_pid"; do
pause 1
done