fix(monitor): seed known working strategies

This commit is contained in:
Zeusina committed 2026-10-10 14:12:13 +03:00
1 parent 324c5efa79
commit a997baa553
3 files changed
+93

No files matched your search

+1
View File
@@ -7,6 +7,7 @@
- **2026-10-10 remote acceptance:** target image built from pinned base and started in a fresh directory/state volume; host SOCKS port `11080` was free, old container stayed untouched. Initial managed status confirmed generation `0`, `groups=none`, no active candidate IDs, and `NFQWS2_ENABLE=0`. Automated frontend scan of `youtube.com` completed with `total=4323, working=0`; candidate check produced an empty `strategies` array, recorded one failed run, and bootstrap error caused monitor/entrypoint exit. No strategy was applied. New target remains stopped with its fresh, strategy-free state preserved. This exposed that managed bootstrap should scan the actual Player API host (`www.youtube.com`) and ordinary no-candidate errors must not terminate ongoing monitoring. Updated code/tests are in progress; rerun required. Old container and Xray were not changed. - **2026-10-10 remote acceptance:** target image built from pinned base and started in a fresh directory/state volume; host SOCKS port `11080` was free, old container stayed untouched. Initial managed status confirmed generation `0`, `groups=none`, no active candidate IDs, and `NFQWS2_ENABLE=0`. Automated frontend scan of `youtube.com` completed with `total=4323, working=0`; candidate check produced an empty `strategies` array, recorded one failed run, and bootstrap error caused monitor/entrypoint exit. No strategy was applied. New target remains stopped with its fresh, strategy-free state preserved. This exposed that managed bootstrap should scan the actual Player API host (`www.youtube.com`) and ordinary no-candidate errors must not terminate ongoing monitoring. Updated code/tests are in progress; rerun required. Old container and Xray were not changed.
- **2026-10-10 remote retry:** scanning the correct `www.youtube.com` host still produced `working=0`. Diagnosis: scan left NFQUEUE workers 200–207 in the isolated target namespace; the subsequent `check --no-conflict-cleanup` preserved those workers and every candidate immediately failed. A manual `check` after a standalone scan confirmed the queue-conflict failure. Stopped only the new target (old target/Xray untouched), preserving its empty state. Managed recovery now opts into blockcheck conflict cleanup because it runs only after stopping zapret in the dedicated isolated namespace; ordinary CLI callers retain `--no-conflict-cleanup`. Local test/build pass; remote rerun pending. - **2026-10-10 remote retry:** scanning the correct `www.youtube.com` host still produced `working=0`. Diagnosis: scan left NFQUEUE workers 200–207 in the isolated target namespace; the subsequent `check --no-conflict-cleanup` preserved those workers and every candidate immediately failed. A manual `check` after a standalone scan confirmed the queue-conflict failure. Stopped only the new target (old target/Xray untouched), preserving its empty state. Managed recovery now opts into blockcheck conflict cleanup because it runs only after stopping zapret in the dedicated isolated namespace; ordinary CLI callers retain `--no-conflict-cleanup`. Local test/build pass; remote rerun pending.
- **2026-10-10 retry after worker-cleanup fix:** deployed commit `a762763` and recreated only the isolated target, retaining generation-0 empty strategy state. Managed `www.youtube.com` scan completed; subsequent shortlist check returned an empty `strategies` array, and the persistent monitor correctly continued into a later recovery scan instead of exiting. At the last reachable check the target was up, generation 0, no candidates, and 3 probe failures; SSH profile `dev` then disconnected during the next scan. Acceptance remains incomplete; reconnect and inspect current process/status/report before any further action. Old target and Xray remained untouched. - **2026-10-10 retry after worker-cleanup fix:** deployed commit `a762763` and recreated only the isolated target, retaining generation-0 empty strategy state. Managed `www.youtube.com` scan completed; subsequent shortlist check returned an empty `strategies` array, and the persistent monitor correctly continued into a later recovery scan instead of exiting. At the last reachable check the target was up, generation 0, no candidates, and 3 probe failures; SSH profile `dev` then disconnected during the next scan. Acceptance remains incomplete; reconnect and inspect current process/status/report before any further action. Old target and Xray remained untouched.
- **2026-10-10 diagnostic follow-up:** SSH reconnected. Existing SOCKS `127.0.0.1:1080` returned HTTP 200 for `https://www.youtube.com/`; direct host curl and fresh target SOCKS `127.0.0.1:11080` timed out, as expected with the disabled baseline. Independently ran `blockcheckw check` in a clean disposable container on the same target bridge using the existing frontend profile; it passed TLS 1.2 and TLS 1.3, 3/3 each. The default scan corpus omitted this exact profile, while `blockcheckw scan --from-file` found it on the same host in under a second. This isolates the gap to our managed candidate input/search, not network reachability or inability of the strategy to bypass. Stopped only the dedicated target to end repeated full scans; its state remains generation 0 with no applied candidates. Added a managed-only, exact-host revalidated seed before full-corpus fallback; `go test ./...`, build, static deploy guard, entrypoint syntax, and diff checks pass. Remote acceptance rerun pending. Old container/Xray untouched.
- Added `managed monitor` and supervision only in the dedicated target entrypoint: guarded target-owned DB/identity before singleton, policy 5m/3/30m with bounded CLI/env validation, owned-loopback Player API probe/discovery without direct fallback, fixed blockcheck/zapret argv with context cancellation/join. Startup bootstraps only without a qualified active frontend, tries saved scenarios first, then scans; unresolved GGC keeps monitoring; safety/DB errors stop it. SIGTERM waits for monitor restoration before stopping target-local zapret/SS. Local fake CLI/entrypoint tests are present. - Added `managed monitor` and supervision only in the dedicated target entrypoint: guarded target-owned DB/identity before singleton, policy 5m/3/30m with bounded CLI/env validation, owned-loopback Player API probe/discovery without direct fallback, fixed blockcheck/zapret argv with context cancellation/join. Startup bootstraps only without a qualified active frontend, tries saved scenarios first, then scans; unresolved GGC keeps monitoring; safety/DB errors stop it. SIGTERM waits for monitor restoration before stopping target-local zapret/SS. Local fake CLI/entrypoint tests are present.
- Hardened managed recovery artifacts: общий `revalidateCandidate` удаляет старый combined JSON перед каждым check, remove errors/missing/malformed report fail closed; scan shortlist проверяется на regular file и mtime не раньше whole-second scan start (production использует unique temp path). Добавлены fake-only regression tests stale/missing shortlist, stale combined, remove failure и infrastructure classification/restoration. Tests/build/network не запускались; verification deferred orchestrator, CLI/entrypoint не добавлены. - Hardened managed recovery artifacts: общий `revalidateCandidate` удаляет старый combined JSON перед каждым check, remove errors/missing/malformed report fail closed; scan shortlist проверяется на regular file и mtime не раньше whole-second scan start (production использует unique temp path). Добавлены fake-only regression tests stale/missing shortlist, stale combined, remove failure и infrastructure classification/restoration. Tests/build/network не запускались; verification deferred orchestrator, CLI/entrypoint не добавлены.
+51
View File
@@ -1,6 +1,7 @@
package main package main
import ( import (
"bytes"
"context" "context"
"errors" "errors"
"fmt" "fmt"
@@ -13,6 +14,20 @@ import (
"time" "time"
) )
// Managed seeds are only starting hypotheses from an existing known-good
// profile; blockcheckw must re-rank and re-verify them for the exact hostname.
func managedSeedArgs(host string) string {
host = strings.ToLower(host)
switch {
case host == "www.youtube.com":
return "--payload=tls_client_hello --lua-desync=hostfakesplit:disorder_after:ip_autottl=-3,3-20:repeats=1 --lua-desync=multidisorder:pos=1,midsld"
case host == "googlevideo.com" || strings.HasSuffix(host, ".googlevideo.com"):
return "--payload=tls_client_hello --lua-desync=fakeddisorder:pos=midsld:tcp_flags_set=SYN"
default:
return ""
}
}
// runBlockcheck leaves the output file (including partial results on failure) to // runBlockcheck leaves the output file (including partial results on failure) to
// the caller to read and remove. stdout/stderr receive the unmodified streams. // the caller to read and remove. stdout/stderr receive the unmodified streams.
// The caller controls cancellation through ctx; no overall deadline is imposed here. // The caller controls cancellation through ctx; no overall deadline is imposed here.
@@ -24,10 +39,32 @@ func runBlockcheck(ctx context.Context, binary, domain string, stdout, stderr io
// namespace, so blockcheck may clean up its own scan workers before shortlist // namespace, so blockcheck may clean up its own scan workers before shortlist
// checks. Other callers preserve foreign nfqws2 processes by default. // checks. Other callers preserve foreign nfqws2 processes by default.
func runManagedBlockcheck(ctx context.Context, binary, domain string, stdout, stderr io.Writer) (string, error) { func runManagedBlockcheck(ctx context.Context, binary, domain string, stdout, stderr io.Writer) (string, error) {
domain = strings.ToLower(domain)
if args := managedSeedArgs(domain); args != "" {
path, err := runBlockcheckWithConflictCleanupAndSeed(ctx, binary, domain, stdout, stderr, true, args)
if err == nil {
report, readErr := os.ReadFile(path + ".check.json")
group := "frontend"
if domain == "googlevideo.com" || strings.HasSuffix(domain, ".googlevideo.com") {
group = "googlevideo"
}
if readErr == nil {
if _, selectErr := selectBlockcheckArgsForTarget(bytes.NewReader(report), "", group, strings.ToLower(domain)); selectErr == nil {
return path, nil
}
}
_ = os.Remove(path)
_ = os.Remove(path + ".check.json")
}
}
return runBlockcheckWithConflictCleanup(ctx, binary, domain, stdout, stderr, true) return runBlockcheckWithConflictCleanup(ctx, binary, domain, stdout, stderr, true)
} }
func runBlockcheckWithConflictCleanup(ctx context.Context, binary, domain string, stdout, stderr io.Writer, allowConflictCleanup bool) (string, error) { func runBlockcheckWithConflictCleanup(ctx context.Context, binary, domain string, stdout, stderr io.Writer, allowConflictCleanup bool) (string, error) {
return runBlockcheckWithConflictCleanupAndSeed(ctx, binary, domain, stdout, stderr, allowConflictCleanup, "")
}
func runBlockcheckWithConflictCleanupAndSeed(ctx context.Context, binary, domain string, stdout, stderr io.Writer, allowConflictCleanup bool, seedArgs string) (string, error) {
if !validDomain(domain) { if !validDomain(domain) {
return "", fmt.Errorf("invalid DNS hostname: %q", domain) return "", fmt.Errorf("invalid DNS hostname: %q", domain)
} }
@@ -55,7 +92,21 @@ func runBlockcheckWithConflictCleanup(ctx context.Context, binary, domain string
} }
} }
domain = strings.ToLower(domain) domain = strings.ToLower(domain)
var seedPath string
if seedArgs != "" {
rows, err := combinedVanillaRowsForHost(seedArgs, domain)
if err != nil {
return path, err
}
seedPath = filepath.Join(workdir, "managed-seed.txt")
if err := os.WriteFile(seedPath, []byte(rows), 0600); err != nil {
return path, err
}
}
scanArgs := []string{"--workers", "8", "scan", "--domain", domain, "--protocols", "tls12,tls13", "--auto"} scanArgs := []string{"--workers", "8", "scan", "--domain", domain, "--protocols", "tls12,tls13", "--auto"}
if seedPath != "" {
scanArgs = append(scanArgs, "--from-file", seedPath)
}
if !allowConflictCleanup { if !allowConflictCleanup {
scanArgs = append(scanArgs, "--no-conflict-cleanup") scanArgs = append(scanArgs, "--no-conflict-cleanup")
} }
+41
View File
@@ -12,6 +12,7 @@ import (
"os/signal" "os/signal"
"path/filepath" "path/filepath"
"reflect" "reflect"
"slices"
"strconv" "strconv"
"strings" "strings"
"syscall" "syscall"
@@ -227,6 +228,46 @@ func TestRunManagedBlockcheckAllowsWorkerCleanup(t *testing.T) {
} }
} }
func TestRunManagedBlockcheckSeedsKnownProfileThroughBlockcheck(t *testing.T) {
t.Setenv("TMPDIR", t.TempDir())
args := managedSeedArgs("www.youtube.com")
t.Setenv("ZAPRET_TEST_REPORT", combinedReportFixture(args))
argvPath := filepath.Join(t.TempDir(), "argv.jsonl")
t.Setenv("ZAPRET_TEST_ARGV", argvPath)
binary := fakeBlockcheck(t, "tune")
path, err := runManagedBlockcheck(context.Background(), binary, "www.youtube.com", io.Discard, io.Discard)
if err != nil {
t.Fatal(err)
}
defer os.Remove(path)
defer os.Remove(path + ".check.json")
data, err := os.ReadFile(path + ".check.json")
if err != nil {
t.Fatal(err)
}
if _, err := selectBlockcheckArgsForTarget(bytes.NewReader(data), args, "frontend", "www.youtube.com"); err != nil {
t.Fatalf("seed was not verified through blockcheck: %v", err)
}
argv, err := os.ReadFile(argvPath)
if err != nil {
t.Fatal(err)
}
decoder := json.NewDecoder(bytes.NewReader(argv))
var scanArgs, checkArgs []string
if err := decoder.Decode(&scanArgs); err != nil {
t.Fatal(err)
}
if err := decoder.Decode(&checkArgs); err != nil {
t.Fatal(err)
}
if !slices.Contains(scanArgs, "--from-file") || slices.Contains(scanArgs, "--no-conflict-cleanup") || slices.Contains(checkArgs, "--no-conflict-cleanup") {
t.Fatalf("managed seeded commands=%q then %q", scanArgs, checkArgs)
}
if seed := managedSeedArgs("rr3.googlevideo.com"); !strings.Contains(seed, "fakeddisorder") {
t.Fatalf("missing exact-host GGC seed: %q", seed)
}
}
func TestRunBlockcheckValidation(t *testing.T) { func TestRunBlockcheckValidation(t *testing.T) {
for _, domain := range []string{"", "--auto", "a;touch x", "a/b", "a..b", "-a.com", strings.Repeat("a", 64) + ".com"} { for _, domain := range []string{"", "--auto", "a;touch x", "a/b", "a..b", "-a.com", strings.Repeat("a", 64) + ".com"} {
path, err := runBlockcheck(context.Background(), "nonexistent", domain, io.Discard, io.Discard) path, err := runBlockcheck(context.Background(), "nonexistent", domain, io.Discard, io.Discard)