feat: discover and qualify Googlevideo CDNs autonomously
Retain strict three-host native qualification, discover fresh representative inventory automatically, and recheck deployed media before promotion. Document offline-only verification and outstanding field gates. Co-authored-by: GPT-6 <gpt-6@openai.com>
This commit is contained in:
1 parent
9984f21466
commit
3fc8563995
22 files changed
+776
-83
No files matched your search
@@ -0,0 +1,34 @@
|
|||||||
|
# Bounded representative Googlevideo discovery
|
||||||
|
|
||||||
|
The YouTube default inventory is the configured primary plus `aqz-KE-bpKQ` and
|
||||||
|
`jNQXAC9IVRw`, deduplicated. Explicit configured inventories override it. The
|
||||||
|
inventory (one to five IDs) is queried freshly
|
||||||
|
in round-robin order for up to three rounds, bounded by three minutes. No new
|
||||||
|
manual CDN addresses, host rewriting or IP pinning is added.
|
||||||
|
With only one configured video, diversity depends on that video's real Player
|
||||||
|
backend rotation; if it does not rotate, qualification remains unresolved.
|
||||||
|
|
||||||
|
Discovery deduplicates actual ordinary signed MP4 media hosts, refetches expired
|
||||||
|
inventory and selects three hosts passing the existing root/prefix diversity
|
||||||
|
gate. URLs need twenty minutes remaining for the three sequential owned native
|
||||||
|
warmups and verified TLS transfers. URLs stay transient; unresolved diagnostics
|
||||||
|
contain only attempt/host and rejection counts. Discovery is not media proof.
|
||||||
|
|
||||||
|
Production adaptive recovery uses this collector. Existing experimental suffixes
|
||||||
|
retain their one-host operational recheck; promotion requires fresh three-host
|
||||||
|
inventory and the unchanged version-1 owned native proof factory. Insufficient
|
||||||
|
inventory leaves the committed experimental profile intact and retries after
|
||||||
|
the configured cooldown, not the exhausted-strategy two-hour pause. Native GGC
|
||||||
|
scope requires qualified frontend and any supplied asset candidates remain
|
||||||
|
qualified and nonoverlapping; missing avatar inventory is not its prerequisite.
|
||||||
|
Apply checks frontend and all three actual media resources in the newly deployed
|
||||||
|
daemon, including bounded learning and three verified transfers per TLS version.
|
||||||
|
Whole-service monitoring still reports avatar degradation independently.
|
||||||
|
|
||||||
|
Single-CDN version-2 qualification has been withdrawn. Maximum strategy catalogs,
|
||||||
|
new video discovery sources and controller stay-alive changes are separate work.
|
||||||
|
The three-CDN requirement was reaffirmed on 2026-10-13. The common suffix remains
|
||||||
|
authorized extrapolative coverage; only exact measured endpoints have transfer
|
||||||
|
evidence. This candidate is offline unit/fixture-verified only: owned factory,
|
||||||
|
autonomous discovery/promotion and deployed learning still need a scoped actual
|
||||||
|
three-CDN field test. No remote network checks or deployment accompany this update.
|
||||||
@@ -1,5 +1,38 @@
|
|||||||
# Fresh managed-test target contract (isolated acceptance, not production approval)
|
# Fresh managed-test target contract (isolated acceptance, not production approval)
|
||||||
|
|
||||||
|
## Autonomous three-CDN source candidate (2026-10-13): field acceptance pending
|
||||||
|
|
||||||
|
The user's three-CDN requirement is retained; the proposed single-CDN relaxation
|
||||||
|
was withdrawn. Service-mode discovery needs no manual domains: missing/empty
|
||||||
|
`ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS` uses primary `ZAPRET_AUTOTUNE_VIDEO_ID`
|
||||||
|
(default `dQw4w9WgXcQ`) plus `aqz-KE-bpKQ` and `jNQXAC9IVRw`, deduplicated.
|
||||||
|
A nonempty JSON array of 1..5 unique VideoIDs overrides this inventory. VideoIDs
|
||||||
|
are discovery inputs, not proof of distinct CDNs.
|
||||||
|
|
||||||
|
Fresh round-robin Player requests are bounded to three rounds (at most 15 requests)
|
||||||
|
and three minutes. Qualification selects three actual signed MP4 media hosts,
|
||||||
|
with at least two roots and two prefixes, and twenty-minute remaining URL TTL.
|
||||||
|
Only the existing version-1 owned native proof factory authorizes qualification:
|
||||||
|
post-warmup actual media 3/3 on both TLS protocols for each measured endpoint.
|
||||||
|
The common `googlevideo.com` suffix is an authorized extrapolative scope, not
|
||||||
|
measured evidence for every Googlevideo host or cold/uniform behavior.
|
||||||
|
|
||||||
|
Experimental promotion uses the same proof and fresh deployed three-host media
|
||||||
|
checks before committing apply. Insufficient inventory preserves the current
|
||||||
|
experiment and uses configured retry cooldown; genuine exhausted strategy search
|
||||||
|
still waits at least two hours. Qualified frontend is required; avatar success
|
||||||
|
does not block this media stage, but whole-service health remains DEGRADED until
|
||||||
|
actual avatar checks pass. Legacy service-off/exact-host behavior is unchanged.
|
||||||
|
|
||||||
|
This source candidate has only offline unit/fixture verification, not a newly
|
||||||
|
deployed image or field-qualified autonomous promotion. The 2026-10-12 measured
|
||||||
|
results below remain historical exact-endpoint evidence. Build from the committed
|
||||||
|
source archive and field-test the owned factory, three actual CDNs, deployed
|
||||||
|
relearning/apply, rollback and whole-service health before acceptance. Maximum
|
||||||
|
strategy collection and PID1 global-failure stay-alive remain unfinished future
|
||||||
|
increments. Arbitrary declarative service configuration is future work; no new
|
||||||
|
generic policy framework is supplied. See [GOOGLEVIDEO_DISCOVERY.md](GOOGLEVIDEO_DISCOVERY.md).
|
||||||
|
|
||||||
`managed init|reconcile|status|active-ids|apply` is exclusively for a **new dedicated container**
|
`managed init|reconcile|status|active-ids|apply` is exclusively for a **new dedicated container**
|
||||||
which owns its entire config. The existing/dev/production container must refuse
|
which owns its entire config. The existing/dev/production container must refuse
|
||||||
these commands. A role env or CLI-created identity is not enough. Provisioning
|
these commands. A role env or CLI-created identity is not enough. Provisioning
|
||||||
|
|||||||
+32
-1
@@ -1,9 +1,40 @@
|
|||||||
# Прогресс реализации
|
# Прогресс реализации
|
||||||
|
|
||||||
Срез на **2026-10-12**. Этот файл описывает фактическое состояние кода, а не весь объём требований из `Requirements.md` и `Architecture.md`.
|
Срез на **2026-10-13**. Этот файл описывает фактическое состояние кода, а не весь объём требований из `Requirements.md` и `Architecture.md`.
|
||||||
|
|
||||||
## Текущий статус
|
## Текущий статус
|
||||||
|
|
||||||
|
### Autonomous three-CDN source candidate (2026-10-13): offline only
|
||||||
|
|
||||||
|
- User reaffirmed **three actual CDN checks**, discovered by the program without
|
||||||
|
manual domains; the earlier single-CDN/version-2 proposal is withdrawn. Missing
|
||||||
|
or empty service-mode VideoID inventory selects primary (default `dQw4w9WgXcQ`)
|
||||||
|
plus `aqz-KE-bpKQ` and `jNQXAC9IVRw`, deduplicated. A nonempty 1..5 unique-ID
|
||||||
|
override is respected; three videos do not prove three CDN hosts.
|
||||||
|
- Fresh discovery runs up to three round-robin rounds, at most 15 requests in
|
||||||
|
three minutes; it deduplicates actual signed ordinary MP4 hosts, requires
|
||||||
|
twenty-minute remaining URL TTL, and selects three hosts across at least two
|
||||||
|
roots and two prefixes. The unchanged version-1 owned native proof factory
|
||||||
|
requires actual post-warmup media 3/3 on both TLS protocols at each endpoint.
|
||||||
|
Common Googlevideo suffix authorization is extrapolative, not all-host evidence.
|
||||||
|
- Existing experiments can now seek promotion through that shared qualification
|
||||||
|
path, then frontend and fresh deployed three-host media checks under the current
|
||||||
|
transaction/ownership guards. Insufficient inventory leaves the experiment
|
||||||
|
committed and retries at configured cooldown; genuine strategy exhaustion keeps
|
||||||
|
its minimum two-hour pause. Missing avatar does not block the media stage;
|
||||||
|
whole-service monitoring remains DEGRADED until actual avatar health passes.
|
||||||
|
Legacy service-off/exact-host behavior and cancellation safety remain unchanged.
|
||||||
|
- This is an offline unit/fixture-tested source candidate, **not deployed or
|
||||||
|
field-accepted**. Renewed actual three-CDN discovery, owned factory proof,
|
||||||
|
deployed learning/apply/rollback and avatar qualification remain field gates.
|
||||||
|
Earlier statements below that automatic qualification is unimplemented describe
|
||||||
|
those historical RCs, not this new source; their measured results are not new
|
||||||
|
factory evidence. No remote network checks or deployment accompany this update.
|
||||||
|
- Maximum strategy collection and PID1 global-failure stay-alive are still future
|
||||||
|
increments; broad maximum-profile live staging is deferred. Arbitrary declarative
|
||||||
|
service configuration remains a future note, not a new framework. Details:
|
||||||
|
[GOOGLEVIDEO_DISCOVERY.md](GOOGLEVIDEO_DISCOVERY.md).
|
||||||
|
|
||||||
### Final managed RC field acceptance (2026-10-12): DEGRADED, not READY
|
### Final managed RC field acceptance (2026-10-12): DEGRADED, not READY
|
||||||
|
|
||||||
- Supplied final field results, not network checks rerun by this docs update: source `fd25cb89398300bb9849db4b5511af75bf86eccf`, correct managed image `sha256:73e955e7fdc6efaef88c62ebe4f7afeab71c29d29ba3c88b2a2b7547bb647da5`, active container `a4e845a6d7…`, official managed PID 1, isolated SOCKS `127.0.0.1:11082`. The earlier `37823…` prototype image is **not active** and is not this acceptance image. Private sanitized proof: `/home/agent/zapret2-autotune-rc-fd25cb8/proof/final.safe.json`. The later docs commit is not the tested image source.
|
- Supplied final field results, not network checks rerun by this docs update: source `fd25cb89398300bb9849db4b5511af75bf86eccf`, correct managed image `sha256:73e955e7fdc6efaef88c62ebe4f7afeab71c29d29ba3c88b2a2b7547bb647da5`, active container `a4e845a6d7…`, official managed PID 1, isolated SOCKS `127.0.0.1:11082`. The earlier `37823…` prototype image is **not active** and is not this acceptance image. Private sanitized proof: `/home/agent/zapret2-autotune-rc-fd25cb8/proof/final.safe.json`. The later docs commit is not the tested image source.
|
||||||
|
|||||||
@@ -75,9 +75,17 @@ func managedGooglevideoConfig(lookup func(string) (string, bool), primary string
|
|||||||
if mode == QualificationAdaptive && !service {
|
if mode == QualificationAdaptive && !service {
|
||||||
return "", nil, fmt.Errorf("adaptive googlevideo requires ZAPRET_AUTOTUNE_YOUTUBE_SERVICE=1")
|
return "", nil, fmt.Errorf("adaptive googlevideo requires ZAPRET_AUTOTUNE_YOUTUBE_SERVICE=1")
|
||||||
}
|
}
|
||||||
// Only the configured primary is sampled by default; no implicit public playlist.
|
|
||||||
ids := []string{primary}
|
ids := []string{primary}
|
||||||
if raw, ok := lookup("ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS"); ok {
|
// YouTube-specific public discovery samples, not CDN addresses. Explicit
|
||||||
|
// inventories below override these defaults; health/discovery primary stays put.
|
||||||
|
if service {
|
||||||
|
for _, id := range []string{"aqz-KE-bpKQ", "jNQXAC9IVRw"} {
|
||||||
|
if id != primary {
|
||||||
|
ids = append(ids, id)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if raw, ok := lookup("ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS"); ok && raw != "" {
|
||||||
if json.Unmarshal([]byte(raw), &ids) != nil || len(ids) == 0 || len(ids) > 5 {
|
if json.Unmarshal([]byte(raw), &ids) != nil || len(ids) == 0 || len(ids) > 5 {
|
||||||
return "", nil, fmt.Errorf("GOOGLEVIDEO_VIDEO_IDS requires a JSON array of 1..5 video IDs")
|
return "", nil, fmt.Errorf("GOOGLEVIDEO_VIDEO_IDS requires a JSON array of 1..5 video IDs")
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,65 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"os"
|
||||||
|
"reflect"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
// check_config.py provides sanitized actual `docker compose config` output.
|
||||||
|
// This bridges deployment interpolation to the production config parser offline.
|
||||||
|
func TestManagedGooglevideoComposeForwarding(t *testing.T) {
|
||||||
|
path := os.Getenv("AUTOTUNE_COMPOSE_CONFIG_FIXTURE")
|
||||||
|
if path == "" {
|
||||||
|
t.Skip("run deploy/managed-target/check_config.py for Compose fixture")
|
||||||
|
}
|
||||||
|
data, err := os.ReadFile(path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal("compose fixture unavailable")
|
||||||
|
}
|
||||||
|
var cases []struct {
|
||||||
|
Environment map[string]string `json:"environment"`
|
||||||
|
IDs []string `json:"ids"`
|
||||||
|
}
|
||||||
|
if json.Unmarshal(data, &cases) != nil || len(cases) < 6 {
|
||||||
|
t.Fatal("invalid compose fixture")
|
||||||
|
}
|
||||||
|
for i, c := range cases {
|
||||||
|
lookup := func(key string) (string, bool) { value, ok := c.Environment[key]; return value, ok }
|
||||||
|
mode, ids, err := managedGooglevideoConfig(lookup, c.Environment["ZAPRET_AUTOTUNE_VIDEO_ID"])
|
||||||
|
if err != nil || string(mode) != c.Environment["ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE"] || !reflect.DeepEqual(ids, c.IDs) {
|
||||||
|
t.Fatalf("compose factory case %d mismatch", i)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestManagedGooglevideoBlankInventoryDefault(t *testing.T) {
|
||||||
|
for _, value := range []string{"", " ", "\n", "[]", "bad", `["aaaaaaaaaaa"]`} {
|
||||||
|
lookup := func(key string) (string, bool) {
|
||||||
|
switch key {
|
||||||
|
case "ZAPRET_AUTOTUNE_YOUTUBE_SERVICE":
|
||||||
|
return "1", true
|
||||||
|
case "ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE":
|
||||||
|
return "adaptive", true
|
||||||
|
case "ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS":
|
||||||
|
return value, true
|
||||||
|
}
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
mode, ids, err := managedGooglevideoConfig(lookup, "dQw4w9WgXcQ")
|
||||||
|
if value != "" && value != `["aaaaaaaaaaa"]` {
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("invalid nonempty inventory accepted")
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
want := []string{"dQw4w9WgXcQ", "aqz-KE-bpKQ", "jNQXAC9IVRw"}
|
||||||
|
if value != "" {
|
||||||
|
want = []string{"aaaaaaaaaaa"}
|
||||||
|
}
|
||||||
|
if err != nil || mode != QualificationAdaptive || !reflect.DeepEqual(ids, want) {
|
||||||
|
t.Fatal("inventory default/override mismatch")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -64,6 +64,7 @@ func renderManagedConfigPolicy(db *sql.DB, ids []int64, scope GooglevideoScope,
|
|||||||
seen := map[int64]bool{}
|
seen := map[int64]bool{}
|
||||||
assets := map[string]bool{}
|
assets := map[string]bool{}
|
||||||
var otherScopes []string
|
var otherScopes []string
|
||||||
|
nativeSuffix := false
|
||||||
for _, id := range ids {
|
for _, id := range ids {
|
||||||
if id <= 0 || seen[id] {
|
if id <= 0 || seen[id] {
|
||||||
return nil, fmt.Errorf("duplicate candidate %d", id)
|
return nil, fmt.Errorf("duplicate candidate %d", id)
|
||||||
@@ -110,6 +111,7 @@ func renderManagedConfigPolicy(db *sql.DB, ids []int64, scope GooglevideoScope,
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
if c.Group == "googlevideo" && scope == GooglevideoScopeSuffix {
|
if c.Group == "googlevideo" && scope == GooglevideoScopeSuffix {
|
||||||
|
nativeSuffix = c.Mode == QualificationAdaptive
|
||||||
if version < 7 || validateGooglevideoSuffixEvidencePolicy(db, c, fresh) != nil {
|
if version < 7 || validateGooglevideoSuffixEvidencePolicy(db, c, fresh) != nil {
|
||||||
return nil, fmt.Errorf("suffix requires representative qualified hosts and verified media evidence")
|
return nil, fmt.Errorf("suffix requires representative qualified hosts and verified media evidence")
|
||||||
}
|
}
|
||||||
@@ -125,8 +127,8 @@ func renderManagedConfigPolicy(db *sql.DB, ids []int64, scope GooglevideoScope,
|
|||||||
groups[c.Group] = true
|
groups[c.Group] = true
|
||||||
profiles = append(profiles, "--filter-tcp=443 --filter-l7=tls --hostlist-domains="+strings.Join(domains, ",")+" "+c.Args)
|
profiles = append(profiles, "--filter-tcp=443 --filter-l7=tls --hostlist-domains="+strings.Join(domains, ",")+" "+c.Args)
|
||||||
}
|
}
|
||||||
if scope == GooglevideoScopeSuffix && (!groups["googlevideo"] || len(assets) != 2) {
|
if scope == GooglevideoScopeSuffix && (!groups["googlevideo"] || nativeSuffix && !groups["frontend"] || !nativeSuffix && len(assets) != 2) {
|
||||||
return nil, fmt.Errorf("suffix requires googlevideo and both qualified asset profiles")
|
return nil, fmt.Errorf("suffix requires qualified googlevideo and frontend (legacy cold scope requires both assets)")
|
||||||
}
|
}
|
||||||
if scope == GooglevideoScopeExperimental && (!groups["googlevideo"] || !groups["frontend"] || len(assets) == 0) {
|
if scope == GooglevideoScopeExperimental && (!groups["googlevideo"] || !groups["frontend"] || len(assets) == 0) {
|
||||||
return nil, fmt.Errorf("experimental suffix requires frontend and existing qualified assets")
|
return nil, fmt.Errorf("experimental suffix requires frontend and existing qualified assets")
|
||||||
|
|||||||
@@ -62,6 +62,15 @@ func recoverManagedExperimental(ctx context.Context, d managedRecoveryDeps) (int
|
|||||||
if err == nil {
|
if err == nil {
|
||||||
err = d.recheckExperimental(ctx, c, r, d.transaction)
|
err = d.recheckExperimental(ctx, c, r, d.transaction)
|
||||||
}
|
}
|
||||||
|
if err == nil && d.mediaSamples != nil && d.verifyGooglevideo != nil {
|
||||||
|
// One-host operational health never promotes the experiment. Only
|
||||||
|
// fresh representative inventory and a completed owned proof may.
|
||||||
|
qualified, err := qualifyManagedGooglevideo(ctx, d)
|
||||||
|
if err != nil {
|
||||||
|
return id, err
|
||||||
|
}
|
||||||
|
return qualified, nil
|
||||||
|
}
|
||||||
return id, managedRecoveryContextError(ctx, err)
|
return id, managedRecoveryContextError(ctx, err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,87 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/sha256"
|
||||||
|
"database/sql"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
"reflect"
|
||||||
|
)
|
||||||
|
|
||||||
|
func newManagedGooglevideoApplyRecheck(target managedDeps, db *sql.DB) func(context.Context, storedCandidate, []mediaResource, *managedTransaction) error {
|
||||||
|
r := newManagedAssetRuntime(target)
|
||||||
|
guard := r.guard
|
||||||
|
r.guard = func() error {
|
||||||
|
if err := guard(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
lua, err := os.ReadFile("/opt/zapret2/lua/zapret-auto.lua")
|
||||||
|
if err != nil || fmt.Sprintf("%x", sha256.Sum256(lua)) != managedGooglevideoLuaHash {
|
||||||
|
return errors.New("unverified native automation Lua source")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return managedGooglevideoApplyRecheck(r, db)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Runs inside Apply's lock: never ActiveCandidates/Reconcile/restart here.
|
||||||
|
// The newly reloaded daemon learns in-place; a failure rolls back via Apply.
|
||||||
|
func managedGooglevideoApplyRecheck(r managedAssetRuntime, db *sql.DB) func(context.Context, storedCandidate, []mediaResource, *managedTransaction) error {
|
||||||
|
return func(ctx context.Context, c storedCandidate, resources []mediaResource, m *managedTransaction) error {
|
||||||
|
j, err := m.load()
|
||||||
|
if err != nil {
|
||||||
|
return &managedRecoveryFatal{Err: err}
|
||||||
|
}
|
||||||
|
found := false
|
||||||
|
for _, id := range j.PendingCandidates {
|
||||||
|
found = found || id == c.ID
|
||||||
|
}
|
||||||
|
var hosts []string
|
||||||
|
for _, resource := range resources {
|
||||||
|
hosts = append(hosts, resource.host)
|
||||||
|
}
|
||||||
|
if j.State != "prepared" || j.PendingScope != GooglevideoScopeSuffix || !found || c.Mode != QualificationAdaptive || !reflect.DeepEqual(hosts, c.Endpoints) || validateGooglevideoSuffixEvidence(db, c) != nil {
|
||||||
|
return &managedRecoveryFatal{Err: errors.New("deployed media candidate ownership mismatch")}
|
||||||
|
}
|
||||||
|
guard := func() error {
|
||||||
|
if err := r.guard(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := m.owns(j, false); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if err := validateManagedRefs(db, j.PendingCandidates, j.Next, j.PendingScope); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
b, err := m.read(m.Name)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if managedHash(b) != j.Next {
|
||||||
|
return errManagedDrift
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
for _, resource := range resources {
|
||||||
|
err := recheckDeployedExperimental(ctx, resource, guard,
|
||||||
|
func(context.Context) error { return guard() }, // already reloaded; warm current daemon only
|
||||||
|
func(context.Context) (mediaResource, error) { return mediaResource{}, errCandidateNonpass },
|
||||||
|
func(ctx context.Context, resource mediaResource, protocol string) error {
|
||||||
|
_, _, err := collectMediaTransfer(ctx, r.proxy, resource, protocol, r.curl)
|
||||||
|
return err
|
||||||
|
})
|
||||||
|
if errors.Is(err, errNoQualifiedStrategy) && !managedRecoveryTimeoutSafe(err) {
|
||||||
|
var fatal *managedRecoveryFatal
|
||||||
|
if !errors.As(err, &fatal) {
|
||||||
|
return errCandidateNonpass
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,191 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func samplePlayerBody(host string, expires time.Time) []byte {
|
||||||
|
u, _ := json.Marshal(fmt.Sprintf("https://%s/videoplayback?expire=%d&sig=private-secret", host, expires.Unix()))
|
||||||
|
return []byte(fmt.Sprintf(`{"playabilityStatus":{"status":"OK"},"streamingData":{"formats":[{"mimeType":"video/mp4","url":%s}]}}`, u))
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestManagedGooglevideoDiscoveryRotatesFreshInventory(t *testing.T) {
|
||||||
|
ids := []string{"dQw4w9WgXcQ", "aqz-KE-bpKQ", "jNQXAC9IVRw"}
|
||||||
|
hosts := []string{"rr1---sn-alpha.googlevideo.com", "rr2---sn-beta.googlevideo.com", "rr5---sn-gamma.googlevideo.com"}
|
||||||
|
for _, scenario := range []string{"rotating", "unavailable", "expired", "same-root", "unsafe-origin", "invalid-host", "control", "unknown", "mixed-safety", "cancel"} {
|
||||||
|
t.Run(scenario, func(t *testing.T) {
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
calls := 0
|
||||||
|
collect := managedGooglevideoSamples(ids, func(ctx context.Context, id string) ([]byte, error) {
|
||||||
|
if _, ok := ctx.Deadline(); !ok {
|
||||||
|
t.Fatal("unbounded discovery")
|
||||||
|
}
|
||||||
|
if id != ids[calls%3] {
|
||||||
|
t.Fatal("not round robin")
|
||||||
|
}
|
||||||
|
calls++
|
||||||
|
host := hosts[(calls-1)%3]
|
||||||
|
expires := time.Now().Add(time.Hour)
|
||||||
|
if calls <= 3 {
|
||||||
|
switch scenario {
|
||||||
|
case "rotating":
|
||||||
|
host = hosts[0]
|
||||||
|
case "unavailable":
|
||||||
|
return nil, context.DeadlineExceeded
|
||||||
|
case "expired":
|
||||||
|
expires = time.Now().Add(5 * time.Minute)
|
||||||
|
case "unsafe-origin":
|
||||||
|
host += ":444"
|
||||||
|
case "invalid-host":
|
||||||
|
host = "not-a-ggc.googlevideo.com"
|
||||||
|
case "control":
|
||||||
|
host += "\n"
|
||||||
|
case "unknown":
|
||||||
|
return nil, errors.New("runner unavailable")
|
||||||
|
case "mixed-safety":
|
||||||
|
return nil, errors.Join(context.DeadlineExceeded, errors.New("cleanup failed"))
|
||||||
|
case "cancel":
|
||||||
|
cancel()
|
||||||
|
return nil, ctx.Err()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if scenario == "same-root" {
|
||||||
|
host = fmt.Sprintf("rr%d---sn-alpha-x.googlevideo.com", calls)
|
||||||
|
}
|
||||||
|
return samplePlayerBody(host, expires), nil
|
||||||
|
})
|
||||||
|
resources, err := collect(ctx)
|
||||||
|
if scenario == "unknown" || scenario == "mixed-safety" {
|
||||||
|
var fatal *managedRecoveryFatal
|
||||||
|
if !errors.As(err, &fatal) || calls != 1 || len(resources) != 0 {
|
||||||
|
t.Fatal(err, calls)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if scenario == "cancel" {
|
||||||
|
if !errors.Is(err, context.Canceled) || calls != 1 || len(resources) != 0 {
|
||||||
|
t.Fatal(err, calls)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if scenario == "same-root" {
|
||||||
|
var insufficient *InsufficientAdaptiveSamples
|
||||||
|
if !errors.As(err, &insufficient) || insufficient.Attempts != 9 || len(resources) != 0 || strings.Contains(err.Error(), "private-secret") {
|
||||||
|
t.Fatal(err, calls)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err != nil || len(resources) != 3 || calls != 6 {
|
||||||
|
t.Fatal(err, calls, len(resources))
|
||||||
|
}
|
||||||
|
for i, r := range resources {
|
||||||
|
if r.host != hosts[i] || !strings.Contains(r.url, "private-secret") {
|
||||||
|
t.Fatal("changed signed inventory")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestManagedGooglevideoDiscoveryBoundsAndLateCancellation(t *testing.T) {
|
||||||
|
ids := []string{"dQw4w9WgXcQ", "aqz-KE-bpKQ", "jNQXAC9IVRw", "aaaaaaaaaaa", "bbbbbbbbbbb"}
|
||||||
|
calls := 0
|
||||||
|
resources, err := managedGooglevideoSamples(ids, func(context.Context, string) ([]byte, error) {
|
||||||
|
calls++
|
||||||
|
return []byte(`{"playabilityStatus":{"status":"OK"},"streamingData":{"serverAbrStreamingUrl":"private-secret"}}`), nil
|
||||||
|
})(context.Background())
|
||||||
|
var insufficient *InsufficientAdaptiveSamples
|
||||||
|
if !errors.As(err, &insufficient) || calls != 15 || insufficient.Unsupported != 15 || len(resources) != 0 {
|
||||||
|
t.Fatal(err, calls)
|
||||||
|
}
|
||||||
|
for _, invalid := range [][]string{nil, {"bad"}, append(ids, "ccccccccccc")} {
|
||||||
|
_, err := managedGooglevideoSamples(invalid, func(context.Context, string) ([]byte, error) { t.Fatal("invalid config queried"); return nil, nil })(context.Background())
|
||||||
|
var fatal *managedRecoveryFatal
|
||||||
|
if !errors.As(err, &fatal) {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
defer cancel()
|
||||||
|
hosts := []string{"rr1---sn-alpha.googlevideo.com", "rr2---sn-beta.googlevideo.com", "rr5---sn-gamma.googlevideo.com"}
|
||||||
|
calls = 0
|
||||||
|
resources, err = managedGooglevideoSamples(ids[:3], func(context.Context, string) ([]byte, error) {
|
||||||
|
host := hosts[calls]
|
||||||
|
calls++
|
||||||
|
if calls == 3 {
|
||||||
|
cancel()
|
||||||
|
}
|
||||||
|
return samplePlayerBody(host, time.Now().Add(time.Hour)), nil
|
||||||
|
})(ctx)
|
||||||
|
if !errors.Is(err, context.Canceled) || len(resources) != 0 || calls != 3 {
|
||||||
|
t.Fatal(err, calls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExperimentalPromotionRequiresRepresentativeOwnedProof(t *testing.T) {
|
||||||
|
for _, enough := range []bool{false, true} {
|
||||||
|
t.Run(fmt.Sprint(enough), func(t *testing.T) {
|
||||||
|
d, front, _ := recoveryFixture(t, true)
|
||||||
|
r, _, resources, _ := googlevideoRuntimeFixture(t)
|
||||||
|
d.service = true
|
||||||
|
d.frontHealth = d.health
|
||||||
|
thumb := unitAssetCandidate(t, d.db, "i.ytimg.com", "--lua-desync=fake", true)
|
||||||
|
avatar := unitAssetCandidate(t, d.db, "yt3.ggpht.com", "--lua-desync=fake", true)
|
||||||
|
tx, err := d.db.Begin()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
experiment, err := insertStrategyCandidate(tx, "googlevideo", []string{resources[0].host}, managedGooglevideoArgs, nil, QualificationCold)
|
||||||
|
if err != nil {
|
||||||
|
_ = tx.Rollback()
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := tx.Commit(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
old := []int64{front, thumb, avatar, experiment}
|
||||||
|
if err := d.transaction.Apply(d.db, old, GooglevideoScopeExperimental); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
d.resources = func(context.Context) (managedServiceInventory, error) {
|
||||||
|
return managedServiceInventory{media: resources[0], images: map[string]string{"i.ytimg.com": "https://i.ytimg.com/sample.jpg", "yt3.ggpht.com": "https://yt3.ggpht.com/sample.jpg"}, proxy: r.proxy, runner: adaptiveImageRunner}, nil
|
||||||
|
}
|
||||||
|
d.recheckExperimental = func(context.Context, storedCandidate, mediaResource, *managedTransaction) error { return nil }
|
||||||
|
d.mediaSamples = func(context.Context) ([]mediaResource, error) {
|
||||||
|
if enough {
|
||||||
|
return resources, nil
|
||||||
|
}
|
||||||
|
return resources[:1], nil
|
||||||
|
}
|
||||||
|
d.verifyGooglevideo = r.verifyGooglevideo
|
||||||
|
d.recheckGooglevideo = managedGooglevideoApplyRecheck(r, d.db)
|
||||||
|
id, err := recoverManagedExperimental(context.Background(), d)
|
||||||
|
if !enough {
|
||||||
|
if !errors.Is(err, errGooglevideoInsufficientSamples) || id != experiment {
|
||||||
|
t.Fatal(id, err)
|
||||||
|
}
|
||||||
|
assertActiveCandidates(t, d.transaction, d.db, old)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err != nil || id == experiment {
|
||||||
|
t.Fatal(id, err)
|
||||||
|
}
|
||||||
|
assertActiveCandidates(t, d.transaction, d.db, []int64{front, thumb, avatar, id})
|
||||||
|
p, err := readGooglevideoProof(d.db, id)
|
||||||
|
if err != nil || p.Version != 1 || len(p.Hosts) != 3 {
|
||||||
|
t.Fatal(p, err)
|
||||||
|
}
|
||||||
|
c, err := readStrategyCandidate(d.db, experiment)
|
||||||
|
if err != nil || c.Qualified || c.Mode != QualificationCold {
|
||||||
|
t.Fatal("old row changed", c, err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -3,78 +3,137 @@ package main
|
|||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"errors"
|
"errors"
|
||||||
|
"fmt"
|
||||||
"sort"
|
"sort"
|
||||||
"strconv"
|
"strconv"
|
||||||
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Ordinary unresolved state, not a target/ownership guard failure.
|
// Ordinary unresolved state, not a target/ownership guard failure.
|
||||||
type InsufficientAdaptiveSamples struct{}
|
type InsufficientAdaptiveSamples struct {
|
||||||
|
Attempts, DistinctHosts, Unavailable, Unsupported, Rejected int
|
||||||
|
}
|
||||||
|
|
||||||
func (*InsufficientAdaptiveSamples) Error() string {
|
func (e *InsufficientAdaptiveSamples) Error() string {
|
||||||
return "DEGRADED: insufficient fresh ordinary signed GGC samples"
|
return fmt.Sprintf("DEGRADED: insufficient fresh ordinary signed GGC samples (attempts=%d distinct_hosts=%d unavailable=%d unsupported=%d rejected=%d)", e.Attempts, e.DistinctHosts, e.Unavailable, e.Unsupported, e.Rejected)
|
||||||
}
|
}
|
||||||
func (*InsufficientAdaptiveSamples) Unwrap() []error {
|
func (*InsufficientAdaptiveSamples) Unwrap() []error {
|
||||||
return []error{errGooglevideoInsufficientSamples, errNoQualifiedStrategy}
|
return []error{errGooglevideoInsufficientSamples}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Sequential round-robin: at most 15 fresh requests, with no child goroutines.
|
||||||
|
const managedGooglevideoSampleRounds = 3
|
||||||
|
const managedGooglevideoDiscoveryTimeout = 3 * time.Minute
|
||||||
|
|
||||||
|
// Three hosts each need up to 120s warmup + six 35s transfers. Leave margin
|
||||||
|
// for daemon lifecycle as well; short-lived URLs are refetched, never rewritten.
|
||||||
|
const managedGooglevideoSampleTTL = 20 * time.Minute
|
||||||
|
|
||||||
|
func freshGooglevideoSample(r mediaResource) bool {
|
||||||
|
// The same exact GGC hostname syntax as the representative gate; reject bad
|
||||||
|
// inventory before counting it, without fetching any returned URL here.
|
||||||
|
prefix, root, ok := strings.Cut(strings.TrimSuffix(r.host, ".googlevideo.com"), "---sn-")
|
||||||
|
if !ok || !strings.HasSuffix(r.host, ".googlevideo.com") || !validDomain(r.host) || r.host != strings.ToLower(r.host) || root == "" || len(prefix) < 3 || !strings.HasPrefix(prefix, "rr") || strings.Trim(prefix[2:], "0123456789") != "" {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
u, err := privateResourceURL(r.url, r.host)
|
||||||
|
if err != nil || u.Path != "/videoplayback" || (r.kind != "video/mp4" && r.kind != "audio/mp4") {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
q := u.Query()
|
||||||
|
expires, err := strconv.ParseInt(q.Get("expire"), 10, 64)
|
||||||
|
return err == nil && expires > time.Now().Add(managedGooglevideoSampleTTL).Unix() && (q.Get("sig") != "" || q.Get("signature") != "" || q.Get("lsig") != "")
|
||||||
|
}
|
||||||
|
|
||||||
|
// Choose exactly three actual hosts passing the unchanged representative gate.
|
||||||
|
func representativeGooglevideoSamples(byHost map[string]mediaResource) []mediaResource {
|
||||||
|
hosts := make([]string, 0, len(byHost))
|
||||||
|
for host, r := range byHost {
|
||||||
|
if freshGooglevideoSample(r) {
|
||||||
|
hosts = append(hosts, host)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sort.Strings(hosts)
|
||||||
|
for i := 0; i < len(hosts); i++ {
|
||||||
|
for j := i + 1; j < len(hosts); j++ {
|
||||||
|
for k := j + 1; k < len(hosts); k++ {
|
||||||
|
if validateGooglevideoHosts([]string{hosts[i], hosts[j], hosts[k]}) == nil {
|
||||||
|
return []mediaResource{byHost[hosts[i]], byHost[hosts[j]], byHost[hosts[k]]}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func managedGooglevideoSamples(ids []string, request func(context.Context, string) ([]byte, error)) func(context.Context) ([]mediaResource, error) {
|
func managedGooglevideoSamples(ids []string, request func(context.Context, string) ([]byte, error)) func(context.Context) ([]mediaResource, error) {
|
||||||
|
ids = append([]string(nil), ids...)
|
||||||
return func(ctx context.Context) ([]mediaResource, error) {
|
return func(ctx context.Context) ([]mediaResource, error) {
|
||||||
byHost := map[string]mediaResource{}
|
if len(ids) == 0 || len(ids) > 5 || request == nil {
|
||||||
|
return nil, &managedRecoveryFatal{Err: errors.New("invalid sample collector configuration")}
|
||||||
|
}
|
||||||
for _, id := range ids {
|
for _, id := range ids {
|
||||||
body, err := request(ctx, id)
|
if !videoIDPattern.MatchString(id) {
|
||||||
|
return nil, &managedRecoveryFatal{Err: errors.New("invalid sample video ID")}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
bounded, cancel := context.WithTimeout(ctx, managedGooglevideoDiscoveryTimeout)
|
||||||
|
defer cancel()
|
||||||
|
stats := &InsufficientAdaptiveSamples{}
|
||||||
|
byHost := map[string]mediaResource{}
|
||||||
|
for round := 0; round < managedGooglevideoSampleRounds; round++ {
|
||||||
|
for _, id := range ids {
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
return nil, managedRecoveryContextError(ctx, ctx.Err())
|
||||||
|
}
|
||||||
|
if bounded.Err() != nil {
|
||||||
|
return nil, stats
|
||||||
|
}
|
||||||
|
stats.Attempts++
|
||||||
|
body, err := request(bounded, id)
|
||||||
var fatal *managedRecoveryFatal
|
var fatal *managedRecoveryFatal
|
||||||
if ctx.Err() != nil || errors.As(err, &fatal) {
|
if ctx.Err() != nil || errors.As(err, &fatal) {
|
||||||
return nil, managedRecoveryContextError(ctx, err)
|
return nil, managedRecoveryContextError(ctx, err)
|
||||||
}
|
}
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
if !managedResourceTransferNonpass(err, true, false) && !(bounded.Err() != nil && managedRecoveryTimeoutSafe(err)) {
|
||||||
|
return nil, &managedRecoveryFatal{Err: err}
|
||||||
|
}
|
||||||
|
stats.Unavailable++
|
||||||
|
if bounded.Err() != nil {
|
||||||
|
return nil, stats
|
||||||
|
}
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
r, _, err := playerMediaResource(body)
|
r, _, err := playerMediaResource(body)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
stats.Unavailable++
|
||||||
continue
|
continue
|
||||||
} // Missing/SABR is unknown, never fabricated.
|
} // Missing/SABR is unknown, never fabricated.
|
||||||
u, err := privateResourceURL(r.url, r.host)
|
if r.url == "" {
|
||||||
if err != nil || u.Path != "/videoplayback" {
|
stats.Unsupported++
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
q := u.Query()
|
if !freshGooglevideoSample(r) {
|
||||||
expires, err := strconv.ParseInt(q.Get("expire"), 10, 64)
|
stats.Rejected++
|
||||||
if err != nil || expires <= time.Now().Unix()+120 || (q.Get("sig") == "" && q.Get("signature") == "" && q.Get("lsig") == "") {
|
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
byHost[r.host] = r
|
byHost[r.host] = r
|
||||||
|
stats.DistinctHosts = len(byHost)
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
return nil, managedRecoveryContextError(ctx, ctx.Err())
|
||||||
}
|
}
|
||||||
hosts := make([]string, 0, len(byHost))
|
if bounded.Err() != nil {
|
||||||
for host := range byHost {
|
return nil, stats
|
||||||
hosts = append(hosts, host)
|
|
||||||
}
|
|
||||||
sort.Strings(hosts)
|
|
||||||
if validateGooglevideoHosts(hosts) != nil {
|
|
||||||
return nil, &InsufficientAdaptiveSamples{}
|
|
||||||
}
|
|
||||||
resources := make([]mediaResource, 0, len(hosts))
|
|
||||||
for _, host := range hosts {
|
|
||||||
resources = append(resources, byHost[host])
|
|
||||||
}
|
}
|
||||||
|
if resources := representativeGooglevideoSamples(byHost); resources != nil {
|
||||||
return resources, nil
|
return resources, nil
|
||||||
}
|
}
|
||||||
}
|
|
||||||
|
|
||||||
func managedAdaptiveAssetsQualified(d managedRecoveryDeps, ids []int64) (bool, error) {
|
|
||||||
covered := map[string]bool{}
|
|
||||||
for _, id := range ids {
|
|
||||||
c, err := readStrategyCandidate(d.db, id)
|
|
||||||
if err != nil {
|
|
||||||
return false, err
|
|
||||||
}
|
|
||||||
if c.Group == "assets" && c.Qualified {
|
|
||||||
for _, host := range c.Endpoints {
|
|
||||||
covered[host] = true
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
return nil, stats
|
||||||
}
|
}
|
||||||
return covered["i.ytimg.com"] && covered["yt3.ggpht.com"], nil
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func recoverManagedGooglevideo(ctx context.Context, d managedRecoveryDeps) (int64, error) {
|
func recoverManagedGooglevideo(ctx context.Context, d managedRecoveryDeps) (int64, error) {
|
||||||
@@ -86,6 +145,13 @@ func recoverManagedGooglevideo(ctx context.Context, d managedRecoveryDeps) (int6
|
|||||||
if journal.Scope == GooglevideoScopeExperimental {
|
if journal.Scope == GooglevideoScopeExperimental {
|
||||||
return recoverManagedExperimental(ctx, d)
|
return recoverManagedExperimental(ctx, d)
|
||||||
}
|
}
|
||||||
|
return qualifyManagedGooglevideo(ctx, d)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Both autonomous adaptive recovery and an operationally rechecked experiment
|
||||||
|
// use the same unchanged three-host owned proof and transactional apply gate.
|
||||||
|
func qualifyManagedGooglevideo(ctx context.Context, d managedRecoveryDeps) (int64, error) {
|
||||||
|
m := *d.transaction
|
||||||
if !d.service || d.mediaSamples == nil || d.verifyGooglevideo == nil {
|
if !d.service || d.mediaSamples == nil || d.verifyGooglevideo == nil {
|
||||||
return 0, &InsufficientAdaptiveSamples{}
|
return 0, &InsufficientAdaptiveSamples{}
|
||||||
}
|
}
|
||||||
@@ -93,20 +159,6 @@ func recoverManagedGooglevideo(ctx context.Context, d managedRecoveryDeps) (int6
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return 0, &managedRecoveryFatal{Err: err}
|
return 0, &managedRecoveryFatal{Err: err}
|
||||||
}
|
}
|
||||||
ok, err := managedAdaptiveAssetsQualified(d, current)
|
|
||||||
if err != nil {
|
|
||||||
return 0, &managedRecoveryFatal{Err: err}
|
|
||||||
}
|
|
||||||
if !ok {
|
|
||||||
return 0, &managedResourceFailure{"thumbnail", "unknown", nil}
|
|
||||||
}
|
|
||||||
status, err := managedAssetApplyHealth(ctx, d, []string{"i.ytimg.com", "yt3.ggpht.com"})
|
|
||||||
if err != nil {
|
|
||||||
return 0, err
|
|
||||||
}
|
|
||||||
if status != "OK" {
|
|
||||||
return 0, &managedHealthRejection{Status: status}
|
|
||||||
}
|
|
||||||
saved, err := listQualifiedCandidates(d.db, "googlevideo", 10)
|
saved, err := listQualifiedCandidates(d.db, "googlevideo", 10)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return 0, &managedRecoveryFatal{Err: err}
|
return 0, &managedRecoveryFatal{Err: err}
|
||||||
@@ -189,16 +241,31 @@ func recoverManagedGooglevideo(ctx context.Context, d managedRecoveryDeps) (int6
|
|||||||
return d.run(cleanup, managedZapretScript, "restart")
|
return d.run(cleanup, managedZapretScript, "restart")
|
||||||
}
|
}
|
||||||
m.Health = func() (string, error) {
|
m.Health = func() (string, error) {
|
||||||
status, err := d.health(ctx) // Fresh media byte health, not hostname-only.
|
if d.frontHealth == nil || d.recheckGooglevideo == nil {
|
||||||
var failure *managedResourceFailure
|
return "DEGRADED", errors.New("frontend and deployed media health required")
|
||||||
|
}
|
||||||
|
status, err := d.frontHealth(ctx)
|
||||||
|
if err != nil || status != "OK" {
|
||||||
|
return status, managedRecoveryContextError(ctx, err)
|
||||||
|
}
|
||||||
|
deployed, err := readStrategyCandidate(d.db, id)
|
||||||
|
if err == nil {
|
||||||
|
err = d.recheckGooglevideo(ctx, deployed, resources, &m)
|
||||||
|
}
|
||||||
|
if err != nil {
|
||||||
var fatal *managedRecoveryFatal
|
var fatal *managedRecoveryFatal
|
||||||
if ctx.Err() != nil {
|
if ctx.Err() != nil || errors.As(err, &fatal) || errors.Is(err, context.Canceled) {
|
||||||
return "DEGRADED", managedRecoveryContextError(ctx, err)
|
return "DEGRADED", managedRecoveryContextError(ctx, err)
|
||||||
}
|
}
|
||||||
if errors.As(err, &failure) && !errors.As(err, &fatal) && !errors.Is(err, context.Canceled) && !errors.Is(err, context.DeadlineExceeded) {
|
if managedGooglevideoTransferNonpass(err) || errors.Is(err, errCandidateNonpass) {
|
||||||
return "DEGRADED", nil
|
return "DEGRADED", nil
|
||||||
}
|
}
|
||||||
return status, err
|
return "DEGRADED", managedRecoveryContextError(ctx, err)
|
||||||
|
}
|
||||||
|
if ctx.Err() != nil {
|
||||||
|
return "DEGRADED", managedRecoveryContextError(ctx, ctx.Err())
|
||||||
|
}
|
||||||
|
return "OK", nil // Stage readiness only; whole-service monitor is unchanged.
|
||||||
}
|
}
|
||||||
err = m.Apply(d.db, desired, GooglevideoScopeSuffix)
|
err = m.Apply(d.db, desired, GooglevideoScopeSuffix)
|
||||||
var rollback *managedRollbackError
|
var rollback *managedRollbackError
|
||||||
|
|||||||
@@ -13,6 +13,7 @@ import (
|
|||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
|
"time"
|
||||||
)
|
)
|
||||||
|
|
||||||
func TestManagedAdaptiveConfig(t *testing.T) {
|
func TestManagedAdaptiveConfig(t *testing.T) {
|
||||||
@@ -42,7 +43,7 @@ func TestManagedAdaptiveConfig(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func TestManagedAdaptiveRecoveryProductionPath(t *testing.T) {
|
func TestManagedAdaptiveRecoveryProductionPath(t *testing.T) {
|
||||||
for _, scenario := range []string{"qualified", "one-host", "unqualified-assets", "rollback"} {
|
for _, scenario := range []string{"qualified", "one-host", "unqualified-assets", "avatar-missing", "rollback", "apply-bad-body", "apply-mixed-guard", "apply-learns"} {
|
||||||
t.Run(scenario, func(t *testing.T) {
|
t.Run(scenario, func(t *testing.T) {
|
||||||
d, front, events := recoveryFixture(t, true)
|
d, front, events := recoveryFixture(t, true)
|
||||||
r, _, resources, _ := googlevideoRuntimeFixture(t)
|
r, _, resources, _ := googlevideoRuntimeFixture(t)
|
||||||
@@ -66,16 +67,71 @@ func TestManagedAdaptiveRecoveryProductionPath(t *testing.T) {
|
|||||||
return resources, nil
|
return resources, nil
|
||||||
}
|
}
|
||||||
d.verifyGooglevideo = r.verifyGooglevideo
|
d.verifyGooglevideo = r.verifyGooglevideo
|
||||||
if scenario == "rollback" {
|
d.recheckGooglevideo = managedGooglevideoApplyRecheck(r, d.db)
|
||||||
|
applyCalls := 0
|
||||||
|
if strings.HasPrefix(scenario, "apply-") {
|
||||||
|
deployed := r
|
||||||
|
if scenario == "apply-mixed-guard" {
|
||||||
|
deployed.guard = func() error { return errors.Join(context.Canceled, errors.New("guard failed")) }
|
||||||
|
}
|
||||||
|
deployed.curl = func(ctx context.Context, args []string, stdin io.Reader, out, diagnostic io.Writer) error {
|
||||||
|
applyCalls++
|
||||||
|
if scenario == "apply-learns" && applyCalls <= 2 {
|
||||||
|
return context.DeadlineExceeded
|
||||||
|
}
|
||||||
|
var body bytes.Buffer
|
||||||
|
err := r.curl(ctx, args, stdin, &body, diagnostic)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
b := body.Bytes()
|
||||||
|
if scenario == "apply-bad-body" {
|
||||||
|
b = bytes.ReplaceAll(b, []byte("ftyp"), []byte("oops"))
|
||||||
|
}
|
||||||
|
_, err = out.Write(b)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
d.recheckGooglevideo = managedGooglevideoApplyRecheck(deployed, d.db)
|
||||||
|
}
|
||||||
|
if scenario == "avatar-missing" {
|
||||||
d.health = func(context.Context) (string, error) {
|
d.health = func(context.Context) (string, error) {
|
||||||
return "DEGRADED", &managedResourceFailure{"media", resources[0].host, nil}
|
return "DEGRADED", &managedResourceFailure{"avatar", "yt3.ggpht.com", nil}
|
||||||
|
}
|
||||||
|
_, ids, err := managedGooglevideoConfig(func(key string) (string, bool) { return "1", key == "ZAPRET_AUTOTUNE_YOUTUBE_SERVICE" }, "dQw4w9WgXcQ")
|
||||||
|
if err != nil || len(ids) != 3 {
|
||||||
|
t.Fatal(ids, err)
|
||||||
|
}
|
||||||
|
calls := 0
|
||||||
|
d.mediaSamples = managedGooglevideoSamples(ids, func(_ context.Context, id string) ([]byte, error) {
|
||||||
|
if id != ids[calls%3] {
|
||||||
|
t.Fatal("wrong default video")
|
||||||
|
}
|
||||||
|
index := calls % 3
|
||||||
|
if calls < 3 {
|
||||||
|
index = 0
|
||||||
|
}
|
||||||
|
calls++
|
||||||
|
return samplePlayerBody(resources[index].host, time.Now().Add(time.Hour)), nil
|
||||||
|
})
|
||||||
|
}
|
||||||
|
if scenario == "rollback" {
|
||||||
|
d.recheckGooglevideo = func(context.Context, storedCandidate, []mediaResource, *managedTransaction) error {
|
||||||
|
return errCandidateNonpass
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
result, err := recoverManagedTarget(context.Background(), d)
|
result, err := recoverManagedTarget(context.Background(), d)
|
||||||
|
if scenario == "apply-mixed-guard" {
|
||||||
|
var fatal *managedRecoveryFatal
|
||||||
|
if !errors.As(err, &fatal) {
|
||||||
|
t.Fatal("mixed guard downgraded", err)
|
||||||
|
}
|
||||||
|
assertActiveCandidates(t, d.transaction, d.db, old)
|
||||||
|
return
|
||||||
|
}
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
}
|
}
|
||||||
if scenario != "qualified" {
|
if scenario == "one-host" || scenario == "rollback" || scenario == "apply-bad-body" {
|
||||||
if result.GGCError == nil {
|
if result.GGCError == nil {
|
||||||
t.Fatal("unqualified stage succeeded")
|
t.Fatal("unqualified stage succeeded")
|
||||||
}
|
}
|
||||||
@@ -102,6 +158,15 @@ func TestManagedAdaptiveRecoveryProductionPath(t *testing.T) {
|
|||||||
t.Fatal(j, err)
|
t.Fatal(j, err)
|
||||||
}
|
}
|
||||||
assertActiveCandidates(t, d.transaction, d.db, append(old, c.ID))
|
assertActiveCandidates(t, d.transaction, d.db, append(old, c.ID))
|
||||||
|
if scenario == "apply-learns" && applyCalls < 20 {
|
||||||
|
t.Fatal("deployed warmup or full TLS transfers skipped", applyCalls)
|
||||||
|
}
|
||||||
|
if scenario == "avatar-missing" {
|
||||||
|
status, err := d.health(context.Background())
|
||||||
|
if status != "DEGRADED" || err == nil {
|
||||||
|
t.Fatal("whole-service health concealed")
|
||||||
|
}
|
||||||
|
}
|
||||||
if strings.Join(*events, ",") != "stop,start,restart" {
|
if strings.Join(*events, ",") != "stop,start,restart" {
|
||||||
t.Fatal("restore must precede apply", *events)
|
t.Fatal("restore must precede apply", *events)
|
||||||
}
|
}
|
||||||
@@ -136,7 +201,39 @@ func adaptiveImageRunner(_ context.Context, args []string, stdin io.Reader, out,
|
|||||||
|
|
||||||
func TestInsufficientAdaptiveSamplesCooldown(t *testing.T) {
|
func TestInsufficientAdaptiveSamplesCooldown(t *testing.T) {
|
||||||
err := &InsufficientAdaptiveSamples{}
|
err := &InsufficientAdaptiveSamples{}
|
||||||
if !errors.Is(err, errNoQualifiedStrategy) || !errors.Is(err, errGooglevideoInsufficientSamples) {
|
if errors.Is(err, errNoQualifiedStrategy) || !errors.Is(err, errGooglevideoInsufficientSamples) || managedRecoveryCooldown(err, time.Minute) != time.Minute {
|
||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestGenuineNoQualifiedCooldown(t *testing.T) {
|
||||||
|
if managedRecoveryCooldown(errNoQualifiedStrategy, time.Minute) != 2*time.Hour {
|
||||||
|
t.Fatal("lost exhausted-search pause")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestManagedYouTubeDefaultSampleInventory(t *testing.T) {
|
||||||
|
for _, primary := range []string{"dQw4w9WgXcQ", "aqz-KE-bpKQ"} {
|
||||||
|
lookup := func(key string) (string, bool) { return "1", key == "ZAPRET_AUTOTUNE_YOUTUBE_SERVICE" }
|
||||||
|
_, ids, err := managedGooglevideoConfig(lookup, primary)
|
||||||
|
want := []string{primary, "aqz-KE-bpKQ", "jNQXAC9IVRw"}
|
||||||
|
if primary == "aqz-KE-bpKQ" {
|
||||||
|
want = []string{primary, "jNQXAC9IVRw"}
|
||||||
|
}
|
||||||
|
if err != nil || !reflect.DeepEqual(ids, want) {
|
||||||
|
t.Fatal(ids, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
_, ids, err := managedGooglevideoConfig(func(key string) (string, bool) {
|
||||||
|
switch key {
|
||||||
|
case "ZAPRET_AUTOTUNE_YOUTUBE_SERVICE":
|
||||||
|
return "1", true
|
||||||
|
case "ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS":
|
||||||
|
return `["aaaaaaaaaaa"]`, true
|
||||||
|
}
|
||||||
|
return "", false
|
||||||
|
}, "dQw4w9WgXcQ")
|
||||||
|
if err != nil || !reflect.DeepEqual(ids, []string{"aaaaaaaaaaa"}) {
|
||||||
|
t.Fatal(ids, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -57,7 +57,11 @@ func TestManagedGooglevideoFreshPrivateSamples(t *testing.T) {
|
|||||||
t.Fatal(actual, err)
|
t.Fatal(actual, err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if calls != 2*len(ids) {
|
want := 2 * len(ids)
|
||||||
|
if scenario != "diverse" {
|
||||||
|
want *= managedGooglevideoSampleRounds
|
||||||
|
}
|
||||||
|
if calls != want {
|
||||||
t.Fatal("samples were reused", calls)
|
t.Fatal("samples were reused", calls)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -192,9 +192,8 @@ func managedMonitorRecoveryAdapters(target managedDeps, db *sql.DB, scanCommand
|
|||||||
if err == nil && d.service {
|
if err == nil && d.service {
|
||||||
d.verifyAssets = newManagedAssetVerifier(target)
|
d.verifyAssets = newManagedAssetVerifier(target)
|
||||||
d.recheckExperimental = newManagedExperimentalRecheck(target, d)
|
d.recheckExperimental = newManagedExperimentalRecheck(target, d)
|
||||||
if d.adaptive {
|
|
||||||
d.verifyGooglevideo = newManagedGooglevideoVerifier(target)
|
d.verifyGooglevideo = newManagedGooglevideoVerifier(target)
|
||||||
}
|
d.recheckGooglevideo = newManagedGooglevideoApplyRecheck(target, db)
|
||||||
}
|
}
|
||||||
return d, err
|
return d, err
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -40,6 +40,7 @@ type managedRecoveryDeps struct {
|
|||||||
mediaSamples func(context.Context) ([]mediaResource, error)
|
mediaSamples func(context.Context) ([]mediaResource, error)
|
||||||
verifyGooglevideo func(context.Context, storedCandidate, []mediaResource) (GooglevideoQualificationProof, error)
|
verifyGooglevideo func(context.Context, storedCandidate, []mediaResource) (GooglevideoQualificationProof, error)
|
||||||
recheckExperimental func(context.Context, storedCandidate, mediaResource, *managedTransaction) error
|
recheckExperimental func(context.Context, storedCandidate, mediaResource, *managedTransaction) error
|
||||||
|
recheckGooglevideo func(context.Context, storedCandidate, []mediaResource, *managedTransaction) error
|
||||||
}
|
}
|
||||||
|
|
||||||
// Target-local adapter: reuses bounded Player requests without invoking a CLI.
|
// Target-local adapter: reuses bounded Player requests without invoking a CLI.
|
||||||
@@ -120,6 +121,9 @@ func managedRecoveryDependencies(target managedDeps, db *sql.DB, scan cacheScan,
|
|||||||
}
|
}
|
||||||
if d.service {
|
if d.service {
|
||||||
d.frontHealth = d.health
|
d.frontHealth = d.health
|
||||||
|
// Construction is side-effect free; experiments may seek representative
|
||||||
|
// qualification without changing the legacy default exact-host mode.
|
||||||
|
d.mediaSamples = managedGooglevideoSamples(videoIDs, requestVideo)
|
||||||
d.resources = managedServiceResources(cfg, request, target.player)
|
d.resources = managedServiceResources(cfg, request, target.player)
|
||||||
d.health = managedServiceHealth(cfg.SOCKSProxy, d.frontHealth, d.resources, runResourceCurl)
|
d.health = managedServiceHealth(cfg.SOCKSProxy, d.frontHealth, d.resources, runResourceCurl)
|
||||||
d.verifyAssets = unwiredManagedAssetVerifier
|
d.verifyAssets = unwiredManagedAssetVerifier
|
||||||
@@ -279,7 +283,21 @@ func recoverManagedTarget(ctx context.Context, d managedRecoveryDeps) (result ma
|
|||||||
if ctx.Err() != nil || errors.As(healthErr, &fatal) || errors.Is(healthErr, context.Canceled) || errors.Is(healthErr, context.DeadlineExceeded) {
|
if ctx.Err() != nil || errors.As(healthErr, &fatal) || errors.Is(healthErr, context.Canceled) || errors.Is(healthErr, context.DeadlineExceeded) {
|
||||||
return result, managedRecoveryContextError(ctx, healthErr)
|
return result, managedRecoveryContextError(ctx, healthErr)
|
||||||
}
|
}
|
||||||
if errors.As(healthErr, &failure) && assetEndpoint(failure.Host) && j.Scope != GooglevideoScopeExperimental {
|
// A previously authorized native suffix stays adaptive after promotion,
|
||||||
|
// even if the operator's legacy environment still says "cold". Stored
|
||||||
|
// validation permits reconstruction, never fresh proof authorization.
|
||||||
|
if j.Scope == GooglevideoScopeSuffix {
|
||||||
|
for _, id := range j.Candidates {
|
||||||
|
c, err := readStrategyCandidatePolicy(d.db, id, false)
|
||||||
|
if err != nil {
|
||||||
|
return result, &managedRecoveryFatal{Err: err}
|
||||||
|
}
|
||||||
|
if c.Group == "googlevideo" && c.Mode == QualificationAdaptive && c.Qualified {
|
||||||
|
d.adaptive = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if errors.As(healthErr, &failure) && assetEndpoint(failure.Host) && j.Scope != GooglevideoScopeExperimental && !d.adaptive {
|
||||||
return result, repairManagedAssets(ctx, d, failure.Host)
|
return result, repairManagedAssets(ctx, d, failure.Host)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -773,7 +791,7 @@ func wireManagedRecovery(monitor *managedMonitorDeps, recovery managedRecoveryDe
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return &managedRecoveryFatal{Err: err}
|
return &managedRecoveryFatal{Err: err}
|
||||||
}
|
}
|
||||||
if recovery.adaptive && candidate.Group == "googlevideo" && candidate.Mode == QualificationAdaptive && candidate.Qualified {
|
if candidate.Group == "googlevideo" && candidate.Mode == QualificationAdaptive && candidate.Qualified {
|
||||||
journal, e := transaction.load()
|
journal, e := transaction.load()
|
||||||
if e != nil {
|
if e != nil {
|
||||||
return &managedRecoveryFatal{Err: e}
|
return &managedRecoveryFatal{Err: e}
|
||||||
|
|||||||
@@ -34,17 +34,18 @@ func TestManagedRecoveryProductionTransactionHooks(t *testing.T) {
|
|||||||
d.transaction = &managedTransaction{Dir: d.transaction.Dir, Name: "config"}
|
d.transaction = &managedTransaction{Dir: d.transaction.Dir, Name: "config"}
|
||||||
d.service, d.adaptive = true, true
|
d.service, d.adaptive = true, true
|
||||||
d.frontHealth = d.health
|
d.frontHealth = d.health
|
||||||
|
samples := 0
|
||||||
d.mediaSamples = func(context.Context) ([]mediaResource, error) {
|
d.mediaSamples = func(context.Context) ([]mediaResource, error) {
|
||||||
t.Fatal("asset qualification must precede private media discovery")
|
samples++
|
||||||
return nil, nil
|
return nil, &InsufficientAdaptiveSamples{}
|
||||||
}
|
}
|
||||||
d.verifyGooglevideo = func(context.Context, storedCandidate, []mediaResource) (GooglevideoQualificationProof, error) {
|
d.verifyGooglevideo = func(context.Context, storedCandidate, []mediaResource) (GooglevideoQualificationProof, error) {
|
||||||
t.Fatal("unexpected verification")
|
t.Fatal("unexpected verification")
|
||||||
return GooglevideoQualificationProof{}, nil
|
return GooglevideoQualificationProof{}, nil
|
||||||
}
|
}
|
||||||
result, err := recoverManagedTarget(context.Background(), d)
|
result, err := recoverManagedTarget(context.Background(), d)
|
||||||
var failure *managedResourceFailure
|
var failure *InsufficientAdaptiveSamples
|
||||||
if err != nil || result.FrontendID != front || !errors.As(result.GGCError, &failure) {
|
if err != nil || result.FrontendID != front || samples != 1 || !errors.As(result.GGCError, &failure) {
|
||||||
// Never expose private DB contents or error causes.
|
// Never expose private DB contents or error causes.
|
||||||
t.Fatalf("frontend_preserved=%t recovery_error_type=%T ggc_error_type=%T", result.FrontendID == front, err, result.GGCError)
|
t.Fatalf("frontend_preserved=%t recovery_error_type=%T ggc_error_type=%T", result.FrontendID == front, err, result.GGCError)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -93,9 +93,10 @@ func TestManagedExpiredAdaptiveJournalRecovery(t *testing.T) {
|
|||||||
verifies++
|
verifies++
|
||||||
return r.verifyGooglevideo(ctx, c, resources)
|
return r.verifyGooglevideo(ctx, c, resources)
|
||||||
}
|
}
|
||||||
|
d.recheckGooglevideo = managedGooglevideoApplyRecheck(r, d.db)
|
||||||
if scenario == "rollback" {
|
if scenario == "rollback" {
|
||||||
d.health = func(context.Context) (string, error) {
|
d.recheckGooglevideo = func(context.Context, storedCandidate, []mediaResource, *managedTransaction) error {
|
||||||
return "DEGRADED", &managedResourceFailure{"media", resources[0].host, nil}
|
return errCandidateNonpass
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
result, err := recoverManagedTarget(context.Background(), d)
|
result, err := recoverManagedTarget(context.Background(), d)
|
||||||
|
|||||||
@@ -141,7 +141,7 @@ func runManagedMonitor(ctx context.Context, d managedMonitorDeps) (err error) {
|
|||||||
if ctx.Err() != nil {
|
if ctx.Err() != nil {
|
||||||
return ctx.Err()
|
return ctx.Err()
|
||||||
}
|
}
|
||||||
if errors.Is(err, errNoQualifiedStrategy) {
|
if errors.Is(err, errNoQualifiedStrategy) || errors.Is(err, errGooglevideoInsufficientSamples) {
|
||||||
if err := pauseMonitorRecovery(d.db, d.serviceID, d.now(), managedRecoveryCooldown(err, d.cooldown)); err != nil {
|
if err := pauseMonitorRecovery(d.db, d.serviceID, d.now(), managedRecoveryCooldown(err, d.cooldown)); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -3,7 +3,8 @@ ZAPRET_AUTOTUNE_HOST_SOCKS_PORT=11080
|
|||||||
ZAPRET_AUTOTUNE_YOUTUBE_SERVICE=0
|
ZAPRET_AUTOTUNE_YOUTUBE_SERVICE=0
|
||||||
ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE=cold
|
ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE=cold
|
||||||
ZAPRET_AUTOTUNE_VIDEO_ID=dQw4w9WgXcQ
|
ZAPRET_AUTOTUNE_VIDEO_ID=dQw4w9WgXcQ
|
||||||
# Optional JSON array: 1..5 unique 11-character VideoIDs (not signed URLs).
|
# Missing/empty selects automatic YouTube inventory: primary + two public samples.
|
||||||
|
# Nonempty JSON overrides it: 1..5 unique 11-character VideoIDs (not signed URLs).
|
||||||
# ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS='["dQw4w9WgXcQ","aqz-KE-bpKQ","jNQXAC9IVRw"]'
|
# ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS='["dQw4w9WgXcQ","aqz-KE-bpKQ","jNQXAC9IVRw"]'
|
||||||
# Optional target ss-server native DNS; empty retains the base resolver behavior.
|
# Optional target ss-server native DNS; empty retains the base resolver behavior.
|
||||||
SS_DNS_SERVER=
|
SS_DNS_SERVER=
|
||||||
@@ -5,6 +5,29 @@ reconfigure, or stop the existing `zapret2-ss-autotune` container. The target
|
|||||||
starts from a disabled managed baseline; `managed monitor` bootstraps the
|
starts from a disabled managed baseline; `managed monitor` bootstraps the
|
||||||
frontend/API strategy, then discovers and applies an exact-host GGC strategy.
|
frontend/API strategy, then discovers and applies an exact-host GGC strategy.
|
||||||
|
|
||||||
|
## New autonomous three-CDN source candidate (2026-10-13): offline only
|
||||||
|
|
||||||
|
Service-mode source now discovers representative inventory automatically and can
|
||||||
|
promote an existing experiment only through the unchanged version-1 owned native
|
||||||
|
proof factory: **three actual media hosts**, at least two roots and two prefixes,
|
||||||
|
post-warmup 3/3 on both TLS protocols. Discovery is bounded to three rounds,
|
||||||
|
15 requests maximum and three minutes; signed MP4 URLs need twenty minutes left.
|
||||||
|
VideoIDs are not CDN evidence. No manual domain list or single-CDN relaxation is
|
||||||
|
introduced. Missing/empty inventory defaults and explicit overrides are below.
|
||||||
|
|
||||||
|
Apply requires qualified frontend and fresh deployed checks of all three media
|
||||||
|
resources, not avatar readiness. Whole-service health remains DEGRADED until avatar
|
||||||
|
checks pass. Insufficient inventory preserves the committed experiment and uses
|
||||||
|
configured cooldown; true strategy exhaustion retains its minimum two-hour pause.
|
||||||
|
Suffix authorization remains extrapolative, not proof of every Googlevideo host.
|
||||||
|
|
||||||
|
This source is offline unit/fixture-verified only; no new image deployment or
|
||||||
|
autonomous three-CDN field acceptance is claimed. The historical results below
|
||||||
|
do not close that gate. Maximum strategy collection and PID1 global-failure
|
||||||
|
stay-alive are unfinished future increments, as is arbitrary declarative service
|
||||||
|
configuration. See [MANAGED_TARGET.md](../../MANAGED_TARGET.md) and
|
||||||
|
[GOOGLEVIDEO_DISCOVERY.md](../../GOOGLEVIDEO_DISCOVERY.md).
|
||||||
|
|
||||||
## Recorded Googlevideo suffix experiment (2026-10-12)
|
## Recorded Googlevideo suffix experiment (2026-10-12)
|
||||||
|
|
||||||
### Final field follow-up: DEGRADED / NOT READY
|
### Final field follow-up: DEGRADED / NOT READY
|
||||||
@@ -133,9 +156,11 @@ Isolated, accelerated shell supervision fixtures (no network):
|
|||||||
Compose forwards `ZAPRET_AUTOTUNE_YOUTUBE_SERVICE` (`0` or `1`, default `0`),
|
Compose forwards `ZAPRET_AUTOTUNE_YOUTUBE_SERVICE` (`0` or `1`, default `0`),
|
||||||
`ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE` (`cold` or `adaptive`, default `cold`),
|
`ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE` (`cold` or `adaptive`, default `cold`),
|
||||||
`ZAPRET_AUTOTUNE_VIDEO_ID` (default `dQw4w9WgXcQ`) and
|
`ZAPRET_AUTOTUNE_VIDEO_ID` (default `dQw4w9WgXcQ`) and
|
||||||
`ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS` (JSON array of 1..5 unique VideoIDs;
|
`ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS` (optional JSON array of 1..5 unique VideoIDs).
|
||||||
default only the primary ID). Explicit empty/invalid settings are rejected,
|
Missing/empty inventory selects automatic service-mode samples: primary plus
|
||||||
not silently replaced. Adaptive requires service `1`. See `.env.example`.
|
`aqz-KE-bpKQ` and `jNQXAC9IVRw`, deduplicated. A nonempty explicit inventory
|
||||||
|
overrides those defaults, including an intentional single video. Invalid nonempty
|
||||||
|
settings are rejected. Adaptive requires service `1`. See `.env.example`.
|
||||||
Service `0` preserves legacy Player API/exact-host behavior.
|
Service `0` preserves legacy Player API/exact-host behavior.
|
||||||
|
|
||||||
Adaptive qualification is **post-warmup**, distinct from cold qualification;
|
Adaptive qualification is **post-warmup**, distinct from cold qualification;
|
||||||
|
|||||||
@@ -45,9 +45,21 @@ with tempfile.TemporaryDirectory(prefix='target-config-') as tmp:
|
|||||||
assert json.loads(path.read_text())['password'] == base['SS_PASSWORD']
|
assert json.loads(path.read_text())['password'] == base['SS_PASSWORD']
|
||||||
# Render privately, never print the config/password; config does not deploy.
|
# Render privately, never print the config/password; config does not deploy.
|
||||||
compose_env = {k: v for k, v in base.items() if not k.startswith('ZAPRET_AUTOTUNE_')}
|
compose_env = {k: v for k, v in base.items() if not k.startswith('ZAPRET_AUTOTUNE_')}
|
||||||
|
factory_cases = []
|
||||||
for overrides, service, mode, ids in (
|
for overrides, service, mode, ids in (
|
||||||
({}, '0', 'cold', ['dQw4w9WgXcQ']),
|
({}, '0', 'cold', ['dQw4w9WgXcQ']),
|
||||||
({'ZAPRET_AUTOTUNE_VIDEO_ID': 'aqz-KE-bpKQ'}, '0', 'cold', ['aqz-KE-bpKQ']),
|
({'ZAPRET_AUTOTUNE_VIDEO_ID': 'aqz-KE-bpKQ'}, '0', 'cold', ['aqz-KE-bpKQ']),
|
||||||
|
({'ZAPRET_AUTOTUNE_YOUTUBE_SERVICE': '1', 'ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE': 'adaptive'},
|
||||||
|
'1', 'adaptive', ['dQw4w9WgXcQ', 'aqz-KE-bpKQ', 'jNQXAC9IVRw']),
|
||||||
|
({'ZAPRET_AUTOTUNE_YOUTUBE_SERVICE': '1', 'ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE': 'adaptive',
|
||||||
|
'ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS': ''},
|
||||||
|
'1', 'adaptive', ['dQw4w9WgXcQ', 'aqz-KE-bpKQ', 'jNQXAC9IVRw']),
|
||||||
|
({'ZAPRET_AUTOTUNE_YOUTUBE_SERVICE': '1', 'ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE': 'adaptive',
|
||||||
|
'ZAPRET_AUTOTUNE_VIDEO_ID': 'aqz-KE-bpKQ'},
|
||||||
|
'1', 'adaptive', ['aqz-KE-bpKQ', 'jNQXAC9IVRw']),
|
||||||
|
({'ZAPRET_AUTOTUNE_YOUTUBE_SERVICE': '1', 'ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE': 'adaptive',
|
||||||
|
'ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS': '["aaaaaaaaaaa"]'},
|
||||||
|
'1', 'adaptive', ['aaaaaaaaaaa']),
|
||||||
({'ZAPRET_AUTOTUNE_YOUTUBE_SERVICE': '1', 'ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE': 'adaptive',
|
({'ZAPRET_AUTOTUNE_YOUTUBE_SERVICE': '1', 'ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE': 'adaptive',
|
||||||
'ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS': '["dQw4w9WgXcQ","aqz-KE-bpKQ","jNQXAC9IVRw"]',
|
'ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS': '["dQw4w9WgXcQ","aqz-KE-bpKQ","jNQXAC9IVRw"]',
|
||||||
'SS_DNS_SERVER': '8.8.8.8'}, '1', 'adaptive', ['dQw4w9WgXcQ', 'aqz-KE-bpKQ', 'jNQXAC9IVRw']),
|
'SS_DNS_SERVER': '8.8.8.8'}, '1', 'adaptive', ['dQw4w9WgXcQ', 'aqz-KE-bpKQ', 'jNQXAC9IVRw']),
|
||||||
@@ -60,7 +72,15 @@ with tempfile.TemporaryDirectory(prefix='target-config-') as tmp:
|
|||||||
settings = json.loads(result.stdout)['services']['target']['environment']
|
settings = json.loads(result.stdout)['services']['target']['environment']
|
||||||
assert settings['ZAPRET_AUTOTUNE_YOUTUBE_SERVICE'] == service
|
assert settings['ZAPRET_AUTOTUNE_YOUTUBE_SERVICE'] == service
|
||||||
assert settings['ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE'] == mode
|
assert settings['ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE'] == mode
|
||||||
assert json.loads(settings['ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS']) == ids
|
assert settings['ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS'] == overrides.get('ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS', '')
|
||||||
assert settings['SS_DNS_SERVER'] == overrides.get('SS_DNS_SERVER', '')
|
assert settings['SS_DNS_SERVER'] == overrides.get('SS_DNS_SERVER', '')
|
||||||
assert settings['ZAPRET_AUTOTUNE_SOCKS_PROXY'] == 'socks5h://127.0.0.1:1080'
|
assert settings['ZAPRET_AUTOTUNE_SOCKS_PROXY'] == 'socks5h://127.0.0.1:1080'
|
||||||
|
factory_cases.append({'environment': {k: v for k, v in settings.items()
|
||||||
|
if k.startswith('ZAPRET_AUTOTUNE_')}, 'ids': ids})
|
||||||
|
# Only sanitized fixture settings cross into Go; never persist the password.
|
||||||
|
fixture = root / 'compose-factory.json'
|
||||||
|
fixture.write_text(json.dumps(factory_cases))
|
||||||
|
subprocess.run(['go', 'test', './cmd/zapret-autotune', '-run', '^TestManagedGooglevideoComposeForwarding$',
|
||||||
|
'-count=1'], cwd=ROOT, env=dict(env, AUTOTUNE_COMPOSE_CONFIG_FIXTURE=str(fixture)),
|
||||||
|
check=True, timeout=60)
|
||||||
print('target config fixtures passed')
|
print('target config fixtures passed')
|
||||||
@@ -26,7 +26,7 @@ for setting in (
|
|||||||
'ZAPRET_AUTOTUNE_YOUTUBE_SERVICE: ${ZAPRET_AUTOTUNE_YOUTUBE_SERVICE-0}',
|
'ZAPRET_AUTOTUNE_YOUTUBE_SERVICE: ${ZAPRET_AUTOTUNE_YOUTUBE_SERVICE-0}',
|
||||||
'ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE: ${ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE-cold}',
|
'ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE: ${ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE-cold}',
|
||||||
'ZAPRET_AUTOTUNE_VIDEO_ID: ${ZAPRET_AUTOTUNE_VIDEO_ID-dQw4w9WgXcQ}',
|
'ZAPRET_AUTOTUNE_VIDEO_ID: ${ZAPRET_AUTOTUNE_VIDEO_ID-dQw4w9WgXcQ}',
|
||||||
"ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS: '${ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS-[\"${ZAPRET_AUTOTUNE_VIDEO_ID-dQw4w9WgXcQ}\"]}'",
|
"ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS: '${ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS:-}'",
|
||||||
):
|
):
|
||||||
assert setting in compose, setting
|
assert setting in compose, setting
|
||||||
assert entrypoint.index('target-config-check || exit 64') < entrypoint.index('cat > "$config_dir/server.json"')
|
assert entrypoint.index('target-config-check || exit 64') < entrypoint.index('cat > "$config_dir/server.json"')
|
||||||
|
|||||||
@@ -16,7 +16,7 @@ services:
|
|||||||
ZAPRET_AUTOTUNE_YOUTUBE_SERVICE: ${ZAPRET_AUTOTUNE_YOUTUBE_SERVICE-0}
|
ZAPRET_AUTOTUNE_YOUTUBE_SERVICE: ${ZAPRET_AUTOTUNE_YOUTUBE_SERVICE-0}
|
||||||
ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE: ${ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE-cold}
|
ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE: ${ZAPRET_AUTOTUNE_GOOGLEVIDEO_MODE-cold}
|
||||||
ZAPRET_AUTOTUNE_VIDEO_ID: ${ZAPRET_AUTOTUNE_VIDEO_ID-dQw4w9WgXcQ}
|
ZAPRET_AUTOTUNE_VIDEO_ID: ${ZAPRET_AUTOTUNE_VIDEO_ID-dQw4w9WgXcQ}
|
||||||
ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS: '${ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS-["${ZAPRET_AUTOTUNE_VIDEO_ID-dQw4w9WgXcQ}"]}'
|
ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS: '${ZAPRET_AUTOTUNE_GOOGLEVIDEO_VIDEO_IDS:-}'
|
||||||
SOCKS_PORT: "1080"
|
SOCKS_PORT: "1080"
|
||||||
ZAPRET_AUTOTUNE_TARGET_ROLE: managed-test
|
ZAPRET_AUTOTUNE_TARGET_ROLE: managed-test
|
||||||
ZAPRET_AUTOTUNE_TARGET_ID: zapret-autotune-managed-v1
|
ZAPRET_AUTOTUNE_TARGET_ID: zapret-autotune-managed-v1
|
||||||
|
|||||||
Reference in new issue
Block a user