feat: extract versioned YouTube service adapter

Wire pure Player parsing and strict qualification planning through a compiled-in kind/version registry. Fail closed before discovery requests when the required module is absent or mismatched.

Require sealed module-bound v2 evidence for new writes while preserving fresh trusted v1 apply eligibility and historical rollback. Keep legacy SOCKS5h routing, separate suffix/asset policy and public configuration unchanged; generic controller cutover and field acceptance remain pending.

Co-authored-by: GPT-6 <gpt-6@openai.com>
This commit is contained in:
ZeusinaandGPT-6 Bot committed 2026-10-15 16:23:54 +03:00
1 parent d324b4137b
commit f4853173b3
21 files changed
+1174 -72

No files matched your search

+34
View File
@@ -125,3 +125,37 @@ within a 90-second Docker stop limit (the supplied Compose grants 240 seconds).
Bounded final reaps/worst-case cleanup were verified offline; measured remote
normal stops were 1.45s, 0.59s and 0.57s, not an 80-second runtime worst-case test.
These changes do not claim to fix legacy YouTube exit 78.
### Internal service-adapter boundary (local extraction, field acceptance pending)
Legacy YouTube execution now explicitly selects compiled-in `youtube/1` via
`cmd/zapret-autotune/service_adapters.go`. `internal/adapter/youtube` parses
bounded Player media JSON and supplies signed-MP4 freshness/diversity semantics
and the owned qualification plan. Unknown/absent kind or version fails closed,
without a generic HTTP fallback. The shared `internal/adapter` DTOs are internal
only: signed URLs/inventory and plan caps have `json:"-"` fields.
The existing controller still owns discovery requests/retries, transport,
origin/SNI/certificate/TLS checks, worker guards/cleanup and sealing after join.
Legacy probes retain SOCKS5h and the owned encrypted-resolver guard; they do not
run the generic all-A/AAAA approval/connect-to IP pinning path. Full request-policy
capabilities (including future metadata POST) and parity are prerequisites before
generic cutover, not guarantees supplied by this minimal extraction.
Public generic checks and their HMAC/state formats have not migrated to this
module interface. Suffix/experimental authorization is separate from measured
endpoints, and avatar/thumbnail health remains independent. A module plan is
neither proof nor permission to claim READY.
New adaptive proof v2 seals adapter kind/version/policy digest along with the
measured facts. Unknown identities, mismatched policy, missing attestation and
unsealed DTOs cannot publish adaptive evidence. Legacy v1 is explicitly the
immutable historical YouTube policy for journal reconstruction, not a request
to reinterpret old evidence using the latest module. Expired historical proofs
can reconstruct their owned config. Fresh trusted stored v1 retains its original
apply eligibility; expiry still blocks fresh apply. Only new producer writes
require sealed module-bound v2; public/unsealed DTOs cannot publish evidence.
Discovery/asset/health entrypoints preflight the required kind/version/policy
before requests. Missing/mismatched modules are fatal configuration failures,
not retryable inventory absence or a genuine-no-strategy cooldown.
No public module field, dynamic plugin ABI, controller cutover, same-UID sandbox,
remote YouTube acceptance, or fix for legacy exit 78 is claimed here.
+104
View File
@@ -695,3 +695,107 @@ Root-private proof references (retained reports, not logs copied into this doc):
readiness, transactional generic apply/profile-journal migration, immutable
rotation and noninterrupting search remain outside this producer acceptance;
the later bounded apply/replay gates are recorded separately above.
## Internal YouTube adapter slice (local; field acceptance pending)
`internal/adapter` is a small closed compiled-in contract (`Binding`, ephemeral
`Resource`/`Plan`, `Module`); `internal/adapter/youtube` implements `youtube/1`.
The explicit kind/version registry in `service_adapters.go` has no init hooks,
dynamic loading, public configuration, HTTP fallback or module privileges.
Player media parsing (`playerMediaResource`) and owned native qualification
(`verifyGooglevideo`) actually call it. Discovery retry/rotation/cancellation and
representative selection remain in the legacy controller; signed MP4 freshness
and the host/TLS qualification plan are pure service semantics in the module.
Core origin/TLS validation, shared image/media transfer code, bounded worker
lifecycle and post-cleanup/join sealing remain authoritative. Legacy media and
metadata routing retains SOCKS5h with the owned encrypted-resolver guard and
exact origin/SNI/certificate/redirect checks. It does **not** use public generic
`runResourceProtocol`'s all-A/AAAA approval and connect-to IP pinning. That inherited
ceiling is unchanged, not a pinning guarantee: full request-policy capabilities
(including future metadata POST) and parity are prerequisites to generic cutover.
No new DNS/IP routing policy is introduced by this extraction. Both TLS
versions must learn separately, then each has three measured MP4 samples per
representative host, minimum 1024 bytes (not an exact 64 KiB requirement).
Measured endpoints do not authorize a suffix: existing exact/suffix/experimental
policy and independent thumbnail/avatar qualification are unchanged. The module
cannot manufacture evidence, request wildcard apply or claim YouTube READY.
New adaptive proof **v2** includes adapter kind/version/immutable policy SHA256
inside the producer seal. Public JSON cannot recreate that seal. Missing or
mismatched binding/version/digest fails closed; v1 with v2 fields is rejected.
Historical proof **v1** explicitly uses the pinned legacy YouTube v1 contract,
including historical freshness-independent journal reconstruction. Its config
bytes/hash and journal refs remain stable. Fresh stored trusted v1 proofs retain
their original apply/qualification eligibility, including unchanged attestation
freshness and media gates. Expired v1 can reconstruct but cannot authorize fresh
apply. Only **new production/persistence** requires sealed, module-bound v2;
neither sealed-new-v1 nor public unsealed v1/v2 DTOs can publish new evidence.
Cold v1 evidence, SQL schema 7, native arguments,
public GET/HEAD v1/v2, state keys and HMAC formats are unchanged.
This slice is not full generic-controller migration, native-engine runtime
acceptance, a security sandbox (including against same-UID actors), or a fix for
unresolved avatars/legacy exit 78. No legacy path is deleted. Local tests and
archived baseline parity are recorded separately; no remote validation or commit
is part of this extraction review.
Discovery preflight resolves the required module identity and policy digest
before any Player/WWW/avatar/front/media requests, including injected callbacks,
legacy proposal discovery and managed discovery hooks. Missing/mismatched modules
produce a fixed private sentinel wrapped as a fatal configuration failure, not
retryable unavailable inventory, a transfer-negative or genuine no-strategy pause.
The compiled production registry remains fixed; synchronized test replacement
does not add public runtime module configuration or loading.
### Local verification for this extraction (2026-10-15)
- Fresh serial `go test -json -p 1 ./... -parallel 1 -count=1 -timeout 600s`:
399 top-level / 1737 including subtest PASS events, zero failures.
- Affected serial race run (`-race -p 1 -parallel 1 -count=1`) covers adapters,
Player/discovery, owned qualification/proof gates, historical journal parity,
independent assets and generic resource core: 61 top-level / 307 including
subtest PASS events, zero failures. Full regex and logs are retained at
`/tmp/opencode/youtube-adapter-fixes/{full,race}.json`.
- Archived `d324b4137b2e95f1c5f4d6c8240470e3f8749f0c` and current builds each
passed 275 affected test/subtest events. The portable legacy parser/journal
corpus ran **unmodified** on both: file SHA256
`7b8eea9c9bbcdd8f0fadde3f78596d41e9d032a0a4f7b2ca90d1223d106584d5`.
Both reconstructed historical config SHA256
`172c7377388334e4baa2dc1009270478dc271a7b2f00d28cfada631f44a35364`,
retaining journal bytes/refs after reconcile and ActiveCandidates validation.
An additional unchanged portable v1 corpus passed 2 top-level / 8 including
subtest events on **each** build: fresh stored v1 shortlist/apply eligibility,
expired fresh-apply refusal, and staged/backup/prepared/replaced/reloaded/healthy
crash rollback with expired historical proofs. Backup/config bytes and hashes
are stable; pre-prepare journal bytes are unchanged and post-prepare generation/
reload/ref transitions follow the original contract. Corpus SHA256:
`6df53b12af5d80d164ea3201a614a5fd5723561dc7b2d6bc2d2aa37f845f053d`.
New negative tests separately reject even internally sealed NEW v1 writes and
public unsealed DTOs; missing/changed module identity/policy makes zero Player,
WWW/avatar/media/front/worker calls and does not become a two-hour absence.
- All four `python3 deploy/managed-target/check_{static,config,startup_diagnostics,supervision}.py`
checks passed serially; supervision was bounded with `timeout 600s`, no retry.
`go vet ./...`, `CGO_ENABLED=0 go build`, `sh -n` on all three entrypoint/
supervision scripts, and `git diff --check` passed. Build SHA256:
`8327faf86098145bb41306b3de895cdd622aeccdcda90f8ec73e19a759c8f98a`.
- Ownership-sensitive final runs used `sudo -n` EUID 0 **inside the existing
user namespace**, mapping UID 0 to parent/host UID 100000, not initial-host
root. The initial extra single-UID `unshare -Ur` run failed ownership fixtures
(`chown: invalid argument` / `invalid private qualification data`); a stale
test assertion also expected newly produced proof v1 and was updated to v2.
Those prior extraction failures are retained in
`/tmp/opencode/youtube-adapter-review/go-full.json`, not reported as passes. Vet's
initial unkeyed shared-DTO literals were fixed before final successful checks.
Reproduction commands (run serially; root is only for offline ownership fixtures):
```sh
pattern='Test(YoutubeLegacy|ManagedGooglevideo|ManagedAdaptive|ExperimentalPromotion|Googlevideo|Resource(Core|Discovery|Contract)|Player|Discover|ManagedAsset|Asset)'
sudo -n env GOPROXY=off GOTOOLCHAIN=local GOMODCACHE="$(go env GOMODCACHE)" "$(command -v go)" test -json -p 1 ./... -parallel 1 -count=1 -timeout 600s
sudo -n env GOPROXY=off GOTOOLCHAIN=local GOMODCACHE="$(go env GOMODCACHE)" "$(command -v go)" test -race -json -p 1 ./internal/adapter/... ./cmd/zapret-autotune -parallel 1 -run "${pattern}|Test(ServiceAdapter|LegacyYoutubeV1|QualificationPlan|SignedSample|DiscoverResponse)" -count=1 -timeout 600s
# Baseline/current parity: the same $pattern, without -race, on both trees.
# Baseline tree: git archive d324b4137b2e95f1c5f4d6c8240470e3f8749f0c;
# add identical youtube_legacy_parity_test.go and youtube_legacy_apply_parity_test.go.
# The additional portable v1 corpus runs separately with -run '^TestLegacyYoutubeV1'.
```
+26 -3
View File
@@ -9,6 +9,8 @@ import (
"math"
"strings"
"time"
"zapret-autotune/internal/adapter/youtube"
)
// Additive only: old proposals are not independently qualified candidates.
@@ -351,6 +353,7 @@ type GooglevideoHostProof struct {
}
type GooglevideoQualificationProof struct {
Adapter *youtube.Binding `json:"adapter,omitempty"`
Version int `json:"version"`
Mode QualificationMode `json:"mode"`
SourceKind string `json:"source_kind"`
@@ -371,7 +374,25 @@ func (p GooglevideoQualificationProof) validate(c storedCandidate) error {
}
func (p GooglevideoQualificationProof) validateStored(c storedCandidate) error {
if p.Version != 1 || p.Mode != c.Mode || p.Args != c.Args || len(p.Hosts) != len(c.Endpoints) || len(p.Hosts) < 3 {
// Missing binding is explicitly the immutable legacy YouTube v1 policy,
// never the current registry default. Retain historical reconstruction.
if p.Version == 1 {
if p.Adapter != nil {
return fmt.Errorf("legacy proof cannot carry adapter binding")
}
} else {
if p.Mode != QualificationAdaptive || p.Adapter == nil {
return fmt.Errorf("version-bound adaptive proof required")
}
module, err := lookupServiceAdapter(p.Adapter.Kind, p.Adapter.Version)
if err != nil {
return err
}
if *p.Adapter != module.Binding() {
return fmt.Errorf("adapter policy mismatch")
}
}
if (p.Version != 1 && p.Version != 2) || p.Mode != c.Mode || p.Args != c.Args || len(p.Hosts) != len(c.Endpoints) || len(p.Hosts) < 3 {
return fmt.Errorf("invalid representative proof definition")
}
if (p.Mode == QualificationCold && p.SourceKind != "media-range") || (p.Mode == QualificationAdaptive && p.SourceKind != NativeCircularPostWarmup) || (p.Mode != QualificationCold && p.Mode != QualificationAdaptive) {
@@ -380,7 +401,7 @@ func (p GooglevideoQualificationProof) validateStored(c storedCandidate) error {
if err := (&candidate{Args: p.Args}).validateArgs(); err != nil {
return err
}
if err := validateGooglevideoHosts(c.Endpoints); err != nil {
if err := validateLegacyYoutubeV1Hosts(c.Endpoints); err != nil {
return err
}
if p.Mode == QualificationCold && p.Attestation != nil {
@@ -494,7 +515,9 @@ func validateAdaptiveGooglevideoEvidence(db *sql.DB, c storedCandidate, fresh bo
// One transactional insert, no cold candidate_evidence summaries. A private
// producer seal is required as well as valid DTO facts and stored definition.
func saveAdaptiveGooglevideoEvidence(tx *sql.Tx, id int64, proof GooglevideoQualificationProof) error {
if proof.producerSeal == nil || *proof.producerSeal != googlevideoProofSeal(proof) {
// Existing trusted v1 rows retain their original read/apply contract. Only
// this new-write boundary requires the version-bound producer protocol.
if proof.Version != 2 || proof.producerSeal == nil || *proof.producerSeal != googlevideoProofSeal(proof) {
return fmt.Errorf("completed owned collector attestation required")
}
var c storedCandidate
+3
View File
@@ -148,6 +148,9 @@ func discoverYoutube(args []string, stdout, stderr io.Writer, runner playerRunne
}
func requestPlayer(videoID, proxy string, runner playerRunner) ([]byte, error) {
if _, err := requiredYoutubeAdapter(); err != nil {
return nil, err
}
payload, _ := json.Marshal(map[string]interface{}{
"videoId": videoID,
"context": map[string]interface{}{"client": map[string]interface{}{
+4
View File
@@ -44,6 +44,10 @@ func googlevideoProposal(flags []string, stdout, stderr io.Writer, discover play
fmt.Fprintln(stderr, "error: tune googlevideo", err)
return 2
}
if _, err := requiredYoutubeAdapter(); err != nil {
fmt.Fprintln(stderr, "error: tune googlevideo", err)
return 1
}
started := time.Now()
db, err := openStore(databasePath())
if err != nil {
@@ -71,6 +71,9 @@ type managedServiceInventory struct {
func managedServiceResources(cfg healthConfig, player func(context.Context) ([]byte, error), page playerRunner) func(context.Context) (managedServiceInventory, error) {
return func(ctx context.Context) (managedServiceInventory, error) {
i := managedServiceInventory{images: make(map[string]string), proxy: cfg.SOCKSProxy, runner: runResourceCurl}
if _, err := requiredYoutubeAdapter(); err != nil {
return i, err
}
if err := ctx.Err(); err != nil {
return i, managedRecoveryContextError(ctx, err)
}
@@ -105,6 +108,9 @@ func managedServiceResources(cfg healthConfig, player func(context.Context) ([]b
}
i.media, _, err = playerMediaResource(body)
if err != nil {
if errors.Is(err, errServiceAdapterUnavailable) {
return i, err
}
i.resourceErr = &managedResourceFailure{"media", "unknown", err}
}
for attempt := 0; attempt < 2; attempt++ {
@@ -152,6 +158,9 @@ func managedServiceResources(cfg healthConfig, player func(context.Context) ([]b
func managedServiceHealth(proxy string, front func(context.Context) (string, error), inventory func(context.Context) (managedServiceInventory, error), runner subprocessRunner) func(context.Context) (string, error) {
return func(ctx context.Context) (string, error) {
if _, err := requiredYoutubeAdapter(); err != nil {
return "DEGRADED", err
}
status, err := front(ctx)
if err != nil || status != "OK" {
return "DEGRADED", &managedResourceFailure{"front", managedFrontendHost, err}
+6
View File
@@ -597,6 +597,9 @@ func managedPrestartReconcile(m *managedTransaction, d managedDeps) (err error)
var errManagedScopeUnresolved = errors.New("managed deployment policy unresolved; explicit reconcile or fresh managed state required")
func managedActivate(m *managedTransaction, d managedDeps, ids []int64, expected map[int64]string, scopes ...GooglevideoScope) error {
if _, err := requiredYoutubeAdapter(); err != nil {
return err
}
info, err := os.Stat(d.script)
if err != nil || !info.Mode().IsRegular() || info.Mode().Perm()&0111 == 0 {
return fmt.Errorf("target zapret daemon init executable unavailable")
@@ -606,6 +609,9 @@ func managedActivate(m *managedTransaction, d managedDeps, ids []int64, expected
return err
}
m.Health = func() (string, error) {
if _, err := requiredYoutubeAdapter(); err != nil {
return "", err
}
portText, _ := d.lookup("SOCKS_PORT")
port, err := strconv.Atoi(portText)
if err != nil {
@@ -86,6 +86,9 @@ func managedExperimentalRequest(dir string, consume bool, apply func() error) er
// Runs before the PID-1-owned monitor starts; never races a recovery worker.
// A real Player-discovered identity is saved WITHOUT cold/adaptive evidence.
func managedExperimentalSuffix(m *managedTransaction, d managedDeps) error {
if _, err := requiredYoutubeAdapter(); err != nil {
return err
}
singleton, err := acquireManagedMonitorSingleton(d)
if err != nil {
return err
@@ -179,7 +179,7 @@ func TestExperimentalPromotionRequiresRepresentativeOwnedProof(t *testing.T) {
}
assertActiveCandidates(t, d.transaction, d.db, []int64{front, thumb, avatar, id})
p, err := readGooglevideoProof(d.db, id)
if err != nil || p.Version != 1 || len(p.Hosts) != 3 {
if err != nil || p.Version != 2 || p.Adapter == nil || len(p.Hosts) != 3 {
t.Fatal(p, err)
}
c, err := readStrategyCandidate(d.db, experiment)
@@ -5,9 +5,10 @@ import (
"errors"
"fmt"
"sort"
"strconv"
"strings"
"time"
"zapret-autotune/internal/adapter/youtube"
)
// Ordinary unresolved state, not a target/ownership guard failure.
@@ -28,7 +29,7 @@ const managedGooglevideoDiscoveryTimeout = 3 * time.Minute
// Three hosts each need up to 120s warmup + six 35s transfers. Leave margin
// for daemon lifecycle as well; short-lived URLs are refetched, never rewritten.
const managedGooglevideoSampleTTL = 20 * time.Minute
const managedGooglevideoSampleTTL = youtube.InventoryTTL
func freshGooglevideoSample(r mediaResource) bool {
// The same exact GGC hostname syntax as the representative gate; reject bad
@@ -37,13 +38,10 @@ func freshGooglevideoSample(r mediaResource) bool {
if !ok || !strings.HasSuffix(r.host, ".googlevideo.com") || !validDomain(r.host) || r.host != strings.ToLower(r.host) || root == "" || len(prefix) < 3 || !strings.HasPrefix(prefix, "rr") || strings.Trim(prefix[2:], "0123456789") != "" {
return false
}
u, err := privateResourceURL(r.url, r.host)
if err != nil || u.Path != "/videoplayback" || (r.kind != "video/mp4" && r.kind != "audio/mp4") {
if _, err := privateResourceURL(r.url, r.host); err != nil {
return false
}
q := u.Query()
expires, err := strconv.ParseInt(q.Get("expire"), 10, 64)
return err == nil && expires > time.Now().Add(managedGooglevideoSampleTTL).Unix() && (q.Get("sig") != "" || q.Get("signature") != "" || q.Get("lsig") != "")
return youtube.SignedSampleFresh(youtube.Resource{URL: r.url, Host: r.host, Kind: r.kind}, time.Now(), managedGooglevideoSampleTTL)
}
// Choose exactly three actual hosts passing the unchanged representative gate.
@@ -70,6 +68,9 @@ func representativeGooglevideoSamples(byHost map[string]mediaResource) []mediaRe
func managedGooglevideoSamples(ids []string, request func(context.Context, string) ([]byte, error)) func(context.Context) ([]mediaResource, error) {
ids = append([]string(nil), ids...)
return func(ctx context.Context) ([]mediaResource, error) {
if _, err := requiredYoutubeAdapter(); err != nil {
return nil, err
}
if len(ids) == 0 || len(ids) > 5 || request == nil {
return nil, &managedRecoveryFatal{Err: errors.New("invalid sample collector configuration")}
}
@@ -108,6 +109,9 @@ func managedGooglevideoSamples(ids []string, request func(context.Context, strin
}
r, _, err := playerMediaResource(body)
if err != nil {
if errors.Is(err, errServiceAdapterUnavailable) {
return nil, err
}
stats.Unavailable++
continue
} // Missing/SABR is unknown, never fabricated.
@@ -9,9 +9,10 @@ import (
"os"
"os/exec"
"reflect"
"strconv"
"strings"
"time"
"zapret-autotune/internal/adapter/youtube"
)
// ponytail: one audited composite only. New native variants need an explicit
@@ -93,45 +94,47 @@ func newManagedGooglevideoVerifier(target managedDeps) func(context.Context, sto
}
func (r managedAssetRuntime) verifyGooglevideo(ctx context.Context, c storedCandidate, resources []mediaResource) (GooglevideoQualificationProof, error) {
p := GooglevideoQualificationProof{Version: 1, Mode: QualificationAdaptive, SourceKind: NativeCircularPostWarmup, Args: c.Args}
p := GooglevideoQualificationProof{Version: 2, Mode: QualificationAdaptive, SourceKind: NativeCircularPostWarmup, Args: c.Args}
if c.Group != "googlevideo" || c.Mode != QualificationAdaptive {
return p, errors.New("adaptive googlevideo definition required")
}
if err := validateNativeGooglevideoArgs(c.Args); err != nil {
return p, err
}
byHost := map[string]mediaResource{}
for _, resource := range resources {
u, err := privateResourceURL(resource.url, resource.host)
if err != nil || u.Path != "/videoplayback" || (resource.kind != "video/mp4" && resource.kind != "audio/mp4") {
return p, errGooglevideoInsufficientSamples
module, err := requiredYoutubeAdapter()
if err != nil {
return p, err
}
q := u.Query()
expires, err := strconv.ParseInt(q.Get("expire"), 10, 64)
if err != nil || expires <= time.Now().Unix()+120 || (q.Get("sig") == "" && q.Get("signature") == "" && q.Get("lsig") == "") {
binding := module.Binding()
p.Adapter = &binding
byHost := map[string]mediaResource{}
var samples []youtube.Resource
for _, resource := range resources {
if _, err := privateResourceURL(resource.url, resource.host); err != nil {
return p, errGooglevideoInsufficientSamples
}
byHost[resource.host] = resource
samples = append(samples, youtube.Resource{URL: resource.url, Host: resource.host, Kind: resource.kind})
}
// Validate representative host diversity using the shared gate before capture.
for _, host := range c.Endpoints {
if _, ok := byHost[host]; !ok {
if validateGooglevideoHosts(c.Endpoints) != nil {
return p, errGooglevideoInsufficientSamples
}
plan, err := module.PlanQualification(c.Endpoints, samples, time.Now())
if err != nil {
return p, errGooglevideoInsufficientSamples
}
for _, host := range plan.Hosts {
p.Hosts = append(p.Hosts, GooglevideoHostProof{Host: host})
}
if err := validateGooglevideoHosts(c.Endpoints); err != nil {
return p, errGooglevideoInsufficientSamples
}
err := r.withOwnedWorker(ctx, c.Endpoints, c.Args, func(probeCtx context.Context) error {
err = r.withOwnedWorker(ctx, c.Endpoints, c.Args, func(probeCtx context.Context) error {
for i := range p.Hosts {
h := &p.Hosts[i]
h.WarmupLimit = 10
h.WarmupLimit = plan.WarmupLimit
start := time.Now()
warmCtx, cancel := context.WithTimeout(probeCtx, 120*time.Second)
warmCtx, cancel := context.WithTimeout(probeCtx, plan.WarmupTimeout)
learned := map[string]bool{}
for h.WarmupAttempts < h.WarmupLimit && len(learned) < 2 && warmCtx.Err() == nil {
protocol := []string{"tls12", "tls13"}[h.WarmupAttempts%2]
for h.WarmupAttempts < h.WarmupLimit && len(learned) < len(plan.Protocols) && warmCtx.Err() == nil {
protocol := plan.Protocols[h.WarmupAttempts%len(plan.Protocols)]
h.WarmupAttempts++
_, pass, err := collectMediaTransfer(warmCtx, r.proxy, byHost[h.Host], protocol, r.curl)
if err != nil {
@@ -140,7 +143,7 @@ func (r managedAssetRuntime) verifyGooglevideo(ctx context.Context, c storedCand
cancel()
return err
}
} else if pass.Bytes < 1024 {
} else if pass.Bytes < plan.MinBytes {
h.WarmupFailures++
} else {
learned[protocol] = true
@@ -151,11 +154,11 @@ func (r managedAssetRuntime) verifyGooglevideo(ctx context.Context, c storedCand
if probeCtx.Err() != nil {
return probeCtx.Err()
}
if len(learned) != 2 || h.WarmupMS > 120000 {
if len(learned) != len(plan.Protocols) || h.WarmupMS > float64(plan.WarmupTimeout/time.Millisecond) {
return errCandidateNonpass
}
for _, protocol := range []string{"tls12", "tls13"} {
for n := 0; n < 3; n++ {
for _, protocol := range plan.Protocols {
for n := 0; n < plan.Samples; n++ {
_, pass, err := collectMediaTransfer(probeCtx, r.proxy, byHost[h.Host], protocol, r.curl)
if err != nil {
if managedGooglevideoTransferNonpass(err) {
@@ -163,7 +166,7 @@ func (r managedAssetRuntime) verifyGooglevideo(ctx context.Context, c storedCand
}
return err
}
if pass.Bytes < 1024 {
if pass.Bytes < plan.MinBytes {
return errCandidateNonpass
}
h.Passes = append(h.Passes, pass)
@@ -239,16 +242,28 @@ func managedResourceTransferNonpass(err error, metadata, image bool) bool {
}
func validateGooglevideoHosts(hosts []string) error {
seen, roots, prefixes := map[string]bool{}, map[string]bool{}, map[string]bool{}
for _, host := range hosts {
prefix, root, ok := strings.Cut(strings.TrimSuffix(host, ".googlevideo.com"), "---sn-")
if !ok || !strings.HasSuffix(host, ".googlevideo.com") || !validDomain(host) || host != strings.ToLower(host) || seen[host] || root == "" || len(prefix) < 3 || !strings.HasPrefix(prefix, "rr") || strings.Trim(prefix[2:], "0123456789") != "" {
if !validDomain(host) {
return errGooglevideoInsufficientSamples
}
seen[host], roots[strings.Split(root, "-")[0]], prefixes[prefix] = true, true, true
}
if len(hosts) < 3 || len(roots) < 2 || len(prefixes) < 2 {
module, err := requiredYoutubeAdapter()
if err != nil {
return err
}
if module.ValidateHosts(hosts) != nil {
return errGooglevideoInsufficientSamples
}
return nil
}
// Historical v1 reconstruction must not depend on a future registry default.
// This pinned contract remains available even when the execution module is absent.
func validateLegacyYoutubeV1Hosts(hosts []string) error {
for _, host := range hosts {
if !validDomain(host) {
return errGooglevideoInsufficientSamples
}
}
return (youtube.Module{}).ValidateHosts(hosts)
}
+6
View File
@@ -47,6 +47,9 @@ type managedRecoveryDeps struct {
// The injected command runner receives only fixed script/argv from orchestration.
func managedRecoveryDependencies(target managedDeps, db *sql.DB, scan cacheScan, check cacheCheck, run func(context.Context, string, ...string) error) (managedRecoveryDeps, error) {
d := managedRecoveryDeps{db: db, scan: scan, check: check, run: run}
if _, err := requiredYoutubeAdapter(); err != nil {
return d, err
}
if target.lookup == nil || target.player == nil || db == nil || scan == nil || check == nil || run == nil {
return d, fmt.Errorf("complete target-local recovery adapter dependencies required")
}
@@ -59,6 +62,9 @@ func managedRecoveryDependencies(target managedDeps, db *sql.DB, scan cacheScan,
}
d.transaction = &managedTransaction{Dir: target.directory, Name: "config"}
requestVideo := func(ctx context.Context, videoID string) ([]byte, error) {
if _, err := requiredYoutubeAdapter(); err != nil {
return nil, err
}
if err := ctx.Err(); err != nil {
return nil, err
}
+8 -30
View File
@@ -190,39 +190,17 @@ type mediaCapability struct {
// SABR is deliberately unknown/not supported, never fabricated as readiness.
func playerMediaResource(body []byte) (mediaResource, mediaCapability, error) {
var p struct {
PlayabilityStatus struct {
Status string `json:"status"`
} `json:"playabilityStatus"`
StreamingData struct {
Formats []struct {
URL string `json:"url"`
MimeType string `json:"mimeType"`
} `json:"formats"`
AdaptiveFormats []struct {
URL string `json:"url"`
MimeType string `json:"mimeType"`
} `json:"adaptiveFormats"`
} `json:"streamingData"`
}
if len(body) > playerResponseLimit || json.Unmarshal(body, &p) != nil || p.PlayabilityStatus.Status != "OK" {
return mediaResource{}, mediaCapability{}, errors.New("invalid playable metadata")
}
for _, f := range append(p.StreamingData.Formats, p.StreamingData.AdaptiveFormats...) {
u, err := url.Parse(f.URL)
module, err := requiredYoutubeAdapter()
if err != nil {
continue
return mediaResource{}, mediaCapability{}, err
}
host := u.Hostname()
if !strings.HasSuffix(host, ".googlevideo.com") {
continue
resources, err := module.DiscoverResponse(body)
if err != nil {
return mediaResource{}, mediaCapability{}, err
}
if _, err := privateResourceURL(f.URL, host); err != nil {
continue
}
kind, _, err := mime.ParseMediaType(f.MimeType)
if err == nil && (kind == "video/mp4" || kind == "audio/mp4") {
return mediaResource{f.URL, host, kind}, mediaCapability{OrdinaryURL: true}, nil
for _, r := range resources {
if _, err := privateResourceURL(r.URL, r.Host); err == nil {
return mediaResource{r.URL, r.Host, r.Kind}, mediaCapability{OrdinaryURL: true}, nil
}
}
return mediaResource{}, mediaCapability{}, nil
+54
View File
@@ -0,0 +1,54 @@
package main
import (
"errors"
"sync"
"zapret-autotune/internal/adapter"
"zapret-autotune/internal/adapter/youtube"
)
// Trusted compiled-in dispatch, deliberately not a public config/plugin API.
// An empty registry is a supported fail-closed seam; no HTTP fallback exists.
type serviceAdapterKey struct {
kind string
version int
}
type serviceAdapterRegistry map[serviceAdapterKey]adapter.Module
var errServiceAdapterUnavailable = errors.New("required service adapter unavailable")
// The registry is immutable in production. The lock permits race-free test
// replacement while request callbacks/owned lifecycle children are running.
var serviceAdapters = struct {
sync.RWMutex
modules serviceAdapterRegistry
}{modules: serviceAdapterRegistry{{youtube.Kind, youtube.Version}: youtube.Module{}}}
func requiredYoutubeAdapter() (adapter.Module, error) {
serviceAdapters.RLock()
m, err := serviceAdapters.modules.lookup(youtube.Kind, youtube.Version)
serviceAdapters.RUnlock()
if err != nil || m.Binding() != (youtube.Module{}).Binding() {
return nil, &managedRecoveryFatal{Err: errServiceAdapterUnavailable}
}
return m, nil
}
func (r serviceAdapterRegistry) lookup(kind string, version int) (adapter.Module, error) {
m, ok := r[serviceAdapterKey{kind, version}]
if !ok || m == nil {
return nil, errServiceAdapterUnavailable
}
b := m.Binding()
if b.Kind != kind || b.Version != version {
return nil, errServiceAdapterUnavailable
}
return m, nil
}
func lookupServiceAdapter(kind string, version int) (adapter.Module, error) {
serviceAdapters.RLock()
defer serviceAdapters.RUnlock()
return serviceAdapters.modules.lookup(kind, version)
}
@@ -0,0 +1,284 @@
package main
import (
"context"
"encoding/json"
"errors"
"io"
"testing"
"time"
"zapret-autotune/internal/adapter"
"zapret-autotune/internal/adapter/youtube"
)
func replaceServiceAdapters(t *testing.T, modules serviceAdapterRegistry) {
t.Helper()
serviceAdapters.Lock()
previous := serviceAdapters.modules
serviceAdapters.modules = modules
serviceAdapters.Unlock()
t.Cleanup(func() {
serviceAdapters.Lock()
serviceAdapters.modules = previous
serviceAdapters.Unlock()
})
}
func TestServiceAdapterDispatchFailClosed(t *testing.T) {
for _, key := range []serviceAdapterKey{{"unknown", 1}, {youtube.Kind, 0}, {youtube.Kind, 2}} {
if _, err := lookupServiceAdapter(key.kind, key.version); err == nil {
t.Fatal("unsupported adapter accepted")
}
}
if _, err := (serviceAdapterRegistry{}).lookup(youtube.Kind, youtube.Version); err == nil {
t.Fatal("absent adapter accepted")
}
replaceServiceAdapters(t, serviceAdapterRegistry{})
if _, _, err := playerMediaResource([]byte(`{"playabilityStatus":{"status":"OK"}}`)); err == nil {
t.Fatal("discovery bypassed registry")
}
r, c, resources, events := googlevideoRuntimeFixture(t)
if _, err := r.verifyGooglevideo(context.Background(), c, resources); err == nil || len(*events) != 0 {
t.Fatal("qualification bypassed registry")
}
}
type mismatchedServiceAdapter struct{ adapter.Module }
func (m mismatchedServiceAdapter) Binding() adapter.Binding {
b := m.Module.Binding()
b.Version++
return b
}
func TestServiceAdapterIdentityBeforeExecution(t *testing.T) {
replaceServiceAdapters(t, serviceAdapterRegistry{{youtube.Kind, youtube.Version}: mismatchedServiceAdapter{youtube.Module{}}})
r, c, resources, events := googlevideoRuntimeFixture(t)
if _, err := r.verifyGooglevideo(context.Background(), c, resources); err == nil || len(*events) != 0 {
t.Fatal("identity mismatch reached worker")
}
}
func TestServiceAdapterStoredProofRefusals(t *testing.T) {
r, c, resources, _ := googlevideoRuntimeFixture(t)
p, err := r.verifyGooglevideo(context.Background(), c, resources)
if err != nil {
t.Fatal(err)
}
db := testStore(t)
tx, err := db.Begin()
if err != nil {
t.Fatal(err)
}
defer tx.Rollback()
id, err := insertStrategyCandidate(tx, c.Group, c.Endpoints, c.Args, nil, c.Mode)
if err != nil {
t.Fatal(err)
}
if err := saveAdaptiveGooglevideoEvidence(tx, id, p); err != nil {
t.Fatal(err)
}
if err := tx.Commit(); err != nil {
t.Fatal(err)
}
front := managedCandidateFixture(t, db, "frontend", "www.youtube.com", "--lua-desync=multisplit", true)
for _, mode := range []string{"kind", "version", "digest", "missing", "unknown-proof", "downgrade", "attestation"} {
t.Run(mode, func(t *testing.T) {
raw, _ := json.Marshal(p)
var proof GooglevideoQualificationProof
if err := json.Unmarshal(raw, &proof); err != nil {
t.Fatal(err)
}
switch mode {
case "kind":
proof.Adapter.Kind = "unknown"
case "version":
proof.Adapter.Version++
case "digest":
proof.Adapter.PolicySHA256 = "changed"
case "missing":
proof.Adapter = nil
case "unknown-proof":
proof.Version = 3
case "downgrade":
proof.Version = 1
case "attestation":
proof.Attestation = nil
}
if proof.validateStored(c) == nil {
t.Fatal("malformed proof reconstructed")
}
raw, _ = json.Marshal(proof)
if _, err := db.Exec("UPDATE googlevideo_scope_evidence SET proof=? WHERE candidate_id=?", string(raw), id); err != nil {
t.Fatal(err)
}
m, reloads := managedFixture(t)
if err := m.Apply(db, []int64{front, id}, GooglevideoScopeSuffix); err == nil || *reloads != 0 {
t.Fatal("invalid proof reached apply", err, *reloads)
}
})
}
}
func TestServiceAdapterProofBoundary(t *testing.T) {
r, c, resources, _ := googlevideoRuntimeFixture(t)
p, err := r.verifyGooglevideo(context.Background(), c, resources)
if err != nil {
t.Fatal(err)
}
if p.Version != 2 || p.Adapter == nil || *p.Adapter != (youtube.Module{}).Binding() {
t.Fatal("missing module identity")
}
seal := googlevideoProofSeal(p)
for _, mutate := range []func(){
func() { p.Adapter.Kind = "unknown" },
func() { p.Adapter.Version++ },
func() { p.Adapter.PolicySHA256 = "changed" },
} {
original := *p.Adapter
mutate()
if p.validateStored(c) == nil || googlevideoProofSeal(p) == seal {
t.Fatal("module boundary not bound to evidence")
}
*p.Adapter = original
}
// Reads retain the original v1 contract, including fresh apply eligibility.
p.Version, p.Adapter = 1, nil
raw, err := json.Marshal(p)
if err != nil {
t.Fatal(err)
}
legacy, err := decodeGooglevideoProof(string(raw))
if err != nil || legacy.validateStored(c) != nil {
t.Fatal("legacy reconstruction lost", err)
}
if legacy.validate(c) != nil {
t.Fatal("fresh trusted legacy adaptive proof lost apply eligibility")
}
legacy.Attestation.CheckedAt = time.Now().Add(-25 * time.Hour)
if legacy.validate(c) == nil {
t.Fatal("expired legacy adaptive proof authorized new apply")
}
replaceServiceAdapters(t, serviceAdapterRegistry{})
if legacy.validateStored(c) != nil {
t.Fatal("historical reconstruction depends on current execution registry")
}
}
type changedPolicyServiceAdapter struct{ adapter.Module }
func (m changedPolicyServiceAdapter) Binding() adapter.Binding {
b := m.Module.Binding()
b.PolicySHA256 = "changed"
return b
}
func TestServiceAdapterDiscoveryPreflight(t *testing.T) {
for _, mode := range []string{"missing", "version", "digest"} {
t.Run(mode, func(t *testing.T) {
modules := serviceAdapterRegistry{}
if mode == "version" {
modules[serviceAdapterKey{youtube.Kind, youtube.Version}] = mismatchedServiceAdapter{youtube.Module{}}
}
if mode == "digest" {
modules[serviceAdapterKey{youtube.Kind, youtube.Version}] = changedPolicyServiceAdapter{youtube.Module{}}
}
calls := 0
player := func(context.Context, []string, io.Writer, io.Writer) error { calls++; return nil }
request := func(context.Context) ([]byte, error) { calls++; return nil, nil }
target, _, _ := managedCLIFixture(t)
target.player = player
target.verifySOCKS = func(int) error { calls++; return nil }
r, _, _ := recoveryFixture(t, false)
bound, err := managedRecoveryDependencies(target, r.db, r.scan, r.check, r.run)
if err != nil {
t.Fatal(err)
}
calls = 0 // Construction guard is allowed before removing the module.
replaceServiceAdapters(t, modules)
cfg := healthConfig{VideoID: "dQw4w9WgXcQ", SOCKSProxy: "socks5://127.0.0.1:1080"}
assertBlocked := func(err error) {
t.Helper()
var fatal *managedRecoveryFatal
if !errors.Is(err, errServiceAdapterUnavailable) || !errors.As(err, &fatal) || errors.Is(err, errNoQualifiedStrategy) || errors.Is(err, errGooglevideoInsufficientSamples) || managedResourceTransferNonpass(err, true, true) {
t.Fatal("adapter configuration became a retryable absence", err)
}
if calls != 0 {
t.Fatal("preflight made network calls", calls)
}
}
_, err = requestPlayer(cfg.VideoID, cfg.SOCKSProxy, player)
assertBlocked(err)
if managedResourceTransferNonpass(errors.Join(err, context.DeadlineExceeded), true, true) || managedRecoveryCooldown(errors.Join(err, errNoQualifiedStrategy), time.Minute) != time.Minute {
t.Fatal("mixed fatal adapter failure became transfer-negative/2h absence")
}
_, err = bound.discover(context.Background())
assertBlocked(err)
_, err = bound.health(context.Background())
assertBlocked(err)
_, err = managedGooglevideoSamples([]string{cfg.VideoID}, func(ctx context.Context, _ string) ([]byte, error) { return request(ctx) })(context.Background())
assertBlocked(err)
inventory := managedServiceResources(cfg, request, player)
_, err = inventory(context.Background())
assertBlocked(err)
_, err = managedServiceHealth(cfg.SOCKSProxy, func(context.Context) (string, error) { calls++; return "OK", nil }, inventory, func(context.Context, []string, io.Reader, io.Writer, io.Writer) error { calls++; return nil })(context.Background())
assertBlocked(err)
if code := discoverYoutube([]string{"--video-id", cfg.VideoID, "--proxy", cfg.SOCKSProxy}, io.Discard, io.Discard, player); code != 1 || calls != 0 {
t.Fatal("legacy CLI bypassed preflight", code, calls)
}
_, err = managedRecoveryDependencies(target, r.db, r.scan, r.check, r.run)
assertBlocked(err)
assertBlocked(managedExperimentalSuffix(nil, managedDeps{}))
assertBlocked(managedActivate(nil, managedDeps{}, nil, nil))
for _, saved := range []bool{false, true} {
code := googlevideoProposal([]string{"--video-id", cfg.VideoID, "--proxy", cfg.SOCKSProxy}, io.Discard, io.Discard, player, func(context.Context, string, string, io.Writer, io.Writer) (string, error) { calls++; return "", nil }, func(context.Context, string, string, string, string, io.Writer, io.Writer) error { calls++; return nil }, saved)
if code != 1 || calls != 0 {
t.Fatal("proposal pipeline bypassed preflight", code, calls)
}
}
runtime, c, resources, events := googlevideoRuntimeFixture(t)
_, err = runtime.verifyGooglevideo(context.Background(), c, resources)
assertBlocked(err)
if len(*events) != 0 {
t.Fatal("preflight launched worker")
}
})
}
}
func TestServiceAdapterLegacyNewWriteRefused(t *testing.T) {
r, c, resources, _ := googlevideoRuntimeFixture(t)
p, err := r.verifyGooglevideo(context.Background(), c, resources)
if err != nil {
t.Fatal(err)
}
p.Version, p.Adapter = 1, nil
if p.validate(c) != nil {
t.Fatal("trusted legacy read invalid")
}
// Even an internally constructed seal cannot authorize NEW v1 publication.
seal := googlevideoProofSeal(p)
p.producerSeal = &seal
db := testStore(t)
tx, err := db.Begin()
if err != nil {
t.Fatal(err)
}
defer tx.Rollback()
id, err := insertStrategyCandidate(tx, c.Group, c.Endpoints, c.Args, nil, c.Mode)
if err != nil {
t.Fatal(err)
}
if saveAdaptiveGooglevideoEvidence(tx, id, p) == nil {
t.Fatal("new sealed v1 write accepted")
}
raw, _ := json.Marshal(p)
var dto GooglevideoQualificationProof
if err := json.Unmarshal(raw, &dto); err != nil {
t.Fatal(err)
}
if saveAdaptiveGooglevideoEvidence(tx, id, dto) == nil {
t.Fatal("unsealed v1 DTO write accepted")
}
}
@@ -0,0 +1,168 @@
package main
// Portable trusted-history fixtures: run unchanged on archived d324b41 too.
// Direct SQL below models an existing trusted v1 database, not an import API.
import (
"bytes"
"context"
"database/sql"
"encoding/json"
"errors"
"reflect"
"testing"
"time"
)
func legacyYoutubeV1Fixture(t *testing.T, db *sql.DB) (int64, GooglevideoQualificationProof) {
t.Helper()
r, c, resources, _ := googlevideoRuntimeFixture(t)
p, err := r.verifyGooglevideo(context.Background(), c, resources)
if err != nil {
t.Fatal(err)
}
raw, _ := json.Marshal(p)
var fields map[string]json.RawMessage
if err := json.Unmarshal(raw, &fields); err != nil {
t.Fatal(err)
}
delete(fields, "adapter")
fields["version"] = json.RawMessage(`1`)
raw, _ = json.Marshal(fields)
legacy, err := decodeGooglevideoProof(string(raw))
if err != nil || legacy.validate(c) != nil {
t.Fatal("trusted v1 contract invalid", err)
}
tx, err := db.Begin()
if err != nil {
t.Fatal(err)
}
defer tx.Rollback()
id, err := insertStrategyCandidate(tx, c.Group, c.Endpoints, c.Args, nil, c.Mode)
if err != nil {
t.Fatal(err)
}
if _, err := tx.Exec("INSERT INTO googlevideo_scope_evidence(candidate_id,proof) VALUES (?,?)", id, string(raw)); err != nil {
t.Fatal(err)
}
if err := tx.Commit(); err != nil {
t.Fatal(err)
}
return id, legacy
}
func expireLegacyYoutubeV1(t *testing.T, db *sql.DB, id int64, proof GooglevideoQualificationProof) {
t.Helper()
proof.Attestation.CheckedAt = time.Now().Add(-25 * time.Hour)
raw, _ := json.Marshal(proof)
if _, err := db.Exec("UPDATE googlevideo_scope_evidence SET proof=? WHERE candidate_id=?", string(raw), id); err != nil {
t.Fatal(err)
}
}
func TestLegacyYoutubeV1FreshApplyEligibility(t *testing.T) {
db := testStore(t)
id, proof := legacyYoutubeV1Fixture(t, db)
c, err := readStrategyCandidate(db, id)
if err != nil || !c.Qualified {
t.Fatal("fresh stored v1 lost qualification", err)
}
saved, err := listQualifiedCandidates(db, "googlevideo", 10)
if err != nil || len(saved) != 1 || saved[0].ID != id {
t.Fatal("fresh stored v1 disappeared from shortlist", err)
}
front := managedCandidateFixture(t, db, "frontend", "www.youtube.com", "--lua-desync=multisplit", true)
ids := []int64{front, id} // Native suffix still does not require assets.
m, reloads := managedFixture(t)
if err := m.Apply(db, ids, GooglevideoScopeSuffix); err != nil {
t.Fatal("fresh v1 apply rejected", err)
}
config, err := m.read(m.Name)
if err != nil {
t.Fatal(err)
}
expireLegacyYoutubeV1(t, db, id, proof)
c, err = readStrategyCandidate(db, id)
if err != nil || c.Qualified {
t.Fatal("expired v1 retained fresh qualification", err)
}
before := *reloads
if err := m.Apply(db, ids, GooglevideoScopeSuffix); err == nil || *reloads != before {
t.Fatal("expired v1 activated")
}
rendered, err := renderStoredManagedConfigScope(db, ids, GooglevideoScopeSuffix)
if err != nil || !bytes.Equal(config, rendered) {
t.Fatal("expired historical reconstruction changed", err)
}
}
func TestLegacyYoutubeV1PreparedRollback(t *testing.T) {
for _, point := range []string{"staged", "backup", "prepared", "replaced", "reloaded", "healthy"} {
t.Run(point, func(t *testing.T) {
db := testStore(t)
id, proof := legacyYoutubeV1Fixture(t, db)
front := managedCandidateFixture(t, db, "frontend", "www.youtube.com", "--lua-desync=multisplit", true)
ids := []int64{front, id}
m, reloads := managedFixture(t)
if err := m.Apply(db, ids, GooglevideoScopeSuffix); err != nil {
t.Fatal(err)
}
before, err := m.load()
if err != nil {
t.Fatal(err)
}
journal, err := m.read(managedJournalName)
if err != nil {
t.Fatal(err)
}
config, err := m.read(m.Name)
if err != nil {
t.Fatal(err)
}
nextFront := managedCandidateFixture(t, db, "frontend", "www.youtube.com", "--lua-desync=fake", true)
m.fault = func(stage string) error {
if stage == point {
return errors.New("fixture crash")
}
return nil
}
if err := m.Apply(db, []int64{nextFront, id}, GooglevideoScopeSuffix); err == nil {
t.Fatal("fault missed")
}
if point != "staged" {
backup, err := m.read(managedBackupName)
if err != nil || !bytes.Equal(backup, config) {
t.Fatal("legacy backup changed", err)
}
}
expireLegacyYoutubeV1(t, db, id, proof)
m.fault = nil
priorReloads := *reloads
if err := m.Reconcile(db); err != nil {
t.Fatal("expired v1 prepared recovery failed", err)
}
after, err := m.load()
if err != nil {
t.Fatal(err)
}
wantGeneration, wantReloads := before.Generation+1, priorReloads+1
if point == "staged" || point == "backup" {
wantGeneration, wantReloads = before.Generation, priorReloads
}
if after.State != "committed" || after.Hash != before.Hash || after.Scope != before.Scope || !reflect.DeepEqual(after.Candidates, before.Candidates) || len(after.PendingCandidates) != 0 || after.PendingScope != "" || after.Generation != wantGeneration || *reloads != wantReloads {
t.Fatal("legacy rollback contract changed", after, *reloads)
}
restored, err := m.read(m.Name)
if err != nil || !bytes.Equal(restored, config) {
t.Fatal("rollback config bytes changed", err)
}
if point == "staged" || point == "backup" {
afterJournal, err := m.read(managedJournalName)
if err != nil || !bytes.Equal(journal, afterJournal) {
t.Fatal("pre-prepare journal changed", err)
}
}
assertActiveCandidates(t, m, db, ids)
t.Logf("legacy restored config SHA256=%s", after.Hash)
})
}
}
@@ -0,0 +1,133 @@
package main
// This corpus also runs unmodified against archived d324b41. It uses only
// pre-extraction APIs, with explicit expected normalized facts, not wrappers.
import (
"bytes"
"context"
"encoding/json"
"fmt"
"strings"
"testing"
"time"
)
func TestYoutubeLegacyParserParity(t *testing.T) {
good := "https://rr1---sn-alpha.googlevideo.com/videoplayback?expire=1800000000&sig=private-secret"
format := func(raw, mime string) map[string]string { return map[string]string{"url": raw, "mimeType": mime} }
for _, tc := range []struct {
name string
formats, adaptive []map[string]string
want string
}{
{"ordinary-first", []map[string]string{format(good, "audio/mp4; codecs=mp4a")}, []map[string]string{format(good+"&second=1", "video/mp4")}, "audio/mp4"},
{"nonmp4-fallback", []map[string]string{format(good, "video/webm")}, []map[string]string{format(good, "video/mp4")}, "video/mp4"},
{"userinfo", []map[string]string{format(strings.Replace(good, "https://", "https://user@", 1), "video/mp4")}, nil, ""},
{"fragment", []map[string]string{format(good+"#secret", "video/mp4")}, nil, ""},
{"http", []map[string]string{format(strings.Replace(good, "https:", "http:", 1), "video/mp4")}, nil, ""},
{"invalid-dns", []map[string]string{format(strings.Replace(good, "rr1---sn-alpha", "bad_host", 1), "video/mp4")}, nil, ""},
{"sabr-only", nil, nil, ""},
} {
t.Run(tc.name, func(t *testing.T) {
body, _ := json.Marshal(map[string]interface{}{"playabilityStatus": map[string]string{"status": "OK"}, "streamingData": map[string]interface{}{"formats": tc.formats, "adaptiveFormats": tc.adaptive, "serverAbrStreamingUrl": good}})
r, capability, err := playerMediaResource(body)
if err != nil || r.kind != tc.want || capability.OrdinaryURL != (tc.want != "") || capability.SABRSupported {
t.Fatal(r.kind, capability, err)
}
if tc.want != "" && (r.url != good || r.host != "rr1---sn-alpha.googlevideo.com") {
t.Fatal("normalized resource changed")
}
})
}
for _, body := range [][]byte{[]byte(`{"secret":`), bytes.Repeat([]byte("x"), playerResponseLimit+1)} {
if _, _, err := playerMediaResource(body); err == nil || err.Error() != "invalid playable metadata" {
t.Fatal("metadata error contract changed", err)
}
}
for _, delta := range []time.Duration{-time.Hour, 19 * time.Minute, 21 * time.Minute} {
r := mediaResource{fmt.Sprintf("https://rr1---sn-alpha.googlevideo.com/videoplayback?expire=%d&lsig=secret", time.Now().Add(delta).Unix()), "rr1---sn-alpha.googlevideo.com", "video/mp4"}
if freshGooglevideoSample(r) != (delta > managedGooglevideoSampleTTL) {
t.Fatal("inventory freshness changed", delta)
}
}
}
func TestYoutubeLegacyJournalParity(t *testing.T) {
r, c, resources, _ := googlevideoRuntimeFixture(t)
p, err := r.verifyGooglevideo(context.Background(), c, resources)
if err != nil {
t.Fatal(err)
}
db := testStore(t)
tx, err := db.Begin()
if err != nil {
t.Fatal(err)
}
defer tx.Rollback()
id, err := insertStrategyCandidate(tx, c.Group, c.Endpoints, c.Args, nil, c.Mode)
if err != nil {
t.Fatal(err)
}
if err := saveAdaptiveGooglevideoEvidence(tx, id, p); err != nil {
t.Fatal(err)
}
if err := tx.Commit(); err != nil {
t.Fatal(err)
}
front := managedCandidateFixture(t, db, "frontend", "www.youtube.com", "--lua-desync=multisplit", true)
m, _ := managedFixture(t)
ids := []int64{front, id} // No new asset requirement for native suffix policy.
if err := m.Apply(db, ids, GooglevideoScopeSuffix); err != nil {
t.Fatal(err)
}
before, err := m.load()
if err != nil {
t.Fatal(err)
}
journalBytes, err := m.read(managedJournalName)
if err != nil {
t.Fatal(err)
}
config, err := m.read(m.Name)
if err != nil {
t.Fatal(err)
}
// Convert the actual producer facts to the exact historical v1 wire shape.
p.Attestation.CheckedAt = time.Now().Add(-25 * time.Hour)
raw, _ := json.Marshal(p)
var fields map[string]json.RawMessage
if err := json.Unmarshal(raw, &fields); err != nil {
t.Fatal(err)
}
delete(fields, "adapter")
fields["version"] = json.RawMessage(`1`)
raw, _ = json.Marshal(fields)
if _, err := db.Exec("UPDATE googlevideo_scope_evidence SET proof=? WHERE candidate_id=?", string(raw), id); err != nil {
t.Fatal(err)
}
rendered, err := renderStoredManagedConfigScope(db, ids, GooglevideoScopeSuffix)
if err != nil || !bytes.Equal(config, rendered) || managedHash(rendered) != before.Hash {
t.Fatal("historical render/hash changed", err)
}
if err := m.Reconcile(db); err != nil {
t.Fatal(err)
}
active, err := m.ActiveCandidates(db)
if err != nil || len(active) != 2 || active[0] != front || active[1] != id {
t.Fatal(active, err)
}
after, err := m.load()
if err != nil {
t.Fatal(err)
}
a, _ := json.Marshal(before)
b, _ := json.Marshal(after)
if !bytes.Equal(a, b) {
t.Fatal("historical journal changed")
}
afterBytes, err := m.read(managedJournalName)
if err != nil || !bytes.Equal(journalBytes, afterBytes) {
t.Fatal("historical journal bytes changed", err)
}
t.Logf("historical config SHA256=%s", before.Hash)
}
+35
View File
@@ -0,0 +1,35 @@
// Package adapter defines the closed internal contract used by compiled-in
// service adapters. It is not the public resource-check configuration API.
package adapter
import "time"
type Binding struct {
Kind string `json:"kind"`
Version int `json:"version"`
PolicySHA256 string `json:"policy_sha256"`
}
// Resource and Plan are ephemeral; neither private inputs nor policy caps belong
// in evidence. Only core-validated, measured facts can become a sealed proof.
type Resource struct {
URL string `json:"-"`
Host string `json:"-"`
Kind string `json:"-"`
}
type Plan struct {
Hosts []string `json:"-"`
Protocols [2]string `json:"-"`
WarmupLimit int `json:"-"`
WarmupTimeout time.Duration `json:"-"`
Samples int `json:"-"`
MinBytes int64 `json:"-"`
}
type Module interface {
Binding() Binding
DiscoverResponse([]byte) ([]Resource, error)
ValidateHosts([]string) error
PlanQualification([]string, []Resource, time.Time) (Plan, error)
}
+29
View File
@@ -0,0 +1,29 @@
# Internal YouTube adapter
The compiled-in `youtube/1` module parses bounded Player JSON and supplies the
ordinary signed MP4 qualification policy and ordered host/TLS sample plan.
The existing YouTube execution paths select it explicitly through the core's
kind/version registry. Unsupported or absent modules fail closed.
The core retains origin validation, HTTPS/SNI/certificate checks, transport
framing/body bounds, retries/cancellation, worker ownership, cleanup/join and
proof sealing. Module output is not evidence or permission to claim PASS.
Signed URLs remain ephemeral and cannot serialize through the internal contract.
New adaptive proof v2 binds the module kind, version and immutable policy digest
inside its producer seal. Unbound proof v1 explicitly means the historical
YouTube v1 policy: fresh trusted stored proofs retain their original apply
eligibility; expired proofs reconstruct history but cannot authorize fresh apply.
Only new producer/persistence writes require sealed module-bound v2 evidence.
Cold historical evidence is unchanged. Discovery entrypoints preflight the module
before requests; absence/mismatched identity or policy is a fatal configuration
failure, not unavailable inventory or a no-strategy cooldown.
Legacy routing remains SOCKS5h with encrypted-resolver ownership guards and
origin/SNI/certificate/TLS checks, not generic all-answer approval/IP pinning.
Full request-policy capabilities and parity are prerequisites to generic cutover.
This is a real internal legacy-execution integration, not a fully generic
controller, public v3 configuration, native addon, runtime plugin or readiness
claim. Another compiled-in module can implement the closed internal contract;
public GET/HEAD v1/v2 configuration remains unchanged.
+120
View File
@@ -0,0 +1,120 @@
// Package youtube contains pure service semantics, not transport or worker ownership.
package youtube
import (
"crypto/sha256"
"encoding/json"
"errors"
"fmt"
"mime"
"net/url"
"strconv"
"strings"
"time"
"zapret-autotune/internal/adapter"
)
const Kind = "youtube"
const Version = 1
const InventoryTTL = 20 * time.Minute
// Immutable v1 policy. Changes require another module version, not a new meaning
// for existing evidence. Legacy proof v1 uses this same historical policy.
const policyV1 = "youtube/1;ggc>=3;roots>=2;prefixes>=2;ordinary-signed-mp4;fresh>120s;inventory>20m;tls12,tls13;warmup<=10,120s;steady=3/protocol;min=1024"
type Binding = adapter.Binding
type Module struct{}
func (Module) Binding() Binding {
return Binding{Kind: Kind, Version: Version, PolicySHA256: fmt.Sprintf("%x", sha256.Sum256([]byte(policyV1)))}
}
// Resource is ephemeral: never serialize signed URLs or inventory metadata.
type Resource = adapter.Resource
// DiscoverResponse preserves player format order. The core must validate every
// returned origin before selecting or fetching it. SABR is not supported.
func (Module) DiscoverResponse(body []byte) ([]Resource, error) {
type format struct {
URL string `json:"url"`
MimeType string `json:"mimeType"`
}
var p struct {
PlayabilityStatus struct {
Status string `json:"status"`
} `json:"playabilityStatus"`
StreamingData struct {
Formats []format `json:"formats"`
AdaptiveFormats []format `json:"adaptiveFormats"`
} `json:"streamingData"`
}
if len(body) > 1<<20 || json.Unmarshal(body, &p) != nil || p.PlayabilityStatus.Status != "OK" {
return nil, errors.New("invalid playable metadata")
}
var resources []Resource
for _, f := range append(p.StreamingData.Formats, p.StreamingData.AdaptiveFormats...) {
u, err := url.Parse(f.URL)
if err != nil || !strings.HasSuffix(u.Hostname(), ".googlevideo.com") {
continue
}
kind, _, err := mime.ParseMediaType(f.MimeType)
if err == nil && (kind == "video/mp4" || kind == "audio/mp4") {
resources = append(resources, Resource{URL: f.URL, Host: u.Hostname(), Kind: kind})
}
}
return resources, nil
}
// ValidateHosts checks service diversity; DNS/origin policy remains in the core.
func (Module) ValidateHosts(hosts []string) error {
seen, roots, prefixes := map[string]bool{}, map[string]bool{}, map[string]bool{}
for _, host := range hosts {
prefix, root, ok := strings.Cut(strings.TrimSuffix(host, ".googlevideo.com"), "---sn-")
if !ok || !strings.HasSuffix(host, ".googlevideo.com") || host != strings.ToLower(host) || seen[host] || root == "" || len(prefix) < 3 || !strings.HasPrefix(prefix, "rr") || strings.Trim(prefix[2:], "0123456789") != "" {
return errors.New("insufficient representative GGC hosts")
}
seen[host], roots[strings.Split(root, "-")[0]], prefixes[prefix] = true, true, true
}
if len(hosts) < 3 || len(roots) < 2 || len(prefixes) < 2 {
return errors.New("insufficient representative GGC hosts")
}
return nil
}
type Plan = adapter.Plan
// PlanQualification grants no PASS capability. Core validates origins first,
// executes the ordered schedule, validates measured facts, and seals after join.
func (m Module) PlanQualification(hosts []string, resources []Resource, now time.Time) (Plan, error) {
p := Plan{Hosts: append([]string(nil), hosts...), Protocols: [2]string{"tls12", "tls13"}, WarmupLimit: 10, WarmupTimeout: 120 * time.Second, Samples: 3, MinBytes: 1024}
if err := m.ValidateHosts(hosts); err != nil {
return Plan{}, err
}
seen := map[string]bool{}
for _, r := range resources {
if !SignedSampleFresh(r, now, 120*time.Second) {
return Plan{}, errors.New("insufficient fresh signed media samples")
}
seen[r.Host] = true
}
for _, host := range hosts {
if !seen[host] {
return Plan{}, errors.New("missing representative sample")
}
}
return p, nil
}
// SignedSampleFresh checks service semantics only, not permission to fetch.
// The core independently validates HTTPS origins before calling this helper.
func SignedSampleFresh(r Resource, now time.Time, ttl time.Duration) bool {
u, err := url.Parse(r.URL)
if err != nil || u.Path != "/videoplayback" || (r.Kind != "video/mp4" && r.Kind != "audio/mp4") {
return false
}
q := u.Query()
expire, err := strconv.ParseInt(q.Get("expire"), 10, 64)
return err == nil && expire > now.Add(ttl).Unix() && (q.Get("sig") != "" || q.Get("signature") != "" || q.Get("lsig") != "")
}
+90
View File
@@ -0,0 +1,90 @@
package youtube
import (
"encoding/json"
"fmt"
"strings"
"testing"
"time"
)
func TestDiscoverResponse(t *testing.T) {
m := Module{}
resources, err := m.DiscoverResponse([]byte(`{"playabilityStatus":{"status":"OK"},"streamingData":{"formats":[{"url":"https://rr1.unit.googlevideo.com/videoplayback?sig=secret","mimeType":"audio/mp4"}],"adaptiveFormats":[{"url":"https://rr2.unit.googlevideo.com/videoplayback?sig=secret","mimeType":"video/mp4; codecs=avc1"}]}}`))
if err != nil || len(resources) != 2 || resources[0].Kind != "audio/mp4" || resources[1].Kind != "video/mp4" {
t.Fatal(resources, err)
}
resources, err = m.DiscoverResponse([]byte(`{"playabilityStatus":{"status":"OK"},"streamingData":{"serverAbrStreamingUrl":"https://rr1.unit.googlevideo.com/sabr?sig=secret"}}`))
if err != nil || len(resources) != 0 {
t.Fatal("SABR fabricated ordinary readiness")
}
for _, body := range []string{"{", strings.Repeat(" ", (1<<20)+1), `{"playabilityStatus":{"status":"ERROR"}}`} {
if _, err := m.DiscoverResponse([]byte(body)); err == nil || strings.Contains(err.Error(), "secret") {
t.Fatal("invalid metadata accepted or disclosed")
}
}
}
func TestQualificationPlan(t *testing.T) {
m, now := Module{}, time.Unix(1800000000, 0)
if m.Binding().PolicySHA256 != "2cfc521a9784c574ceee16430aa485b9c0e9e64795486a23df5117b6265d89d5" {
t.Fatal("immutable youtube/1 policy changed: introduce a new version")
}
hosts := []string{"rr1---sn-alpha.googlevideo.com", "rr2---sn-beta.googlevideo.com", "rr5---sn-gamma.googlevideo.com"}
var resources []Resource
for _, host := range hosts {
resources = append(resources, Resource{URL: fmt.Sprintf("https://%s/videoplayback?expire=%d&sig=secret", host, now.Unix()+121), Host: host, Kind: "video/mp4"})
}
p, err := m.PlanQualification(hosts, resources, now)
if err != nil || strings.Join(p.Hosts, ",") != strings.Join(hosts, ",") || p.Protocols != [2]string{"tls12", "tls13"} || p.Samples != 3 || p.MinBytes != 1024 || p.WarmupLimit != 10 || p.WarmupTimeout != 120*time.Second {
t.Fatal(p, err)
}
for _, mode := range []string{"expired", "unsigned", "sabr", "missing", "diversity"} {
t.Run(mode, func(t *testing.T) {
r, h := append([]Resource(nil), resources...), append([]string(nil), hosts...)
switch mode {
case "expired":
r[0].URL = strings.Replace(r[0].URL, "1800000121", "1800000120", 1)
case "unsigned":
r[0].URL = strings.Split(r[0].URL, "&sig=")[0]
case "sabr":
r[0].URL = strings.Replace(r[0].URL, "videoplayback", "sabr", 1)
case "missing":
r = r[:2]
case "diversity":
h[1] = h[0]
}
if _, err := m.PlanQualification(h, r, now); err == nil {
t.Fatal("invalid sample accepted")
}
})
}
encoded, err := json.Marshal(struct {
Resources []Resource
Plan Plan
}{resources, p})
if err != nil || strings.Contains(string(encoded), "secret") || strings.Contains(string(encoded), hosts[0]) {
t.Fatal("ephemeral data serialized", string(encoded), err)
}
}
func TestSignedSampleFresh(t *testing.T) {
now := time.Unix(1800000000, 900000000)
for _, tc := range []struct {
query string
want bool
}{
{"expire=1800000121&sig=secret", true},
{"expire=1800000120&sig=secret", false},
{"expire=1800000121&signature=secret", true},
{"expire=1800000121&lsig=secret", true},
{"expire=1800000121", false},
{"expire=invalid&sig=secret", false},
{"expire=9223372036854775808&sig=secret", false},
} {
r := Resource{URL: "https://rr1---sn-alpha.googlevideo.com/videoplayback?" + tc.query, Kind: "video/mp4"}
if SignedSampleFresh(r, now, 120*time.Second) != tc.want {
t.Fatal("signed expiry boundary changed")
}
}
}