feat: extract versioned YouTube service adapter
Wire pure Player parsing and strict qualification planning through a compiled-in kind/version registry. Fail closed before discovery requests when the required module is absent or mismatched. Require sealed module-bound v2 evidence for new writes while preserving fresh trusted v1 apply eligibility and historical rollback. Keep legacy SOCKS5h routing, separate suffix/asset policy and public configuration unchanged; generic controller cutover and field acceptance remain pending. Co-authored-by: GPT-6 <gpt-6@openai.com>
This commit is contained in:
1 parent
d324b4137b
commit
f4853173b3
21 files changed
+1174
-72
No files matched your search
@@ -125,3 +125,37 @@ within a 90-second Docker stop limit (the supplied Compose grants 240 seconds).
|
|||||||
Bounded final reaps/worst-case cleanup were verified offline; measured remote
|
Bounded final reaps/worst-case cleanup were verified offline; measured remote
|
||||||
normal stops were 1.45s, 0.59s and 0.57s, not an 80-second runtime worst-case test.
|
normal stops were 1.45s, 0.59s and 0.57s, not an 80-second runtime worst-case test.
|
||||||
These changes do not claim to fix legacy YouTube exit 78.
|
These changes do not claim to fix legacy YouTube exit 78.
|
||||||
|
|
||||||
|
### Internal service-adapter boundary (local extraction, field acceptance pending)
|
||||||
|
|
||||||
|
Legacy YouTube execution now explicitly selects compiled-in `youtube/1` via
|
||||||
|
`cmd/zapret-autotune/service_adapters.go`. `internal/adapter/youtube` parses
|
||||||
|
bounded Player media JSON and supplies signed-MP4 freshness/diversity semantics
|
||||||
|
and the owned qualification plan. Unknown/absent kind or version fails closed,
|
||||||
|
without a generic HTTP fallback. The shared `internal/adapter` DTOs are internal
|
||||||
|
only: signed URLs/inventory and plan caps have `json:"-"` fields.
|
||||||
|
|
||||||
|
The existing controller still owns discovery requests/retries, transport,
|
||||||
|
origin/SNI/certificate/TLS checks, worker guards/cleanup and sealing after join.
|
||||||
|
Legacy probes retain SOCKS5h and the owned encrypted-resolver guard; they do not
|
||||||
|
run the generic all-A/AAAA approval/connect-to IP pinning path. Full request-policy
|
||||||
|
capabilities (including future metadata POST) and parity are prerequisites before
|
||||||
|
generic cutover, not guarantees supplied by this minimal extraction.
|
||||||
|
Public generic checks and their HMAC/state formats have not migrated to this
|
||||||
|
module interface. Suffix/experimental authorization is separate from measured
|
||||||
|
endpoints, and avatar/thumbnail health remains independent. A module plan is
|
||||||
|
neither proof nor permission to claim READY.
|
||||||
|
|
||||||
|
New adaptive proof v2 seals adapter kind/version/policy digest along with the
|
||||||
|
measured facts. Unknown identities, mismatched policy, missing attestation and
|
||||||
|
unsealed DTOs cannot publish adaptive evidence. Legacy v1 is explicitly the
|
||||||
|
immutable historical YouTube policy for journal reconstruction, not a request
|
||||||
|
to reinterpret old evidence using the latest module. Expired historical proofs
|
||||||
|
can reconstruct their owned config. Fresh trusted stored v1 retains its original
|
||||||
|
apply eligibility; expiry still blocks fresh apply. Only new producer writes
|
||||||
|
require sealed module-bound v2; public/unsealed DTOs cannot publish evidence.
|
||||||
|
Discovery/asset/health entrypoints preflight the required kind/version/policy
|
||||||
|
before requests. Missing/mismatched modules are fatal configuration failures,
|
||||||
|
not retryable inventory absence or a genuine-no-strategy cooldown.
|
||||||
|
No public module field, dynamic plugin ABI, controller cutover, same-UID sandbox,
|
||||||
|
remote YouTube acceptance, or fix for legacy exit 78 is claimed here.
|
||||||
@@ -695,3 +695,107 @@ Root-private proof references (retained reports, not logs copied into this doc):
|
|||||||
readiness, transactional generic apply/profile-journal migration, immutable
|
readiness, transactional generic apply/profile-journal migration, immutable
|
||||||
rotation and noninterrupting search remain outside this producer acceptance;
|
rotation and noninterrupting search remain outside this producer acceptance;
|
||||||
the later bounded apply/replay gates are recorded separately above.
|
the later bounded apply/replay gates are recorded separately above.
|
||||||
|
|
||||||
|
## Internal YouTube adapter slice (local; field acceptance pending)
|
||||||
|
|
||||||
|
`internal/adapter` is a small closed compiled-in contract (`Binding`, ephemeral
|
||||||
|
`Resource`/`Plan`, `Module`); `internal/adapter/youtube` implements `youtube/1`.
|
||||||
|
The explicit kind/version registry in `service_adapters.go` has no init hooks,
|
||||||
|
dynamic loading, public configuration, HTTP fallback or module privileges.
|
||||||
|
Player media parsing (`playerMediaResource`) and owned native qualification
|
||||||
|
(`verifyGooglevideo`) actually call it. Discovery retry/rotation/cancellation and
|
||||||
|
representative selection remain in the legacy controller; signed MP4 freshness
|
||||||
|
and the host/TLS qualification plan are pure service semantics in the module.
|
||||||
|
|
||||||
|
Core origin/TLS validation, shared image/media transfer code, bounded worker
|
||||||
|
lifecycle and post-cleanup/join sealing remain authoritative. Legacy media and
|
||||||
|
metadata routing retains SOCKS5h with the owned encrypted-resolver guard and
|
||||||
|
exact origin/SNI/certificate/redirect checks. It does **not** use public generic
|
||||||
|
`runResourceProtocol`'s all-A/AAAA approval and connect-to IP pinning. That inherited
|
||||||
|
ceiling is unchanged, not a pinning guarantee: full request-policy capabilities
|
||||||
|
(including future metadata POST) and parity are prerequisites to generic cutover.
|
||||||
|
No new DNS/IP routing policy is introduced by this extraction. Both TLS
|
||||||
|
versions must learn separately, then each has three measured MP4 samples per
|
||||||
|
representative host, minimum 1024 bytes (not an exact 64 KiB requirement).
|
||||||
|
Measured endpoints do not authorize a suffix: existing exact/suffix/experimental
|
||||||
|
policy and independent thumbnail/avatar qualification are unchanged. The module
|
||||||
|
cannot manufacture evidence, request wildcard apply or claim YouTube READY.
|
||||||
|
|
||||||
|
New adaptive proof **v2** includes adapter kind/version/immutable policy SHA256
|
||||||
|
inside the producer seal. Public JSON cannot recreate that seal. Missing or
|
||||||
|
mismatched binding/version/digest fails closed; v1 with v2 fields is rejected.
|
||||||
|
Historical proof **v1** explicitly uses the pinned legacy YouTube v1 contract,
|
||||||
|
including historical freshness-independent journal reconstruction. Its config
|
||||||
|
bytes/hash and journal refs remain stable. Fresh stored trusted v1 proofs retain
|
||||||
|
their original apply/qualification eligibility, including unchanged attestation
|
||||||
|
freshness and media gates. Expired v1 can reconstruct but cannot authorize fresh
|
||||||
|
apply. Only **new production/persistence** requires sealed, module-bound v2;
|
||||||
|
neither sealed-new-v1 nor public unsealed v1/v2 DTOs can publish new evidence.
|
||||||
|
Cold v1 evidence, SQL schema 7, native arguments,
|
||||||
|
public GET/HEAD v1/v2, state keys and HMAC formats are unchanged.
|
||||||
|
|
||||||
|
This slice is not full generic-controller migration, native-engine runtime
|
||||||
|
acceptance, a security sandbox (including against same-UID actors), or a fix for
|
||||||
|
unresolved avatars/legacy exit 78. No legacy path is deleted. Local tests and
|
||||||
|
archived baseline parity are recorded separately; no remote validation or commit
|
||||||
|
is part of this extraction review.
|
||||||
|
|
||||||
|
Discovery preflight resolves the required module identity and policy digest
|
||||||
|
before any Player/WWW/avatar/front/media requests, including injected callbacks,
|
||||||
|
legacy proposal discovery and managed discovery hooks. Missing/mismatched modules
|
||||||
|
produce a fixed private sentinel wrapped as a fatal configuration failure, not
|
||||||
|
retryable unavailable inventory, a transfer-negative or genuine no-strategy pause.
|
||||||
|
The compiled production registry remains fixed; synchronized test replacement
|
||||||
|
does not add public runtime module configuration or loading.
|
||||||
|
|
||||||
|
### Local verification for this extraction (2026-10-15)
|
||||||
|
|
||||||
|
- Fresh serial `go test -json -p 1 ./... -parallel 1 -count=1 -timeout 600s`:
|
||||||
|
399 top-level / 1737 including subtest PASS events, zero failures.
|
||||||
|
- Affected serial race run (`-race -p 1 -parallel 1 -count=1`) covers adapters,
|
||||||
|
Player/discovery, owned qualification/proof gates, historical journal parity,
|
||||||
|
independent assets and generic resource core: 61 top-level / 307 including
|
||||||
|
subtest PASS events, zero failures. Full regex and logs are retained at
|
||||||
|
`/tmp/opencode/youtube-adapter-fixes/{full,race}.json`.
|
||||||
|
- Archived `d324b4137b2e95f1c5f4d6c8240470e3f8749f0c` and current builds each
|
||||||
|
passed 275 affected test/subtest events. The portable legacy parser/journal
|
||||||
|
corpus ran **unmodified** on both: file SHA256
|
||||||
|
`7b8eea9c9bbcdd8f0fadde3f78596d41e9d032a0a4f7b2ca90d1223d106584d5`.
|
||||||
|
Both reconstructed historical config SHA256
|
||||||
|
`172c7377388334e4baa2dc1009270478dc271a7b2f00d28cfada631f44a35364`,
|
||||||
|
retaining journal bytes/refs after reconcile and ActiveCandidates validation.
|
||||||
|
An additional unchanged portable v1 corpus passed 2 top-level / 8 including
|
||||||
|
subtest events on **each** build: fresh stored v1 shortlist/apply eligibility,
|
||||||
|
expired fresh-apply refusal, and staged/backup/prepared/replaced/reloaded/healthy
|
||||||
|
crash rollback with expired historical proofs. Backup/config bytes and hashes
|
||||||
|
are stable; pre-prepare journal bytes are unchanged and post-prepare generation/
|
||||||
|
reload/ref transitions follow the original contract. Corpus SHA256:
|
||||||
|
`6df53b12af5d80d164ea3201a614a5fd5723561dc7b2d6bc2d2aa37f845f053d`.
|
||||||
|
New negative tests separately reject even internally sealed NEW v1 writes and
|
||||||
|
public unsealed DTOs; missing/changed module identity/policy makes zero Player,
|
||||||
|
WWW/avatar/media/front/worker calls and does not become a two-hour absence.
|
||||||
|
- All four `python3 deploy/managed-target/check_{static,config,startup_diagnostics,supervision}.py`
|
||||||
|
checks passed serially; supervision was bounded with `timeout 600s`, no retry.
|
||||||
|
`go vet ./...`, `CGO_ENABLED=0 go build`, `sh -n` on all three entrypoint/
|
||||||
|
supervision scripts, and `git diff --check` passed. Build SHA256:
|
||||||
|
`8327faf86098145bb41306b3de895cdd622aeccdcda90f8ec73e19a759c8f98a`.
|
||||||
|
- Ownership-sensitive final runs used `sudo -n` EUID 0 **inside the existing
|
||||||
|
user namespace**, mapping UID 0 to parent/host UID 100000, not initial-host
|
||||||
|
root. The initial extra single-UID `unshare -Ur` run failed ownership fixtures
|
||||||
|
(`chown: invalid argument` / `invalid private qualification data`); a stale
|
||||||
|
test assertion also expected newly produced proof v1 and was updated to v2.
|
||||||
|
Those prior extraction failures are retained in
|
||||||
|
`/tmp/opencode/youtube-adapter-review/go-full.json`, not reported as passes. Vet's
|
||||||
|
initial unkeyed shared-DTO literals were fixed before final successful checks.
|
||||||
|
|
||||||
|
Reproduction commands (run serially; root is only for offline ownership fixtures):
|
||||||
|
|
||||||
|
```sh
|
||||||
|
pattern='Test(YoutubeLegacy|ManagedGooglevideo|ManagedAdaptive|ExperimentalPromotion|Googlevideo|Resource(Core|Discovery|Contract)|Player|Discover|ManagedAsset|Asset)'
|
||||||
|
sudo -n env GOPROXY=off GOTOOLCHAIN=local GOMODCACHE="$(go env GOMODCACHE)" "$(command -v go)" test -json -p 1 ./... -parallel 1 -count=1 -timeout 600s
|
||||||
|
sudo -n env GOPROXY=off GOTOOLCHAIN=local GOMODCACHE="$(go env GOMODCACHE)" "$(command -v go)" test -race -json -p 1 ./internal/adapter/... ./cmd/zapret-autotune -parallel 1 -run "${pattern}|Test(ServiceAdapter|LegacyYoutubeV1|QualificationPlan|SignedSample|DiscoverResponse)" -count=1 -timeout 600s
|
||||||
|
# Baseline/current parity: the same $pattern, without -race, on both trees.
|
||||||
|
# Baseline tree: git archive d324b4137b2e95f1c5f4d6c8240470e3f8749f0c;
|
||||||
|
# add identical youtube_legacy_parity_test.go and youtube_legacy_apply_parity_test.go.
|
||||||
|
# The additional portable v1 corpus runs separately with -run '^TestLegacyYoutubeV1'.
|
||||||
|
```
|
||||||
@@ -9,6 +9,8 @@ import (
|
|||||||
"math"
|
"math"
|
||||||
"strings"
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"zapret-autotune/internal/adapter/youtube"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Additive only: old proposals are not independently qualified candidates.
|
// Additive only: old proposals are not independently qualified candidates.
|
||||||
@@ -351,6 +353,7 @@ type GooglevideoHostProof struct {
|
|||||||
}
|
}
|
||||||
|
|
||||||
type GooglevideoQualificationProof struct {
|
type GooglevideoQualificationProof struct {
|
||||||
|
Adapter *youtube.Binding `json:"adapter,omitempty"`
|
||||||
Version int `json:"version"`
|
Version int `json:"version"`
|
||||||
Mode QualificationMode `json:"mode"`
|
Mode QualificationMode `json:"mode"`
|
||||||
SourceKind string `json:"source_kind"`
|
SourceKind string `json:"source_kind"`
|
||||||
@@ -371,7 +374,25 @@ func (p GooglevideoQualificationProof) validate(c storedCandidate) error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (p GooglevideoQualificationProof) validateStored(c storedCandidate) error {
|
func (p GooglevideoQualificationProof) validateStored(c storedCandidate) error {
|
||||||
if p.Version != 1 || p.Mode != c.Mode || p.Args != c.Args || len(p.Hosts) != len(c.Endpoints) || len(p.Hosts) < 3 {
|
// Missing binding is explicitly the immutable legacy YouTube v1 policy,
|
||||||
|
// never the current registry default. Retain historical reconstruction.
|
||||||
|
if p.Version == 1 {
|
||||||
|
if p.Adapter != nil {
|
||||||
|
return fmt.Errorf("legacy proof cannot carry adapter binding")
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
if p.Mode != QualificationAdaptive || p.Adapter == nil {
|
||||||
|
return fmt.Errorf("version-bound adaptive proof required")
|
||||||
|
}
|
||||||
|
module, err := lookupServiceAdapter(p.Adapter.Kind, p.Adapter.Version)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if *p.Adapter != module.Binding() {
|
||||||
|
return fmt.Errorf("adapter policy mismatch")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (p.Version != 1 && p.Version != 2) || p.Mode != c.Mode || p.Args != c.Args || len(p.Hosts) != len(c.Endpoints) || len(p.Hosts) < 3 {
|
||||||
return fmt.Errorf("invalid representative proof definition")
|
return fmt.Errorf("invalid representative proof definition")
|
||||||
}
|
}
|
||||||
if (p.Mode == QualificationCold && p.SourceKind != "media-range") || (p.Mode == QualificationAdaptive && p.SourceKind != NativeCircularPostWarmup) || (p.Mode != QualificationCold && p.Mode != QualificationAdaptive) {
|
if (p.Mode == QualificationCold && p.SourceKind != "media-range") || (p.Mode == QualificationAdaptive && p.SourceKind != NativeCircularPostWarmup) || (p.Mode != QualificationCold && p.Mode != QualificationAdaptive) {
|
||||||
@@ -380,7 +401,7 @@ func (p GooglevideoQualificationProof) validateStored(c storedCandidate) error {
|
|||||||
if err := (&candidate{Args: p.Args}).validateArgs(); err != nil {
|
if err := (&candidate{Args: p.Args}).validateArgs(); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
if err := validateGooglevideoHosts(c.Endpoints); err != nil {
|
if err := validateLegacyYoutubeV1Hosts(c.Endpoints); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
if p.Mode == QualificationCold && p.Attestation != nil {
|
if p.Mode == QualificationCold && p.Attestation != nil {
|
||||||
@@ -494,7 +515,9 @@ func validateAdaptiveGooglevideoEvidence(db *sql.DB, c storedCandidate, fresh bo
|
|||||||
// One transactional insert, no cold candidate_evidence summaries. A private
|
// One transactional insert, no cold candidate_evidence summaries. A private
|
||||||
// producer seal is required as well as valid DTO facts and stored definition.
|
// producer seal is required as well as valid DTO facts and stored definition.
|
||||||
func saveAdaptiveGooglevideoEvidence(tx *sql.Tx, id int64, proof GooglevideoQualificationProof) error {
|
func saveAdaptiveGooglevideoEvidence(tx *sql.Tx, id int64, proof GooglevideoQualificationProof) error {
|
||||||
if proof.producerSeal == nil || *proof.producerSeal != googlevideoProofSeal(proof) {
|
// Existing trusted v1 rows retain their original read/apply contract. Only
|
||||||
|
// this new-write boundary requires the version-bound producer protocol.
|
||||||
|
if proof.Version != 2 || proof.producerSeal == nil || *proof.producerSeal != googlevideoProofSeal(proof) {
|
||||||
return fmt.Errorf("completed owned collector attestation required")
|
return fmt.Errorf("completed owned collector attestation required")
|
||||||
}
|
}
|
||||||
var c storedCandidate
|
var c storedCandidate
|
||||||
|
|||||||
@@ -148,6 +148,9 @@ func discoverYoutube(args []string, stdout, stderr io.Writer, runner playerRunne
|
|||||||
}
|
}
|
||||||
|
|
||||||
func requestPlayer(videoID, proxy string, runner playerRunner) ([]byte, error) {
|
func requestPlayer(videoID, proxy string, runner playerRunner) ([]byte, error) {
|
||||||
|
if _, err := requiredYoutubeAdapter(); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
payload, _ := json.Marshal(map[string]interface{}{
|
payload, _ := json.Marshal(map[string]interface{}{
|
||||||
"videoId": videoID,
|
"videoId": videoID,
|
||||||
"context": map[string]interface{}{"client": map[string]interface{}{
|
"context": map[string]interface{}{"client": map[string]interface{}{
|
||||||
|
|||||||
@@ -44,6 +44,10 @@ func googlevideoProposal(flags []string, stdout, stderr io.Writer, discover play
|
|||||||
fmt.Fprintln(stderr, "error: tune googlevideo", err)
|
fmt.Fprintln(stderr, "error: tune googlevideo", err)
|
||||||
return 2
|
return 2
|
||||||
}
|
}
|
||||||
|
if _, err := requiredYoutubeAdapter(); err != nil {
|
||||||
|
fmt.Fprintln(stderr, "error: tune googlevideo", err)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
started := time.Now()
|
started := time.Now()
|
||||||
db, err := openStore(databasePath())
|
db, err := openStore(databasePath())
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
@@ -71,6 +71,9 @@ type managedServiceInventory struct {
|
|||||||
func managedServiceResources(cfg healthConfig, player func(context.Context) ([]byte, error), page playerRunner) func(context.Context) (managedServiceInventory, error) {
|
func managedServiceResources(cfg healthConfig, player func(context.Context) ([]byte, error), page playerRunner) func(context.Context) (managedServiceInventory, error) {
|
||||||
return func(ctx context.Context) (managedServiceInventory, error) {
|
return func(ctx context.Context) (managedServiceInventory, error) {
|
||||||
i := managedServiceInventory{images: make(map[string]string), proxy: cfg.SOCKSProxy, runner: runResourceCurl}
|
i := managedServiceInventory{images: make(map[string]string), proxy: cfg.SOCKSProxy, runner: runResourceCurl}
|
||||||
|
if _, err := requiredYoutubeAdapter(); err != nil {
|
||||||
|
return i, err
|
||||||
|
}
|
||||||
if err := ctx.Err(); err != nil {
|
if err := ctx.Err(); err != nil {
|
||||||
return i, managedRecoveryContextError(ctx, err)
|
return i, managedRecoveryContextError(ctx, err)
|
||||||
}
|
}
|
||||||
@@ -105,6 +108,9 @@ func managedServiceResources(cfg healthConfig, player func(context.Context) ([]b
|
|||||||
}
|
}
|
||||||
i.media, _, err = playerMediaResource(body)
|
i.media, _, err = playerMediaResource(body)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
if errors.Is(err, errServiceAdapterUnavailable) {
|
||||||
|
return i, err
|
||||||
|
}
|
||||||
i.resourceErr = &managedResourceFailure{"media", "unknown", err}
|
i.resourceErr = &managedResourceFailure{"media", "unknown", err}
|
||||||
}
|
}
|
||||||
for attempt := 0; attempt < 2; attempt++ {
|
for attempt := 0; attempt < 2; attempt++ {
|
||||||
@@ -152,6 +158,9 @@ func managedServiceResources(cfg healthConfig, player func(context.Context) ([]b
|
|||||||
|
|
||||||
func managedServiceHealth(proxy string, front func(context.Context) (string, error), inventory func(context.Context) (managedServiceInventory, error), runner subprocessRunner) func(context.Context) (string, error) {
|
func managedServiceHealth(proxy string, front func(context.Context) (string, error), inventory func(context.Context) (managedServiceInventory, error), runner subprocessRunner) func(context.Context) (string, error) {
|
||||||
return func(ctx context.Context) (string, error) {
|
return func(ctx context.Context) (string, error) {
|
||||||
|
if _, err := requiredYoutubeAdapter(); err != nil {
|
||||||
|
return "DEGRADED", err
|
||||||
|
}
|
||||||
status, err := front(ctx)
|
status, err := front(ctx)
|
||||||
if err != nil || status != "OK" {
|
if err != nil || status != "OK" {
|
||||||
return "DEGRADED", &managedResourceFailure{"front", managedFrontendHost, err}
|
return "DEGRADED", &managedResourceFailure{"front", managedFrontendHost, err}
|
||||||
|
|||||||
@@ -597,6 +597,9 @@ func managedPrestartReconcile(m *managedTransaction, d managedDeps) (err error)
|
|||||||
var errManagedScopeUnresolved = errors.New("managed deployment policy unresolved; explicit reconcile or fresh managed state required")
|
var errManagedScopeUnresolved = errors.New("managed deployment policy unresolved; explicit reconcile or fresh managed state required")
|
||||||
|
|
||||||
func managedActivate(m *managedTransaction, d managedDeps, ids []int64, expected map[int64]string, scopes ...GooglevideoScope) error {
|
func managedActivate(m *managedTransaction, d managedDeps, ids []int64, expected map[int64]string, scopes ...GooglevideoScope) error {
|
||||||
|
if _, err := requiredYoutubeAdapter(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
info, err := os.Stat(d.script)
|
info, err := os.Stat(d.script)
|
||||||
if err != nil || !info.Mode().IsRegular() || info.Mode().Perm()&0111 == 0 {
|
if err != nil || !info.Mode().IsRegular() || info.Mode().Perm()&0111 == 0 {
|
||||||
return fmt.Errorf("target zapret daemon init executable unavailable")
|
return fmt.Errorf("target zapret daemon init executable unavailable")
|
||||||
@@ -606,6 +609,9 @@ func managedActivate(m *managedTransaction, d managedDeps, ids []int64, expected
|
|||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
m.Health = func() (string, error) {
|
m.Health = func() (string, error) {
|
||||||
|
if _, err := requiredYoutubeAdapter(); err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
portText, _ := d.lookup("SOCKS_PORT")
|
portText, _ := d.lookup("SOCKS_PORT")
|
||||||
port, err := strconv.Atoi(portText)
|
port, err := strconv.Atoi(portText)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
@@ -86,6 +86,9 @@ func managedExperimentalRequest(dir string, consume bool, apply func() error) er
|
|||||||
// Runs before the PID-1-owned monitor starts; never races a recovery worker.
|
// Runs before the PID-1-owned monitor starts; never races a recovery worker.
|
||||||
// A real Player-discovered identity is saved WITHOUT cold/adaptive evidence.
|
// A real Player-discovered identity is saved WITHOUT cold/adaptive evidence.
|
||||||
func managedExperimentalSuffix(m *managedTransaction, d managedDeps) error {
|
func managedExperimentalSuffix(m *managedTransaction, d managedDeps) error {
|
||||||
|
if _, err := requiredYoutubeAdapter(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
singleton, err := acquireManagedMonitorSingleton(d)
|
singleton, err := acquireManagedMonitorSingleton(d)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
|
|||||||
@@ -179,7 +179,7 @@ func TestExperimentalPromotionRequiresRepresentativeOwnedProof(t *testing.T) {
|
|||||||
}
|
}
|
||||||
assertActiveCandidates(t, d.transaction, d.db, []int64{front, thumb, avatar, id})
|
assertActiveCandidates(t, d.transaction, d.db, []int64{front, thumb, avatar, id})
|
||||||
p, err := readGooglevideoProof(d.db, id)
|
p, err := readGooglevideoProof(d.db, id)
|
||||||
if err != nil || p.Version != 1 || len(p.Hosts) != 3 {
|
if err != nil || p.Version != 2 || p.Adapter == nil || len(p.Hosts) != 3 {
|
||||||
t.Fatal(p, err)
|
t.Fatal(p, err)
|
||||||
}
|
}
|
||||||
c, err := readStrategyCandidate(d.db, experiment)
|
c, err := readStrategyCandidate(d.db, experiment)
|
||||||
|
|||||||
@@ -5,9 +5,10 @@ import (
|
|||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"sort"
|
"sort"
|
||||||
"strconv"
|
|
||||||
"strings"
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"zapret-autotune/internal/adapter/youtube"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Ordinary unresolved state, not a target/ownership guard failure.
|
// Ordinary unresolved state, not a target/ownership guard failure.
|
||||||
@@ -28,7 +29,7 @@ const managedGooglevideoDiscoveryTimeout = 3 * time.Minute
|
|||||||
|
|
||||||
// Three hosts each need up to 120s warmup + six 35s transfers. Leave margin
|
// Three hosts each need up to 120s warmup + six 35s transfers. Leave margin
|
||||||
// for daemon lifecycle as well; short-lived URLs are refetched, never rewritten.
|
// for daemon lifecycle as well; short-lived URLs are refetched, never rewritten.
|
||||||
const managedGooglevideoSampleTTL = 20 * time.Minute
|
const managedGooglevideoSampleTTL = youtube.InventoryTTL
|
||||||
|
|
||||||
func freshGooglevideoSample(r mediaResource) bool {
|
func freshGooglevideoSample(r mediaResource) bool {
|
||||||
// The same exact GGC hostname syntax as the representative gate; reject bad
|
// The same exact GGC hostname syntax as the representative gate; reject bad
|
||||||
@@ -37,13 +38,10 @@ func freshGooglevideoSample(r mediaResource) bool {
|
|||||||
if !ok || !strings.HasSuffix(r.host, ".googlevideo.com") || !validDomain(r.host) || r.host != strings.ToLower(r.host) || root == "" || len(prefix) < 3 || !strings.HasPrefix(prefix, "rr") || strings.Trim(prefix[2:], "0123456789") != "" {
|
if !ok || !strings.HasSuffix(r.host, ".googlevideo.com") || !validDomain(r.host) || r.host != strings.ToLower(r.host) || root == "" || len(prefix) < 3 || !strings.HasPrefix(prefix, "rr") || strings.Trim(prefix[2:], "0123456789") != "" {
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
u, err := privateResourceURL(r.url, r.host)
|
if _, err := privateResourceURL(r.url, r.host); err != nil {
|
||||||
if err != nil || u.Path != "/videoplayback" || (r.kind != "video/mp4" && r.kind != "audio/mp4") {
|
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
q := u.Query()
|
return youtube.SignedSampleFresh(youtube.Resource{URL: r.url, Host: r.host, Kind: r.kind}, time.Now(), managedGooglevideoSampleTTL)
|
||||||
expires, err := strconv.ParseInt(q.Get("expire"), 10, 64)
|
|
||||||
return err == nil && expires > time.Now().Add(managedGooglevideoSampleTTL).Unix() && (q.Get("sig") != "" || q.Get("signature") != "" || q.Get("lsig") != "")
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// Choose exactly three actual hosts passing the unchanged representative gate.
|
// Choose exactly three actual hosts passing the unchanged representative gate.
|
||||||
@@ -70,6 +68,9 @@ func representativeGooglevideoSamples(byHost map[string]mediaResource) []mediaRe
|
|||||||
func managedGooglevideoSamples(ids []string, request func(context.Context, string) ([]byte, error)) func(context.Context) ([]mediaResource, error) {
|
func managedGooglevideoSamples(ids []string, request func(context.Context, string) ([]byte, error)) func(context.Context) ([]mediaResource, error) {
|
||||||
ids = append([]string(nil), ids...)
|
ids = append([]string(nil), ids...)
|
||||||
return func(ctx context.Context) ([]mediaResource, error) {
|
return func(ctx context.Context) ([]mediaResource, error) {
|
||||||
|
if _, err := requiredYoutubeAdapter(); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
if len(ids) == 0 || len(ids) > 5 || request == nil {
|
if len(ids) == 0 || len(ids) > 5 || request == nil {
|
||||||
return nil, &managedRecoveryFatal{Err: errors.New("invalid sample collector configuration")}
|
return nil, &managedRecoveryFatal{Err: errors.New("invalid sample collector configuration")}
|
||||||
}
|
}
|
||||||
@@ -108,6 +109,9 @@ func managedGooglevideoSamples(ids []string, request func(context.Context, strin
|
|||||||
}
|
}
|
||||||
r, _, err := playerMediaResource(body)
|
r, _, err := playerMediaResource(body)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
if errors.Is(err, errServiceAdapterUnavailable) {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
stats.Unavailable++
|
stats.Unavailable++
|
||||||
continue
|
continue
|
||||||
} // Missing/SABR is unknown, never fabricated.
|
} // Missing/SABR is unknown, never fabricated.
|
||||||
|
|||||||
@@ -9,9 +9,10 @@ import (
|
|||||||
"os"
|
"os"
|
||||||
"os/exec"
|
"os/exec"
|
||||||
"reflect"
|
"reflect"
|
||||||
"strconv"
|
|
||||||
"strings"
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"zapret-autotune/internal/adapter/youtube"
|
||||||
)
|
)
|
||||||
|
|
||||||
// ponytail: one audited composite only. New native variants need an explicit
|
// ponytail: one audited composite only. New native variants need an explicit
|
||||||
@@ -93,45 +94,47 @@ func newManagedGooglevideoVerifier(target managedDeps) func(context.Context, sto
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (r managedAssetRuntime) verifyGooglevideo(ctx context.Context, c storedCandidate, resources []mediaResource) (GooglevideoQualificationProof, error) {
|
func (r managedAssetRuntime) verifyGooglevideo(ctx context.Context, c storedCandidate, resources []mediaResource) (GooglevideoQualificationProof, error) {
|
||||||
p := GooglevideoQualificationProof{Version: 1, Mode: QualificationAdaptive, SourceKind: NativeCircularPostWarmup, Args: c.Args}
|
p := GooglevideoQualificationProof{Version: 2, Mode: QualificationAdaptive, SourceKind: NativeCircularPostWarmup, Args: c.Args}
|
||||||
if c.Group != "googlevideo" || c.Mode != QualificationAdaptive {
|
if c.Group != "googlevideo" || c.Mode != QualificationAdaptive {
|
||||||
return p, errors.New("adaptive googlevideo definition required")
|
return p, errors.New("adaptive googlevideo definition required")
|
||||||
}
|
}
|
||||||
if err := validateNativeGooglevideoArgs(c.Args); err != nil {
|
if err := validateNativeGooglevideoArgs(c.Args); err != nil {
|
||||||
return p, err
|
return p, err
|
||||||
}
|
}
|
||||||
byHost := map[string]mediaResource{}
|
module, err := requiredYoutubeAdapter()
|
||||||
for _, resource := range resources {
|
if err != nil {
|
||||||
u, err := privateResourceURL(resource.url, resource.host)
|
return p, err
|
||||||
if err != nil || u.Path != "/videoplayback" || (resource.kind != "video/mp4" && resource.kind != "audio/mp4") {
|
|
||||||
return p, errGooglevideoInsufficientSamples
|
|
||||||
}
|
}
|
||||||
q := u.Query()
|
binding := module.Binding()
|
||||||
expires, err := strconv.ParseInt(q.Get("expire"), 10, 64)
|
p.Adapter = &binding
|
||||||
if err != nil || expires <= time.Now().Unix()+120 || (q.Get("sig") == "" && q.Get("signature") == "" && q.Get("lsig") == "") {
|
byHost := map[string]mediaResource{}
|
||||||
|
var samples []youtube.Resource
|
||||||
|
for _, resource := range resources {
|
||||||
|
if _, err := privateResourceURL(resource.url, resource.host); err != nil {
|
||||||
return p, errGooglevideoInsufficientSamples
|
return p, errGooglevideoInsufficientSamples
|
||||||
}
|
}
|
||||||
byHost[resource.host] = resource
|
byHost[resource.host] = resource
|
||||||
|
samples = append(samples, youtube.Resource{URL: resource.url, Host: resource.host, Kind: resource.kind})
|
||||||
}
|
}
|
||||||
// Validate representative host diversity using the shared gate before capture.
|
if validateGooglevideoHosts(c.Endpoints) != nil {
|
||||||
for _, host := range c.Endpoints {
|
|
||||||
if _, ok := byHost[host]; !ok {
|
|
||||||
return p, errGooglevideoInsufficientSamples
|
return p, errGooglevideoInsufficientSamples
|
||||||
}
|
}
|
||||||
|
plan, err := module.PlanQualification(c.Endpoints, samples, time.Now())
|
||||||
|
if err != nil {
|
||||||
|
return p, errGooglevideoInsufficientSamples
|
||||||
|
}
|
||||||
|
for _, host := range plan.Hosts {
|
||||||
p.Hosts = append(p.Hosts, GooglevideoHostProof{Host: host})
|
p.Hosts = append(p.Hosts, GooglevideoHostProof{Host: host})
|
||||||
}
|
}
|
||||||
if err := validateGooglevideoHosts(c.Endpoints); err != nil {
|
err = r.withOwnedWorker(ctx, c.Endpoints, c.Args, func(probeCtx context.Context) error {
|
||||||
return p, errGooglevideoInsufficientSamples
|
|
||||||
}
|
|
||||||
err := r.withOwnedWorker(ctx, c.Endpoints, c.Args, func(probeCtx context.Context) error {
|
|
||||||
for i := range p.Hosts {
|
for i := range p.Hosts {
|
||||||
h := &p.Hosts[i]
|
h := &p.Hosts[i]
|
||||||
h.WarmupLimit = 10
|
h.WarmupLimit = plan.WarmupLimit
|
||||||
start := time.Now()
|
start := time.Now()
|
||||||
warmCtx, cancel := context.WithTimeout(probeCtx, 120*time.Second)
|
warmCtx, cancel := context.WithTimeout(probeCtx, plan.WarmupTimeout)
|
||||||
learned := map[string]bool{}
|
learned := map[string]bool{}
|
||||||
for h.WarmupAttempts < h.WarmupLimit && len(learned) < 2 && warmCtx.Err() == nil {
|
for h.WarmupAttempts < h.WarmupLimit && len(learned) < len(plan.Protocols) && warmCtx.Err() == nil {
|
||||||
protocol := []string{"tls12", "tls13"}[h.WarmupAttempts%2]
|
protocol := plan.Protocols[h.WarmupAttempts%len(plan.Protocols)]
|
||||||
h.WarmupAttempts++
|
h.WarmupAttempts++
|
||||||
_, pass, err := collectMediaTransfer(warmCtx, r.proxy, byHost[h.Host], protocol, r.curl)
|
_, pass, err := collectMediaTransfer(warmCtx, r.proxy, byHost[h.Host], protocol, r.curl)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -140,7 +143,7 @@ func (r managedAssetRuntime) verifyGooglevideo(ctx context.Context, c storedCand
|
|||||||
cancel()
|
cancel()
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
} else if pass.Bytes < 1024 {
|
} else if pass.Bytes < plan.MinBytes {
|
||||||
h.WarmupFailures++
|
h.WarmupFailures++
|
||||||
} else {
|
} else {
|
||||||
learned[protocol] = true
|
learned[protocol] = true
|
||||||
@@ -151,11 +154,11 @@ func (r managedAssetRuntime) verifyGooglevideo(ctx context.Context, c storedCand
|
|||||||
if probeCtx.Err() != nil {
|
if probeCtx.Err() != nil {
|
||||||
return probeCtx.Err()
|
return probeCtx.Err()
|
||||||
}
|
}
|
||||||
if len(learned) != 2 || h.WarmupMS > 120000 {
|
if len(learned) != len(plan.Protocols) || h.WarmupMS > float64(plan.WarmupTimeout/time.Millisecond) {
|
||||||
return errCandidateNonpass
|
return errCandidateNonpass
|
||||||
}
|
}
|
||||||
for _, protocol := range []string{"tls12", "tls13"} {
|
for _, protocol := range plan.Protocols {
|
||||||
for n := 0; n < 3; n++ {
|
for n := 0; n < plan.Samples; n++ {
|
||||||
_, pass, err := collectMediaTransfer(probeCtx, r.proxy, byHost[h.Host], protocol, r.curl)
|
_, pass, err := collectMediaTransfer(probeCtx, r.proxy, byHost[h.Host], protocol, r.curl)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
if managedGooglevideoTransferNonpass(err) {
|
if managedGooglevideoTransferNonpass(err) {
|
||||||
@@ -163,7 +166,7 @@ func (r managedAssetRuntime) verifyGooglevideo(ctx context.Context, c storedCand
|
|||||||
}
|
}
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
if pass.Bytes < 1024 {
|
if pass.Bytes < plan.MinBytes {
|
||||||
return errCandidateNonpass
|
return errCandidateNonpass
|
||||||
}
|
}
|
||||||
h.Passes = append(h.Passes, pass)
|
h.Passes = append(h.Passes, pass)
|
||||||
@@ -239,16 +242,28 @@ func managedResourceTransferNonpass(err error, metadata, image bool) bool {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func validateGooglevideoHosts(hosts []string) error {
|
func validateGooglevideoHosts(hosts []string) error {
|
||||||
seen, roots, prefixes := map[string]bool{}, map[string]bool{}, map[string]bool{}
|
|
||||||
for _, host := range hosts {
|
for _, host := range hosts {
|
||||||
prefix, root, ok := strings.Cut(strings.TrimSuffix(host, ".googlevideo.com"), "---sn-")
|
if !validDomain(host) {
|
||||||
if !ok || !strings.HasSuffix(host, ".googlevideo.com") || !validDomain(host) || host != strings.ToLower(host) || seen[host] || root == "" || len(prefix) < 3 || !strings.HasPrefix(prefix, "rr") || strings.Trim(prefix[2:], "0123456789") != "" {
|
|
||||||
return errGooglevideoInsufficientSamples
|
return errGooglevideoInsufficientSamples
|
||||||
}
|
}
|
||||||
seen[host], roots[strings.Split(root, "-")[0]], prefixes[prefix] = true, true, true
|
|
||||||
}
|
}
|
||||||
if len(hosts) < 3 || len(roots) < 2 || len(prefixes) < 2 {
|
module, err := requiredYoutubeAdapter()
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if module.ValidateHosts(hosts) != nil {
|
||||||
return errGooglevideoInsufficientSamples
|
return errGooglevideoInsufficientSamples
|
||||||
}
|
}
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Historical v1 reconstruction must not depend on a future registry default.
|
||||||
|
// This pinned contract remains available even when the execution module is absent.
|
||||||
|
func validateLegacyYoutubeV1Hosts(hosts []string) error {
|
||||||
|
for _, host := range hosts {
|
||||||
|
if !validDomain(host) {
|
||||||
|
return errGooglevideoInsufficientSamples
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return (youtube.Module{}).ValidateHosts(hosts)
|
||||||
|
}
|
||||||
@@ -47,6 +47,9 @@ type managedRecoveryDeps struct {
|
|||||||
// The injected command runner receives only fixed script/argv from orchestration.
|
// The injected command runner receives only fixed script/argv from orchestration.
|
||||||
func managedRecoveryDependencies(target managedDeps, db *sql.DB, scan cacheScan, check cacheCheck, run func(context.Context, string, ...string) error) (managedRecoveryDeps, error) {
|
func managedRecoveryDependencies(target managedDeps, db *sql.DB, scan cacheScan, check cacheCheck, run func(context.Context, string, ...string) error) (managedRecoveryDeps, error) {
|
||||||
d := managedRecoveryDeps{db: db, scan: scan, check: check, run: run}
|
d := managedRecoveryDeps{db: db, scan: scan, check: check, run: run}
|
||||||
|
if _, err := requiredYoutubeAdapter(); err != nil {
|
||||||
|
return d, err
|
||||||
|
}
|
||||||
if target.lookup == nil || target.player == nil || db == nil || scan == nil || check == nil || run == nil {
|
if target.lookup == nil || target.player == nil || db == nil || scan == nil || check == nil || run == nil {
|
||||||
return d, fmt.Errorf("complete target-local recovery adapter dependencies required")
|
return d, fmt.Errorf("complete target-local recovery adapter dependencies required")
|
||||||
}
|
}
|
||||||
@@ -59,6 +62,9 @@ func managedRecoveryDependencies(target managedDeps, db *sql.DB, scan cacheScan,
|
|||||||
}
|
}
|
||||||
d.transaction = &managedTransaction{Dir: target.directory, Name: "config"}
|
d.transaction = &managedTransaction{Dir: target.directory, Name: "config"}
|
||||||
requestVideo := func(ctx context.Context, videoID string) ([]byte, error) {
|
requestVideo := func(ctx context.Context, videoID string) ([]byte, error) {
|
||||||
|
if _, err := requiredYoutubeAdapter(); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
if err := ctx.Err(); err != nil {
|
if err := ctx.Err(); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -190,39 +190,17 @@ type mediaCapability struct {
|
|||||||
|
|
||||||
// SABR is deliberately unknown/not supported, never fabricated as readiness.
|
// SABR is deliberately unknown/not supported, never fabricated as readiness.
|
||||||
func playerMediaResource(body []byte) (mediaResource, mediaCapability, error) {
|
func playerMediaResource(body []byte) (mediaResource, mediaCapability, error) {
|
||||||
var p struct {
|
module, err := requiredYoutubeAdapter()
|
||||||
PlayabilityStatus struct {
|
|
||||||
Status string `json:"status"`
|
|
||||||
} `json:"playabilityStatus"`
|
|
||||||
StreamingData struct {
|
|
||||||
Formats []struct {
|
|
||||||
URL string `json:"url"`
|
|
||||||
MimeType string `json:"mimeType"`
|
|
||||||
} `json:"formats"`
|
|
||||||
AdaptiveFormats []struct {
|
|
||||||
URL string `json:"url"`
|
|
||||||
MimeType string `json:"mimeType"`
|
|
||||||
} `json:"adaptiveFormats"`
|
|
||||||
} `json:"streamingData"`
|
|
||||||
}
|
|
||||||
if len(body) > playerResponseLimit || json.Unmarshal(body, &p) != nil || p.PlayabilityStatus.Status != "OK" {
|
|
||||||
return mediaResource{}, mediaCapability{}, errors.New("invalid playable metadata")
|
|
||||||
}
|
|
||||||
for _, f := range append(p.StreamingData.Formats, p.StreamingData.AdaptiveFormats...) {
|
|
||||||
u, err := url.Parse(f.URL)
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
continue
|
return mediaResource{}, mediaCapability{}, err
|
||||||
}
|
}
|
||||||
host := u.Hostname()
|
resources, err := module.DiscoverResponse(body)
|
||||||
if !strings.HasSuffix(host, ".googlevideo.com") {
|
if err != nil {
|
||||||
continue
|
return mediaResource{}, mediaCapability{}, err
|
||||||
}
|
}
|
||||||
if _, err := privateResourceURL(f.URL, host); err != nil {
|
for _, r := range resources {
|
||||||
continue
|
if _, err := privateResourceURL(r.URL, r.Host); err == nil {
|
||||||
}
|
return mediaResource{r.URL, r.Host, r.Kind}, mediaCapability{OrdinaryURL: true}, nil
|
||||||
kind, _, err := mime.ParseMediaType(f.MimeType)
|
|
||||||
if err == nil && (kind == "video/mp4" || kind == "audio/mp4") {
|
|
||||||
return mediaResource{f.URL, host, kind}, mediaCapability{OrdinaryURL: true}, nil
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return mediaResource{}, mediaCapability{}, nil
|
return mediaResource{}, mediaCapability{}, nil
|
||||||
|
|||||||
@@ -0,0 +1,54 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"errors"
|
||||||
|
"sync"
|
||||||
|
|
||||||
|
"zapret-autotune/internal/adapter"
|
||||||
|
"zapret-autotune/internal/adapter/youtube"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Trusted compiled-in dispatch, deliberately not a public config/plugin API.
|
||||||
|
// An empty registry is a supported fail-closed seam; no HTTP fallback exists.
|
||||||
|
type serviceAdapterKey struct {
|
||||||
|
kind string
|
||||||
|
version int
|
||||||
|
}
|
||||||
|
type serviceAdapterRegistry map[serviceAdapterKey]adapter.Module
|
||||||
|
|
||||||
|
var errServiceAdapterUnavailable = errors.New("required service adapter unavailable")
|
||||||
|
|
||||||
|
// The registry is immutable in production. The lock permits race-free test
|
||||||
|
// replacement while request callbacks/owned lifecycle children are running.
|
||||||
|
var serviceAdapters = struct {
|
||||||
|
sync.RWMutex
|
||||||
|
modules serviceAdapterRegistry
|
||||||
|
}{modules: serviceAdapterRegistry{{youtube.Kind, youtube.Version}: youtube.Module{}}}
|
||||||
|
|
||||||
|
func requiredYoutubeAdapter() (adapter.Module, error) {
|
||||||
|
serviceAdapters.RLock()
|
||||||
|
m, err := serviceAdapters.modules.lookup(youtube.Kind, youtube.Version)
|
||||||
|
serviceAdapters.RUnlock()
|
||||||
|
if err != nil || m.Binding() != (youtube.Module{}).Binding() {
|
||||||
|
return nil, &managedRecoveryFatal{Err: errServiceAdapterUnavailable}
|
||||||
|
}
|
||||||
|
return m, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (r serviceAdapterRegistry) lookup(kind string, version int) (adapter.Module, error) {
|
||||||
|
m, ok := r[serviceAdapterKey{kind, version}]
|
||||||
|
if !ok || m == nil {
|
||||||
|
return nil, errServiceAdapterUnavailable
|
||||||
|
}
|
||||||
|
b := m.Binding()
|
||||||
|
if b.Kind != kind || b.Version != version {
|
||||||
|
return nil, errServiceAdapterUnavailable
|
||||||
|
}
|
||||||
|
return m, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func lookupServiceAdapter(kind string, version int) (adapter.Module, error) {
|
||||||
|
serviceAdapters.RLock()
|
||||||
|
defer serviceAdapters.RUnlock()
|
||||||
|
return serviceAdapters.modules.lookup(kind, version)
|
||||||
|
}
|
||||||
@@ -0,0 +1,284 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"io"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"zapret-autotune/internal/adapter"
|
||||||
|
"zapret-autotune/internal/adapter/youtube"
|
||||||
|
)
|
||||||
|
|
||||||
|
func replaceServiceAdapters(t *testing.T, modules serviceAdapterRegistry) {
|
||||||
|
t.Helper()
|
||||||
|
serviceAdapters.Lock()
|
||||||
|
previous := serviceAdapters.modules
|
||||||
|
serviceAdapters.modules = modules
|
||||||
|
serviceAdapters.Unlock()
|
||||||
|
t.Cleanup(func() {
|
||||||
|
serviceAdapters.Lock()
|
||||||
|
serviceAdapters.modules = previous
|
||||||
|
serviceAdapters.Unlock()
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestServiceAdapterDispatchFailClosed(t *testing.T) {
|
||||||
|
for _, key := range []serviceAdapterKey{{"unknown", 1}, {youtube.Kind, 0}, {youtube.Kind, 2}} {
|
||||||
|
if _, err := lookupServiceAdapter(key.kind, key.version); err == nil {
|
||||||
|
t.Fatal("unsupported adapter accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if _, err := (serviceAdapterRegistry{}).lookup(youtube.Kind, youtube.Version); err == nil {
|
||||||
|
t.Fatal("absent adapter accepted")
|
||||||
|
}
|
||||||
|
replaceServiceAdapters(t, serviceAdapterRegistry{})
|
||||||
|
if _, _, err := playerMediaResource([]byte(`{"playabilityStatus":{"status":"OK"}}`)); err == nil {
|
||||||
|
t.Fatal("discovery bypassed registry")
|
||||||
|
}
|
||||||
|
r, c, resources, events := googlevideoRuntimeFixture(t)
|
||||||
|
if _, err := r.verifyGooglevideo(context.Background(), c, resources); err == nil || len(*events) != 0 {
|
||||||
|
t.Fatal("qualification bypassed registry")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type mismatchedServiceAdapter struct{ adapter.Module }
|
||||||
|
|
||||||
|
func (m mismatchedServiceAdapter) Binding() adapter.Binding {
|
||||||
|
b := m.Module.Binding()
|
||||||
|
b.Version++
|
||||||
|
return b
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestServiceAdapterIdentityBeforeExecution(t *testing.T) {
|
||||||
|
replaceServiceAdapters(t, serviceAdapterRegistry{{youtube.Kind, youtube.Version}: mismatchedServiceAdapter{youtube.Module{}}})
|
||||||
|
r, c, resources, events := googlevideoRuntimeFixture(t)
|
||||||
|
if _, err := r.verifyGooglevideo(context.Background(), c, resources); err == nil || len(*events) != 0 {
|
||||||
|
t.Fatal("identity mismatch reached worker")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestServiceAdapterStoredProofRefusals(t *testing.T) {
|
||||||
|
r, c, resources, _ := googlevideoRuntimeFixture(t)
|
||||||
|
p, err := r.verifyGooglevideo(context.Background(), c, resources)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
db := testStore(t)
|
||||||
|
tx, err := db.Begin()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer tx.Rollback()
|
||||||
|
id, err := insertStrategyCandidate(tx, c.Group, c.Endpoints, c.Args, nil, c.Mode)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := saveAdaptiveGooglevideoEvidence(tx, id, p); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := tx.Commit(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
front := managedCandidateFixture(t, db, "frontend", "www.youtube.com", "--lua-desync=multisplit", true)
|
||||||
|
for _, mode := range []string{"kind", "version", "digest", "missing", "unknown-proof", "downgrade", "attestation"} {
|
||||||
|
t.Run(mode, func(t *testing.T) {
|
||||||
|
raw, _ := json.Marshal(p)
|
||||||
|
var proof GooglevideoQualificationProof
|
||||||
|
if err := json.Unmarshal(raw, &proof); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
switch mode {
|
||||||
|
case "kind":
|
||||||
|
proof.Adapter.Kind = "unknown"
|
||||||
|
case "version":
|
||||||
|
proof.Adapter.Version++
|
||||||
|
case "digest":
|
||||||
|
proof.Adapter.PolicySHA256 = "changed"
|
||||||
|
case "missing":
|
||||||
|
proof.Adapter = nil
|
||||||
|
case "unknown-proof":
|
||||||
|
proof.Version = 3
|
||||||
|
case "downgrade":
|
||||||
|
proof.Version = 1
|
||||||
|
case "attestation":
|
||||||
|
proof.Attestation = nil
|
||||||
|
}
|
||||||
|
if proof.validateStored(c) == nil {
|
||||||
|
t.Fatal("malformed proof reconstructed")
|
||||||
|
}
|
||||||
|
raw, _ = json.Marshal(proof)
|
||||||
|
if _, err := db.Exec("UPDATE googlevideo_scope_evidence SET proof=? WHERE candidate_id=?", string(raw), id); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
m, reloads := managedFixture(t)
|
||||||
|
if err := m.Apply(db, []int64{front, id}, GooglevideoScopeSuffix); err == nil || *reloads != 0 {
|
||||||
|
t.Fatal("invalid proof reached apply", err, *reloads)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestServiceAdapterProofBoundary(t *testing.T) {
|
||||||
|
r, c, resources, _ := googlevideoRuntimeFixture(t)
|
||||||
|
p, err := r.verifyGooglevideo(context.Background(), c, resources)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if p.Version != 2 || p.Adapter == nil || *p.Adapter != (youtube.Module{}).Binding() {
|
||||||
|
t.Fatal("missing module identity")
|
||||||
|
}
|
||||||
|
seal := googlevideoProofSeal(p)
|
||||||
|
for _, mutate := range []func(){
|
||||||
|
func() { p.Adapter.Kind = "unknown" },
|
||||||
|
func() { p.Adapter.Version++ },
|
||||||
|
func() { p.Adapter.PolicySHA256 = "changed" },
|
||||||
|
} {
|
||||||
|
original := *p.Adapter
|
||||||
|
mutate()
|
||||||
|
if p.validateStored(c) == nil || googlevideoProofSeal(p) == seal {
|
||||||
|
t.Fatal("module boundary not bound to evidence")
|
||||||
|
}
|
||||||
|
*p.Adapter = original
|
||||||
|
}
|
||||||
|
// Reads retain the original v1 contract, including fresh apply eligibility.
|
||||||
|
p.Version, p.Adapter = 1, nil
|
||||||
|
raw, err := json.Marshal(p)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
legacy, err := decodeGooglevideoProof(string(raw))
|
||||||
|
if err != nil || legacy.validateStored(c) != nil {
|
||||||
|
t.Fatal("legacy reconstruction lost", err)
|
||||||
|
}
|
||||||
|
if legacy.validate(c) != nil {
|
||||||
|
t.Fatal("fresh trusted legacy adaptive proof lost apply eligibility")
|
||||||
|
}
|
||||||
|
legacy.Attestation.CheckedAt = time.Now().Add(-25 * time.Hour)
|
||||||
|
if legacy.validate(c) == nil {
|
||||||
|
t.Fatal("expired legacy adaptive proof authorized new apply")
|
||||||
|
}
|
||||||
|
replaceServiceAdapters(t, serviceAdapterRegistry{})
|
||||||
|
if legacy.validateStored(c) != nil {
|
||||||
|
t.Fatal("historical reconstruction depends on current execution registry")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type changedPolicyServiceAdapter struct{ adapter.Module }
|
||||||
|
|
||||||
|
func (m changedPolicyServiceAdapter) Binding() adapter.Binding {
|
||||||
|
b := m.Module.Binding()
|
||||||
|
b.PolicySHA256 = "changed"
|
||||||
|
return b
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestServiceAdapterDiscoveryPreflight(t *testing.T) {
|
||||||
|
for _, mode := range []string{"missing", "version", "digest"} {
|
||||||
|
t.Run(mode, func(t *testing.T) {
|
||||||
|
modules := serviceAdapterRegistry{}
|
||||||
|
if mode == "version" {
|
||||||
|
modules[serviceAdapterKey{youtube.Kind, youtube.Version}] = mismatchedServiceAdapter{youtube.Module{}}
|
||||||
|
}
|
||||||
|
if mode == "digest" {
|
||||||
|
modules[serviceAdapterKey{youtube.Kind, youtube.Version}] = changedPolicyServiceAdapter{youtube.Module{}}
|
||||||
|
}
|
||||||
|
calls := 0
|
||||||
|
player := func(context.Context, []string, io.Writer, io.Writer) error { calls++; return nil }
|
||||||
|
request := func(context.Context) ([]byte, error) { calls++; return nil, nil }
|
||||||
|
target, _, _ := managedCLIFixture(t)
|
||||||
|
target.player = player
|
||||||
|
target.verifySOCKS = func(int) error { calls++; return nil }
|
||||||
|
r, _, _ := recoveryFixture(t, false)
|
||||||
|
bound, err := managedRecoveryDependencies(target, r.db, r.scan, r.check, r.run)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
calls = 0 // Construction guard is allowed before removing the module.
|
||||||
|
replaceServiceAdapters(t, modules)
|
||||||
|
cfg := healthConfig{VideoID: "dQw4w9WgXcQ", SOCKSProxy: "socks5://127.0.0.1:1080"}
|
||||||
|
assertBlocked := func(err error) {
|
||||||
|
t.Helper()
|
||||||
|
var fatal *managedRecoveryFatal
|
||||||
|
if !errors.Is(err, errServiceAdapterUnavailable) || !errors.As(err, &fatal) || errors.Is(err, errNoQualifiedStrategy) || errors.Is(err, errGooglevideoInsufficientSamples) || managedResourceTransferNonpass(err, true, true) {
|
||||||
|
t.Fatal("adapter configuration became a retryable absence", err)
|
||||||
|
}
|
||||||
|
if calls != 0 {
|
||||||
|
t.Fatal("preflight made network calls", calls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
_, err = requestPlayer(cfg.VideoID, cfg.SOCKSProxy, player)
|
||||||
|
assertBlocked(err)
|
||||||
|
if managedResourceTransferNonpass(errors.Join(err, context.DeadlineExceeded), true, true) || managedRecoveryCooldown(errors.Join(err, errNoQualifiedStrategy), time.Minute) != time.Minute {
|
||||||
|
t.Fatal("mixed fatal adapter failure became transfer-negative/2h absence")
|
||||||
|
}
|
||||||
|
_, err = bound.discover(context.Background())
|
||||||
|
assertBlocked(err)
|
||||||
|
_, err = bound.health(context.Background())
|
||||||
|
assertBlocked(err)
|
||||||
|
_, err = managedGooglevideoSamples([]string{cfg.VideoID}, func(ctx context.Context, _ string) ([]byte, error) { return request(ctx) })(context.Background())
|
||||||
|
assertBlocked(err)
|
||||||
|
inventory := managedServiceResources(cfg, request, player)
|
||||||
|
_, err = inventory(context.Background())
|
||||||
|
assertBlocked(err)
|
||||||
|
_, err = managedServiceHealth(cfg.SOCKSProxy, func(context.Context) (string, error) { calls++; return "OK", nil }, inventory, func(context.Context, []string, io.Reader, io.Writer, io.Writer) error { calls++; return nil })(context.Background())
|
||||||
|
assertBlocked(err)
|
||||||
|
if code := discoverYoutube([]string{"--video-id", cfg.VideoID, "--proxy", cfg.SOCKSProxy}, io.Discard, io.Discard, player); code != 1 || calls != 0 {
|
||||||
|
t.Fatal("legacy CLI bypassed preflight", code, calls)
|
||||||
|
}
|
||||||
|
_, err = managedRecoveryDependencies(target, r.db, r.scan, r.check, r.run)
|
||||||
|
assertBlocked(err)
|
||||||
|
assertBlocked(managedExperimentalSuffix(nil, managedDeps{}))
|
||||||
|
assertBlocked(managedActivate(nil, managedDeps{}, nil, nil))
|
||||||
|
for _, saved := range []bool{false, true} {
|
||||||
|
code := googlevideoProposal([]string{"--video-id", cfg.VideoID, "--proxy", cfg.SOCKSProxy}, io.Discard, io.Discard, player, func(context.Context, string, string, io.Writer, io.Writer) (string, error) { calls++; return "", nil }, func(context.Context, string, string, string, string, io.Writer, io.Writer) error { calls++; return nil }, saved)
|
||||||
|
if code != 1 || calls != 0 {
|
||||||
|
t.Fatal("proposal pipeline bypassed preflight", code, calls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
runtime, c, resources, events := googlevideoRuntimeFixture(t)
|
||||||
|
_, err = runtime.verifyGooglevideo(context.Background(), c, resources)
|
||||||
|
assertBlocked(err)
|
||||||
|
if len(*events) != 0 {
|
||||||
|
t.Fatal("preflight launched worker")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestServiceAdapterLegacyNewWriteRefused(t *testing.T) {
|
||||||
|
r, c, resources, _ := googlevideoRuntimeFixture(t)
|
||||||
|
p, err := r.verifyGooglevideo(context.Background(), c, resources)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
p.Version, p.Adapter = 1, nil
|
||||||
|
if p.validate(c) != nil {
|
||||||
|
t.Fatal("trusted legacy read invalid")
|
||||||
|
}
|
||||||
|
// Even an internally constructed seal cannot authorize NEW v1 publication.
|
||||||
|
seal := googlevideoProofSeal(p)
|
||||||
|
p.producerSeal = &seal
|
||||||
|
db := testStore(t)
|
||||||
|
tx, err := db.Begin()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer tx.Rollback()
|
||||||
|
id, err := insertStrategyCandidate(tx, c.Group, c.Endpoints, c.Args, nil, c.Mode)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if saveAdaptiveGooglevideoEvidence(tx, id, p) == nil {
|
||||||
|
t.Fatal("new sealed v1 write accepted")
|
||||||
|
}
|
||||||
|
raw, _ := json.Marshal(p)
|
||||||
|
var dto GooglevideoQualificationProof
|
||||||
|
if err := json.Unmarshal(raw, &dto); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if saveAdaptiveGooglevideoEvidence(tx, id, dto) == nil {
|
||||||
|
t.Fatal("unsealed v1 DTO write accepted")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,168 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
// Portable trusted-history fixtures: run unchanged on archived d324b41 too.
|
||||||
|
// Direct SQL below models an existing trusted v1 database, not an import API.
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"database/sql"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"reflect"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func legacyYoutubeV1Fixture(t *testing.T, db *sql.DB) (int64, GooglevideoQualificationProof) {
|
||||||
|
t.Helper()
|
||||||
|
r, c, resources, _ := googlevideoRuntimeFixture(t)
|
||||||
|
p, err := r.verifyGooglevideo(context.Background(), c, resources)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
raw, _ := json.Marshal(p)
|
||||||
|
var fields map[string]json.RawMessage
|
||||||
|
if err := json.Unmarshal(raw, &fields); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
delete(fields, "adapter")
|
||||||
|
fields["version"] = json.RawMessage(`1`)
|
||||||
|
raw, _ = json.Marshal(fields)
|
||||||
|
legacy, err := decodeGooglevideoProof(string(raw))
|
||||||
|
if err != nil || legacy.validate(c) != nil {
|
||||||
|
t.Fatal("trusted v1 contract invalid", err)
|
||||||
|
}
|
||||||
|
tx, err := db.Begin()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer tx.Rollback()
|
||||||
|
id, err := insertStrategyCandidate(tx, c.Group, c.Endpoints, c.Args, nil, c.Mode)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := tx.Exec("INSERT INTO googlevideo_scope_evidence(candidate_id,proof) VALUES (?,?)", id, string(raw)); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := tx.Commit(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
return id, legacy
|
||||||
|
}
|
||||||
|
|
||||||
|
func expireLegacyYoutubeV1(t *testing.T, db *sql.DB, id int64, proof GooglevideoQualificationProof) {
|
||||||
|
t.Helper()
|
||||||
|
proof.Attestation.CheckedAt = time.Now().Add(-25 * time.Hour)
|
||||||
|
raw, _ := json.Marshal(proof)
|
||||||
|
if _, err := db.Exec("UPDATE googlevideo_scope_evidence SET proof=? WHERE candidate_id=?", string(raw), id); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLegacyYoutubeV1FreshApplyEligibility(t *testing.T) {
|
||||||
|
db := testStore(t)
|
||||||
|
id, proof := legacyYoutubeV1Fixture(t, db)
|
||||||
|
c, err := readStrategyCandidate(db, id)
|
||||||
|
if err != nil || !c.Qualified {
|
||||||
|
t.Fatal("fresh stored v1 lost qualification", err)
|
||||||
|
}
|
||||||
|
saved, err := listQualifiedCandidates(db, "googlevideo", 10)
|
||||||
|
if err != nil || len(saved) != 1 || saved[0].ID != id {
|
||||||
|
t.Fatal("fresh stored v1 disappeared from shortlist", err)
|
||||||
|
}
|
||||||
|
front := managedCandidateFixture(t, db, "frontend", "www.youtube.com", "--lua-desync=multisplit", true)
|
||||||
|
ids := []int64{front, id} // Native suffix still does not require assets.
|
||||||
|
m, reloads := managedFixture(t)
|
||||||
|
if err := m.Apply(db, ids, GooglevideoScopeSuffix); err != nil {
|
||||||
|
t.Fatal("fresh v1 apply rejected", err)
|
||||||
|
}
|
||||||
|
config, err := m.read(m.Name)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
expireLegacyYoutubeV1(t, db, id, proof)
|
||||||
|
c, err = readStrategyCandidate(db, id)
|
||||||
|
if err != nil || c.Qualified {
|
||||||
|
t.Fatal("expired v1 retained fresh qualification", err)
|
||||||
|
}
|
||||||
|
before := *reloads
|
||||||
|
if err := m.Apply(db, ids, GooglevideoScopeSuffix); err == nil || *reloads != before {
|
||||||
|
t.Fatal("expired v1 activated")
|
||||||
|
}
|
||||||
|
rendered, err := renderStoredManagedConfigScope(db, ids, GooglevideoScopeSuffix)
|
||||||
|
if err != nil || !bytes.Equal(config, rendered) {
|
||||||
|
t.Fatal("expired historical reconstruction changed", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestLegacyYoutubeV1PreparedRollback(t *testing.T) {
|
||||||
|
for _, point := range []string{"staged", "backup", "prepared", "replaced", "reloaded", "healthy"} {
|
||||||
|
t.Run(point, func(t *testing.T) {
|
||||||
|
db := testStore(t)
|
||||||
|
id, proof := legacyYoutubeV1Fixture(t, db)
|
||||||
|
front := managedCandidateFixture(t, db, "frontend", "www.youtube.com", "--lua-desync=multisplit", true)
|
||||||
|
ids := []int64{front, id}
|
||||||
|
m, reloads := managedFixture(t)
|
||||||
|
if err := m.Apply(db, ids, GooglevideoScopeSuffix); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
before, err := m.load()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
journal, err := m.read(managedJournalName)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
config, err := m.read(m.Name)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
nextFront := managedCandidateFixture(t, db, "frontend", "www.youtube.com", "--lua-desync=fake", true)
|
||||||
|
m.fault = func(stage string) error {
|
||||||
|
if stage == point {
|
||||||
|
return errors.New("fixture crash")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if err := m.Apply(db, []int64{nextFront, id}, GooglevideoScopeSuffix); err == nil {
|
||||||
|
t.Fatal("fault missed")
|
||||||
|
}
|
||||||
|
if point != "staged" {
|
||||||
|
backup, err := m.read(managedBackupName)
|
||||||
|
if err != nil || !bytes.Equal(backup, config) {
|
||||||
|
t.Fatal("legacy backup changed", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
expireLegacyYoutubeV1(t, db, id, proof)
|
||||||
|
m.fault = nil
|
||||||
|
priorReloads := *reloads
|
||||||
|
if err := m.Reconcile(db); err != nil {
|
||||||
|
t.Fatal("expired v1 prepared recovery failed", err)
|
||||||
|
}
|
||||||
|
after, err := m.load()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
wantGeneration, wantReloads := before.Generation+1, priorReloads+1
|
||||||
|
if point == "staged" || point == "backup" {
|
||||||
|
wantGeneration, wantReloads = before.Generation, priorReloads
|
||||||
|
}
|
||||||
|
if after.State != "committed" || after.Hash != before.Hash || after.Scope != before.Scope || !reflect.DeepEqual(after.Candidates, before.Candidates) || len(after.PendingCandidates) != 0 || after.PendingScope != "" || after.Generation != wantGeneration || *reloads != wantReloads {
|
||||||
|
t.Fatal("legacy rollback contract changed", after, *reloads)
|
||||||
|
}
|
||||||
|
restored, err := m.read(m.Name)
|
||||||
|
if err != nil || !bytes.Equal(restored, config) {
|
||||||
|
t.Fatal("rollback config bytes changed", err)
|
||||||
|
}
|
||||||
|
if point == "staged" || point == "backup" {
|
||||||
|
afterJournal, err := m.read(managedJournalName)
|
||||||
|
if err != nil || !bytes.Equal(journal, afterJournal) {
|
||||||
|
t.Fatal("pre-prepare journal changed", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
assertActiveCandidates(t, m, db, ids)
|
||||||
|
t.Logf("legacy restored config SHA256=%s", after.Hash)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,133 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
// This corpus also runs unmodified against archived d324b41. It uses only
|
||||||
|
// pre-extraction APIs, with explicit expected normalized facts, not wrappers.
|
||||||
|
import (
|
||||||
|
"bytes"
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestYoutubeLegacyParserParity(t *testing.T) {
|
||||||
|
good := "https://rr1---sn-alpha.googlevideo.com/videoplayback?expire=1800000000&sig=private-secret"
|
||||||
|
format := func(raw, mime string) map[string]string { return map[string]string{"url": raw, "mimeType": mime} }
|
||||||
|
for _, tc := range []struct {
|
||||||
|
name string
|
||||||
|
formats, adaptive []map[string]string
|
||||||
|
want string
|
||||||
|
}{
|
||||||
|
{"ordinary-first", []map[string]string{format(good, "audio/mp4; codecs=mp4a")}, []map[string]string{format(good+"&second=1", "video/mp4")}, "audio/mp4"},
|
||||||
|
{"nonmp4-fallback", []map[string]string{format(good, "video/webm")}, []map[string]string{format(good, "video/mp4")}, "video/mp4"},
|
||||||
|
{"userinfo", []map[string]string{format(strings.Replace(good, "https://", "https://user@", 1), "video/mp4")}, nil, ""},
|
||||||
|
{"fragment", []map[string]string{format(good+"#secret", "video/mp4")}, nil, ""},
|
||||||
|
{"http", []map[string]string{format(strings.Replace(good, "https:", "http:", 1), "video/mp4")}, nil, ""},
|
||||||
|
{"invalid-dns", []map[string]string{format(strings.Replace(good, "rr1---sn-alpha", "bad_host", 1), "video/mp4")}, nil, ""},
|
||||||
|
{"sabr-only", nil, nil, ""},
|
||||||
|
} {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
body, _ := json.Marshal(map[string]interface{}{"playabilityStatus": map[string]string{"status": "OK"}, "streamingData": map[string]interface{}{"formats": tc.formats, "adaptiveFormats": tc.adaptive, "serverAbrStreamingUrl": good}})
|
||||||
|
r, capability, err := playerMediaResource(body)
|
||||||
|
if err != nil || r.kind != tc.want || capability.OrdinaryURL != (tc.want != "") || capability.SABRSupported {
|
||||||
|
t.Fatal(r.kind, capability, err)
|
||||||
|
}
|
||||||
|
if tc.want != "" && (r.url != good || r.host != "rr1---sn-alpha.googlevideo.com") {
|
||||||
|
t.Fatal("normalized resource changed")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
for _, body := range [][]byte{[]byte(`{"secret":`), bytes.Repeat([]byte("x"), playerResponseLimit+1)} {
|
||||||
|
if _, _, err := playerMediaResource(body); err == nil || err.Error() != "invalid playable metadata" {
|
||||||
|
t.Fatal("metadata error contract changed", err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, delta := range []time.Duration{-time.Hour, 19 * time.Minute, 21 * time.Minute} {
|
||||||
|
r := mediaResource{fmt.Sprintf("https://rr1---sn-alpha.googlevideo.com/videoplayback?expire=%d&lsig=secret", time.Now().Add(delta).Unix()), "rr1---sn-alpha.googlevideo.com", "video/mp4"}
|
||||||
|
if freshGooglevideoSample(r) != (delta > managedGooglevideoSampleTTL) {
|
||||||
|
t.Fatal("inventory freshness changed", delta)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestYoutubeLegacyJournalParity(t *testing.T) {
|
||||||
|
r, c, resources, _ := googlevideoRuntimeFixture(t)
|
||||||
|
p, err := r.verifyGooglevideo(context.Background(), c, resources)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
db := testStore(t)
|
||||||
|
tx, err := db.Begin()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer tx.Rollback()
|
||||||
|
id, err := insertStrategyCandidate(tx, c.Group, c.Endpoints, c.Args, nil, c.Mode)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := saveAdaptiveGooglevideoEvidence(tx, id, p); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if err := tx.Commit(); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
front := managedCandidateFixture(t, db, "frontend", "www.youtube.com", "--lua-desync=multisplit", true)
|
||||||
|
m, _ := managedFixture(t)
|
||||||
|
ids := []int64{front, id} // No new asset requirement for native suffix policy.
|
||||||
|
if err := m.Apply(db, ids, GooglevideoScopeSuffix); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
before, err := m.load()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
journalBytes, err := m.read(managedJournalName)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
config, err := m.read(m.Name)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
// Convert the actual producer facts to the exact historical v1 wire shape.
|
||||||
|
p.Attestation.CheckedAt = time.Now().Add(-25 * time.Hour)
|
||||||
|
raw, _ := json.Marshal(p)
|
||||||
|
var fields map[string]json.RawMessage
|
||||||
|
if err := json.Unmarshal(raw, &fields); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
delete(fields, "adapter")
|
||||||
|
fields["version"] = json.RawMessage(`1`)
|
||||||
|
raw, _ = json.Marshal(fields)
|
||||||
|
if _, err := db.Exec("UPDATE googlevideo_scope_evidence SET proof=? WHERE candidate_id=?", string(raw), id); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
rendered, err := renderStoredManagedConfigScope(db, ids, GooglevideoScopeSuffix)
|
||||||
|
if err != nil || !bytes.Equal(config, rendered) || managedHash(rendered) != before.Hash {
|
||||||
|
t.Fatal("historical render/hash changed", err)
|
||||||
|
}
|
||||||
|
if err := m.Reconcile(db); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
active, err := m.ActiveCandidates(db)
|
||||||
|
if err != nil || len(active) != 2 || active[0] != front || active[1] != id {
|
||||||
|
t.Fatal(active, err)
|
||||||
|
}
|
||||||
|
after, err := m.load()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
a, _ := json.Marshal(before)
|
||||||
|
b, _ := json.Marshal(after)
|
||||||
|
if !bytes.Equal(a, b) {
|
||||||
|
t.Fatal("historical journal changed")
|
||||||
|
}
|
||||||
|
afterBytes, err := m.read(managedJournalName)
|
||||||
|
if err != nil || !bytes.Equal(journalBytes, afterBytes) {
|
||||||
|
t.Fatal("historical journal bytes changed", err)
|
||||||
|
}
|
||||||
|
t.Logf("historical config SHA256=%s", before.Hash)
|
||||||
|
}
|
||||||
@@ -0,0 +1,35 @@
|
|||||||
|
// Package adapter defines the closed internal contract used by compiled-in
|
||||||
|
// service adapters. It is not the public resource-check configuration API.
|
||||||
|
package adapter
|
||||||
|
|
||||||
|
import "time"
|
||||||
|
|
||||||
|
type Binding struct {
|
||||||
|
Kind string `json:"kind"`
|
||||||
|
Version int `json:"version"`
|
||||||
|
PolicySHA256 string `json:"policy_sha256"`
|
||||||
|
}
|
||||||
|
|
||||||
|
// Resource and Plan are ephemeral; neither private inputs nor policy caps belong
|
||||||
|
// in evidence. Only core-validated, measured facts can become a sealed proof.
|
||||||
|
type Resource struct {
|
||||||
|
URL string `json:"-"`
|
||||||
|
Host string `json:"-"`
|
||||||
|
Kind string `json:"-"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type Plan struct {
|
||||||
|
Hosts []string `json:"-"`
|
||||||
|
Protocols [2]string `json:"-"`
|
||||||
|
WarmupLimit int `json:"-"`
|
||||||
|
WarmupTimeout time.Duration `json:"-"`
|
||||||
|
Samples int `json:"-"`
|
||||||
|
MinBytes int64 `json:"-"`
|
||||||
|
}
|
||||||
|
|
||||||
|
type Module interface {
|
||||||
|
Binding() Binding
|
||||||
|
DiscoverResponse([]byte) ([]Resource, error)
|
||||||
|
ValidateHosts([]string) error
|
||||||
|
PlanQualification([]string, []Resource, time.Time) (Plan, error)
|
||||||
|
}
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
# Internal YouTube adapter
|
||||||
|
|
||||||
|
The compiled-in `youtube/1` module parses bounded Player JSON and supplies the
|
||||||
|
ordinary signed MP4 qualification policy and ordered host/TLS sample plan.
|
||||||
|
The existing YouTube execution paths select it explicitly through the core's
|
||||||
|
kind/version registry. Unsupported or absent modules fail closed.
|
||||||
|
|
||||||
|
The core retains origin validation, HTTPS/SNI/certificate checks, transport
|
||||||
|
framing/body bounds, retries/cancellation, worker ownership, cleanup/join and
|
||||||
|
proof sealing. Module output is not evidence or permission to claim PASS.
|
||||||
|
Signed URLs remain ephemeral and cannot serialize through the internal contract.
|
||||||
|
|
||||||
|
New adaptive proof v2 binds the module kind, version and immutable policy digest
|
||||||
|
inside its producer seal. Unbound proof v1 explicitly means the historical
|
||||||
|
YouTube v1 policy: fresh trusted stored proofs retain their original apply
|
||||||
|
eligibility; expired proofs reconstruct history but cannot authorize fresh apply.
|
||||||
|
Only new producer/persistence writes require sealed module-bound v2 evidence.
|
||||||
|
Cold historical evidence is unchanged. Discovery entrypoints preflight the module
|
||||||
|
before requests; absence/mismatched identity or policy is a fatal configuration
|
||||||
|
failure, not unavailable inventory or a no-strategy cooldown.
|
||||||
|
|
||||||
|
Legacy routing remains SOCKS5h with encrypted-resolver ownership guards and
|
||||||
|
origin/SNI/certificate/TLS checks, not generic all-answer approval/IP pinning.
|
||||||
|
Full request-policy capabilities and parity are prerequisites to generic cutover.
|
||||||
|
|
||||||
|
This is a real internal legacy-execution integration, not a fully generic
|
||||||
|
controller, public v3 configuration, native addon, runtime plugin or readiness
|
||||||
|
claim. Another compiled-in module can implement the closed internal contract;
|
||||||
|
public GET/HEAD v1/v2 configuration remains unchanged.
|
||||||
@@ -0,0 +1,120 @@
|
|||||||
|
// Package youtube contains pure service semantics, not transport or worker ownership.
|
||||||
|
package youtube
|
||||||
|
|
||||||
|
import (
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"mime"
|
||||||
|
"net/url"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"zapret-autotune/internal/adapter"
|
||||||
|
)
|
||||||
|
|
||||||
|
const Kind = "youtube"
|
||||||
|
const Version = 1
|
||||||
|
const InventoryTTL = 20 * time.Minute
|
||||||
|
|
||||||
|
// Immutable v1 policy. Changes require another module version, not a new meaning
|
||||||
|
// for existing evidence. Legacy proof v1 uses this same historical policy.
|
||||||
|
const policyV1 = "youtube/1;ggc>=3;roots>=2;prefixes>=2;ordinary-signed-mp4;fresh>120s;inventory>20m;tls12,tls13;warmup<=10,120s;steady=3/protocol;min=1024"
|
||||||
|
|
||||||
|
type Binding = adapter.Binding
|
||||||
|
|
||||||
|
type Module struct{}
|
||||||
|
|
||||||
|
func (Module) Binding() Binding {
|
||||||
|
return Binding{Kind: Kind, Version: Version, PolicySHA256: fmt.Sprintf("%x", sha256.Sum256([]byte(policyV1)))}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Resource is ephemeral: never serialize signed URLs or inventory metadata.
|
||||||
|
type Resource = adapter.Resource
|
||||||
|
|
||||||
|
// DiscoverResponse preserves player format order. The core must validate every
|
||||||
|
// returned origin before selecting or fetching it. SABR is not supported.
|
||||||
|
func (Module) DiscoverResponse(body []byte) ([]Resource, error) {
|
||||||
|
type format struct {
|
||||||
|
URL string `json:"url"`
|
||||||
|
MimeType string `json:"mimeType"`
|
||||||
|
}
|
||||||
|
var p struct {
|
||||||
|
PlayabilityStatus struct {
|
||||||
|
Status string `json:"status"`
|
||||||
|
} `json:"playabilityStatus"`
|
||||||
|
StreamingData struct {
|
||||||
|
Formats []format `json:"formats"`
|
||||||
|
AdaptiveFormats []format `json:"adaptiveFormats"`
|
||||||
|
} `json:"streamingData"`
|
||||||
|
}
|
||||||
|
if len(body) > 1<<20 || json.Unmarshal(body, &p) != nil || p.PlayabilityStatus.Status != "OK" {
|
||||||
|
return nil, errors.New("invalid playable metadata")
|
||||||
|
}
|
||||||
|
var resources []Resource
|
||||||
|
for _, f := range append(p.StreamingData.Formats, p.StreamingData.AdaptiveFormats...) {
|
||||||
|
u, err := url.Parse(f.URL)
|
||||||
|
if err != nil || !strings.HasSuffix(u.Hostname(), ".googlevideo.com") {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
kind, _, err := mime.ParseMediaType(f.MimeType)
|
||||||
|
if err == nil && (kind == "video/mp4" || kind == "audio/mp4") {
|
||||||
|
resources = append(resources, Resource{URL: f.URL, Host: u.Hostname(), Kind: kind})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return resources, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// ValidateHosts checks service diversity; DNS/origin policy remains in the core.
|
||||||
|
func (Module) ValidateHosts(hosts []string) error {
|
||||||
|
seen, roots, prefixes := map[string]bool{}, map[string]bool{}, map[string]bool{}
|
||||||
|
for _, host := range hosts {
|
||||||
|
prefix, root, ok := strings.Cut(strings.TrimSuffix(host, ".googlevideo.com"), "---sn-")
|
||||||
|
if !ok || !strings.HasSuffix(host, ".googlevideo.com") || host != strings.ToLower(host) || seen[host] || root == "" || len(prefix) < 3 || !strings.HasPrefix(prefix, "rr") || strings.Trim(prefix[2:], "0123456789") != "" {
|
||||||
|
return errors.New("insufficient representative GGC hosts")
|
||||||
|
}
|
||||||
|
seen[host], roots[strings.Split(root, "-")[0]], prefixes[prefix] = true, true, true
|
||||||
|
}
|
||||||
|
if len(hosts) < 3 || len(roots) < 2 || len(prefixes) < 2 {
|
||||||
|
return errors.New("insufficient representative GGC hosts")
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
type Plan = adapter.Plan
|
||||||
|
|
||||||
|
// PlanQualification grants no PASS capability. Core validates origins first,
|
||||||
|
// executes the ordered schedule, validates measured facts, and seals after join.
|
||||||
|
func (m Module) PlanQualification(hosts []string, resources []Resource, now time.Time) (Plan, error) {
|
||||||
|
p := Plan{Hosts: append([]string(nil), hosts...), Protocols: [2]string{"tls12", "tls13"}, WarmupLimit: 10, WarmupTimeout: 120 * time.Second, Samples: 3, MinBytes: 1024}
|
||||||
|
if err := m.ValidateHosts(hosts); err != nil {
|
||||||
|
return Plan{}, err
|
||||||
|
}
|
||||||
|
seen := map[string]bool{}
|
||||||
|
for _, r := range resources {
|
||||||
|
if !SignedSampleFresh(r, now, 120*time.Second) {
|
||||||
|
return Plan{}, errors.New("insufficient fresh signed media samples")
|
||||||
|
}
|
||||||
|
seen[r.Host] = true
|
||||||
|
}
|
||||||
|
for _, host := range hosts {
|
||||||
|
if !seen[host] {
|
||||||
|
return Plan{}, errors.New("missing representative sample")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return p, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// SignedSampleFresh checks service semantics only, not permission to fetch.
|
||||||
|
// The core independently validates HTTPS origins before calling this helper.
|
||||||
|
func SignedSampleFresh(r Resource, now time.Time, ttl time.Duration) bool {
|
||||||
|
u, err := url.Parse(r.URL)
|
||||||
|
if err != nil || u.Path != "/videoplayback" || (r.Kind != "video/mp4" && r.Kind != "audio/mp4") {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
q := u.Query()
|
||||||
|
expire, err := strconv.ParseInt(q.Get("expire"), 10, 64)
|
||||||
|
return err == nil && expire > now.Add(ttl).Unix() && (q.Get("sig") != "" || q.Get("signature") != "" || q.Get("lsig") != "")
|
||||||
|
}
|
||||||
@@ -0,0 +1,90 @@
|
|||||||
|
package youtube
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestDiscoverResponse(t *testing.T) {
|
||||||
|
m := Module{}
|
||||||
|
resources, err := m.DiscoverResponse([]byte(`{"playabilityStatus":{"status":"OK"},"streamingData":{"formats":[{"url":"https://rr1.unit.googlevideo.com/videoplayback?sig=secret","mimeType":"audio/mp4"}],"adaptiveFormats":[{"url":"https://rr2.unit.googlevideo.com/videoplayback?sig=secret","mimeType":"video/mp4; codecs=avc1"}]}}`))
|
||||||
|
if err != nil || len(resources) != 2 || resources[0].Kind != "audio/mp4" || resources[1].Kind != "video/mp4" {
|
||||||
|
t.Fatal(resources, err)
|
||||||
|
}
|
||||||
|
resources, err = m.DiscoverResponse([]byte(`{"playabilityStatus":{"status":"OK"},"streamingData":{"serverAbrStreamingUrl":"https://rr1.unit.googlevideo.com/sabr?sig=secret"}}`))
|
||||||
|
if err != nil || len(resources) != 0 {
|
||||||
|
t.Fatal("SABR fabricated ordinary readiness")
|
||||||
|
}
|
||||||
|
for _, body := range []string{"{", strings.Repeat(" ", (1<<20)+1), `{"playabilityStatus":{"status":"ERROR"}}`} {
|
||||||
|
if _, err := m.DiscoverResponse([]byte(body)); err == nil || strings.Contains(err.Error(), "secret") {
|
||||||
|
t.Fatal("invalid metadata accepted or disclosed")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestQualificationPlan(t *testing.T) {
|
||||||
|
m, now := Module{}, time.Unix(1800000000, 0)
|
||||||
|
if m.Binding().PolicySHA256 != "2cfc521a9784c574ceee16430aa485b9c0e9e64795486a23df5117b6265d89d5" {
|
||||||
|
t.Fatal("immutable youtube/1 policy changed: introduce a new version")
|
||||||
|
}
|
||||||
|
hosts := []string{"rr1---sn-alpha.googlevideo.com", "rr2---sn-beta.googlevideo.com", "rr5---sn-gamma.googlevideo.com"}
|
||||||
|
var resources []Resource
|
||||||
|
for _, host := range hosts {
|
||||||
|
resources = append(resources, Resource{URL: fmt.Sprintf("https://%s/videoplayback?expire=%d&sig=secret", host, now.Unix()+121), Host: host, Kind: "video/mp4"})
|
||||||
|
}
|
||||||
|
p, err := m.PlanQualification(hosts, resources, now)
|
||||||
|
if err != nil || strings.Join(p.Hosts, ",") != strings.Join(hosts, ",") || p.Protocols != [2]string{"tls12", "tls13"} || p.Samples != 3 || p.MinBytes != 1024 || p.WarmupLimit != 10 || p.WarmupTimeout != 120*time.Second {
|
||||||
|
t.Fatal(p, err)
|
||||||
|
}
|
||||||
|
for _, mode := range []string{"expired", "unsigned", "sabr", "missing", "diversity"} {
|
||||||
|
t.Run(mode, func(t *testing.T) {
|
||||||
|
r, h := append([]Resource(nil), resources...), append([]string(nil), hosts...)
|
||||||
|
switch mode {
|
||||||
|
case "expired":
|
||||||
|
r[0].URL = strings.Replace(r[0].URL, "1800000121", "1800000120", 1)
|
||||||
|
case "unsigned":
|
||||||
|
r[0].URL = strings.Split(r[0].URL, "&sig=")[0]
|
||||||
|
case "sabr":
|
||||||
|
r[0].URL = strings.Replace(r[0].URL, "videoplayback", "sabr", 1)
|
||||||
|
case "missing":
|
||||||
|
r = r[:2]
|
||||||
|
case "diversity":
|
||||||
|
h[1] = h[0]
|
||||||
|
}
|
||||||
|
if _, err := m.PlanQualification(h, r, now); err == nil {
|
||||||
|
t.Fatal("invalid sample accepted")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
encoded, err := json.Marshal(struct {
|
||||||
|
Resources []Resource
|
||||||
|
Plan Plan
|
||||||
|
}{resources, p})
|
||||||
|
if err != nil || strings.Contains(string(encoded), "secret") || strings.Contains(string(encoded), hosts[0]) {
|
||||||
|
t.Fatal("ephemeral data serialized", string(encoded), err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSignedSampleFresh(t *testing.T) {
|
||||||
|
now := time.Unix(1800000000, 900000000)
|
||||||
|
for _, tc := range []struct {
|
||||||
|
query string
|
||||||
|
want bool
|
||||||
|
}{
|
||||||
|
{"expire=1800000121&sig=secret", true},
|
||||||
|
{"expire=1800000120&sig=secret", false},
|
||||||
|
{"expire=1800000121&signature=secret", true},
|
||||||
|
{"expire=1800000121&lsig=secret", true},
|
||||||
|
{"expire=1800000121", false},
|
||||||
|
{"expire=invalid&sig=secret", false},
|
||||||
|
{"expire=9223372036854775808&sig=secret", false},
|
||||||
|
} {
|
||||||
|
r := Resource{URL: "https://rr1---sn-alpha.googlevideo.com/videoplayback?" + tc.query, Kind: "video/mp4"}
|
||||||
|
if SignedSampleFresh(r, now, 120*time.Second) != tc.want {
|
||||||
|
t.Fatal("signed expiry boundary changed")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
Reference in new issue
Block a user