fix: discover avatar metadata from trusted watch pages
Fall back only on missing homepage metadata, preserve private bounded inventory and fail-closed error classification. Record supplied field results without claiming image or family qualification. Co-authored-by: GPT-6 <gpt-6@openai.com>
This commit is contained in:
1 parent
a60f62de22
commit
50a78608ce
6 files changed
+223
-21
No files matched your search
+16
-2
@@ -432,12 +432,26 @@ not strategy absence; mixed, guard, infrastructure and cancellation errors do
|
|||||||
not authorize relearning. Unrelated missing image inventory may coexist with
|
not authorize relearning. Unrelated missing image inventory may coexist with
|
||||||
usable media, but successful media recovery does not repair or qualify it.
|
usable media, but successful media recovery does not repair or qualify it.
|
||||||
|
|
||||||
|
Avatar inventory first parses fresh homepage `ytInitialData`. Only absent metadata
|
||||||
|
or no allowed avatar permits one fallback to the configured video's watch page,
|
||||||
|
using a validated video ID, the same trusted HTTPS WWW origin and SOCKS proxy,
|
||||||
|
bounded time/body size, no redirects, and discarded diagnostics. Malformed JSON,
|
||||||
|
transfer errors and safety/cancellation failures do not trigger fallback. Both
|
||||||
|
pages and discovered resource URLs remain ephemeral; discovering an allowed URL
|
||||||
|
is not image-transfer qualification or a guarantee of full-family coverage.
|
||||||
|
|
||||||
`managed status` reports active-profile `covered=` separately from qualified
|
`managed status` reports active-profile `covered=` separately from qualified
|
||||||
`groups` and `unresolved_groups`; `managed active-ids` adds `covered_groups` and
|
`groups` and `unresolved_groups`; `managed active-ids` adds `covered_groups` and
|
||||||
`googlevideo_scope` to its JSON. An experimental Googlevideo profile can be
|
`googlevideo_scope` to its JSON. An experimental Googlevideo profile can be
|
||||||
covered yet qualification-unresolved; one image profile does not establish full
|
covered yet qualification-unresolved; one image profile does not establish full
|
||||||
asset-inventory success. Status is read-only and reports live health NOT CHECKED.
|
asset-inventory success. Status is read-only and reports live health NOT CHECKED.
|
||||||
The new autonomous recovery/classifier/status behavior is source-only pending
|
Reported `a60f62d` field checks passed restart/recreation and status coverage checks.
|
||||||
field checks, not a deployment or production-acceptance claim. Generic production
|
A live upgrade hit runner `BrokenPipe` and automatically rolled back to the same
|
||||||
|
last-good image; successful autonomous media recovery remains unaccepted. Fresh
|
||||||
|
Range media passed once on TLS1.2, then failed on repeat; no TLS1.3 media PASS is
|
||||||
|
claimed, and the field failure's cause is unknown. Reported watch metadata had an
|
||||||
|
allowed avatar URL absent from the homepage, but this new conditional fallback
|
||||||
|
is source-only and neither field-deployed nor avatar-image-qualified. See
|
||||||
|
[PROGRESS.md](PROGRESS.md) for the remaining gates. Generic production
|
||||||
media behavior and signed-URL/SS secret privacy remain unchanged, as do the
|
media behavior and signed-URL/SS secret privacy remain unchanged, as do the
|
||||||
persisted minimum two-hour strategy-absence pause and 90-minute whole-job budget.
|
persisted minimum two-hour strategy-absence pause and 90-minute whole-job budget.
|
||||||
+7
-5
@@ -10,16 +10,18 @@
|
|||||||
- Fresh signed media on `rr16---sn-n8v7znsz.googlevideo.com` returned HTTP206, 65536 bytes, MP4 `ftyp`, TLS1.2. On **2026-10-12 the user confirmed browser videos work** with this live RC and accepted one shared profile for Googlevideo, including initial learning. Browser images also appeared, but that does not qualify every image route.
|
- Fresh signed media on `rr16---sn-n8v7znsz.googlevideo.com` returned HTTP206, 65536 bytes, MP4 `ftyp`, TLS1.2. On **2026-10-12 the user confirmed browser videos work** with this live RC and accepted one shared profile for Googlevideo, including initial learning. Browser images also appeared, but that does not qualify every image route.
|
||||||
- One suffix profile is a **composite native circular A/E profile with two strategies and per-host learning**, not one uniform static strategy. Explicit operator experimental policy is durable across restart with hash/rollback guards; it is neither family-qualified nor `READY`, and no all-host cold 3/3 result is claimed.
|
- One suffix profile is a **composite native circular A/E profile with two strategies and per-host learning**, not one uniform static strategy. Explicit operator experimental policy is durable across restart with hash/rollback guards; it is neither family-qualified nor `READY`, and no all-host cold 3/3 result is claimed.
|
||||||
- Source now wires autonomous experimental media recovery in either configured service mode. Healthy actual-resource probes short-circuit recovery; a validated media transfer failure permits one guarded restart of the same committed profiles and bounded native relearning in the persistent deployed daemon (10 attempts / 120s), followed by fresh signed media 3/3 on both TLS protocols. Learning stays in daemon RAM, not a temporary verification worker; expired inventory is refreshed, never rewritten or persisted. Missing inventory is degraded, not strategy absence; guard/infrastructure/cancellation and mixed failures remain fail-closed. No experimental candidate is replaced or promoted to qualified.
|
- Source now wires autonomous experimental media recovery in either configured service mode. Healthy actual-resource probes short-circuit recovery; a validated media transfer failure permits one guarded restart of the same committed profiles and bounded native relearning in the persistent deployed daemon (10 attempts / 120s), followed by fresh signed media 3/3 on both TLS protocols. Learning stays in daemon RAM, not a temporary verification worker; expired inventory is refreshed, never rewritten or persisted. Missing inventory is degraded, not strategy absence; guard/infrastructure/cancellation and mixed failures remain fail-closed. No experimental candidate is replaced or promoted to qualified.
|
||||||
- Status distinguishes active-profile coverage (`covered=` / JSON `covered_groups`, `googlevideo_scope`) from qualified `groups` and `unresolved_groups`. Experimental Googlevideo remains unresolved for qualification even when operational media is healthy; a thumbnail profile does not mean the full asset inventory is qualified. Status itself performs no live health check. New recovery/classifier/status code is pending field acceptance; no new deployment, network test, family-qualified/`READY` or production claim is made. Generic production media handling, signed-URL/SS privacy, persisted minimum two-hour strategy-absence pause and the separate 90-minute whole-job budget are unchanged.
|
- Status distinguishes active-profile coverage (`covered=` / JSON `covered_groups`, `googlevideo_scope`) from qualified `groups` and `unresolved_groups`. Experimental Googlevideo remains unresolved for qualification even when operational media is healthy; a thumbnail profile does not mean the full asset inventory is qualified. Status itself performs no live health check. Generic production media handling, signed-URL/SS privacy, persisted minimum two-hour strategy-absence pause and the separate 90-minute whole-job budget are unchanged.
|
||||||
- Local source verification: fresh `go test ./... -count=1`, affected experimental/resource-classifier/status/monitor regression tests under `-race`, and `git diff --check` passed. These are unit/fixture checks only, not new field evidence.
|
- **Reported field follow-up for `a60f62d`:** privileged checks passed restart/recreation and covered-versus-qualified status checks. The live upgrade encountered a runner `BrokenPipe` and automatically rolled back to the same last-good image; this is not a successful autonomous media-recovery acceptance. Fresh Range media passed once on TLS1.2, then failed on repeat; no TLS1.3 media PASS is claimed. The underlying field failure remains unknown. These are supplied field results, not network checks rerun by this source update.
|
||||||
|
- Avatar discovery now conditionally falls back from a homepage with absent metadata/no allowed avatar to one fresh watch page for the validated configured video ID, on the same trusted HTTPS WWW origin and proxy. Malformed JSON, request failures, cancellation and safety failures do not authorize fallback. Reported watch metadata contained an allowed avatar URL where the homepage did not; the updated fallback has not been field-deployed or image-qualified. URL discovery alone does not qualify the avatar route/profile or close a family gate.
|
||||||
|
- Local source verification for the watch fallback: fresh `go test ./... -count=1 -timeout=180s`, focused avatar inventory/parser/classifier, experimental production-probe and private-inventory regression tests under `-race`, and `git diff --check` passed. These are unit/fixture checks only, not new field evidence.
|
||||||
|
|
||||||
Remaining acceptance gates (not asserted defects):
|
Remaining acceptance gates (not asserted defects):
|
||||||
|
|
||||||
1. Representative fresh signed media evidence: at least three unique hosts across at least two roots/prefixes, post-warmup 3/3 for both TLS protocols plus actual media transfer. Do not impose uniform static/cold behavior on the accepted learning experiment or label it normally qualified without evidence.
|
1. Representative fresh signed media evidence: at least three unique hosts across at least two roots/prefixes, post-warmup 3/3 for both TLS protocols plus actual media transfer. Do not impose uniform static/cold behavior on the accepted learning experiment or label it normally qualified without evidence.
|
||||||
2. Qualify the avatar route/profile: only the thumbnail image profile is currently active; browser-visible images do not close this gate.
|
2. Qualify the avatar route/profile: only the thumbnail image profile is currently active; browser-visible images do not close this gate.
|
||||||
3. Field-accept actual restart/recreation recovery after the all-Googlevideo suffix change, including renewed native learning; older exact-host lifecycle PASS does not cover this RC.
|
3. Restart/recreation checks are reported PASS for `a60f62d`; renewed learning with repeatable actual media success remains an open gate.
|
||||||
4. Field-accept the new autonomous same-profile experimental recovery and resource-error classifier without replacing or promoting operator-approved scope. Automatic family qualification is not implemented.
|
4. Field-accept the new autonomous same-profile experimental recovery and resource-error classifier without replacing or promoting operator-approved scope. Unit coverage exists; live upgrade/rollback observations do not prove successful recovery. Automatic family qualification is not implemented.
|
||||||
5. Field-check the new covered-versus-qualified status fields against the live journal and asset inventory; no family-qualified/`READY` or blanket production claim yet.
|
5. Covered-versus-qualified field checks are reported PASS for `a60f62d`; validate the new conditional watch fallback and qualify actual avatar image transfers separately. No family-qualified/`READY` or blanket production claim yet.
|
||||||
|
|
||||||
### Earlier isolated-target acceptance (2026-10-11)
|
### Earlier isolated-target acceptance (2026-10-11)
|
||||||
|
|
||||||
|
|||||||
@@ -42,6 +42,15 @@ func (e *managedResourceFailure) Unwrap() error { return e.Err }
|
|||||||
// generic fatal wrapper at a live-health boundary. Missing resources remain
|
// generic fatal wrapper at a live-health boundary. Missing resources remain
|
||||||
// DEGRADED; joined guard/infrastructure/cancellation failures retain their type.
|
// DEGRADED; joined guard/infrastructure/cancellation failures retain their type.
|
||||||
func managedResourceHealthError(ctx context.Context, err error) error {
|
func managedResourceHealthError(ctx context.Context, err error) error {
|
||||||
|
// Only a direct, known parser absence is missing inventory. Never flatten
|
||||||
|
// joined guard/infrastructure failures into an unrelated image absence.
|
||||||
|
if ctx.Err() == nil {
|
||||||
|
if failure, ok := err.(*managedResourceFailure); ok && failure.ProbeKind == "avatar" && failure.Host == "yt3.ggpht.com" {
|
||||||
|
if code, ok := failure.Err.(avatarMetadataError); ok && (code == "markerAbsent" || code == "jsonInvalid" || code == "noAllowedAvatar") {
|
||||||
|
return &managedResourceFailure{ProbeKind: failure.ProbeKind, Host: failure.Host}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
var failure *managedResourceFailure
|
var failure *managedResourceFailure
|
||||||
var collector *resourceError
|
var collector *resourceError
|
||||||
if ctx.Err() == nil && errors.As(err, &failure) && (failure.ProbeKind == "media" || assetEndpoint(failure.Host) && failure.ProbeKind == managedImageProbeKind(failure.Host)) && errors.As(failure.Err, &collector) && managedResourceTransferNonpass(err, assetEndpoint(failure.Host)) {
|
if ctx.Err() == nil && errors.As(err, &failure) && (failure.ProbeKind == "media" || assetEndpoint(failure.Host) && failure.ProbeKind == managedImageProbeKind(failure.Host)) && errors.As(failure.Err, &collector) && managedResourceTransferNonpass(err, assetEndpoint(failure.Host)) {
|
||||||
@@ -87,24 +96,41 @@ func managedServiceResources(cfg healthConfig, player func(context.Context) ([]b
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
i.resourceErr = &managedResourceFailure{"media", "unknown", err}
|
i.resourceErr = &managedResourceFailure{"media", "unknown", err}
|
||||||
}
|
}
|
||||||
|
for attempt := 0; attempt < 2; attempt++ {
|
||||||
|
origin := "https://www.youtube.com/"
|
||||||
|
if attempt == 1 {
|
||||||
|
if !videoIDPattern.MatchString(cfg.VideoID) {
|
||||||
|
return i, &managedRecoveryFatal{Err: errors.New("invalid video ID")}
|
||||||
|
}
|
||||||
|
origin = "https://www.youtube.com/watch?v=" + cfg.VideoID
|
||||||
|
}
|
||||||
bounded, cancel := context.WithTimeout(ctx, 35*time.Second)
|
bounded, cancel := context.WithTimeout(ctx, 35*time.Second)
|
||||||
defer cancel()
|
|
||||||
var output bytes.Buffer
|
var output bytes.Buffer
|
||||||
out := &playerBoundedWriter{dst: &output, limit: 5 << 20}
|
out := &playerBoundedWriter{dst: &output, limit: 5 << 20}
|
||||||
// Fixed root origin; no redirects, credentials, user URL or diagnostics.
|
// Fixed trusted WWW origin; no redirects, credentials or diagnostics.
|
||||||
err = page(bounded, []string{"--disable", "--silent", "--show-error", "--fail", "--noproxy", "", "--proxy", cfg.SOCKSProxy, "--proto", "=https", "--max-time", "30", "--max-filesize", "5242880", "https://www.youtube.com/"}, out, io.Discard)
|
err = page(bounded, []string{"--disable", "--silent", "--show-error", "--fail", "--noproxy", "", "--proxy", cfg.SOCKSProxy, "--proto", "=https", "--max-time", "30", "--max-filesize", "5242880", origin}, out, io.Discard)
|
||||||
|
requestErr := bounded.Err()
|
||||||
|
cancel()
|
||||||
if ctx.Err() != nil {
|
if ctx.Err() != nil {
|
||||||
return i, managedRecoveryContextError(ctx, err)
|
return i, managedRecoveryContextError(ctx, err)
|
||||||
}
|
}
|
||||||
if err != nil || bounded.Err() != nil || out.exceeded {
|
if err != nil || requestErr != nil || out.exceeded {
|
||||||
cause := errors.Join(err, bounded.Err())
|
cause := errors.Join(err, requestErr)
|
||||||
if cause == nil || out.exceeded {
|
if out.exceeded {
|
||||||
cause = errors.Join(cause, errors.New("fresh root metadata request failed"))
|
cause = errors.Join(cause, errors.New("fresh metadata exceeds limit"))
|
||||||
}
|
}
|
||||||
return i, managedResourceHealthError(ctx, &managedResourceFailure{"avatar", "yt3.ggpht.com", resourceFailure(bounded, cause)})
|
return i, managedResourceHealthError(ctx, &managedResourceFailure{"avatar", "yt3.ggpht.com", resourceFailure(ctx, cause)})
|
||||||
}
|
}
|
||||||
if raw, e := avatarResourceFromPage(output.Bytes()); e == nil {
|
raw, parseErr := avatarResourceFromPage(output.Bytes())
|
||||||
|
if parseErr == nil {
|
||||||
i.images["yt3.ggpht.com"] = raw
|
i.images["yt3.ggpht.com"] = raw
|
||||||
|
return i, nil
|
||||||
|
}
|
||||||
|
// Only missing structural metadata permits one fresh watch request.
|
||||||
|
var missing avatarMetadataError
|
||||||
|
if attempt == 1 || !errors.As(parseErr, &missing) || (missing != "markerAbsent" && missing != "noAllowedAvatar") {
|
||||||
|
return i, managedResourceHealthError(ctx, &managedResourceFailure{"avatar", "yt3.ggpht.com", parseErr})
|
||||||
|
}
|
||||||
}
|
}
|
||||||
return i, nil
|
return i, nil
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,149 @@
|
|||||||
|
package main
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"errors"
|
||||||
|
"io"
|
||||||
|
"os/exec"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestAvatarPageClassification(t *testing.T) {
|
||||||
|
for _, tc := range []struct{ body, code string }{
|
||||||
|
{`<html></html>`, "markerAbsent"},
|
||||||
|
{`var ytInitialData = {broken};`, "jsonInvalid"},
|
||||||
|
{`var ytInitialData = {"url":"https://yt3.ggpht.com.attacker.invalid/avatar"};`, "noAllowedAvatar"},
|
||||||
|
{`var ytInitialData = {"url":"http://yt3.ggpht.com/avatar"};`, "noAllowedAvatar"},
|
||||||
|
{`var ytInitialData = {"url":"https://user@yt3.ggpht.com/avatar"};`, "noAllowedAvatar"},
|
||||||
|
{`var ytInitialData = {"url":"https://yt3.ggpht.com/avatar?private=fixture"};`, "valid"},
|
||||||
|
} {
|
||||||
|
t.Run(tc.code, func(t *testing.T) {
|
||||||
|
raw, err := avatarResourceFromPage([]byte(tc.body))
|
||||||
|
if tc.code == "valid" {
|
||||||
|
if err != nil || raw == "" {
|
||||||
|
t.Fatal("missing validated resource", err)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
var classified avatarMetadataError
|
||||||
|
if raw != "" || !errors.As(err, &classified) || string(classified) != tc.code {
|
||||||
|
t.Fatal("unexpected classification", err)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestManagedAvatarInventoryFallback(t *testing.T) {
|
||||||
|
exit := func(code string) error { return exec.Command("sh", "-c", "exit "+code).Run() }
|
||||||
|
for _, tc := range []struct {
|
||||||
|
name, home, watch, video string
|
||||||
|
pageErr, watchErr error
|
||||||
|
calls int
|
||||||
|
avatar, fatal bool
|
||||||
|
}{
|
||||||
|
{name: "homeNoAvatarWatchHasValidatedURL", home: `var ytInitialData = {};`, watch: `var ytInitialData = {"url":"https://yt3.ggpht.com/fresh?private=fixture"};`, calls: 2, avatar: true},
|
||||||
|
{name: "homeAlreadyValid", home: `var ytInitialData = {"url":"https://yt3.ggpht.com/fresh"};`, calls: 1, avatar: true},
|
||||||
|
{name: "bothMissing", home: `var ytInitialData = {};`, watch: `<html></html>`, calls: 2},
|
||||||
|
{name: "malformedHome", home: `var ytInitialData = {bad};`, calls: 1},
|
||||||
|
{name: "malformedWatch", home: `var ytInitialData = {};`, watch: `var ytInitialData = {bad};`, calls: 2},
|
||||||
|
{name: "watchHTTPMissing", home: `var ytInitialData = {};`, watchErr: exit("22"), calls: 2},
|
||||||
|
{name: "watchTimeout", home: `var ytInitialData = {};`, watchErr: exit("28"), calls: 2},
|
||||||
|
{name: "watchWrongCN", home: `var ytInitialData = {};`, watchErr: exit("60"), calls: 2},
|
||||||
|
{name: "watchMixedGuard", home: `var ytInitialData = {};`, watchErr: errors.Join(exit("28"), errors.New("owned proxy guard failed")), calls: 2, fatal: true},
|
||||||
|
{name: "untrustedWatchURL", watch: `var ytInitialData = {"url":"https://yt3.ggpht.com.evil.invalid/avatar"};`, calls: 2},
|
||||||
|
{name: "invalidVideo", video: "bad&injection", calls: 1, fatal: true},
|
||||||
|
{name: "timeoutNoFallback", pageErr: exit("28"), calls: 1},
|
||||||
|
{name: "wrongCNNoFallback", pageErr: exit("60"), calls: 1},
|
||||||
|
{name: "mixedGuardNoFallback", pageErr: errors.Join(exit("28"), errors.New("owned proxy guard failed")), calls: 1, fatal: true},
|
||||||
|
{name: "oversizedHomeNoFallback", home: strings.Repeat("x", (5<<20)+1), calls: 1, fatal: true},
|
||||||
|
} {
|
||||||
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
cfg := healthConfig{SOCKSProxy: "socks5://127.0.0.1:11080", VideoID: "dQw4w9WgXcQ"}
|
||||||
|
if tc.video != "" {
|
||||||
|
cfg.VideoID = tc.video
|
||||||
|
}
|
||||||
|
calls := 0
|
||||||
|
resources := managedServiceResources(cfg, func(context.Context) ([]byte, error) {
|
||||||
|
return []byte(`{"playabilityStatus":{"status":"OK"},"videoDetails":{"thumbnail":{"thumbnails":[{"url":"https://i.ytimg.com/fresh"}]}},"streamingData":{"formats":[{"url":"https://fresh.googlevideo.com/private","mimeType":"video/mp4"}]}}`), nil
|
||||||
|
}, func(ctx context.Context, args []string, out, diagnostic io.Writer) error {
|
||||||
|
calls++
|
||||||
|
if _, ok := ctx.Deadline(); !ok || diagnostic != io.Discard {
|
||||||
|
t.Fatal("unbounded metadata or diagnostics")
|
||||||
|
}
|
||||||
|
joined := strings.Join(args, " ")
|
||||||
|
if strings.Contains(joined, "private=") || strings.Contains(joined, "--insecure") || strings.Contains(joined, "--location") || !strings.Contains(joined, "--proto =https") || !strings.Contains(joined, "--max-filesize 5242880") || !strings.Contains(joined, "--proxy "+cfg.SOCKSProxy) {
|
||||||
|
t.Fatal("unsafe metadata request")
|
||||||
|
}
|
||||||
|
body, origin := tc.home, "https://www.youtube.com/"
|
||||||
|
if calls == 2 {
|
||||||
|
body, origin = tc.watch, "https://www.youtube.com/watch?v="+cfg.VideoID
|
||||||
|
}
|
||||||
|
if args[len(args)-1] != origin {
|
||||||
|
t.Fatal("unexpected metadata origin")
|
||||||
|
}
|
||||||
|
if tc.pageErr != nil {
|
||||||
|
return tc.pageErr
|
||||||
|
}
|
||||||
|
if calls == 2 && tc.watchErr != nil {
|
||||||
|
return tc.watchErr
|
||||||
|
}
|
||||||
|
_, err := io.WriteString(out, body)
|
||||||
|
return err
|
||||||
|
})
|
||||||
|
i, err := resources(context.Background())
|
||||||
|
var fatal *managedRecoveryFatal
|
||||||
|
if calls != tc.calls || (i.images["yt3.ggpht.com"] != "") != tc.avatar || errors.As(err, &fatal) != tc.fatal {
|
||||||
|
t.Fatal("fallback or guard contract", calls, err)
|
||||||
|
}
|
||||||
|
if (err == nil) != tc.avatar || i.images["i.ytimg.com"] == "" || i.media.url == "" || i.resourceErr != nil {
|
||||||
|
t.Fatal("missing avatar masked or partial inventory lost", err)
|
||||||
|
}
|
||||||
|
if err != nil && strings.Contains(err.Error(), "private") {
|
||||||
|
t.Fatal("private metadata leaked")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestManagedAvatarMetadataHealthClassification(t *testing.T) {
|
||||||
|
for _, code := range []avatarMetadataError{"markerAbsent", "jsonInvalid", "noAllowedAvatar"} {
|
||||||
|
failure := &managedResourceFailure{"avatar", "yt3.ggpht.com", code}
|
||||||
|
err := managedResourceHealthError(context.Background(), failure)
|
||||||
|
var missing *managedResourceFailure
|
||||||
|
if !errors.As(err, &missing) || missing.Err != nil {
|
||||||
|
t.Fatal("known metadata absence not normalized", code)
|
||||||
|
}
|
||||||
|
for _, mixed := range []error{
|
||||||
|
errors.Join(failure, errors.New("owned proxy guard failed")),
|
||||||
|
&managedResourceFailure{"avatar", "yt3.ggpht.com", errors.Join(code, errors.New("collector unavailable"))},
|
||||||
|
&managedResourceFailure{"avatar", "yt3.ggpht.com", errors.Join(code, context.Canceled)},
|
||||||
|
} {
|
||||||
|
err := managedResourceHealthError(context.Background(), mixed)
|
||||||
|
if !errors.As(err, &missing) || missing.Err == nil {
|
||||||
|
t.Fatal("mixed safety failure normalized", code)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
unknown := avatarMetadataError("unknown")
|
||||||
|
err := managedResourceHealthError(context.Background(), &managedResourceFailure{"avatar", "yt3.ggpht.com", unknown})
|
||||||
|
if !errors.Is(err, unknown) {
|
||||||
|
t.Fatal("unknown parser error normalized")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestManagedAvatarInventoryCancellationNoFallback(t *testing.T) {
|
||||||
|
ctx, cancel := context.WithCancel(context.Background())
|
||||||
|
cancel()
|
||||||
|
calls := 0
|
||||||
|
resources := managedServiceResources(healthConfig{VideoID: "dQw4w9WgXcQ"}, func(context.Context) ([]byte, error) {
|
||||||
|
return []byte(`{"playabilityStatus":{"status":"OK"}}`), nil
|
||||||
|
}, func(ctx context.Context, _ []string, _, _ io.Writer) error {
|
||||||
|
calls++
|
||||||
|
return ctx.Err()
|
||||||
|
})
|
||||||
|
i, err := resources(ctx)
|
||||||
|
if calls != 1 || !errors.Is(err, context.Canceled) || i.images["yt3.ggpht.com"] != "" {
|
||||||
|
t.Fatal("cancellation lost or fallback attempted", calls, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -41,10 +41,11 @@ func TestExperimentalProductionProbeCollectorClassification(t *testing.T) {
|
|||||||
}
|
}
|
||||||
knownTimeout := exec.Command("sh", "-c", "exit 28").Run()
|
knownTimeout := exec.Command("sh", "-c", "exit 28").Run()
|
||||||
if strings.HasPrefix(mode, "metadata-") {
|
if strings.HasPrefix(mode, "metadata-") {
|
||||||
d.resources = managedServiceResources(healthConfig{SOCKSProxy: "socks5://127.0.0.1:11080"}, func(context.Context) ([]byte, error) {
|
d.resources = managedServiceResources(healthConfig{SOCKSProxy: "socks5://127.0.0.1:11080", VideoID: "dQw4w9WgXcQ"}, func(context.Context) ([]byte, error) {
|
||||||
return []byte(fmt.Sprintf(`{"playabilityStatus":{"status":"OK"},"streamingData":{"formats":[{"url":%q,"mimeType":"video/mp4"}]}}`, experimentalResourceFixture().url)), nil
|
return []byte(fmt.Sprintf(`{"playabilityStatus":{"status":"OK"},"streamingData":{"formats":[{"url":%q,"mimeType":"video/mp4"}]}}`, experimentalResourceFixture().url)), nil
|
||||||
}, func(_ context.Context, args []string, out, diagnostic io.Writer) error {
|
}, func(_ context.Context, args []string, out, diagnostic io.Writer) error {
|
||||||
if args[len(args)-1] != "https://www.youtube.com/" || diagnostic != io.Discard {
|
origin := args[len(args)-1]
|
||||||
|
if (origin != "https://www.youtube.com/" && !(mode == "metadata-malformed" && origin == "https://www.youtube.com/watch?v=dQw4w9WgXcQ")) || diagnostic != io.Discard {
|
||||||
t.Fatal("metadata origin/diagnostics changed")
|
t.Fatal("metadata origin/diagnostics changed")
|
||||||
}
|
}
|
||||||
switch mode {
|
switch mode {
|
||||||
|
|||||||
@@ -33,6 +33,11 @@ func runResourceCurl(ctx context.Context, args []string, stdin io.Reader, out, d
|
|||||||
|
|
||||||
type resourceError struct{ cause error }
|
type resourceError struct{ cause error }
|
||||||
|
|
||||||
|
// Safe metadata classifications contain neither page content nor resource URLs.
|
||||||
|
type avatarMetadataError string
|
||||||
|
|
||||||
|
func (e avatarMetadataError) Error() string { return string(e) }
|
||||||
|
|
||||||
func (e *resourceError) PublicError() string { return "resource qualification failed" }
|
func (e *resourceError) PublicError() string { return "resource qualification failed" }
|
||||||
func (e *resourceError) Error() string { return e.PublicError() }
|
func (e *resourceError) Error() string { return e.PublicError() }
|
||||||
func (e *resourceError) Unwrap() error { return e.cause }
|
func (e *resourceError) Unwrap() error { return e.cause }
|
||||||
@@ -78,7 +83,7 @@ func avatarResource(metadata []byte) (string, error) {
|
|||||||
}
|
}
|
||||||
var root interface{}
|
var root interface{}
|
||||||
if json.Unmarshal(metadata, &root) != nil {
|
if json.Unmarshal(metadata, &root) != nil {
|
||||||
return "", errors.New("invalid metadata")
|
return "", avatarMetadataError("jsonInvalid")
|
||||||
}
|
}
|
||||||
var visit func(interface{}, int) string
|
var visit func(interface{}, int) string
|
||||||
visit = func(v interface{}, depth int) string {
|
visit = func(v interface{}, depth int) string {
|
||||||
@@ -109,7 +114,7 @@ func avatarResource(metadata []byte) (string, error) {
|
|||||||
if raw := visit(root, 0); raw != "" {
|
if raw := visit(root, 0); raw != "" {
|
||||||
return raw, nil
|
return raw, nil
|
||||||
}
|
}
|
||||||
return "", errors.New("avatar unavailable")
|
return "", avatarMetadataError("noAllowedAvatar")
|
||||||
}
|
}
|
||||||
|
|
||||||
// The WWW page is transient; decode the embedded JSON structurally and never
|
// The WWW page is transient; decode the embedded JSON structurally and never
|
||||||
@@ -118,15 +123,20 @@ func avatarResourceFromPage(page []byte) (string, error) {
|
|||||||
if len(page) > 5<<20 {
|
if len(page) > 5<<20 {
|
||||||
return "", errors.New("page exceeds limit")
|
return "", errors.New("page exceeds limit")
|
||||||
}
|
}
|
||||||
|
found := false
|
||||||
for _, marker := range []string{"var ytInitialData = ", "ytInitialData = ", "window[\"ytInitialData\"] = "} {
|
for _, marker := range []string{"var ytInitialData = ", "ytInitialData = ", "window[\"ytInitialData\"] = "} {
|
||||||
if i := bytes.Index(page, []byte(marker)); i >= 0 {
|
if i := bytes.Index(page, []byte(marker)); i >= 0 {
|
||||||
|
found = true
|
||||||
var object json.RawMessage
|
var object json.RawMessage
|
||||||
if json.NewDecoder(bytes.NewReader(page[i+len(marker):])).Decode(&object) == nil {
|
if json.NewDecoder(bytes.NewReader(page[i+len(marker):])).Decode(&object) == nil {
|
||||||
return avatarResource(object)
|
return avatarResource(object)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return "", errors.New("structural avatar metadata unavailable")
|
if found {
|
||||||
|
return "", avatarMetadataError("jsonInvalid")
|
||||||
|
}
|
||||||
|
return "", avatarMetadataError("markerAbsent")
|
||||||
}
|
}
|
||||||
|
|
||||||
var resourceContentRange = regexp.MustCompile(`^bytes 0-([0-9]+)/([0-9]+)$`)
|
var resourceContentRange = regexp.MustCompile(`^bytes 0-([0-9]+)/([0-9]+)$`)
|
||||||
|
|||||||
Reference in new issue
Block a user