Your message dated Tue, 28 May 2024 19:07:07 -0400 (EDT)
with message-id <[email protected]>
and subject line Re: Bug#1071386: yt-dlp: “501 Tor is not an HTTP Proxy” error
when SOCKS proxying was requested
has caused the Debian Bug report #1071386,
regarding yt-dlp: “501 Tor is not an HTTP Proxy” error when SOCKS proxying was
requested
to be marked as done.
This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.
(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact [email protected]
immediately.)
--
1071386: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=1071386
Debian Bug Tracking System
Contact [email protected] with problems
--- Begin Message ---
Package: yt-dlp
Version: 2023.03.04-1
Severity: normal
Tags: upstream
X-Debbugs-Cc: [email protected]
The app incorrectly interprets a SOCKS proxy as an HTTP proxy. The
host machine has both kinds of proxies configured for Tor as follows:
* SOCKS proxy listening on port 9050
* HTTP proxy listening on port 8118
This is the result of an attempt to use the socks5h:// scheme:
===8<----------------------------------------
$ yt-dlp --proxy socks5h://127.0.0.1:9050 --max-filesize 1M
http://ng27owmagn5amdm7l5s3rsqxwscl5ynppnis5dqcasogkyxcfqn7psid.onion/watch?v=-Zj50DmBFp0
[youtube] Extracting URL:
http://ng27owmagn5amdm7l5s3rsqxwscl5ynppnis5dqcasogkyxcfqn7psid.onion/watch?v=-Zj50DmBFp0
[youtube] -Zj50DmBFp0: Downloading webpage
WARNING: [youtube] Unable to download webpage: <urlopen error Tunnel
connection failed: 501 Tor is not an HTTP Proxy>
[youtube] -Zj50DmBFp0: Downloading android player API JSON
WARNING: [youtube] <urlopen error Tunnel connection failed: 501 Tor is not an
HTTP Proxy>. Retrying (1/3)...
[youtube] -Zj50DmBFp0: Downloading android player API JSON
WARNING: [youtube] <urlopen error Tunnel connection failed: 501 Tor is not an
HTTP Proxy>. Retrying (2/3)...
[youtube] -Zj50DmBFp0: Downloading android player API JSON
WARNING: [youtube] <urlopen error Tunnel connection failed: 501 Tor is not an
HTTP Proxy>. Retrying (3/3)...
[youtube] -Zj50DmBFp0: Downloading android player API JSON
[youtube] -Zj50DmBFp0: Downloading iframe API JS
WARNING: [youtube] Unable to download webpage: <urlopen error Tunnel
connection failed: 501 Tor is not an HTTP Proxy>
[youtube] -Zj50DmBFp0: Downloading web player API JSON
WARNING: [youtube] <urlopen error Tunnel connection failed: 501 Tor is not an
HTTP Proxy>. Retrying (1/3)...
[youtube] -Zj50DmBFp0: Downloading web player API JSON
WARNING: [youtube] <urlopen error Tunnel connection failed: 501 Tor is not an
HTTP Proxy>. Retrying (2/3)...
[youtube] -Zj50DmBFp0: Downloading web player API JSON
WARNING: [youtube] <urlopen error Tunnel connection failed: 501 Tor is not an
HTTP Proxy>. Retrying (3/3)...
[youtube] -Zj50DmBFp0: Downloading web player API JSON
WARNING: [youtube] Unable to download API page: <urlopen error Tunnel
connection failed: 501 Tor is not an HTTP Proxy> (caused by
URLError(OSError('Tunnel connection failed: 501 Tor is not an HTTP Proxy')))
ERROR: [youtube] -Zj50DmBFp0: Unable to download API page: <urlopen error
Tunnel connection failed: 501 Tor is not an HTTP Proxy> (caused by
URLError(OSError('Tunnel connection failed: 501 Tor is not an HTTP Proxy')))
===8<----------------------------------------
The error message itself is erroneous nonsense. The user never claimed
Tor to *be* an HTTP proxy. Proxies are wholly independent of Tor and
should be treated genericly. yt-dlp is trying to be smart in working
out that an onion URL implies Tor, but really all it needs to know is
that the “h” in SOCKS5h means to make the proxy handle the DNS
resolution.
Workaround:
Supplying --proxy http://127.0.0.1:8118 works for me because I
happen to have an HTTP proxy configured as well. But the man page
explicitly states that both socks and http proxies are supported.
-- System Information:
Debian Release: 12.5
APT prefers stable-updates
APT policy: (990, 'stable-updates'), (990, 'stable-security'), (990,
'stable'), (500, 'oldstable')
Architecture: amd64 (x86_64)
Foreign Architectures: i386
Kernel: Linux 5.10.0-28-amd64 (SMP w/2 CPU threads)
Kernel taint flags: TAINT_OOT_MODULE, TAINT_UNSIGNED_MODULE
Locale: LANG=en_US.UTF-8, LC_CTYPE=en_US.UTF-8 (charmap=UTF-8), LANGUAGE not set
Shell: /bin/sh linked to /usr/bin/dash
Init: systemd (via /run/systemd/system)
LSM: AppArmor: enabled
Versions of packages yt-dlp depends on:
ii python3 3.11.2-1+b1
ii python3-brotli 1.0.9-2+b6
ii python3-certifi 2022.9.24-1
ii python3-mutagen 1.46.0-1
ii python3-pkg-resources 66.1.1-1
ii python3-pycryptodome 3.11.0+dfsg1-4
ii python3-websockets 10.4-1
Versions of packages yt-dlp recommends:
ii aria2 1.36.0-1
ii ca-certificates 20230311
ii curl 7.88.1-10+deb12u5
ii ffmpeg 7:5.1.4-0+deb12u1
ii python3-pyxattr 0.8.1-1
ii rtmpdump 2.4+20151223.gitfa8646d.1-2+b2
ii wget 1.21.3-1+b2
Versions of packages yt-dlp suggests:
pn libfribidi-bin | bidiv <none>
ii mpv 0.35.1-4
pn phantomjs <none>
-- no debconf information
--- End Message ---
--- Begin Message ---
Package: yt-dlp
Version: 2023.09.24-1
Howdy,
On Sat, 18 May 2024, Manny wrote:
The app incorrectly interprets a SOCKS proxy as an HTTP proxy. The
host machine has both kinds of proxies configured for Tor as follows:
There were a few socks related fixes in [1], that might have been fixed in
there as well. I can tell you that the version of yt-dlp from
stable-backports works fine with Tor's socks proxy though.
[1]:
https://github.com/yt-dlp/yt-dlp/commit/20fbbd9249a2f26c7ae579bde5ba5d69aa8fac69
* SOCKS proxy listening on port 9050
* HTTP proxy listening on port 8118
This is the result of an attempt to use the socks5h:// scheme:
===8<----------------------------------------
$ yt-dlp --proxy socks5h://127.0.0.1:9050 --max-filesize 1M
http://ng27owmagn5amdm7l5s3rsqxwscl5ynppnis5dqcasogkyxcfqn7psid.onion/watch?v=-Zj50DmBFp0
[youtube] Extracting URL:
http://ng27owmagn5amdm7l5s3rsqxwscl5ynppnis5dqcasogkyxcfqn7psid.onion/watch?v=-Zj50DmBFp0
[youtube] -Zj50DmBFp0: Downloading webpage
WARNING: [youtube] Unable to download webpage: <urlopen error Tunnel connection
failed: 501 Tor is not an HTTP Proxy>
The error message itself is erroneous nonsense. The user never claimed
Tor to *be* an HTTP proxy. Proxies are wholly independent of Tor and
should be treated genericly. yt-dlp is trying to be smart in working
out that an onion URL implies Tor, but really all it needs to know is
that the “h” in SOCKS5h means to make the proxy handle the DNS
resolution.
That's not exactly accurate since the message is itself from Tor[2] and
not yt-dlp. The latter most likely missed a case and just fell back to
http proxies.
[2]:
https://gitlab.torproject.org/tpo/core/tor/-/blob/main/src/core/proto/proto_socks.c?ref_type=heads#L914
Workaround:
Supplying --proxy http://127.0.0.1:8118 works for me because I
happen to have an HTTP proxy configured as well. But the man page
explicitly states that both socks and http proxies are supported.
I find enough applications don't have socks support that using tinyproxy
and/or torsocks is still mandatory at this point. Either that workaround
or pulling from -backports will work here as well.
~Unit 193
Unit193 @ Libera
Unit193 @ OFTC
--- End Message ---
