[issue213721] urllib handles proxy badly

[admin]

Change by admin :


___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue219822] urllib doesn't like unicode

[admin]

Change by admin :


___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue532007] urllib ftp broken if Win32 proxy

[admin]

Change by admin :


--
github: None -> 36284

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue401529] urllib and redirects

[admin]

Change by admin :


--
github: None -> 33124

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue420753] Patch for bug #420725 urllib MIME header

[admin]

Change by admin :


--
github: None -> 34450

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue535285] urllib, fragment identifiers and 404s

[admin]

Change by admin :


--
github: None -> 36332

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue403640] incomplete proxy handling in URLLIB

[admin]

Change by admin :


--
github: None -> 33859

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue402364] Fix for119822: Allow Unicode in urllib

[admin]

Change by admin :


--
github: None -> 33472

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue212634] urllib doesn't look at self.version as documented

[admin]

Change by admin :


___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue525945] urllib: Defering open call for file urls

[admin]

Change by admin :


--
github: None -> 36203

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue511073] urllib problems

[admin]

Change by admin :


--
github: None -> 36005

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue495693] urllib doesn't support passive FTP

[admin]

Change by admin :


--
github: None -> 35810

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue486247] ftplib/URLLib time outs on transfers

[admin]

Change by admin :


--
github: None -> 35606

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue463286] Problem in urllib. No strip attribute

[admin]

Change by admin :


--
github: None -> 35209

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue214427] bug with urlencode method from urllib

[admin]

Change by admin :


--
github: None -> 33112

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue401530] urllib and redirects

[admin]

Change by admin :


--
github: None -> 33125

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue445815] urllib doesn't handle proxy exceptions

[admin]

Change by admin :


--
github: None -> 34857

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue426866] urllib and socket fail with MS proxy

[admin]

Change by admin :


--
github: None -> 34538

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue419459] urllib adds extra CRLF in posted data

[admin]

Change by admin :


--
github: None -> 34426

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue432501] Problem with urllib and proxies / Win32

[admin]

Change by admin :


--
github: None -> 34618

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue426504] urllib fail with MS proxy

[admin]

Change by admin :


--
github: None -> 34531

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue423721] failure in urllib exception handling

[admin]

Change by admin :


--
github: None -> 34498

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue420725] urllib: MIME header for local files

[admin]

Change by admin :


--
github: None -> 34449

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue403236] urllib: Encode special characters in user and password

[admin]

Change by admin :


--
github: None -> 33717

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue232010] urllib and httplib fails to open url

[admin]

Change by admin :


--
github: None -> 33913

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue402363] Allow Unicode in urllib

[admin]

Change by admin :


--
github: None -> 33471

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue402453] Modified urlencode in urllib to accept more types

[admin]

Change by admin :


--
github: None -> 33495

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue219707] urllib failure when return code not 200

[admin]

Change by admin :


___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue219123] Use of ftplib or urllib causes page fault on exit

[admin]

Change by admin :


___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue214427] bug with urlencode method from urllib

[admin]

Change by admin :


___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue214256] Python 2.0b1, Win2K - urllib failure

[admin]

Change by admin :


___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue401529] urllib and redirects

[admin]

Change by admin :


___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue401530] urllib and redirects

[admin]

Change by admin :


___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue400837] urllib patch to simplify POST form query

[admin]

Change by admin :


___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue400873] Use registry to find proxies for urllib on Win32

[admin]

Change by admin :


___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue402453] Modified urlencode in urllib to accept more types

[admin]

Change by admin :


--
github: None -> 33495

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue402364] Fix for119822: Allow Unicode in urllib

[admin]

Change by admin :


--
github: None -> 33472

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue402363] Allow Unicode in urllib

[admin]

Change by admin :


--
github: None -> 33471

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue219707] urllib failure when return code not 200

[admin]

Change by admin :


--
github: None -> 33411

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue219123] Use of ftplib or urllib causes page fault on exit

[admin]

Change by admin :


--
github: None -> 33400

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue219822] urllib doesn't like unicode

[admin]

Change by admin :


--
github: None -> 33419

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue213721] urllib handles proxy badly

[admin]

Change by admin :


--
github: None -> 33049

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue214256] Python 2.0b1, Win2K - urllib failure

[admin]

Change by admin :


--
github: None -> 33092

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue212634] urllib doesn't look at self.version as documented

[admin]

Change by admin :


--
github: None -> 32978

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue400873] Use registry to find proxies for urllib on Win32

[admin]

Change by admin :


--
github: None -> 32616

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue400837] urllib patch to simplify POST form query

[admin]

Change by admin :


--
github: None -> 32597

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46517] Review exception handling in urllib

[Irit Katriel]

Irit Katriel  added the comment:

> The author probably also wants the TypeError initially raised from the 
> "len(query)" and "query[0]" operations to get the same "not a valid . . ." 
> message.

I see.


I didn’t realise it’s deprecated, I guess we’ll leave it alone then. Thanks.

--
resolution:  -> wont fix
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46517] Review exception handling in urllib

[Martin Panter]

Martin Panter  added the comment:

The linked code is for urllib.parse.urlencode, looking something like

try:
if len(query) and not isinstance(query[0], tuple):
raise TypeError
except TypeError:
ty, va, tb = sys.exc_info()
raise TypeError("not a valid non-string sequence "
"or mapping object").with_traceback(tb)

I guess it raises twice so that the error message is not duplicated in the 
code. The author probably also wants the TypeError initially raised from the 
"len(query)" and "query[0]" operations to get the same "not a valid . . ." 
message.

Regarding the OSError, originally it was catching socket.error and raising 
IOError. I guess someone only wanted the caller to have catch IOError and not 
need to import the socket module. Later these exception types became aliases of 
each other.

Anyway, the URLopener class is documented as deprecated, so is it really worth 
changing anything in that?

--
nosy: +martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46517] Review exception handling in urllib

[Irit Katriel]

Irit Katriel  added the comment:

Note that test.support has special handling for urllib's nested exception 
structure:

https://github.com/python/cpython/blob/3.10/Lib/test/support/socket_helper.py#L250

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46517] Review exception handling in urllib

[Irit Katriel]

Change by Irit Katriel :


--
title: Review exception handling in urllib.parse -> Review exception handling 
in urllib

___
Python tracker 
<https://bugs.python.org/issue46517>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34629] Python3 regression for urllib(2).urlopen(...).fp for chunked http responses

[Irit Katriel]

Change by Irit Katriel :


--
nosy: +orsenthil

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20559] urllib/http fail to sanitize a non-ascii url

[STINNER Victor]

Change by STINNER Victor :


--
nosy:  -vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20559] urllib/http fail to sanitize a non-ascii url

[Irit Katriel]

Irit Katriel  added the comment:

Reproduced on 3.11.

--
nosy: +iritkatriel
versions: +Python 3.10, Python 3.11, Python 3.9 -Python 3.3

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue4733] Add a "decode to declared encoding" version of urlopen to urllib

[Daniel Diniz]

Daniel Diniz  added the comment:

As Victor notes, this is a controversial issue. And I'll add that the need for 
this feature seems not to have been brought up up in over a decade. So I'm 
closing this.

--
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36338] urlparse of urllib returns wrong hostname

[STINNER Victor]

Change by STINNER Victor :


--
nosy:  -vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36338] urlparse of urllib returns wrong hostname

[Irit Katriel]

Irit Katriel  added the comment:

It produces a deprecation warning on 3.11, but still does the same.

>>> urlparse('http://benign.com\[attacker.com]').hostname
:1: DeprecationWarning: invalid escape sequence '\['
'attacker.com'

--
nosy: +iritkatriel
versions: +Python 3.10, Python 3.11 -Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6471] errno and strerror attributes incorrectly set on socket errors wrapped by urllib

[Irit Katriel]

Irit Katriel  added the comment:

Reproduced on 3.11:

>>> from urllib.request import urlopen
>>> try:
... urlopen('http://www.pythonfoobarbaz.org')
... except Exception as exc:
... err = exc
... print('err:', err)
... print('repr(err):', repr(err))
... print('err.errno:', err.errno)
... print('err.strerror:', err.strerror)
... print('err.reason:', err.reason)
... print('err.reason.errno:', err.reason.errno)
... print('err.reason.strerror:', err.reason.strerror)
... 
err: 
repr(err): URLError(gaierror(8, 'nodename nor servname provided, or not known'))
err.errno: None
err.strerror: None
err.reason: [Errno 8] nodename nor servname provided, or not known
err.reason.errno: 8
err.reason.strerror: nodename nor servname provided, or not known

--
nosy: +iritkatriel
versions: +Python 3.10, Python 3.11, Python 3.9 -Python 2.6, Python 2.7, Python 
3.1, Python 3.2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45795] urllib http client vulnerable to DOS attack

[Senthil Kumaran]

Senthil Kumaran  added the comment:

Hi Muhammad, 

I haven't gotten to this. urllib doesn't maintain a client state during 
multiple request / response.

The code is available here 
https://github.com/python/cpython/tree/main/Lib/urllib

--

___
Python tracker 
<https://bugs.python.org/issue45795>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45795] urllib http client vulnerable to DOS attack

[Muhammad Farhan]

Muhammad Farhan  added the comment:

Hi,
Hope all of you are doing good. Looks like you guys are not interested in this 
issue. Can you please provide me the source code for yhe urllib, I will fix it 
myself

--

___
Python tracker 
<https://bugs.python.org/issue45795>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45795] urllib http client vulnerable to DOS attack

[Muhammad Farhan]

Muhammad Farhan  added the comment:

Maximum time in seconds that you allow the whole operation to take. This is 
useful for preventing your batch jobs from hanging for hours due to slow 
networks or links going down. Since 7.32.0, this option accepts decimal values, 
but the actual timeout will decrease in accuracy as the specified timeout 
increases in decimal precision.

If this option is used several times, the last one will be used.

Examples:

 curl --max-time 10 https://example.com
 curl --max-time 2.92 https://example.com

Ref: 
https://curl.se/docs/manpage.html#-m

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45795] urllib http client vulnerable to DOS attack

[Christian Heimes]

Christian Heimes  added the comment:

Please don't post screenshots. Screenshots are neither accessible nor 
searchable. It's better to link to documentation and copy the relevant 
sentences here.

--
nosy: +christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45795] urllib http client vulnerable to DOS attack

[Muhammad Farhan]

Muhammad Farhan  added the comment:

So, the idea is to make timeout for the whole operation and it should not reset 
in any case.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45795] urllib http client vulnerable to DOS attack

[Muhammad Farhan]

Muhammad Farhan  added the comment:

See the max_time.png and curl.png

--
Added file: https://bugs.python.org/file50449/curl.png

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45795] urllib http client vulnerable to DOS attack

[Muhammad Farhan]

Muhammad Farhan  added the comment:

Yes, other clients like curl does not reset the timeout

See the attached screenshots for references.

--
Added file: https://bugs.python.org/file50448/max_time.png

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45795] urllib http client vulnerable to DOS attack

[Senthil Kumaran]

Senthil Kumaran  added the comment:

> Timeout value should not be reset after client receives a data(bytes), 
> because it can easily be abused to achieve DOS.

Interesting. I looked the server example.

Does clients like curl have something like this too?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45795] urllib http client vulnerable to DOS attack

[Ned Deily]

Change by Ned Deily :


--
nosy: +orsenthil

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45795] urllib http client vulnerable to DOS attack

[Muhammad Farhan]

Muhammad Farhan  added the comment:

Is any one going to respond?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45795] urllib http client vulnerable to DOS attack

[Muhammad Farhan]

New submission from Muhammad Farhan :

Hi, During my recent tests I have discovered that the urllib http client 
(urllib.request.urlopen()) is vulnerable to DOS attack using a simple but 
effective trick.

I am attaching a file named server.py download it and run it using latest 
version of python. After running it execute the following python code in python 
interactive mode. (python -i)

import urllib.request
request = urllib.request.Request('http://127.0.0.1:1338')
response = urllib.request.urlopen(req, timeout=1)

DOS limit: We can achieve DOS for unlimited time.
 
How to fix?
Implement a good logic for timeout in urllib.request.urlopen(url, timeout). 
Timeout value should not be reset after client receives a data(bytes), because 
it can easily be abused to achieve DOS.

--
components: Library (Lib)
files: server.py
messages: 406220
nosy: haqsek2
priority: normal
severity: normal
status: open
title: urllib http client vulnerable to DOS attack
type: security
versions: Python 3.10
Added file: https://bugs.python.org/file50436/server.py

___
Python tracker 
<https://bugs.python.org/issue45795>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45466] Simple curl/wget-like download functionality in urllib (like http offers server)

[Tom Pohl]

Change by Tom Pohl :


--
nosy:  -tom.pohl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45466] Simple curl/wget-like download functionality in urllib (like http offers server)

[STINNER Victor]

STINNER Victor  added the comment:

I don't think that urllib.request is a great library. There are better 
replacements like https://www.python-httpx.org/ or 
https://urllib3.readthedocs.io/

urllib.request API is not great and it doesn't support HTTP2 (nor HTTP3).

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45466] Simple curl/wget-like download functionality in urllib (like http offers server)

[Eric V. Smith]

Eric V. Smith  added the comment:

The core devs have decided to reject this. Basically for a few reasons:

- the possibility of introducing security vulnerabilities
- the ongoing maintenance burden

--
nosy: +eric.smith
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45466] Simple curl/wget-like download functionality in urllib (like http offers server)

[Raymond Hettinger]

Raymond Hettinger  added the comment:

> If curl/wget are available, great, but often slim images 
> don't offer that.

I concur with Christian.  For the most part, the standard library aims to be a 
collection of resources helpful for building applications like curl and wget.  
The applications themselves should live on the Python Package Index (PyPI).  
Also, as Christian points out, this is a non-trivial project with a large scope.


> The urllib could provide a very simple download functionality
> (like http offers a simple server):
>
>from urllib.request import urlopen
>data = urlopen('https://.../install-poetry.py').read()
># print or save data

This is a simpler request and is within our scope; however, I not sure how it 
differs from the existing functionality in urllib.request.urlretrieve().  Can 
you clarify what extra functionality is being requested?

--
nosy: +rhettinger

___
Python tracker 
<https://bugs.python.org/issue45466>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45466] Simple curl/wget-like download functionality in urllib (like http offers server)

[Christian Heimes]

Christian Heimes  added the comment:

I fear that you underestimate both the complexity of this feature request and 
the amount of demand from users this feature is going to create. Once we start 
to offer a command line client for urllib, users **will** ask for more options, 
e.g. POST requests, custom headers, various TLS/SSL options and so on. There is 
a ton of hidden complexity in a download tool. curl has over 240 (!) command 
line options for a reason.

--
nosy: +christian.heimes

___
Python tracker 
<https://bugs.python.org/issue45466>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45466] Simple curl/wget-like download functionality in urllib (like http offers server)

[Tom Pohl]

Tom Pohl  added the comment:

Thanks, Terry, for the hint.

The idea got some support on python-ideas, so I thought it is worthwhile to do 
a PR. As a first-time contributor, I now have to wait for approval for the 
pipeline to run...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45466] Simple curl/wget-like download functionality in urllib (like http offers server)

[Terry J. Reedy]

Change by Terry J. Reedy :


--
nosy:  -terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45466] Simple curl/wget-like download functionality in urllib (like http offers server)

[Roundup Robot]

Change by Roundup Robot :


--
keywords: +patch
nosy: +python-dev
nosy_count: 2.0 -> 3.0
pull_requests: +27479
stage: test needed -> patch review
pull_request: https://github.com/python/cpython/pull/29217

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45466] Simple curl/wget-like download functionality in urllib (like http offers server)

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Posting the idea to python-ideas list might generate interest and ideas.

--
nosy: +terry.reedy
stage:  -> test needed
versions: +Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45466] Simple curl/wget-like download functionality in urllib (like http offers server)

[Tom Pohl]

New submission from Tom Pohl :

In the context of building Docker images, it is often required to download 
stuff. If curl/wget are available, great, but often slim images don't offer 
that.

The urllib could provide a very simple download functionality (like http offers 
a simple server):

from urllib.request import urlopen
data = urlopen('https://.../install-poetry.py').read()
# print or save data

If there's some interest, I could open a PR.

--
components: Library (Lib)
messages: 403888
nosy: tom.pohl
priority: normal
severity: normal
status: open
title: Simple curl/wget-like download functionality in urllib (like http offers 
server)
type: enhancement

___
Python tracker 
<https://bugs.python.org/issue45466>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44022] CVE-2021-3737: urllib http client possible infinite loop on a 100 Continue response

[STINNER Victor]

STINNER Victor  added the comment:

I'm not sure why the fix in the main branch was not listed here:

commit 47895e31b6f626bc6ce47d175fe9d43c1098909d
Author: Gen Xu 
Date:   Wed May 5 15:42:41 2021 -0700

bpo-44022: Fix http client infinite line reading (DoS) after a HTTP 100 
Continue (GH-25916)

Fixes http.client potential denial of service where it could get stuck 
reading lines from a malicious server after a 100 Continue response.

Co-authored-by: Gregory P. Smith 

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44022] CVE-2021-3737: urllib http client possible infinite loop on a 100 Continue response

[STINNER Victor]

STINNER Victor  added the comment:

I created 
https://python-security.readthedocs.io/vuln/urllib-100-continue-loop.html to 
track the issue.

--

___
Python tracker 
<https://bugs.python.org/issue44022>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44022] CVE-2021-3737: urllib http client possible infinite loop on a 100 Continue response

[STINNER Victor]

STINNER Victor  added the comment:

Matej Cepl: "Is there a CVE for this?"

Yes, CVE-2021-3737 was assigned to this issue.

* https://access.redhat.com/security/cve/CVE-2021-3737
* https://bugzilla.redhat.com/show_bug.cgi?id=1995162

--
nosy: +vstinner
title: urllib http client possible infinite loop on a 100 Continue response -> 
CVE-2021-3737: urllib http client possible infinite loop on a 100 Continue 
response

___
Python tracker 
<https://bugs.python.org/issue44022>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44022] urllib http client possible infinite loop on a 100 Continue response

[Matej Cepl]

Matej Cepl  added the comment:

Is there a CVE for this?

--
nosy: +mcepl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33003] [doc] urllib: Document parse_http_list

[Irit Katriel]

Change by Irit Katriel :


--
title: urllib: Document parse_http_list -> [doc] urllib: Document 
parse_http_list
versions: +Python 3.10, Python 3.11, Python 3.9 -Python 2.7, Python 3.4, Python 
3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 
<https://bugs.python.org/issue33003>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44022] urllib http client possible infinite loop on a 100 Continue response

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset 0389426fa4af4dfc8b1d7f3f291932d928392d8b by Miss Islington (bot) 
in branch '3.8':
bpo-44022: Improve the regression test. (GH-26503) (#26506)
https://github.com/python/cpython/commit/0389426fa4af4dfc8b1d7f3f291932d928392d8b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44022] urllib http client possible infinite loop on a 100 Continue response

[miss-islington]

miss-islington  added the comment:


New changeset 7ac7a0c0f03c60934bc924ee144db170a0e0161f by Sergey Fedoseev in 
branch 'main':
bpo-44022: Fix Sphinx role in NEWS entry (GH-27033)
https://github.com/python/cpython/commit/7ac7a0c0f03c60934bc924ee144db170a0e0161f


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44022] urllib http client possible infinite loop on a 100 Continue response

[Sergey Fedoseev]

Change by Sergey Fedoseev :


--
nosy: +sir-sigurd
nosy_count: 8.0 -> 9.0
pull_requests: +25593
pull_request: https://github.com/python/cpython/pull/27033

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44520] In Lib/urllib/parse.py quote_from_bytes, exception is thrown if bs = None

[Eric V. Smith]

Eric V. Smith  added the comment:

I agree this should be closed (and I'm closing it). I don't see any reason why 
this function should do something other than raise TypeError if given bs=None. 
If you want that behavior, write a small wrapper function.

The "if not bs" check appears to be an optimization for the case of zero-length 
input. Hopefully the code would continue to work without that test (or instead 
testing for len(bs)==0), but in my opinion it's not worth the risk of removing 
or changing it.

--
stage:  -> resolved
status: pending -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44520] In Lib/urllib/parse.py quote_from_bytes, exception is thrown if bs = None

[Irit Katriel]

Irit Katriel  added the comment:

I meant:

... it will be true for bs which is an empty bytes().


You are thinking of b'' and None as if they are the same thing. They are not.  
If this was a check for None it would be "if bs is None" and not "if not bs".

--
status: pending -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44520] In Lib/urllib/parse.py quote_from_bytes, exception is thrown if bs = None

[Irit Katriel]

Change by Irit Katriel :


--
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44520] In Lib/urllib/parse.py quote_from_bytes, exception is thrown if bs = None

[Irit Katriel]

Change by Irit Katriel :


--
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44520] In Lib/urllib/parse.py quote_from_bytes, exception is thrown if bs = None

[Irit Katriel]

Irit Katriel  added the comment:

There is still a point in the "if not bs:" check, it will be true for bs which 
is an empty string.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44520] In Lib/urllib/parse.py quote_from_bytes, exception is thrown if bs = None

[Олег Масло]

Олег Масло  added the comment:

If you pass None to the quote_from_bytes function, then there is no point in 
the "if not bs" check, because it won't even reach it. 

This function is not with dynamic behavior, which violates python concepts. If 
you pass a string instead of bytes, it will throw a TypeError exception, it's 
ok. But if for some reason you need to pass None, and this happens, then the 
function does not behave as expected.

Why even check that bs is not None, if this can never be? And if it does, there 
will always be a TypeError exception.

--
status: pending -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44520] In Lib/urllib/parse.py quote_from_bytes, exception is thrown if bs = None

[Irit Katriel]

Irit Katriel  added the comment:

The documentation states that this function accepts bytes:

https://docs.python.org/3/library/urllib.parse.html#urllib.parse.quote_from_bytes

None is not of type bytes, so raising a TypeError is not unreasonable. It would 
certainly be wrong to return any string. It could have returned None but that 
doesn't make a massive usability difference and it's not worth changing now.

I propose to close this as 'not a bug'.

--
resolution:  -> not a bug
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44520] In Lib/urllib/parse.py quote_from_bytes, exception is thrown if bs = None

[Irit Katriel]

Irit Katriel  added the comment:

As I wrote on the PR, and again on this issue, a PR that makes a behavior 
change (like this one) is not complete without a unit test (which fails before 
the change and passes after the change, and can show what the bug being fixed 
is).  You could add such a test.

In this case, it is also not clear that the current behavior is a bug at all, 
and if it is then what the fix should be (you propose to map None to ''. But 
why not 'None'? Why not 'Mary had a little lamb'?  I suggested to return None 
rather than some arbitrary string). You could explain why you think it's a bug 
and why you think '' is the correct return value.

Once you do write the test and there is consensus that it is a bug and we agree 
what the fix should be, it should be some core dev's top priority to review and 
merge the PR (as opposed to reviewing and merging another PR). So you could 
push it forward by explaining why this bug is important.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44520] In Lib/urllib/parse.py quote_from_bytes, exception is thrown if bs = None

[Олег Масло]

Олег Масло  added the comment:

What are the next actions? Do I need to do something or are we waiting for 
something?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34629] Python3 regression for urllib(2).urlopen(...).fp for chunked http responses

[Jonathan Schweder]

Jonathan Schweder  added the comment:

Hello @tkruse, I have made some research and found that when using the Chunked 
transfer encoding [1], each chunk is preceded by its size in bytes, something 
that really happen if you check the content of one downloaded file from the 
example you provided [2]. So far, I would say that this is not a bug, it is 
just how the transfer encoding works.

[1]: https://en.wikipedia.org/wiki/Chunked_transfer_encoding
[2]: https://gist.github.com/jaswdr/95b2adc519d986c00b17f6572d470f2a

--
nosy: +jaswdr

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44520] In Lib/urllib/parse.py quote_from_bytes, exception is thrown if bs = None

[Irit Katriel]

Irit Katriel  added the comment:

I don’t know if it needs to change, but if anything I would map None to None 
and not to ‘’.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44520] In Lib/urllib/parse.py quote_from_bytes, exception is thrown if bs = None

[Олег Масло]

Олег Масло  added the comment:

That is, all the libraries already created need to follow your advice? :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44520] In Lib/urllib/parse.py quote_from_bytes, exception is thrown if bs = None

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

If you want to interpret None as an empty string, you can just write 
quote_from_bytes(data or b'').

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue44520] In Lib/urllib/parse.py quote_from_bytes, exception is thrown if bs = None

[Eric V. Smith]

Eric V. Smith  added the comment:

Clarifying the title.

--
nosy: +eric.smith
title: exception is thrown if  bs = None -> In Lib/urllib/parse.py 
quote_from_bytes, exception is thrown if  bs = None

___
Python tracker 
<https://bugs.python.org/issue44520>
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com