[issue32958] socket module calls with long host names can fail with idna codec error

[Gregory P. Smith]

Gregory P. Smith  added the comment:

ablack: the basic auth username:password@ part of the string is not part of a 
hostname.  What code are you seeing that is trying to send that to a name 
resolver rather than stripping the obviously private info up through the @ sign?

--
nosy: +gregory.p.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32958] socket module calls with long host names can fail with idna codec error

[Ben Darnell]

Ben Darnell  added the comment:

[I'm coming here from https://github.com/tornadoweb/tornado/pull/3010)

UnicodeError is a subclass of ValueError, so I don't see what value that change 
would provide. The thing that's surprising to me is that it's not a 
`socket.herror` (or `gaierror` for socket.getaddrinfo). I guess the docs don't 
formally say that `herror`/`gaierror` is the *only* possible error from these 
functions, but `gaierror` was the only error I was catching so the unexpected 
UnicodeError escaped the layer that was intended to handle it. 

I do think that in the special case of `getaddrinfo` with the `AI_NUMERICHOST` 
flag it should be handled differently: in that mode there is no network access 
necessary and it's reasonable to assume that the only possible error is a 
`gaierror` with `EAI_NONAME`. 

I'd like to at least see better documentation about what errors are possible 
from this family of functions.

--
nosy: +Ben.Darnell

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32958] socket module calls with long host names can fail with idna codec error

[Alex Vandiver]

Alex Vandiver  added the comment:

It seems reasonable to fail on hostnames that are too long -- but it feels like 
the weirdness is that it is categorized as a UnicodeError, and not as, say, a 
ValueError.

Would a re-categorization as ValueError seem like a reasonable adjustment here?

--
nosy: +alexmv

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32958] socket module calls with long host names can fail with idna codec error

[--]

Change by -- :


--
nosy: +midopa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32958] socket module calls with long host names can fail with idna codec error

[Aaron Black]

Aaron Black  added the comment:

joseph.hackman

I don't think that the 63 character limit on a label is the problem 
specifically, merely it's application. 

The crux of my issue was that credentials passed with the url in a basic-authy 
fashion (as some services require) count against the label length. For example, 
this would trigger the error:

h = 
"https://ablack:very_long_api_key_0123456789012345678901234567890123456789012345678901234567890...@www.example.com";

Since the first label would be treated as:
 
"ablack:very_long_api_key_0123456789012345678901234567890123456789012345678901234567890123@www"

My specific issue goes away if any text up to / including an "@" in the first 
label section is not included in the label validation. I don't know off hand if 
that information is supposed to be included per the label in the DNS spec 
though.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32958] socket module calls with long host names can fail with idna codec error

[Joseph Hackman]

Joseph Hackman  added the comment:

According to the DNS standard, hostnames with more than 63 characters per label 
(the sections between .) are not allowed 
[https://tools.ietf.org/html/rfc1035#section-2.3.1].

That said, enforcing that at the codec level might be the wrong choice. I threw 
together a quick patch moving the limits up to 250, and nothing blew up. It's 
unclear what the general usefulness of such a change would be, since DNS 
servers probably couldn't handle those requests anyway.

As for the original issue, if anybody is still doing something like that, could 
they provide a full example URL? I was unable to reproduce on HTTP (failed in a 
different place), or FTP.

--
nosy: +joseph.hackman

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32958] socket module calls with long host names can fail with idna codec error

[Steve Bowman]

Steve Bowman  added the comment:

When will this issue be fixed?  Thanks!

--
nosy: +sdbowman

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32958] socket module calls with long host names can fail with idna codec error

[Ned Deily]

Change by Ned Deily :


--
nosy:  -ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32958] socket module calls with long host names can fail with idna codec error

[R. David Murray]

Change by R. David Murray :


--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32958] socket module calls with long host names can fail with idna codec error

[Matt Eaton]

Matt Eaton  added the comment:

Using Ubuntu 16.04 with the 3.6.0 tag I was also able to reproduce the same 
error reported:

import socket

h = 
"0123456789012345678901234567890123456789012345678901234567890123.example.com"
socket.gethostbyname(h)

Traceback (most recent call last):
  File 
"/home/agnosticdev/Documents/code/python/python-dev/cpython-3_6_0/Lib/encodings/idna.py",
 line 165, in encode
raise UnicodeError("label empty or too long")
UnicodeError: label empty or too long

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "host_test.py", line 8, in 
socket.gethostbyname(h)
UnicodeError: encoding with 'idna' codec failed (UnicodeError: label empty or 
too long)


It looks like the hostname being 64 characters long is the issue in that it 
cannot be encoded.  Thus falling into the UnicodeError being raised in idna.py:
# ASCII name: fast path
labels = result.split(b'.')
for label in labels[:-1]:
if not (0 < len(label) < 64):
raise UnicodeError("label empty or too long")
if len(labels[-1]) >= 64:
raise UnicodeError("label too long")
return result, len(input)

I did some work on this to try and resolve this, but ultimately it was not 
worth committing so I wanted to report my findings.

--
nosy: +agnosticdev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32958] socket module calls with long host names can fail with idna codec error

[Aaron Black]

Aaron Black  added the comment:

Just to be clear, I don't know if the socket needs to support 64 character long 
host name sections, so here's an example url that is at the root of my problem 
that I'm pretty sure it should support:

>>> import socket
>>> h = 
>>> "username:long_api_key0123456789012345678901234567890123456...@www.example.com"
>>> socket.gethostbyname(h)
Traceback (most recent call last):
  File "/Users/ablack/miniconda3/lib/python3.6/encodings/idna.py", line 165, in 
encode
raise UnicodeError("label empty or too long")
UnicodeError: label empty or too long

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "", line 1, in 
UnicodeError: encoding with 'idna' codec failed (UnicodeError: label empty or 
too long)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32958] socket module calls with long host names can fail with idna codec error

[Ned Deily]

Ned Deily  added the comment:

Thanks for the report.  The behavior you see can be further isolated to 
socket.gethostbyname:

>>> import socket
>>> h = 
>>> "0123456789012345678901234567890123456789012345678901234567890123.example.com"
>>> socket.gethostbyname(h)
Traceback (most recent call last):
  File "/usr/lib/python3.6/encodings/idna.py", line 165, in encode
raise UnicodeError("label empty or too long")
UnicodeError: label empty or too long

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "", line 1, in 
UnicodeError: encoding with 'idna' codec failed (UnicodeError: label empty or 
too long)

Other socket module calls accepting host names fail similarly, such as 
getaddrinfo.

--
nosy: +ned.deily
stage:  -> needs patch
title: Urllib proxy_bypass crashes for urls containing long basic auth strings 
-> socket module calls with long host names can fail with idna codec error
type: crash -> 
versions: +Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com