[issue27612] socket.gethostbyname resolving octal IP addresses incorrectly

[Eryk Sun]

Eryk Sun  added the comment:

Update from my previous comment in 2016: in Python 3.7+, the socket module's 
setipaddr() function calls Winsock inet_pton() instead of inet_addr(), and 
falls back on getaddrinfo(). Neither supports octal addresses. At least using 
octal fails instead of mistakenly parsing as decimal.

--
versions: +Python 3.10, Python 3.8, Python 3.9 -Python 2.7, Python 3.3, Python 
3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27612] socket.gethostbyname resolving octal IP addresses incorrectly

[R. David Murray]

R. David Murray added the comment:

koobs' results are also interesting, since they indicate that *something* 
changed on the python side that affected this for freebsd.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27612] socket.gethostbyname resolving octal IP addresses incorrectly

[R. David Murray]

R. David Murray added the comment:

There's also the fact that Eryk pointed out that there are different ways to 
implement this on Windows, so there might be something we want to "fix" there.  
It seems like we're not consistent in how we handle addresses in the various 
socket module functions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27612] socket.gethostbyname resolving octal IP addresses incorrectly

[STINNER Victor]

STINNER Victor added the comment:

> I didn't, but an attacker leveraged this to bypass security.

Ah, that's a real use case. Can you please rephrase the issue title to make it 
more explicit?

Because in this issue, it's not obvious to me if octal addressses must be 
accepted on all platforms, or rejected on all platforms.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27612] socket.gethostbyname resolving octal IP addresses incorrectly

[Matt Robenolt]

Matt Robenolt added the comment:

> Why do you need octal addresses? What is your use case? :-p

I didn't, but an attacker leveraged this to bypass security. We had checks 
against `127.0.0.1`, but this resolved to `177.0.0.1` incorrectly, bypassing 
the check. We were using `socket.gethostbyname` which yielded this.

See https://github.com/getsentry/sentry/pull/3787 for a little bit more context.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27612] socket.gethostbyname resolving octal IP addresses incorrectly

[STINNER Victor]

STINNER Victor added the comment:

> However, if someone wants to investigate further and finds a fix, we will 
> evaluate it.

IMHO the best fix is to document that the exact behaviour depends on the 
platform, and that only IPv4 decimal and IPv6 hexadecimal are portable. Corner 
cases like IPv4 octal addresses are not portable, you should write your own 
parser.

Note: I checked ipaddress, it doesn't seem to support the funny octal addresses 
format.

Why do you need octal addresses? What is your use case? :-p

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27612] socket.gethostbyname resolving octal IP addresses incorrectly

[STINNER Victor]

STINNER Victor added the comment:

I don't understand the point of the issue. Is it a documentation issue?

Python doesn't parse anything: it's a thin wrapper on top of the standard C 
library. If you want to complain, report the issue to the maintainers of your C 
library ;-)

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27612] socket.gethostbyname resolving octal IP addresses incorrectly

[Ronald Oussoren]

Ronald Oussoren added the comment:

For what it is worth: the relevant standard says that octal and hexadecimal 
addresses should be accepted (POSIX getaddrinfo refers to inet_addr for numeric 
IP addresses and that says that octal and hexadecimal numbers are valid in IP 
addresses), see:

http://pubs.opengroup.org/onlinepubs/9699919799/functions/inet_addr.html#

Adding a implementation note to the documentation might be useful, but it 
should IMHO only mention that the platform getaddrinfo is used in the 
implementation for the Python functions and should not mention specific 
platforms because we don't have the processes to keep such specific notes 
up-to-date.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27612] socket.gethostbyname resolving octal IP addresses incorrectly

[koobs]

koobs added the comment:

@David 

The symptoms from FreeBSD look a little different:

Only gethostbyname affected only on 2.7 and 3.3 on all freebsd versions (9, 10, 
11). 

Python 3.2 was not tested (freebsd port was deleted), but likely affected as 
well

Feels/Appears like a gethostbyname fix or other change affecting gethostbyname 
in 3.4, missing merges to 3.3, (likely 3.2) and 2.7.

Full test matrix attached

--
versions: +Python 3.3
Added file: 
http://bugs.python.org/file43890/socket-test-freebsd-9-10-11-python-27-33-34-35.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27612] socket.gethostbyname resolving octal IP addresses incorrectly

[Eryk Sun]

Eryk Sun added the comment:

socket.gethostbyname calls the internal function setipaddr, which tries to 
avoid a name resolution by first calling either inet_pton or inet_addr. 
Otherwise it calls getaddrinfo.

Windows
---

setipaddr calls inet_addr, which supports octal [1]. ctypes example:

ws2_32 = ctypes.WinDLL('ws2_32')
in_addr = ctypes.c_ubyte * 4
ws2_32.inet_addr.restype = in_addr

>>> ws2_32.inet_addr(b'0177...0001')[:]
[127, 0, 0, 1]

3.5+ could call inet_pton since it was added in Vista. However, it does not 
support octal:

>>> addr = in_addr()
>>> ws2_32.inet_pton(socket.AF_INET, b'0177...0001', addr)
0
>>> ws2_32.inet_pton(socket.AF_INET, b'127.0.0.1', addr)
1
>>> addr[:]
[127, 0, 0, 1]

socket.inet_pton instead calls WSAStringToAddressA, which does support octal:

>>> list(socket.inet_pton(socket.AF_INET, '0177...0001'))
[127, 0, 0, 1]

socket.gethostbyname_ex calls gethostbyname since gethostbyname_r isn't 
defined. This does not support octal and errors out:

>>> socket.gethostbyname_ex('0177...0001')
Traceback (most recent call last):
  File "", line 1, in 
socket.herror: [Errno 11001] host not found

getaddrinfo also does not support octal and errors out:

>>> socket.getaddrinfo('0177...0001', None)[0]
Traceback (most recent call last):
  File "", line 1, in 
  File "C:\Program Files\Python35\lib\socket.py", line 732, in getaddrinfo
for res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno 11001] getaddrinfo failed

>>> ctypes.FormatError(11001)
'No such host is known.'

[1]: https://msdn.microsoft.com/en-us/library/ms738563#internet_addresses

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27612] socket.gethostbyname resolving octal IP addresses incorrectly

[Matt Robenolt]

Matt Robenolt added the comment:

Ah, I just confirmed broken behavior in macOS as well using `getaddrinfo()` in 
C.

I guess I'd be ok with python ignoring this as well. Maybe worth a change to 
documentation to note this?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27612] socket.gethostbyname resolving octal IP addresses incorrectly

[Xiang Zhang]

Xiang Zhang added the comment:

A similar bug report can be seen at 
https://github.com/dotnet/corefx/issues/8362. There someone makes a conclusion 
that getaddrinfo (Python seems to use getaddrinfo to implement gethostbyname) 
doesn't work correctly with octal form. They finally ignore this inconsistent 
behaviour.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27612] socket.gethostbyname resolving octal IP addresses incorrectly

[Matt Robenolt]

Matt Robenolt added the comment:

Is it worth investigating the different behavior then with `getaddrinfo` 
between platforms? As far as I know, that's the only method that works with 
both ipv6 and will tell you "here are all the IP addresses this resolves to".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27612] socket.gethostbyname resolving octal IP addresses incorrectly

[R. David Murray]

R. David Murray added the comment:

Hmm.  Since gethostbyname is a deprecated interface, perhaps there is nothing 
to do here.

However, if someone wants to investigate further and finds a fix, we will 
evaluate it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27612] socket.gethostbyname resolving octal IP addresses incorrectly

[Matt Robenolt]

Matt Robenolt added the comment:

And lastly, it seems that `socket.gethostbyname_ex` _does_ work correctly on 
both platforms.

```
>>> socket.gethostbyname_ex('0177...0001')
('0177...0001', [], ['127.0.0.1'])
```

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27612] socket.gethostbyname resolving octal IP addresses incorrectly

[Matt Robenolt]

Matt Robenolt added the comment:

Sorry, to add a data point, in C, `gethostbyname` also does the correct thing 
on macOS.

See:

```
#include 
#include 
#include 
#include 
#include 
#include 
#include 

int main(int argc, char *argv[]) {
int i;
struct hostent *lh = gethostbyname("0177...0001");
struct in_addr **addr_list;

if (lh) {
addr_list = (struct in_addr **)lh->h_addr_list;
for (i=0; addr_list[i] != NULL; i++) {
printf("%s", inet_ntoa(*addr_list[i]));
}
printf("\n");
} else {
herror("gethostbyname");
}

return 0;
}
```

So I'm not sure this is platform specific.

Either way, `socket.gethostbyname` is wrong on both linux and macOS. I'm a bit 
lost with what's going on here though, admittedly. :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27612] socket.gethostbyname resolving octal IP addresses incorrectly

[Xiang Zhang]

Xiang Zhang added the comment:

On Linux, it seems it's not an accident. inet_addr(3) explicitly says it can 
handle octal or haxadecimal forms.

--
nosy: +xiang.zhang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27612] socket.gethostbyname resolving octal IP addresses incorrectly

[R. David Murray]

R. David Murray added the comment:

To clarify: by platform OS issue, I mean that the octal-conversion-or-not is 
none of Python's doing, it is done by the C library call that gethostbyname is 
a thin wrapper around.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27612] socket.gethostbyname resolving octal IP addresses incorrectly

[R. David Murray]

R. David Murray added the comment:

This would appear to be a platform OS issue.  Is it "broken" also for FreeBSD?  
(I put broken in quotes because  interpreting ocatal isn't part of the posix 
speck for gethostbyname.  It could even be an accident that it works on Linux.

I'm not going to close this yet, since it might be worth a doc issue, or at 
least documenting here what the status of this is on FreeBSD.

--
nosy: +koobs, r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27612] socket.gethostbyname resolving octal IP addresses incorrectly

[SilentGhost]

Changes by SilentGhost :


--
components: +Macintosh
nosy: +ned.deily, ronaldoussoren

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27612] socket.gethostbyname resolving octal IP addresses incorrectly

[Matt Robenolt]

New submission from Matt Robenolt:

This also affects socket.getaddrinfo on macOS only, but is fine on Linux. I've 
not tested on Windows to see behavior there.

Given the IP address `0177...0001`, which is a valid octal format 
representing `127.0.0.1`, we can see varying results. Confirmed in both python 
2.7 and 3.5.

First, socket.gethostbyname is always wrong, and always returns `177.0.0.1`:

```
>>> socket.gethostbyname('0177...0001')
'177.0.0.1'
```

This can be seen on both Linux and macOS.

With `socket.getaddrinfo`, resolution is correct on Linux, but the bad 
177.0.0.1 on macOS.

Linux:
```
>>> socket.getaddrinfo('0177...0001', None)[0]
(2, 1, 6, '', ('127.0.0.1', 0))
```

macOS:
```
>>> socket.getaddrinfo('0177...0001', None)[0]
(2, 2, 17, '', ('177.0.0.1', 0))
```

This behavior exists in both 2.7.12 and 3.5.2 at least. I haven't tested many 
others, but I assume pretty universal.

--
components: Library (Lib)
messages: 271237
nosy: mattrobenolt
priority: normal
severity: normal
status: open
title: socket.gethostbyname resolving octal IP addresses incorrectly
type: behavior
versions: Python 2.7, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com