[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.

[Yury Selivanov]

Change by Yury Selivanov :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.

[Yury Selivanov]

Yury Selivanov  added the comment:


New changeset 0442599961f966a3dc7f3fe6a3c0d5765fcf2082 by Yury Selivanov (Miss 
Islington (bot)) in branch '3.7':
bpo-32221: makeipaddr(): remove interface part + speedup (GH-5449) (GH-5449) 
(#5641)
https://github.com/python/cpython/commit/0442599961f966a3dc7f3fe6a3c0d5765fcf2082


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.

[miss-islington]

Change by miss-islington :


--
pull_requests: +5441

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.

[Yury Selivanov]

Yury Selivanov  added the comment:


New changeset 7766b96ab80b04509bbac708ee5ecf3c1c5934fc by Yury Selivanov 
(Коренберг Марк) in branch 'master':
bpo-32221: makeipaddr(): remove interface part + speedup (GH-5449) (#5449)
https://github.com/python/cpython/commit/7766b96ab80b04509bbac708ee5ecf3c1c5934fc


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.

[Марк Коренберг]

Change by Марк Коренберг :


--
pull_requests: +5281

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.

[Yury Selivanov]

Yury Selivanov  added the comment:

Mark, I think this still land in beta-2.  Please try to figure out what's going 
on on Mac OS (I have no time to take care of this myself).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.

[Yury Selivanov]

Yury Selivanov  added the comment:


New changeset 0ceb717689b04c0540d78c1ba93c0572c66c0994 by Yury Selivanov in 
branch 'master':
Revert "bpo-32221: makeipaddr(): remove interface part + speedup (GH-4724)" 
(#5394)
https://github.com/python/cpython/commit/0ceb717689b04c0540d78c1ba93c0572c66c0994


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.

[Yury Selivanov]

Yury Selivanov  added the comment:

Mark,

Two tests fail on MacOS X now (newly added tests).  I'm reverting the change 
for now.  If you figure this out, please re-submit a PR for a quick approval.


==
ERROR: test_getaddrinfo_ipv6_scopeid (test.test_socket.GeneralModuleTests)
--
Traceback (most recent call last):
  File "/Users/yury/dev/pydev/cpython/Lib/test/test_socket.py", line 1612, in 
test_getaddrinfo_ipv6_scopeid
ifindex = socket.if_nametoindex(test_interface)
OSError: no interface with this name

==
ERROR: test_getnameinfo_ipv6_scopeid (test.test_socket.GeneralModuleTests)
--
Traceback (most recent call last):
  File "/Users/yury/dev/pydev/cpython/Lib/test/test_socket.py", line 1636, in 
test_getnameinfo_ipv6_scopeid
nameinfo = socket.getnameinfo(sockaddr, socket.NI_NUMERICHOST | 
socket.NI_NUMERICSERV)
socket.gaierror: [Errno 4] Non-recoverable failure in name resolution

--

--
resolution: fixed -> 
stage: resolved -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.

[Yury Selivanov]

Change by Yury Selivanov :


--
pull_requests: +5229
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.

[Yury Selivanov]

Yury Selivanov  added the comment:

Merged.  Thanks, Mark!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.

[Yury Selivanov]

Yury Selivanov  added the comment:


New changeset 47c0b1f7d4115e6f15e6776c1f91d28e7d96fe0c by Yury Selivanov 
(Коренберг Марк) in branch 'master':
bpo-32221: makeipaddr(): remove interface part + speedup (GH-4724)
https://github.com/python/cpython/commit/47c0b1f7d4115e6f15e6776c1f91d28e7d96fe0c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.

[Yury Selivanov]

Yury Selivanov  added the comment:

> So, PR is ready. Please review.

I understand, however I still don't have time for it at the moment. If another 
core dev wants to take initiative, please by all means do.

I can/will take another look at this after January 15.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.

[Марк Коренберг]

Марк Коренберг  added the comment:

So, PR is ready. Please review.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.

[Yury Selivanov]

Yury Selivanov  added the comment:

The PR looks good, but I don't have the capacity right now to merge it. Ill 
need some time for research to double check if it's ok to drop scope id. Will 
try to find it in the next few days.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.

[Марк Коренберг]

Марк Коренберг  added the comment:

No, Improvement for typical case (not scoped IPv6 addresses) was not the 
target, but fortunatelly it also sped up.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.

[Yury Selivanov]

Yury Selivanov  added the comment:

I thought the improvements were way bigger than 200nsec, no?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.

[Марк Коренберг]

Марк Коренберг  added the comment:

After my patch:

$ python3.7 -m timeit --setup 'import socket; s = 
socket.socket(socket.AF_INET6, socket.SOCK_DGRAM); s.bind(("ff02::1de:c0db", 
1234, 0, 2))' 's.getsockname()'
50 loops, best of 5: 613 nsec per loop

$ python3.7 -m timeit --setup 'import socket; s = 
socket.socket(socket.AF_INET6, socket.SOCK_DGRAM); s.bind(("::", 1235, 0, 0))' 
's.getsockname()'
50 loops, best of 5: 420 nsec per loop

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Wow.  That's a heavy price to pay indeed.

Unfortunately I'm not competent on IPv6.  I'm cc-ing other core developers so 
that they may chime in.

--
nosy: +gregory.p.smith, neologix, yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.

[Марк Коренберг]

Марк Коренберг  added the comment:

So it may affect applications, that work with UDP and .recvfrom()

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.

[Марк Коренберг]

Марк Коренберг  added the comment:

Original (not patched) python:

```
In [1]s = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
In [2]: s.bind(('ff02::1de:c0db', 1234, 0, 2))
In [3]: timeit s.getsockname()
The slowest run took 12.06 times longer than the fastest. This could mean that 
an intermediate result is being cached.
10 loops, best of 3: 3.8 µs per loop

In [5]: d = socket.socket(socket.AF_INET6, socket.SOCK_DGRAM)
In [6]: d.bind(('::', 1235, 0, 0))
In [7]: timeit d.getsockname()
The slowest run took 23.18 times longer than the fastest. This could mean that 
an intermediate result is being cached.
100 loops, best of 3: 532 ns per loop
```

On patched version, times for both cases should be the same exactly (i.e. 532 
ns)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> `recvfrom` from multicast socket is painfull slow.

What do you mean? Can you give results for a benchmark of your choice?  How 
much does your PR speed it up?

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.

[Antoine Pitrou]

Change by Antoine Pitrou :


--
versions:  -Python 3.6, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.

[Марк Коренберг]

Марк Коренберг  added the comment:

Please look at my PR

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.

[Марк Коренберг]

Change by Марк Коренберг :


--
keywords: +patch
pull_requests: +4631
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32221] Converting ipv6 address to string representation using getnameinfo() is wrong.

[Марк Коренберг]

New submission from Марк Коренберг :

https://github.com/python/cpython/pull/4724


`recvfrom` from multicast socket is painfull slow. In fact, it returns sender 
address in form:

`('fe80::941f:f6ff:fe04:c560%qwe', 42133, 0, 78)`
which is superfluous, since interface-name part (`%qwe`) is not actually used. 
Actually, scopeid (`78`) signify interface/scope/zone_id. This tuple can be 
used for `.sendto()` either with this interface-name-part or without.

The problem is in the performance. For each `recvrfom()`, `getnameinfo()` 
internally converts interface index to interface name using three syscalls, 
i.e. `socket(), getsockopt()?, close()` , which slows down receiving (I have 
not measured result, but see additional syscalls in `strace`).

In order to convert from tuple to string-based full address one may use 
`getnameinfo()`:
As you can see, initial interface is ignored (but without my patch it is also 
validated uselessly):
```
In[1]: socket.getnameinfo(('fe80::941f:f6ff:fe04:c560%qwe', 42133, 0, 78), 
socket.NI_NUMERICHOST)
Out[1]: ('fe80::941f:f6ff:fe04:c560%qwe', '42133')
In[2]: socket.getnameinfo(('fe80::941f:f6ff:fe04:c560', 42133, 0, 78), 
socket.NI_NUMERICHOST)
Out[2]: ('fe80::941f:f6ff:fe04:c560%qwe', '42133')
```

--
components: Library (Lib)
messages: 307651
nosy: socketpair
priority: normal
severity: normal
status: open
title: Converting ipv6 address to string representation using getnameinfo() is 
wrong.
type: performance
versions: Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com