[issue23133] Pickling of ipaddress classes

[Roundup Robot]

Roundup Robot added the comment:

New changeset 712ac77b772b by Serhiy Storchaka in branch 'default':
Fixed tests for issue #23133 (pickling of IPv4Network was not tested).
https://hg.python.org/cpython/rev/712ac77b772b

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23133] Pickling of ipaddress classes

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Thank you Antoine.

And here is comparison of pickle size.

Unpatched:
>>> len(pickle.dumps([ipaddress.ip_address('192.0.2.%s'%i) for i in range(1, 
>>> 101)]))
2971
>>> len(pickle.dumps([ipaddress.ip_address('2001:db8::%x'%i) for i in range(1, 
>>> 101)]))
4071
>>> len(pickle.dumps([ipaddress.ip_interface('192.0.2.%s/27'%i) for i in 
>>> range(1, 101)]))
19341
>>> len(pickle.dumps([ipaddress.ip_interface('2001:db8::%x/124'%i) for i in 
>>> range(1, 101)]))
22741
>>> len(pickle.dumps([ipaddress.ip_network('192.0.2.%s/27'%(i&-32)) for i in 
>>> range(1, 101)]))
10614
>>> len(pickle.dumps([ipaddress.ip_interface('2001:db8::%x/124'%(i&-32)) for i 
>>> in range(1, 101)]))
22741

Patched:
>>> len(pickle.dumps([ipaddress.ip_address('192.0.2.%s'%i) for i in range(1, 
>>> 101)]))
1531
>>> len(pickle.dumps([ipaddress.ip_address('2001:db8::%x'%i) for i in range(1, 
>>> 101)]))
2631
>>> len(pickle.dumps([ipaddress.ip_interface('192.0.2.%s/27'%i) for i in 
>>> range(1, 101)]))
2963
>>> len(pickle.dumps([ipaddress.ip_interface('2001:db8::%x/124'%i) for i in 
>>> range(1, 101)]))
3256
>>> len(pickle.dumps([ipaddress.ip_network('192.0.2.%s/27'%(i&-32)) for i in 
>>> range(1, 101)]))
2938
>>> len(pickle.dumps([ipaddress.ip_interface('2001:db8::%x/124'%(i&-32)) for i 
>>> in range(1, 101)]))
3209

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23133] Pickling of ipaddress classes

[Roundup Robot]

Roundup Robot added the comment:

New changeset 781b54f7bccc by Serhiy Storchaka in branch 'default':
Issue #23133: Pickling of ipaddress objects now produces more compact and
https://hg.python.org/cpython/rev/781b54f7bccc

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23133] Pickling of ipaddress classes

[Antoine Pitrou]

Antoine Pitrou added the comment:

The patch looks fine to me.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23133] Pickling of ipaddress classes

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Then I'll remove it. Could you please make a review of optimized patch?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23133] Pickling of ipaddress classes

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
assignee:  -> serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23133] Pickling of ipaddress classes

[Antoine Pitrou]

Antoine Pitrou added the comment:

I don't understand what the test is for. I think it's safe it's remove it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23133] Pickling of ipaddress classes

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

ipaddress_pickle_3.patch breaks one test (testMissingAddressVersion). Is this 
test needed?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23133] Pickling of ipaddress classes

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


Added file: http://bugs.python.org/file37728/ipaddress_pickle_3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23133] Pickling of ipaddress classes

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


Removed file: http://bugs.python.org/file37726/ipaddress_pickle_3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23133] Pickling of ipaddress classes

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


Added file: http://bugs.python.org/file37727/ipaddress_pickle_2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23133] Pickling of ipaddress classes

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


Removed file: http://bugs.python.org/file37725/ipaddress_lightweight_2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23133] Pickling of ipaddress classes

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


Added file: http://bugs.python.org/file37726/ipaddress_pickle_3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23133] Pickling of ipaddress classes

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Yes, pickling (and especially unpickling) ints is more efficient, but the code 
will more complex. Interfaces should be pickled as strings for backward 
compatibility, and interfaces are subclasses of addresses.

Here are microbenchmarks:

./python -m timeit -s "import ipaddress, pickle; ips = 
[ipaddress.ip_address('192.0.2.%s'%i) for i in range(1, 101)]" -- 
"pickle.dumps(ips)"
./python -m timeit -s "import ipaddress, pickle; ips = 
[ipaddress.ip_address('2001:db8::%x'%i) for i in range(1, 101)]" -- 
"pickle.dumps(ips)"
./python -m timeit -s "import ipaddress, pickle; ips = 
[ipaddress.ip_address('192.0.2.%s'%i) for i in range(1, 101)]; pickled = 
pickle.dumps(ips)" -- "pickle.loads(pickled)"
./python -m timeit -s "import ipaddress, pickle; ips = 
[ipaddress.ip_address('2001:db8::%x'%i) for i in range(1, 101)]; pickled = 
pickle.dumps(ips)" -- "pickle.loads(pickled)"

Results for unpatched module:
1000 loops, best of 3: 1.56 msec per loop
1000 loops, best of 3: 1.62 msec per loop
1000 loops, best of 3: 1.08 msec per loop
1000 loops, best of 3: 1.09 msec per loop

With ipaddress_pickle.patch:
100 loops, best of 3: 3.43 msec per loop
100 loops, best of 3: 10.6 msec per loop
100 loops, best of 3: 7.76 msec per loop
100 loops, best of 3: 8.58 msec per loop

With ipaddress_pickle_2.patch:
1000 loops, best of 3: 1.11 msec per loop
1000 loops, best of 3: 1.16 msec per loop
1000 loops, best of 3: 1.88 msec per loop
100 loops, best of 3: 2.05 msec per loop

With ipaddress_pickle_3.patch:
1000 loops, best of 3: 1.12 msec per loop
1000 loops, best of 3: 1.15 msec per loop
1000 loops, best of 3: 1.13 msec per loop
1000 loops, best of 3: 1.15 msec per loop

--
Added file: http://bugs.python.org/file37725/ipaddress_lightweight_2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23133] Pickling of ipaddress classes

[Antoine Pitrou]

Antoine Pitrou added the comment:

Patch looks good to me. For further efficiency, addresses could be pickled as 
ints (but beware of interfaces and networks).

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23133] Pickling of ipaddress classes

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

Currently ipaddress classes support pickling, but the pickling is not efficient 
and is implementation depened. Proposed patch makes pickling more compact and 
implementation agnostic.

--
components: Library (Lib)
files: ipaddress_pickle.patch
keywords: patch
messages: 233201
nosy: ncoghlan, pmoody, serhiy.storchaka
priority: normal
severity: normal
stage: patch review
status: open
title: Pickling of ipaddress classes
type: enhancement
versions: Python 3.5
Added file: http://bugs.python.org/file37564/ipaddress_pickle.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com