New submission from Markus: I found the code used to collapse addresses to be very slow on a large number (64k) of island addresses which are not collapseable.
The code at https://github.com/python/cpython/blob/0f164ccc85ff055a32d11ad00017eff768a79625/Lib/ipaddress.py#L349 was found to be guilty, especially the index lookup. The patch changes the code to discard the index lookup and have _find_address_range return the number of items consumed. That way the set operation to dedup the addresses can be dropped as well. Numbers from the testrig I adapted from http://bugs.python.org/issue20826 with 8k non-consecutive addresses: Execution time: 0.6893927365541458 seconds vs. Execution time: 12.116527611762285 seconds MfG Markus Kötter ---------- components: Library (Lib) files: ipaddress_collapse_non_consecutive_performance.diff keywords: patch messages: 234239 nosy: cmn priority: normal severity: normal status: open title: Faster implementation to collapse non-consecutive ip-addresses type: performance versions: Python 3.5 Added file: http://bugs.python.org/file37762/ipaddress_collapse_non_consecutive_performance.diff _______________________________________ Python tracker <rep...@bugs.python.org> <http://bugs.python.org/issue23266> _______________________________________ _______________________________________________ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
