Hello, I've been working on a patch which speeds up shutil.copy* operations for all 3 major platforms (Linux, Windows, OSX): https://bugs.python.org/issue33671 Since the speedup is quite consistent I'd love to see this merged in, but considering shutil.copy* is quite crucial I wanted to hear other folk's opinion first. Attached patch attempts to use platform-specific zero-copy syscalls [1] by default and fallbacks on using plain read() / write() variant in case of immediate failure. In theory this should work fine, in practice I haven't tested it on exotic (e.g. network) filesystems. In order to diminish risks of breakage I think if it would make sense to: - add a global shutil.NO_ZEROCOPY variable defaulting to False - add a "no_zerocopy" argument to all functions involving a copy (copyfile(). copy(), copy2(), copytree(), move())
Thoughts? [1] sendfile() (Linux), fcopyfile() (OSX), CopyFileW (Windows) since the matter is a bit sensitive in terms of potential breakage on exotic / untested (e.g. network) filesystems I want to raise some attention about: https://bugs.python.org/issue33671 Attached patch attempts to use platform-specific zero-copy syscalls by default and fallbacks on using plain read() / write() copy in case of immediate failure. In order to diminish risks I think it would make sense to: - add a global shutil.NO_ZEROCOPY variable defaulting to False - add a "no_zerocopy" argument to all functions involving a copy (copyfile(). copy(), copy2(), copytree(), move()) Thoughts? -- Giampaolo - http://grodola.blogspot.com
_______________________________________________ Python-ideas mailing list Python-ideas@python.org https://mail.python.org/mailman/listinfo/python-ideas Code of Conduct: http://python.org/psf/codeofconduct/
