Hi, On Sun, 8 Mar 2015 at 12:33 Ben Hoyt <benh...@gmail.com> wrote:
> Others: if you want to benchmark this, the simplest way is to use my > os.walk() benchmark.py test program here: > https://github.com/benhoyt/scandir -- it compares the built-in os.walk() > implemented with os.listdir() with a version of walk() implemented with > os.scandir(). I see huge gains on Windows (12-50x) and modest gains on my > Linux VM (3-5x). > I have a MacBook Pro Laptop running OS X 10.10.2. I did the following: - hg update -r 8ef4f75a8018 - patch -p1 < scandir-8.patch - ./configure --with-pydebug && make -j7 I then ran ./python.exe ~/Workspace/python/scandir/benchmark.py and I got: *Creating tree at /Users/rstuart/Workspace/python/scandir/benchtree: depth=4, num_dirs=5, num_files=50* *Using slower ctypes version of scandir* *Comparing against builtin version of os.walk()* *Priming the system's cache...* *Benchmarking walks on /Users/rstuart/Workspace/python/scandir/benchtree, repeat 1/3...* *Benchmarking walks on /Users/rstuart/Workspace/python/scandir/benchtree, repeat 2/3...* *Benchmarking walks on /Users/rstuart/Workspace/python/scandir/benchtree, repeat 3/3...* *os.walk took 0.184s, scandir.walk took 0.158s -- 1.2x as fast* I then did ./python.exe ~/Workspace/python/scandir/benchmark.py -s and got: *Using slower ctypes version of scandir* *Comparing against builtin version of os.walk()* *Priming the system's cache...* *Benchmarking walks on /Users/rstuart/Workspace/python/scandir/benchtree, repeat 1/3...* *Benchmarking walks on /Users/rstuart/Workspace/python/scandir/benchtree, repeat 2/3...* *Benchmarking walks on /Users/rstuart/Workspace/python/scandir/benchtree, repeat 3/3...* *os.walk size 23400, scandir.walk size 23400 -- equal* *os.walk took 0.483s, scandir.walk took 0.463s -- 1.0x as fast* Hope this helps. Cheers Note that the actual CPython version of os.walk() doesn't yet use > os.scandir(). I intend to open a separate issue for that shortly (or Victor > can). But that part should be fairly straight-forward, as I already have a > version available in my GitHub project. > > -Ben > > > On Sat, Mar 7, 2015 at 9:13 PM, Victor Stinner <victor.stin...@gmail.com> > wrote: > >> Hi, >> >> FYI I commited the implementation of os.scandir() written by Ben Hoyt. >> I hope that it will be part of Python 3.5 alpha 2 (Ben just sent the >> final patch today). >> >> Please test this new feature. You may benchmark here. >> http://bugs.python.org/issue22524 contains some benchmark tools and >> benchmark results of older versions of the patch. >> >> The implementation was tested on Windows and Linux. I'm now watching >> for buildbots to see how other platforms like os.scandir(). >> >> Bad news: OpenIndiana doesn't support d_type: the dirent structure has >> no d_type field. I already fixed the implementation to support this >> case. os.scandir() is still useful on OpenIndiana, because the stat >> result is cached in a DirEntry, so only one syscall is required, >> instead of multiple, when multiple DirEntry methods are called (ex: >> entry.is_dir() and not entry.is_symlink()). >> >> Victor >> _______________________________________________ >> Python-Dev mailing list >> Python-Dev@python.org >> https://mail.python.org/mailman/listinfo/python-dev >> > Unsubscribe: >> https://mail.python.org/mailman/options/python-dev/benhoyt%40gmail.com >> > > _______________________________________________ > Python-Dev mailing list > Python-Dev@python.org > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: https://mail.python.org/mailman/options/python-dev/ > ryan.stuart.85%40gmail.com >
_______________________________________________ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
