On 14 Feb 2015 03:43, Nathaniel Smith n...@pobox.com wrote:
On 13 Feb 2015 02:09, Victor Stinner victor.stin...@gmail.com wrote:
A alternative is to add a new _scandir.c module to host the new C
code, and share some code with posixmodule.c: remove static keyword
from required C functions
On 13 February 2015 at 10:07, Victor Stinner victor.stin...@gmail.com wrote:
= IMO the best option is to take the C implementation. What do you think?
FWIW (as I'm not a core dev) I agree. The Windows speedup is huge, and
well worth adding the code. I'm assuming that the majority of the C
code
On 13.02.15 12:07, Victor Stinner wrote:
TL,DR: are you ok to add 800 lines of C code for os.scandir(), 4x
faster than os.listdir() when the file type is checked?
You can try to make Python implementation faster if
1) Don't set attributes to None in constructor.
2) Implement scandir as:
def
Hi,
TL,DR: are you ok to add 800 lines of C code for os.scandir(), 4x
faster than os.listdir() when the file type is checked?
I accepted the PEP 471 (os.scandir) a few months ago, but it is not
implement yet in Python 3.5, because I didn't make a choice on the
implementation.
Ben Hoyt wrote
2015-02-13 11:19 GMT+01:00 Paul Moore p.f.mo...@gmail.com:
On 13 February 2015 at 10:07, Victor Stinner victor.stin...@gmail.com wrote:
= IMO the best option is to take the C implementation. What do you think?
FWIW (as I'm not a core dev) I agree. The Windows speedup is huge, and
well worth
2015-02-13 12:27 GMT+01:00 Serhiy Storchaka storch...@gmail.com:
On 13.02.15 12:07, Victor Stinner wrote:
* C implementation: scandir is at least 3.5x faster than listdir, up
to 44.6x faster on Windows
Results on Windows was obtained in the becnhmark that doesn't drop disk
caches and runs
On 13.02.15 12:07, Victor Stinner wrote:
* C implementation: scandir is at least 3.5x faster than listdir, up
to 44.6x faster on Windows
Results on Windows was obtained in the becnhmark that doesn't drop disk
caches and runs listdir before scandir.
On 13 February 2015 at 20:33, Victor Stinner victor.stin...@gmail.com wrote:
2015-02-13 11:19 GMT+01:00 Paul Moore p.f.mo...@gmail.com:
On 13 February 2015 at 10:07, Victor Stinner victor.stin...@gmail.com
wrote:
= IMO the best option is to take the C implementation. What do you think?
FWIW
2015-02-13 11:52 GMT+01:00 Serhiy Storchaka storch...@gmail.com:
You can try to make Python implementation faster if
1) Don't set attributes to None in constructor.
The class uses __slots__. Setting attributes in the class body is
forbidden when __slots__ is used.
3) Or pass DirEntry to
* C implementation: scandir is at least 3.5x faster than listdir, up
to 44.6x faster on Windows
* C+Python implementation: scandir is not really faster than listdir,
between 1.3x and 1.4x faster
So amusingly, the bottleneck is not so much the cost of system calls,
but the cost of Python
Phone
From: Antoine Pitroumailto:solip...@pitrou.net
Sent: 2/13/2015 5:44
To: python-dev@python.orgmailto:python-dev@python.org
Subject: Re: [Python-Dev] PEP 471 (scandir): Poll to choose the implementation
(full C or C+Python)
On Fri, 13 Feb 2015 08:35:00 -0500
On Fri, 13 Feb 2015 08:35:00 -0500
Ben Hoyt benh...@gmail.com wrote:
If we go ahead with the all C approach, I'd be in favour of
refactoring a little and putting the new scandir code into a separate
C file. There are two ways to do this: a) sticking with a single
Python module and just
2015-02-13 15:36 GMT+01:00 Steve Dower steve.do...@microsoft.com:
I think posixmodule is a great candidate for splitting up by platform rather
than function, as the whole file is packed with ifdef. It's really only
lacking a volunteer to do it, but we could start here (ie. make
Since there are many ways to split this huge file, I agree that it's
just fine to add these 800 lines and *then* think how the huge file
can be splitted. It's a different topic.
That's a good idea. Consider adding the new feature (scandir) and the
larger issue of refactoring posixmodule as
On 02/13/2015 02:07 AM, Victor Stinner wrote:
Hi,
TL,DR: are you ok to add 800 lines of C code for os.scandir(), 4x
faster than os.listdir() when the file type is checked?
+1 for the all-C version.
--
~Ethan~
signature.asc
Description: OpenPGP digital signature
On 13 Feb 2015 02:09, Victor Stinner victor.stin...@gmail.com wrote:
A alternative is to add a new _scandir.c module to host the new C
code, and share some code with posixmodule.c: remove static keyword
from required C functions (functions to convert Windows attributes to
a os.stat_result
I vote for the C implementation.
On Fri, Feb 13, 2015 at 2:07 AM, Victor Stinner victor.stin...@gmail.com
wrote:
Hi,
TL,DR: are you ok to add 800 lines of C code for os.scandir(), 4x
faster than os.listdir() when the file type is checked?
I accepted the PEP 471 (os.scandir) a few months
On Fri, Feb 13, 2015 at 5:07 AM, Victor Stinner victor.stin...@gmail.com
wrote:
Now I see 3 choices:
- take the full C implementation, because it's much faster (at least
3.4x faster!)
- reject the whole PEP 471 (not nice), because it adds too much code
for a minor speedup (not true on
On Fri, 13 Feb 2015 13:27:04 +0200
Serhiy Storchaka storch...@gmail.com wrote:
On 13.02.15 12:07, Victor Stinner wrote:
* C implementation: scandir is at least 3.5x faster than listdir, up
to 44.6x faster on Windows
Results on Windows was obtained in the becnhmark that doesn't drop disk
On Fri, 13 Feb 2015 11:07:03 +0100
Victor Stinner victor.stin...@gmail.com wrote:
* C implementation: scandir is at least 3.5x faster than listdir, up
to 44.6x faster on Windows
* C+Python implementation: scandir is not really faster than listdir,
between 1.3x and 1.4x faster
So amusingly,
20 matches
Mail list logo
