[issue25596] Use scandir() to speed up the glob module

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

os.scandir() cannot handle bytes argument on Windows. If an argument is string, 
os.scandir() yields entries with string names, if an argument is bytes object, 
os.scandir() yields entries with bytes names. Opened issue27998 for adding 
support of bytes paths in os.scandir(). After this will be implemented, a 
workaround cb7ee9d9cddd can be reverted.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25596] Use scandir() to speed up the glob module

[Dima Tisnek]

Dima Tisnek added the comment:

@Serhiy please comment the implications / limitations of the fallback on 
Windows.

Is it that scandir cannot handle bytes argument only?
If argument is unicode, but response set contains bytes paths, will that work?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25596] Use scandir() to speed up the glob module

[Roundup Robot]

Roundup Robot added the comment:

New changeset 48e573e0a610 by Serhiy Storchaka in branch 'default':
Issue #25596: Falls back to listdir in glob for bytes paths on Windows.
https://hg.python.org/cpython/rev/48e573e0a610

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25596] Use scandir() to speed up the glob module

[Guido van Rossum]

Guido van Rossum added the comment:

Thanks Serhiy!

--Guido (mobile)

On Sep 6, 2016 12:35 PM, "Roundup Robot"  wrote:

>
> Roundup Robot added the comment:
>
> New changeset cb7ee9d9cddd by Serhiy Storchaka in branch 'default':
> Issue #25596: Optimized glob() and iglob() functions in the
> https://hg.python.org/cpython/rev/cb7ee9d9cddd
>
> --
> nosy: +python-dev
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25596] Use scandir() to speed up the glob module

[Roundup Robot]

Roundup Robot added the comment:

New changeset cb7ee9d9cddd by Serhiy Storchaka in branch 'default':
Issue #25596: Optimized glob() and iglob() functions in the
https://hg.python.org/cpython/rev/cb7ee9d9cddd

--
nosy: +python-dev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25596] Use scandir() to speed up the glob module

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I think the change in general is approved by GvR. If there are some 
implementation bugs, we can fix them later.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25596] Use scandir() to speed up the glob module

[Dima Tisnek]

Dima Tisnek added the comment:

Who can review Serhiy's patch?

P.S. core of the patch seems good to me, but I'm not an expert on stdlib.

--
nosy: +Dima.Tisnek

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25596] Use scandir() to speed up the glob module

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The patch is completed now. The with statement is used to avoid FD leaks.

--
Added file: http://bugs.python.org/file41895/glob_scandir_5.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25596] Use scandir() to speed up the glob module

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Not this version of the path. It accumulates all scandir results in a list 
before yielding values. Otherwise globbing will consume a lot of open file 
descriptors and could leak them (see issue25994, issue25995).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25596] Use scandir() to speed up the glob module

[Guido van Rossum]

Guido van Rossum added the comment:

Oh well. Too bad. Never mind then.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25596] Use scandir() to speed up the glob module

[Guido van Rossum]

Guido van Rossum added the comment:

Looks like switching to scandir will also resolve 
http://bugs.python.org/issue22167 without further action.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25596] Use scandir() to speed up the glob module

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
title: Use scandir() to speed the glob module -> Use scandir() to speed up the 
glob module

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25596] Use scandir() to speed up the glob module

[Ben Hoyt]

Ben Hoyt added the comment:

Nice! Good to see scandir() getting used to speed up other functions, functions 
that folks are using a ton already, like glob().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com