On Jun 15, 2:35 am, tom f...@thefsb.org wrote:
i can traverse adirectoryusing os.listdir() or os.walk(). but if
adirectoryhas a very large number of files, these methods produce very
large objects talking a lot of memory.
in other languages one can avoid generating such an object by walking
rkl wrote:
I might be a little late with my comment here.
David Beazley in his PyCon'2008 presentation Generator Tricks
For Systems Programmers had this very elegant example of handling an
unlimited numbers of files:
David Beazley's generator stuff is definitely worth recommending
on. I
On Jun 15, 6:35 am, Andre Engels andreeng...@gmail.com wrote:
What kind of directories are those that just a list of files would
result in a very large object? I don't think I have ever seen
directories with more than a few thousand files...
(a...@lucrezia:~/pit/lsa/act:5)$ ls -1 | wc -l
Nick Craig-Wood n...@craig-wood.com writes:
It can be done properly with gccxml though which converts structures
into ctypes definitions.
That sounds interesting.
That said the dirent struct is specified by POSIX so if you get the
correct types for all the individual members then it should
On Jun 15, 6:56 am, Steven D'Aprano
ste...@remove.this.cybersource.com.au wrote:
On Sun, 14 Jun 2009 22:35:50 +0200, Andre Engels wrote:
On Sun, Jun 14, 2009 at 6:35 PM, tomf...@thefsb.org wrote:
i can traverse a directory using os.listdir() or os.walk(). but if a
directory has a very large
Nick Craig-Wood n...@craig-wood.com wrote:
Jean-Paul Calderone exar...@divmod.com wrote:
On Mon, 15 Jun 2009 09:29:33 -0500, Nick Craig-Wood n...@craig-wood.com
wrote:
Hrvoje Niksic hnik...@xemacs.org wrote:
Nick Craig-Wood n...@craig-wood.com writes:
Here is a ctypes
tom f...@thefsb.org wrote:
On Jun 14, 1:35 pm, Tim Golden m...@timgolden.me.uk wrote:
If you're on Windows, you can use the win32file.FindFilesIterator
function from the pywin32 package. (Which wraps the Win32 API
FindFirstFile / FindNextFile pattern).
thanks, tim.
however, i'm
Terry Reedy tjre...@udel.edu writes:
You did not specify version. In Python3, os.walk has become a
generater function. So, to answer your question, use 3.1.
os.walk has been a generator function all along, but that doesn't help
OP because it still uses os.listdir internally. This means that
Nick Craig-Wood n...@craig-wood.com writes:
Here is a ctypes generator listdir for unix-like OSes.
ctypes code scares me with its duplication of the contents of system
headers. I understand its use as a proof of concept, or for hacks one
needs right now, but can anyone seriously propose using
Hrvoje Niksic hnik...@xemacs.org wrote:
Nick Craig-Wood n...@craig-wood.com writes:
Here is a ctypes generator listdir for unix-like OSes.
ctypes code scares me with its duplication of the contents of system
headers. I understand its use as a proof of concept, or for hacks one
needs
On Mon, 15 Jun 2009 09:29:33 -0500, Nick Craig-Wood n...@craig-wood.com wrote:
Hrvoje Niksic hnik...@xemacs.org wrote:
Nick Craig-Wood n...@craig-wood.com writes:
Here is a ctypes generator listdir for unix-like OSes.
ctypes code scares me with its duplication of the contents of system
Christian Heimes wrote:
Terry Reedy wrote:
You did not specify version. In Python3, os.walk has become a generater
function. So, to answer your question, use 3.1.
I'm sorry to inform you that Python 3.x still returns a list, not a
generator.
type(os.walk('.'))
class 'generator'
However,
Jean-Paul Calderone exar...@divmod.com wrote:
On Mon, 15 Jun 2009 09:29:33 -0500, Nick Craig-Wood n...@craig-wood.com
wrote:
Hrvoje Niksic hnik...@xemacs.org wrote:
Nick Craig-Wood n...@craig-wood.com writes:
Here is a ctypes generator listdir for unix-like OSes.
ctypes code
On Mon, 15 Jun 2009 15:35:04 -0400
Terry Reedy tjre...@udel.edu wrote:
Christian Heimes wrote:
Terry Reedy wrote:
You did not specify version. In Python3, os.walk has become a generater
function. So, to answer your question, use 3.1.
I'm sorry to inform you that Python 3.x still
tom wrote:
i can traverse a directory using os.listdir() or os.walk(). but if a
directory has a very large number of files, these methods produce very
large objects talking a lot of memory.
in other languages one can avoid generating such an object by walking
a directory as a liked list. for
On Jun 14, 1:35 pm, Tim Golden m...@timgolden.me.uk wrote:
If you're on Windows, you can use the win32file.FindFilesIterator
function from the pywin32 package. (Which wraps the Win32 API
FindFirstFile / FindNextFile pattern).
thanks, tim.
however, i'm not using windows. freebsd and os x.
--
tom wrote:
On Jun 14, 1:35 pm, Tim Golden m...@timgolden.me.uk wrote:
If you're on Windows, you can use the win32file.FindFilesIterator
function from the pywin32 package. (Which wraps the Win32 API
FindFirstFile / FindNextFile pattern).
thanks, tim.
however, i'm not using windows. freebsd
On Sun, Jun 14, 2009 at 6:35 PM, tomf...@thefsb.org wrote:
i can traverse a directory using os.listdir() or os.walk(). but if a
directory has a very large number of files, these methods produce very
large objects talking a lot of memory.
in other languages one can avoid generating such an
tom schrieb:
i can traverse a directory using os.listdir() or os.walk(). but if a
directory has a very large number of files, these methods produce very
large objects talking a lot of memory.
in other languages one can avoid generating such an object by walking
a directory as a liked list.
tom wrote:
i can traverse a directory using os.listdir() or os.walk(). but if a
directory has a very large number of files, these methods produce very
large objects talking a lot of memory.
in other languages one can avoid generating such an object by walking
a directory as a liked list. for
Andre Engels wrote:
What kind of directories are those that just a list of files would
result in a very large object? I don't think I have ever seen
directories with more than a few thousand files...
I've seen directories with several hundreds of thousand files. Depending
on the file system
Christian Heimes wrote:
tom schrieb:
i can traverse a directory using os.listdir() or os.walk(). but if a
directory has a very large number of files, these methods produce very
large objects talking a lot of memory.
in other languages one can avoid generating such an object by walking
a
Terry Reedy wrote:
You did not specify version. In Python3, os.walk has become a generater
function. So, to answer your question, use 3.1.
I'm sorry to inform you that Python 3.x still returns a list, not a
generator.
ython 3.1rc1+ (py3k:73396, Jun 12 2009, 22:45:18)
[GCC 4.3.3] on linux2
i can traverse a directory using os.listdir() or os.walk(). but if a
directory has a very large number of files, these methods produce very
large objects talking a lot of memory.
in other languages one can avoid generating such an object by walking
a directory as a liked list. for example, in c,
On Sun, 14 Jun 2009 22:35:50 +0200, Andre Engels wrote:
On Sun, Jun 14, 2009 at 6:35 PM, tomf...@thefsb.org wrote:
i can traverse a directory using os.listdir() or os.walk(). but if a
directory has a very large number of files, these methods produce very
large objects talking a lot of memory.
25 matches
Mail list logo
