Re: more os.walk() issues... probably user error

2005-02-16 Thread Kent Johnson
rbt wrote:
##  for fs in fs_objects:
##
##for f in fs[2]:
##if f in file_skip_list:
##print f
##fs[2].remove(f)
##
##for d in fs[1]:
##if d in dir_skip_list:
##print d
##fs[1].remove(d)
Will the changes I made (file and dir removals from os.walk()) be 
reflected in the generator object? Is it safe to remove objects this way 
and pass the results in a generator on to another function? Sorry for 
all the questions, I just like to fully understand something before I 
start doing it with confidence.
Yes. The docs for os.walk() explicitly state, "When topdown is true, the caller can modify the 
dirnames list in-place (perhaps using del or slice assignment), and walk() will only recurse into 
the subdirectories whose names remain in dirnames."

So changes to the dir list affect the iteration; changes to the file list directly affect the value 
you return to the caller.

Kent
--
http://mail.python.org/mailman/listinfo/python-list


Re: more os.walk() issues... probably user error

2005-02-16 Thread rbt
Kent Johnson wrote:
rbt wrote:
rbt wrote:
This function is intended to remove unwanted files and dirs from 
os.walk(). It will return correctly *IF* I leave the 'for fs in 
fs_objects' statement out (basically leave out the entire purpose of 
the function).

It's odd, when the program goes into that statment... even when only 
a 'pass', and nothing else is present, nothing is returned. Why is 
that? I'm testing Python 2.4 on Linux x86 and WinXP. Results are the 
same on either platform.

def build_clean_list(self, path):
file_skip_list = ['search_results.txt']
dir_skip_list = ['dev', 'proc', 'Temporary Internet Files']
fs_objects = os.walk(path, topdown=True)

fs_objects is a generator, not a list. This loop is exhausting 
fs_objects, so when you return fs_objects is at the end of iteration, 
there is nothing left.
That makes sense. Thanks for the explanation. I've never used generators 
before.


##  for fs in fs_objects:
##
##for f in fs[2]:
##if f in file_skip_list:
##print f
##fs[2].remove(f)
##
##for d in fs[1]:
##if d in dir_skip_list:
##print d
##fs[1].remove(d)

Add this here:
 yield fs
and take out the return. This turns build_clean_list() into a generator 
function and you will be able to iterate the result.
I'll try this.
Will the changes I made (file and dir removals from os.walk()) be 
reflected in the generator object? Is it safe to remove objects this way 
and pass the results in a generator on to another function? Sorry for 
all the questions, I just like to fully understand something before I 
start doing it with confidence.

rbt

Kent
return fs_objects

Just to clarify, it's wrong of me to say that 'nothing is returned'... 
in either case, this is what is returned:

Here's what was returned and its type:




But, I can't iterate over the returned object when I descend into the 
for statement I mentioned above.

--
http://mail.python.org/mailman/listinfo/python-list


Re: more os.walk() issues... probably user error

2005-02-16 Thread Kent Johnson
rbt wrote:
rbt wrote:
This function is intended to remove unwanted files and dirs from 
os.walk(). It will return correctly *IF* I leave the 'for fs in 
fs_objects' statement out (basically leave out the entire purpose of 
the function).

It's odd, when the program goes into that statment... even when only a 
'pass', and nothing else is present, nothing is returned. Why is that? 
I'm testing Python 2.4 on Linux x86 and WinXP. Results are the same on 
either platform.

def build_clean_list(self, path):
file_skip_list = ['search_results.txt']
dir_skip_list = ['dev', 'proc', 'Temporary Internet Files']
fs_objects = os.walk(path, topdown=True)
fs_objects is a generator, not a list. This loop is exhausting fs_objects, so when you return 
fs_objects is at the end of iteration, there is nothing left.

##  for fs in fs_objects:
##
##for f in fs[2]:
##if f in file_skip_list:
##print f
##fs[2].remove(f)
##
##for d in fs[1]:
##if d in dir_skip_list:
##print d
##fs[1].remove(d)
Add this here:
 yield fs
and take out the return. This turns build_clean_list() into a generator function and you will be 
able to iterate the result.

Kent
return fs_objects

Just to clarify, it's wrong of me to say that 'nothing is returned'... 
in either case, this is what is returned:

Here's what was returned and its type:




But, I can't iterate over the returned object when I descend into the 
for statement I mentioned above.

--
http://mail.python.org/mailman/listinfo/python-list


Re: more os.walk() issues... probably user error

2005-02-16 Thread Dan Perl

"rbt" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
> def build_clean_list(self, path):
>
> file_skip_list = ['search_results.txt']
> dir_skip_list = ['dev', 'proc', 'Temporary Internet Files']
>
> fs_objects = os.walk(path, topdown=True)
> ##  for fs in fs_objects:
> ##
> ##for f in fs[2]:
> ##if f in file_skip_list:
> ##print f
> ##fs[2].remove(f)
> ##
> ##for d in fs[1]:
> ##if d in dir_skip_list:
> ##print d
> ##fs[1].remove(d)
>
> return fs_objects

Rather as an aside, the idiom for using os.walk is
for dirpath, dirnames, dirfiles in os.walk(path):
for f in dirnames:
if f in file_skip_list:
print f
filenames.remove(f)
if d in dir_skip_list:
print d
dirnames.remove(f)

More crucially for your code, returning the generator object after having 
iterated all the way through it will not do you any good.  The generator has 
an internal state that puts it at "the end of the iteration" so you cannot 
use it to iterate again. 


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: more os.walk() issues... probably user error

2005-02-16 Thread Dan Perl

"rbt" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
> rbt wrote:
>> This function is intended to remove unwanted files and dirs from 
>> os.walk(). It will return correctly *IF* I leave the 'for fs in 
>> fs_objects' statement out (basically leave out the entire purpose of the 
>> function).
>>
>> It's odd, when the program goes into that statment... even when only a 
>> 'pass', and nothing else is present, nothing is returned. Why is that? 
>> I'm testing Python 2.4 on Linux x86 and WinXP. Results are the same on 
>> either platform.
>>
>> def build_clean_list(self, path):
>>
>> file_skip_list = ['search_results.txt']
>> dir_skip_list = ['dev', 'proc', 'Temporary Internet Files']
>>
>> fs_objects = os.walk(path, topdown=True)
>> ##  for fs in fs_objects:
>> ##
>> ##for f in fs[2]:
>> ##if f in file_skip_list:
>> ##print f
>> ##fs[2].remove(f)
>> ##
>> ##for d in fs[1]:
>> ##if d in dir_skip_list:
>> ##print d
>> ##fs[1].remove(d)
>>
>> return fs_objects
>>
>>
>
> Just to clarify, it's wrong of me to say that 'nothing is returned'... in 
> either case, this is what is returned:
>
> Here's what was returned and its type:
> 
> 
> 
> 
>
> But, I can't iterate over the returned object when I descend into the for 
> statement I mentioned above.

What do you mean by not being able to iterate over the returned object? 
What kind of error are you getting?  Have you tried to debug the code?

BTW, os.walk indeed returns a generator.  You should familiarize yourself 
with generators and iterators if you haven't done so yet. 


-- 
http://mail.python.org/mailman/listinfo/python-list


Re: more os.walk() issues... probably user error

2005-02-16 Thread [EMAIL PROTECTED]
That's an easy one: fs_objects is not modified by your ode, so you get
it back as created by os.walk

-- 
http://mail.python.org/mailman/listinfo/python-list


Re: more os.walk() issues... probably user error

2005-02-16 Thread rbt
rbt wrote:
This function is intended to remove unwanted files and dirs from 
os.walk(). It will return correctly *IF* I leave the 'for fs in 
fs_objects' statement out (basically leave out the entire purpose of the 
function).

It's odd, when the program goes into that statment... even when only a 
'pass', and nothing else is present, nothing is returned. Why is that? 
I'm testing Python 2.4 on Linux x86 and WinXP. Results are the same on 
either platform.

def build_clean_list(self, path):
file_skip_list = ['search_results.txt']
dir_skip_list = ['dev', 'proc', 'Temporary Internet Files']
fs_objects = os.walk(path, topdown=True)
##  for fs in fs_objects:
##
##for f in fs[2]:
##if f in file_skip_list:
##print f
##fs[2].remove(f)
##
##for d in fs[1]:
##if d in dir_skip_list:
##print d
##fs[1].remove(d)
return fs_objects

Just to clarify, it's wrong of me to say that 'nothing is returned'... 
in either case, this is what is returned:

Here's what was returned and its type:




But, I can't iterate over the returned object when I descend into the 
for statement I mentioned above.

--
http://mail.python.org/mailman/listinfo/python-list


more os.walk() issues... probably user error

2005-02-16 Thread rbt
This function is intended to remove unwanted files and dirs from 
os.walk(). It will return correctly *IF* I leave the 'for fs in 
fs_objects' statement out (basically leave out the entire purpose of the 
function).

It's odd, when the program goes into that statment... even when only a 
'pass', and nothing else is present, nothing is returned. Why is that? 
I'm testing Python 2.4 on Linux x86 and WinXP. Results are the same on 
either platform.

def build_clean_list(self, path):
file_skip_list = ['search_results.txt']
dir_skip_list = ['dev', 'proc', 'Temporary Internet Files']
fs_objects = os.walk(path, topdown=True)
##  for fs in fs_objects:
##
##for f in fs[2]:
##if f in file_skip_list:
##print f
##fs[2].remove(f)
##
##for d in fs[1]:
##if d in dir_skip_list:
##print d
##fs[1].remove(d)
return fs_objects
--
http://mail.python.org/mailman/listinfo/python-list