Re: efficient text file search -solution

2006-09-12 Thread Sion Arrowsmith
noro [EMAIL PROTECTED] wrote: OK, am not sure why, but fList=file('somefile').read() if fList.find('string') != -1: print 'FOUND' works much much faster. it is strange since i thought 'for line in file('somefile')' is optemized and read pages to the memory, Step back and think about what

efficient text file search.

2006-09-11 Thread noro
Is there a more efficient method to find a string in a text file then: f=file('somefile') for line in f: if 'string' in line: print 'FOUND' ? BTW: does for line in f: read a block of line to te memory or is it simply calls f.readline() many times? thanks amit --

Re: efficient text file search.

2006-09-11 Thread Luuk
noro [EMAIL PROTECTED] schreef in bericht news:[EMAIL PROTECTED] Is there a more efficient method to find a string in a text file then: f=file('somefile') for line in f: if 'string' in line: print 'FOUND' yes, more efficient would be: grep (http://www.gnu.org/software/grep/)

Re: efficient text file search.

2006-09-11 Thread noro
:) via python... Luuk wrote: noro [EMAIL PROTECTED] schreef in bericht news:[EMAIL PROTECTED] Is there a more efficient method to find a string in a text file then: f=file('somefile') for line in f: if 'string' in line: print 'FOUND' yes, more efficient would be:

Re: efficient text file search.

2006-09-11 Thread Luuk
noro [EMAIL PROTECTED] schreef in bericht news:[EMAIL PROTECTED] :) via python... Luuk wrote: noro [EMAIL PROTECTED] schreef in bericht news:[EMAIL PROTECTED] Is there a more efficient method to find a string in a text file then: f=file('somefile') for line in f: if 'string'

Re: efficient text file search.

2006-09-11 Thread Kent Johnson
noro wrote: Is there a more efficient method to find a string in a text file then: f=file('somefile') for line in f: if 'string' in line: print 'FOUND' Probably better to read the whole file at once if it isn't too big: f = file('somefile') data = f.read() if 'string' in data:

Re: efficient text file search.

2006-09-11 Thread Bill Scherer
noro wrote: Is there a more efficient method to find a string in a text file then: f=file('somefile') for line in f: if 'string' in line: print 'FOUND' ? BTW: does for line in f: read a block of line to te memory or is it simply calls f.readline() many times? thanks amit If

Re: efficient text file search.

2006-09-11 Thread Ant
noro wrote: Is there a more efficient method to find a string in a text file then: f=file('somefile') for line in f: if 'string' in line: print 'FOUND' break ^^^ Add a 'break' after the print statement - that way you won't have to read the entire file

Re: efficient text file search.

2006-09-11 Thread John Machin
Luuk wrote: [snip] some googling turned op the following. Second paragraph of chapter 14 of http://www.amk.ca/python/2.1/ [snip] For a fuller discussion of the line I/O changes, see the python-dev summary for January 1-15, 2001 at http://www.amk.ca/python/dev/2001-01-1.html. That is

Re: efficient text file search.

2006-09-11 Thread noro
can you add some more info, or point me to a link, i havn't found anything about binary search in mmap() in python documents. the files are very big... thanks amit Bill Scherer wrote: noro wrote: Is there a more efficient method to find a string in a text file then: f=file('somefile') for

Re: efficient text file search.

2006-09-11 Thread Steve Holden
noro wrote: Bill Scherer wrote: noro wrote: Is there a more efficient method to find a string in a text file then: f=file('somefile') for line in f: if 'string' in line: print 'FOUND' ? BTW: does for line in f: read a block of line to te memory or is it simply calls

Re: efficient text file search.

2006-09-11 Thread Luuk
John Machin [EMAIL PROTECTED] schreef in bericht news:[EMAIL PROTECTED] Luuk wrote: [snip] some googling turned op the following. Second paragraph of chapter 14 of http://www.amk.ca/python/2.1/ [snip] For a fuller discussion of the line I/O changes, see the python-dev summary for

Re: efficient text file search.

2006-09-11 Thread noro
i'm not sure. each line in the text file and an index string. i can sort the file, and use some binary tree search on it. (I need to do a number of searchs). there are 1219137 indexs in the file. so maby a memory efficient sort algorithm is in place. how can mmap help me? is there any fbinary

Re: efficient text file search.

2006-09-11 Thread George Sakkis
noro wrote: Is there a more efficient method to find a string in a text file then: f=file('somefile') for line in f: if 'string' in line: print 'FOUND' ? Is this something you want to do only once for a given file ? The replies so far seem to imply so and in this case I

efficient text file search -solution

2006-09-11 Thread noro
OK, am not sure why, but fList=file('somefile').read() if fList.find('string') != -1: print 'FOUND' works much much faster. it is strange since i thought 'for line in file('somefile')' is optemized and read pages to the memory, i guess not.. George Sakkis wrote: noro wrote: Is there a