Re: what happens when the file begin read is too big for all lines to be read with readlines()

Steve Holden wrote: Xiao Jianfeng wrote: Steven D'Aprano wrote: On Sun, 20 Nov 2005 11:05:53 +0800, Xiao Jianfeng wrote: I have some other questions: when fh will be closed? When all references to the file are no longer in scope: def handle_file(name): fp

Re: what happens when the file begin read is too big for all lines to be read with readlines()

Xiao Jianfeng wrote: First, I must say thanks to all of you. And I'm really sorry that I didn't describe my problem clearly. There are many tokens in the file, every time I find a token, I have to get the data on the next line and do some operation with it. It should be easy for

Re: what happens when the file begin read is too big for all lines to be read with readlines()

[EMAIL PROTECTED] wrote: Xiao Jianfeng wrote: First, I must say thanks to all of you. And I'm really sorry that I didn't describe my problem clearly. There are many tokens in the file, every time I find a token, I have to get the data on the next line and do some operation with it.

Re: what happens when the file begin read is too big for all lines to be read with readlines()

Xiao Jianfeng wrote: I have compared the two methods, (1). for x in fh: (2). read all the file into memory firstly. I have tested the two methods on two files, one is 80M and the second one is 815M. The first method gained a speedup of about 40% for the first file, and a speedup

Re: what happens when the file begin read is too big for all lines to be read with readlines()

[EMAIL PROTECTED] wrote: Xiao Jianfeng wrote: I have compared the two methods, (1). for x in fh: (2). read all the file into memory firstly. I have tested the two methods on two files, one is 80M and the second one is 815M. The first method gained a speedup of about 40% for the

what happens when the file begin read is too big for all lines to be read with readlines()

HI - Sorry for maybe a too simple a question but I googled and also checked my reference O'Reilly Learning Python book and I did not find a satisfactory answer. When I use readlines, what happens if the number of lines is huge?I have a very big file (4GB) I want to read in, but I'm sure

Re: what happens when the file begin read is too big for all lines to be read with readlines()

newer python should use for x in fh:, according to the doc : fh = open(your file) for x in fh: print x which would only read one line at a time. Ross Reyes wrote: HI - Sorry for maybe a too simple a question but I googled and also checked my reference O'Reilly Learning Python book and I did

Re: what happens when the file begin read is too big for all lines to be?read with readlines()

Ross Reyes [EMAIL PROTECTED] wrote: Sorry for maybe a too simple a question but I googled and also checked my reference O'Reilly Learning Python book and I did not find a satisfactory answer. The Python documentation is online, and it's good to get familiar with it:

Re: what happens when the file begin read is too big for all lines to be read with readlines()

Just try it, it is not that hard ... ;-) /Jean Brouwers PS) Here is what happens on Linux: $ limit vmemory 1 $ python ... s = file(bugfile).readlines() Traceback (most recent call last): File stdin, line 1 in ? MemoryError --

Re: what happens when the file begin read is too big for all lines to be read with readlines()

[EMAIL PROTECTED] wrote: newer python should use for x in fh:, according to the doc : fh = open(your file) for x in fh: print x which would only read one line at a time. I have some other questions: when fh will be closed? And what shoud I do if I want to explicitly close the file

Re: what happens when the file begin read is too big for all lines to be read with readlines()

On Sun, 20 Nov 2005 11:05:53 +0800, Xiao Jianfeng wrote: I have some other questions: when fh will be closed? When all references to the file are no longer in scope: def handle_file(name): fp = file(name, r) # reference to file now in scope do_stuff(fp) return fp f =

Re: what happens when the file begin read is too big for all lines to be read with readlines()

Steven D'Aprano wrote: On Sun, 20 Nov 2005 11:05:53 +0800, Xiao Jianfeng wrote: I have some other questions: when fh will be closed? When all references to the file are no longer in scope: def handle_file(name): fp = file(name, r) # reference to file now in scope

Re: what happens when the file begin read is too big for all lines to be read with readlines()

Xiao Jianfeng wrote: Steven D'Aprano wrote: On Sun, 20 Nov 2005 11:05:53 +0800, Xiao Jianfeng wrote: I have some other questions: when fh will be closed? When all references to the file are no longer in scope: def handle_file(name): fp = file(name, r) # reference to file

Re: what happens when the file begin read is too big for all lines to be read with readlines()

On Sun, 20 Nov 2005 12:28:07 +0800, Xiao Jianfeng wrote: Let me introduce my problem I came across last night first. I need to read a file(which may be small or very big) and to check line by line to find a specific token, then the data on the next line will be what I want. If I

Re: what happens when the file begin read is too big for all lines to be read with readlines()

On Sun, 20 Nov 2005 16:10:58 +1100, Steven D'Aprano wrote: def get_line(filename, token): Returns the next line following a token, or None if not found. Leading and trailing whitespace is ignored when looking for the token. fp = file(filename, r) for line in fp: