Re: Reading a file that is changing and getting the new lines
Ouray Viney wrote: Hi: Problem: = I want to read a ASCII text file that can have data appended to it. I have hacked some code together that handles the basics, but it falls short. My code doesn't read in the new lines that could have been added to the end of the file. Not python's fault, more that I don't know how to do it in python. Example scenario: As the python script is running a user/application adds new entries to the end of the test case file, example, adds the following to the file. test4 test5 test6 The python code below, won't pick these changes up. Analysis: I understand why my code doesn't handle the addition of new data. This is simply because the os.open reads all the lines in the beginning and since the data is added after that, the for loop doesn't iterate through it since it doesn't know about it. Please note, my code is not pretty and isn't at all advanced. Code: == import os,string,sys,time # define locals tcListFile=testcase_list.txt fh=open(tcListFile,r) tcList=fh.readlines() fh.close() for tc in tcList: print tc.strip() print sleeping for 2 seconds time.sleep(2) # close the file handle fh.close() text file: == test1 test2 test3 Question: = What is the best way to handle this type of scenario using python? Read your file once. Make a note of the length (X) Close the file Setup a loop with a sleep at the end. Each time round check the file size If it has increased, make a note of it (Y) and open the file, seek to X and read Y - X bytes This gives you the new data, you can then append it to the inital read or whatever. Do bar in mind though that all OS's buffer IO so the program that's generating the file might have a different view of it than your python script i.e. program A might output 300 chars to file but they may not be written to disk until the buffer receives 4096 chars. If this might be a problem to you make sure the program generating the data flushes the cache after every write (sys.stdout.flush() in python). Roger. -- http://mail.python.org/mailman/listinfo/python-list
Reading a file that is changing and getting the new lines
Hi: Problem: = I want to read a ASCII text file that can have data appended to it. I have hacked some code together that handles the basics, but it falls short. My code doesn't read in the new lines that could have been added to the end of the file. Not python's fault, more that I don't know how to do it in python. Example scenario: As the python script is running a user/application adds new entries to the end of the test case file, example, adds the following to the file. test4 test5 test6 The python code below, won't pick these changes up. Analysis: I understand why my code doesn't handle the addition of new data. This is simply because the os.open reads all the lines in the beginning and since the data is added after that, the for loop doesn't iterate through it since it doesn't know about it. Please note, my code is not pretty and isn't at all advanced. Code: == import os,string,sys,time # define locals tcListFile=testcase_list.txt fh=open(tcListFile,r) tcList=fh.readlines() fh.close() for tc in tcList: print tc.strip() print sleeping for 2 seconds time.sleep(2) # close the file handle fh.close() text file: == test1 test2 test3 Question: = What is the best way to handle this type of scenario using python? -- http://mail.python.org/mailman/listinfo/python-list
Re: Reading a file that is changing and getting the new lines
On Dec 1, 3:09 pm, Ouray Viney ovi...@gmail.com wrote: Problem: = I want to read a ASCII text file that can have data appended to it. Example scenario: As the python script is running a user/application adds new entries to the end of the test case file, example, adds the following to the file. Question: = What is the best way to handle this type of scenario using python? There was a thread regarding 'imitating tail -f' recently that you might find useful. The best approach IMO is this one: http://www.dabeaz.com/generators/follow.py ~Sean -- http://mail.python.org/mailman/listinfo/python-list
Re: Reading a file that is changing and getting the new lines
Ouray Viney wrote: Hi: Problem: = I want to read a ASCII text file that can have data appended to it. I have hacked some code together that handles the basics, but it falls short. My code doesn't read in the new lines that could have been added to the end of the file. Not python's fault, more that I don't know how to do it in python. Example scenario: As the python script is running a user/application adds new entries to the end of the test case file, example, adds the following to the file. test4 test5 test6 I need clarification. Is this an independent process, that appends data onto the same file, after the python script has opened it, but while it's still running? Can you give me an example point in your source code that your script might be running when the other application does its append? The python code below, won't pick these changes up. Analysis: I understand why my code doesn't handle the addition of new data. This is simply because the os.open reads all the lines in the os.open() doesn't read the lines, the readlines() call does. Once you've got the data in a list, of course changing the file has no impact on your code. And by the way, you didn't call os.open(), you called open(). They are not interchangeable. beginning and since the data is added after that, the for loop doesn't iterate through it since it doesn't know about it. Please note, my code is not pretty and isn't at all advanced. Code: == import os,string,sys,time # define locals tcListFile=testcase_list.txt fh=open(tcListFile,r) tcList=fh.readlines() fh.close() for tc in tcList: print tc.strip() print sleeping for 2 seconds time.sleep(2) # close the file handle fh.close() text file: == test1 test2 test3 Question: = What is the best way to handle this type of scenario using python? You can solve your first level of problem by simply skipping the readlines() call, and eliminating the first close() call. You close it again later, let that be the only close. You want your loop to actually get a line from the file, not from a list which is frozen in time. Naturally, you'd want a better name than tcList. Next problem is likely to be one of sharing the file. You open it read-only, the other application simultaneously tries to append. Does the OS permit that? You find out. You might actually have to close the file during those sleep() calls, and then do an open/seek to get to the earlier position. You most likely will have to switch from open() to os.open(). The calls are very different. Final problem that I note is how to decide when to quit your loop. Just because you hit eof (end of file), doesn't mean the other application isn't about to append another 50 lines. So you need a different termination condition. I suspect the sharing problems aren't going to be portable between operating systems, so you might want to answer the usual what version of Python, on what OS question that we seem to have for most new questions. DaveA -- http://mail.python.org/mailman/listinfo/python-list
