Hello,
I've been looking for a while for an answer, but so far I haven't been able to turn anything up yet. Basically, what I'd like to do is to use re.finditer to search a large file (or a file stream), but I haven't figured out how to get finditer to work without loading the entire file into memory, or just reading one line at a time (or more complicated buffering).
Can you use mmap?
http://docs.python.org/lib/module-mmap.html
"You can use mmap objects in most places where strings are expected; for example, you can use the re module to search through a memory-mapped file."
Seems applicable, and it should keep your memory use down, but I'm not very experienced with it...
Steve -- http://mail.python.org/mailman/listinfo/python-list
