> Now that you see what I want to do with 1 billion random digits, > please give me your suggestion(s). As I mentioned before, > I'm very new to reading from and writing to files.
The way old text editors used to do this - in the days when we only had 16K RAM etc! was to use buffers and pages. Let's say you have a 100K buffer size and read in the first buffer full of text. When you get down to say 75K you delete the first 50K and load the next 50K from the file. Your cursor is now 25K into a new set of 100K. This allows you to page back a certain amount and page forward a lot. And so you progress losing 50% from top or bottom and loading the next/previous chunk into RAM. Its non-trivial to get right but in your case you are only looking to sequentially process so you only need to look forward so not quite so bad... You need markers to keep your location and other markers to keep track of the buffers location within the file. If you really are only doing sequential processing you can dispense with the buffer/pages and just read small chunks such that you always have two in memory at once. The chunks might only be a few Kb in size depending on the longest sequence you are looking for (a chunk of twice that size is a good guide) The seek() tell() and read() methods are your friends for this kind of work Alan Gauld Author of the Learn To Program website http://www.alan-g.me.uk/ _______________________________________________ Tutor maillist - Tutor@python.org To unsubscribe or change subscription options: http://mail.python.org/mailman/listinfo/tutor
