On Tue, Aug 25, 2009 at 6:46 PM, Matthew Brand<[email protected]> wrote: > I have a 5GB text file with data which I want to process one line at a > time. Is there any way to stream through the file one line at a time > without reading in the whole thing and splitting it into lines? I > could break the file into smaller file and > proc...@split_into_lines@open them, but I wonder if there is an easier > way?
The way most languages implement this involves reading in blocks, finding line boundaries (lines will often span blocks) assembling lines and making them available one at a time to the program. I do not know if anyone has bothered making this kind of facility for J, but J does support reading blocks. See "Indexed Read" at http://www.jsoftware.com/help/dictionary/dx001.htm FYI, -- Raul ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
