Matthew Brand wrote: > Thanks for the links. I tried the fapplylines adverb but the computer > grinds along for 30 minutes or so before I pulled the plug. It ends up > using 10Gb of (mainly virtual) memory. There are 40M lines in my file. > > I will use the unix split command to make lots of little files and > (myverb fapplylines)&.> fname to solve the problem.
There should be little difference between processing lots of small files, and one big file in chunks. What processing is being done? What result is being accumulated? Why not test on a small file first and find out what is taking time - and only then try on the full file? ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
