Jack Andrews wrote: > i've got many data. how can i deal with a readcsv call that consumes > all available memory? > how can i use j to analyse more data than comfortably fits? > i guess i can partition the data in different ways. > i could also use another language to parse the csv, row-by-row, and > invert the table.
You could process the CSV in chunks and dump the parsed data into J binary data files (one per column) using 3!:4 and 3!:5 (for ints and floats, strings and other more complicated types would be a little bit more work) and fappend from 'files', and then map those into memory using explicit map_jmf_ . Robby ---------------------------------------------------------------------- For information about J forums see http://www.jsoftware.com/forums.htm
