Pascal wrote:

>Can you be more specific about the code?
>I assume that you looked into cut ;. ?
>an alternative to boxing might be to strip out the commas, and then run 0&". 
>on the string.  Not sure that is faster though.

I'm sorry for being unclear. I did something like this:

loadd 'tables/csv'
datloc=: '/path/to/csvs/'

ip=: ".> readcsv datloc,'chunk1.csv'
ip=: ip, ".> readcsv datloc,'chunk2.csv'
(etc)

ip is fairly small, but the boxed array read in by readcsv is bloody enormous. 
readcsv is also pretty slow. I solved the problem with chunking the csvs, but 
waiting around for several minutes seemed a very un J-like experience. By 
comparison, the binsearch I needed to do took a fraction of a second (it took 
almost a half hour in R, which has no native binsearch). 

-Scott, who will definitely be taking Eric up on his kind offer
----------------------------------------------------------------------
For information about J forums see http://www.jsoftware.com/forums.htm

Reply via email to