Re: [Haskell-cafe] operating on a hundred files at once

2007-04-10 Thread Chad Scherrer
Hi Jeff, I have a series of NxM numeric tables I'm doing a quick mean/variance/t-test etcetera on. The cell t1 [i,j] corresponds exactly to the cells t2..N [i,j], and so it's perfectly possible to read one item at a time from each of the 100 files and compute the mean/variance etcetera on all

Re: [Haskell-cafe] operating on a hundred files at once

2007-04-10 Thread Ketil Malde
On Tue, 2007-04-10 at 13:16 +1000, Duncan Coutts wrote: Note, that like in your original we read each file twice, once for the mean and once for the variance. As an aside, you can calculate both mean and variance in one pass (and constant space) by calculating the sum of elements 'x', the sum

Re: [Haskell-cafe] operating on a hundred files at once

2007-04-10 Thread Jefferson Heard
Thanks, Ketil. I knew I could calcuate the mean in constant space, but I didn't think about the variance. Much appreciated. On Tue, 2007-04-10 at 08:30 +0200, Ketil Malde wrote: On Tue, 2007-04-10 at 13:16 +1000, Duncan Coutts wrote: Note, that like in your original we read each file

[Haskell-cafe] operating on a hundred files at once

2007-04-09 Thread Jefferson Heard
I have a series of NxM numeric tables I'm doing a quick mean/variance/t-test etcetera on. The cell t1 [i,j] corresponds exactly to the cells t2..N [i,j], and so it's perfectly possible to read one item at a time from each of the 100 files and compute the mean/variance etcetera on all cells that

Re: [Haskell-cafe] operating on a hundred files at once

2007-04-09 Thread Bulat Ziganshin
Hello Jefferson, Monday, April 9, 2007, 9:34:12 PM, you wrote: if you have enough memory available, the fastest way is to read file to memory using bytestring, convert it into array of doubles, repeating this step for all files. then perform your computations. if you will try to read 100 files

Re: [Haskell-cafe] operating on a hundred files at once

2007-04-09 Thread Jefferson Heard
Thanks for the advice. I'm not so much interested in performance here, as this is just a one-off. Disk thrashing or not, these files are only a few hundred K apiece, and I can't imagine that the whole computation will take more than a few minutes. My question is more about how to deal with

Re: [Haskell-cafe] operating on a hundred files at once

2007-04-09 Thread Duncan Coutts
On Mon, 2007-04-09 at 14:40 -0400, Jefferson Heard wrote: Thanks for the advice. I'm not so much interested in performance here, as this is just a one-off. Disk thrashing or not, these files are only a few hundred K apiece, and I can't imagine that the whole computation will take more than a

Re: [Haskell-cafe] operating on a hundred files at once

2007-04-09 Thread Jefferson Heard
It is indeed! Is that to be found in Control.Monad, I take it? On Tue, 2007-04-10 at 08:50 +1000, Duncan Coutts wrote: On Mon, 2007-04-09 at 14:40 -0400, Jefferson Heard wrote: Thanks for the advice. I'm not so much interested in performance here, as this is just a one-off. Disk thrashing

Re: [Haskell-cafe] operating on a hundred files at once

2007-04-09 Thread Stefan O'Rear
On Mon, Apr 09, 2007 at 09:24:30PM -0400, Jefferson Heard wrote: It is indeed! Is that to be found in Control.Monad, I take it? It's in the Prelude, so you don't have to import anything to get it. Stefan ___ Haskell-Cafe mailing list

Re: [Haskell-cafe] operating on a hundred files at once

2007-04-09 Thread Duncan Coutts
On Mon, 2007-04-09 at 21:24 -0400, Jefferson Heard wrote: It is indeed! Is that to be found in Control.Monad, I take it? Yes. Other common derivatives in that module include: mapM f as = sequence (map f as) mapM_ f as = sequence_ (map f as) forM_ = flip mapM_ forM = flip mapM however,