Re: [Haskell-cafe] Re: Lazy IO and closing of file handles

2007-03-21 Thread Claus Reinke
[trigger garbage collection when open runs out of free file descriptors, then try again] so, instead of documenting limitations and workarounds, this issue should be fixed in GHC as well. This may help in some cases but it cannot be relied upon. Finalizers are always run in a separate thread

[Haskell-cafe] Re: Lazy IO and closing of file handles

2007-03-21 Thread Benjamin Franksen
Benjamin Franksen wrote: Bertram Felgenhauer wrote: Having to rely on GC to close the fds quickly enough is another problem; can this be solved on the library side, maybe by performing GCs when running out of FDs? Claus Reinke wrote: in good old Hugs, for instance, we find in function

[Haskell-cafe] Re: Lazy IO and closing of file handles

2007-03-20 Thread Ferenc Wagner
Matthew Brecknell [EMAIL PROTECTED] writes: enumLines :: (a - String - Either a a) - a - FilePath - IO a enumLines iter accum filename = do h - openFile filename ReadMode flip fix accum $ \iterate accum - do try_line - try (hGetLine h) case try_line of Left e -

[Haskell-cafe] Re: Lazy IO and closing of file handles

2007-03-20 Thread Benjamin Franksen
Bertram Felgenhauer wrote: Having to rely on GC to close the fds quickly enough is another problem; can this be solved on the library side, maybe by performing GCs when running out of FDs? Claus Reinke wrote: in good old Hugs, for instance, we find in function newHandle in src/iomonad.c

[Haskell-cafe] Re: Lazy IO and closing of file handles

2007-03-19 Thread Pete Kazmier
Matthew Brecknell [EMAIL PROTECTED] writes: Pete Kazmier: I attempted to read Oleg's fold-stream implementation [1] as this sounds quite appealing to me, but I was completely overwhelmed, especially with all of the various type signatures used. It would be great if one of the regular

[Haskell-cafe] Re: Lazy IO and closing of file handles

2007-03-19 Thread Pete Kazmier
Pete Kazmier [EMAIL PROTECTED] writes: I attempted to read Oleg's fold-stream implementation [1] as this sounds quite appealing to me, but I was completely overwhelmed, especially with all of the various type signatures used. It would be great if one of the regular Haskell bloggers (Tom

Re: [Haskell-cafe] Re: Lazy IO and closing of file handles

2007-03-19 Thread Bryan O'Sullivan
Pete Kazmier wrote: I understand the intent of this code, but I am having a hard time understanding the implementation, specifically the combination of 'fix', 'flip', and 'interate'. I looked up 'fix' and I'm unsure how one can call 'flip' on a function that takes one argument. If you look

Re: [Haskell-cafe] Re: Lazy IO and closing of file handles

2007-03-19 Thread Lennart Augustsson
Here's what happens: fix has type (x-x)-x and that has to match the first argument to flip, namely 'a-b-c'. The only chance of that is if x is actually a function type. Pick x=b-c, now we have fix has type ((b-c)-b-c)-b-c and it matches a-b-c if a=(b-c)-b-c Flip returns b-a-c, and if we

Re: [Haskell-cafe] Re: Lazy IO and closing of file handles

2007-03-19 Thread Isaac Dupree
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Pete Kazmier wrote: Bryan O'Sullivan [EMAIL PROTECTED] writes: Pete Kazmier wrote: I understand the intent of this code, but I am having a hard time understanding the implementation, specifically the combination of 'fix', 'flip', and

[Haskell-cafe] Re: Lazy IO and closing of file handles

2007-03-17 Thread Pete Kazmier
Matthew Brecknell [EMAIL PROTECTED] writes: So here's a test. I don't have any big maildirs handy, so this is based on the simple exercise of printing the first line of each of a large number of files. First, the preamble. import Control.Exception (bracket) import System.Environment import

[Haskell-cafe] Re: Lazy IO and closing of file handles

2007-03-14 Thread Pete Kazmier
[EMAIL PROTECTED] (Donald Bruce Stewart) writes: pete-expires-20070513: When using readFile to process a large number of files, I am exceeding the resource limits for the maximum number of open file descriptors on my system. How can I enhance my program to deal with this situation without

Re: [Haskell-cafe] Re: Lazy IO and closing of file handles

2007-03-14 Thread Donald Bruce Stewart
pete-expires-20070513: [EMAIL PROTECTED] (Donald Bruce Stewart) writes: pete-expires-20070513: When using readFile to process a large number of files, I am exceeding the resource limits for the maximum number of open file descriptors on my system. How can I enhance my program to deal

Re: [Haskell-cafe] Re: Lazy IO and closing of file handles

2007-03-14 Thread Dougal Stanton
Quoth Pete Kazmier, nevermore, the same error regarding max open files. Incidentally, the lazy bytestring version of my program was by far the fastest and used the least amount of memory, but it still crapped out regarding max open files. I've tried the approach you appear to be using and