Re: Effect of large binaries on garbage collection
On Wednesday 12 March 2003 00:49, Manuel M T Chakravarty wrote: Simon Peyton-Jones [EMAIL PROTECTED] wrote, | In the current CVS GHC, undoubtedly the right thing to use is | Foreign.mallocForeignPtr. Internally these are implemented as | MutableByteArray#, so you get fast allocation and GC, but from the | programmer's point of view it's a normal ForeignPtr. I wonder how it is for a random FFI user to discover this. Does some advice belong in the FFI spec, or in GHC's FFI chapter? What is needed is a FFI tutorial (in addition to the spec); similar to how we have the Gentle Introduction to complement the Report. Thank's for the advice from everybody. Actually, on the whole I think the FFI spec is pretty easy to understand, but tutorial would be nice too :-) There are obviously subtleties about it which I had not understood properly. Regards -- Adrian Hey ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Effect of large binaries on garbage collection
Simon Peyton-Jones [EMAIL PROTECTED] wrote, | In the current CVS GHC, undoubtedly the right thing to use is | Foreign.mallocForeignPtr. Internally these are implemented as | MutableByteArray#, so you get fast allocation and GC, but from the | programmer's point of view it's a normal ForeignPtr. I wonder how it is for a random FFI user to discover this. Does some advice belong in the FFI spec, or in GHC's FFI chapter? What is needed is a FFI tutorial (in addition to the spec); similar to how we have the Gentle Introduction to complement the Report. Manuel ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Effect of large binaries on garbage collection
On Thursday 06 March 2003 10:55, Adrian Hey wrote: On Tuesday 04 March 2003 12:36, Simon Peyton-Jones wrote: GHC does not copy big objects, so don't worry about the copying cost. (Instead of copying, it allocates big objects to (a contiguous series of) heap blocks, with no other objects in those blocks. Then the object can move simply by swizzling the heap-block descriptor.) Thanks, looks like it's option (1) then. Could you tell me what Haskell type I have to use be able to pass a pointer to this binary to C land (genuine address, not StablePtr). I don't think the standard FFI allows this at all but, IIRC, the old ghc libraries allowed you to do this with a mutable byte array. Does anybody know the answer to this? please.. pretty please..:-) Sorry if this is a case of me failing to RTFM, but I don't think it is. Paragraph 8.1.1 of the users guide says.. The types ByteArray and MutableByteArray may be used as basic foreign types (see FFI Addendum, Section 3.2). In C land, they map to (char *). I can't find any way in the Base libs (or what's left of the old libs) to create a ByteArray or MutableByteArray, which leads me to suspect that they no longer exist. Should I use something else instead? Thanks -- Adrian Hey ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
RE: Effect of large binaries on garbage collection
Thanks, looks like it's option (1) then. Could you tell me what Haskell type I have to use be able to pass a pointer to this binary to C land (genuine address, not StablePtr). I don't think the standard FFI allows this at all but, IIRC, the old ghc libraries allowed you to do this with a mutable byte array. Does anybody know the answer to this? please.. pretty please..:-) Sorry if this is a case of me failing to RTFM, but I don't think it is. Paragraph 8.1.1 of the users guide says.. The types ByteArray and MutableByteArray may be used as basic foreign types (see FFI Addendum, Section 3.2). In C land, they map to (char *). This is still true for 5.04.x: the ByteArray and MutableArray libs in hslibs (package lang) provide these types, but they are deprecated and will be removed at some point. The functionality which allows ByteArray and MutableByteArrays to be passed to FFI functions has already been removed in CVS. In the current CVS GHC, undoubtedly the right thing to use is Foreign.mallocForeignPtr. Internally these are implemented as MutableByteArray#, so you get fast allocation and GC, but from the programmer's point of view it's a normal ForeignPtr. If you don't mind the not-so-great performance, you can use ForeignPtr with malloc/free in 5.04.x and convert to mallocForeignPtr when the next major release of GHC comes along. Or you can use ByteArray/MutableByteArray for the time being (note that you can only pass one of these to an unsafe FFI function, BTW). Cheers, Simon ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
RE: Effect of large binaries on garbage collection
| In the current CVS GHC, undoubtedly the right thing to use is | Foreign.mallocForeignPtr. Internally these are implemented as | MutableByteArray#, so you get fast allocation and GC, but from the | programmer's point of view it's a normal ForeignPtr. I wonder how it is for a random FFI user to discover this. Does some advice belong in the FFI spec, or in GHC's FFI chapter? Simon ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Effect of large binaries on garbage collection
On Tuesday 04 March 2003 12:36, Simon Peyton-Jones wrote: GHC does not copy big objects, so don't worry about the copying cost. (Instead of copying, it allocates big objects to (a contiguous series of) heap blocks, with no other objects in those blocks. Then the object can move simply by swizzling the heap-block descriptor.) Thanks, looks like it's option (1) then. Could you tell me what Haskell type I have to use be able to pass a pointer to this binary to C land (genuine address, not StablePtr). I don't think the standard FFI allows this at all but, IIRC, the old ghc libraries allowed you to do this with a mutable byte array. Regards -- Adrian Hey ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
RE: Effect of large binaries on garbage collection
GHC does not copy big objects, so don't worry about the copying cost. (Instead of copying, it allocates big objects to (a contiguous series of) heap blocks, with no other objects in those blocks. Then the object can move simply by swizzling the heap-block descriptor.) Simon | -Original Message- | From: Adrian Hey [mailto:[EMAIL PROTECTED] | Sent: 04 March 2003 11:05 | To: [EMAIL PROTECTED] | Subject: Effect of large binaries on garbage collection | | Hello, | | I'm writing a library which will require many blocks of binary data | of various sizes (some very large) to be stored in the heap. I'm a | little worried about the effect this will have on the efficiency of | garbage collection. I'm not sure how ghc gc works these days, but | I seem to remember it's a copying collector by default. If so it seems | a pity to waste time copying 10's of MBytes of binaries at each | collection. | | The options I'm considering are.. | | (1) Use Haskell heap space | Pros: Easy for me | Cons: May slow down gc | AFAICS I can't use anything like realloc | Current FFI proposals seem to prevent me from directly | accessing Haskell heap objects from C land (or have I | misunderstood?). | | (2) Use C heap space | Pros: Easy(ish) to use from C and Haskell ffi | Cons: Unless C heaps have improved a lot since I last looked | (which I doubt), it seems likely I will suffer from slow | allocation and fragmentation problems. | | (3) Write my own sliding heap manager and use finalisers for |garbage collection. |Pros: Can tailor it to work exactly the way I want. |Cons: More work for me, especially if I want the | result to be portable across OS's. | Might be a complete waste of time if my worries | about ghc heap management are groundless :-) | | Any advice? | | Thanks | -- | Adrian Hey | ___ | Glasgow-haskell-users mailing list | [EMAIL PROTECTED] | http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
