Re: Proposal: Pooled memory management
Manuel wrote: [...] * I want to get v1.0 of the spec fixed. We are really only in bug fix mode for quite a while and only the finalizer problems held us back from finishing the spec. That's OK and I understand your motivation. Let's finish v1.0 first. * I am sure there are plenty more useful FFI-related libraries. However, the initial plan was to define basic functionality on top of which more elaborate schemes can be implemented. We need to draw the line somewhere. [...] But I strongly disagree here: The initial plan was to make a very small, but sufficient addition to the language (= foreign import/export), which can be implemented easily on existing systems. In this respect, we have reached our goal quite elegantly IMHO. The related libraries are a totally different beast: Minimality is *not* a design goal here. Otherwise we might be happy with e.g. Foreign.Marshal.Alloc.mallocBytes Foreign.Marshal.Alloc.allocaBytes Foreign.Marshal.Alloc.reallocBytes Foreign.Marshal.Alloc.free Foreign.Marshal.Util.copyBytes Foreign.Marshal.Util.moveBytes alone. Or another example: Why should we include such trivialities like Foreign.Marshal.Error.void or Foreign.Marshal.Util.toBool in the FFI spec? Minimality? Possibility of a lightning-fast special implementation? Definitely not. The basic task of a library (or a collection of related libraries) is to establish a common language and make common notions (like scoped or pooled allocations) explicit. Looking at the OO world, that's the reason for all this pattern hype. In a nutshell: Let's include as many useful patterns in the next FFI spec versions as possible! Otherwise we might talk the same language without recognizing it... Cheers, S. ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: [Simon Marlow simonmar@microsoft.com] RE: cvs commit: fptools/libraries/base/Foreign ForeignPtr.hs
Alastair Reid [EMAIL PROTECTED] wrote, perhaps ForeignPtr should not be an instance of Eq so people can provide their own? Note that if we did this, we'd want to consider adding an operation eqForeignPtr :: FP a - FP a - Bool FP b-- possible variant but not very useful which lets people test equality of the container and not the contents. In fact the more I think on it, the more convinced I am that the Eq instance should compare contaner equality and not contents equality. The reason is that I believe Eq instances should follow the following design rule: Eq instances should compute observational equivalence I believe this is satisfied by all Haskell98 types, all the usual extensions (IORefs and friends) and by derived instances of datatypes. So far, I agree. Just to be clear what I mean by observational equivalence, consider comparing two IORefs x and y using this code: eq x y = do a - readIORef x writeIORef (a+1) b - readIORef x return (x==y) Obviously, this code is a bad way to test if two IORefs are the same IORef but the point is that we can observe the difference between them. Similarily, with ForeignPtrs, adding a finalizer to one and not the other and then watching for when the finalizer runs is a way that we might observe differences between two FPs. There is no way in H98 + FFI to observe that a finalizer has run. In fact, in the finalizer discussion your point was that any means to observe this would lead to races. What I don't mean by observational (in)equivalence is this: One might be able to distinguish two data structures of type [Int] (say), by observing how much memory they consume. This is perfectly true but Haskell semantics doesn't let you observe this so we'll rule any such 'observations' as irrelevant or invalid. In C, we can observe it, and only in C can we observe whether a finalizer was executed. Nevertheless, your point regarding observations is an important one. We must not be able to observe a change in the equality of two values. This is why we cannot define equality of IORef's as the contents of the IORef's. However, the contents (ie, vanilla pointer on which it is based) of a ForeignPtr can never change. Hence, defining the equality of ForeignPtrs via their contents is perfectly valid. Hence, I propose to leave the definition in the spec as it was; ie, the equality of ForeignPtrs is defined via the vanilla pointer that they encapsulate. Cheers, Manuel ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: [Simon Marlow simonmar@microsoft.com] RE: cvs commit: fptools/libraries/base/Foreign ForeignPtr.hs
On Mon, Jan 20, 2003 at 11:03:36PM +1100, Manuel M T Chakravarty wrote: Hence, I propose to leave the definition in the spec as it was; ie, the equality of ForeignPtrs is defined via the vanilla pointer that they encapsulate. However, if you generalize ForeignPtrs (which I hope you will) this would require Eq on the underlying type. I guess this is no great hardship for the types people want to use it with. ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: Finalizers finalized
On Fri, Jan 17, 2003 at 10:23:27AM +1100, Manuel M T Chakravarty wrote: Ross Paterson [EMAIL PROTECTED] wrote, I'd also like to see the addition of mallocForeignPtrArray :: Storable a = Int - IO (ForeignPtr a) to ForeignPtr, to save people from rolling their own. (Maybe the realloc and 0 versions too?) The reason I didn't answer to this earlier is that I wanted to see how many people say that they support this addition. To be honest, I never needed a function like this. Did anybody else? Only me, it seems. But surely if both mallocForeignPtr and mallocArray are useful, this is too. If you don't want to do explicit deallocation, mallocForeignPtr is attractive (and handled specially by GHC). Sooner or later there are going to be arrays, and then if there's no such function you'll have to write your own. ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: Re-exporting modules in the Foreign hierarchy
Sven Panne [EMAIL PROTECTED] writes: Yet another change request for the FFI libraries: For reasons of consistency, I propose to introduce an intermediate module Foreign.Marshal, which re-exports the modules: Foreign.Marshal.Alloc Foreign.Marshal.Array Foreign.Marshal.Error Foreign.Marshal.Pool Foreign.Marshal.Utils Seems reasonable. It is a useful addition, with a trivial implementation, so there's not much to go wrong I think. While I'm at it: What was the reason for the decision that Foreign.C re-exports Foreign.C.Types Foreign.C.String Foreign.C.Error but not Foreign.C.TypesISO? If there wasn't a good one, I propose to add this for consistency again. I was recently pondering the same question. The answer is that Foreign.C.Types already re-exports everything from Foreign.C.TypesISO, and therefore, transitively, Foreign.C also does so. I do seem to remember a proposal at one stage to remove Foreign.C.TypesISO altogether by incorporating it fully into Foreign.C.Types. Given that the latter re-exports everything from the former, is there any reason for TypesISO to remain separate? Regards, Malcolm ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
RE: Proposal: Pooled memory management
Manuel wrote: [...] * I want to get v1.0 of the spec fixed. We are really only in bug fix mode for quite a while and only the finalizer problems held us back from finishing the spec. That's OK and I understand your motivation. Let's finish v1.0 first. I agree, but I don't have any problem with forking the spec at this point. The changes to the current 1.0 RC are going to be very small from now on. In a nutshell: Let's include as many useful patterns in the next FFI spec versions as possible! Otherwise we might talk the same language without recognizing it... So we might well ask what useful new functionality is provided by a pool-style memory manager. One feature is performance: compared to using malloc/free, a pool is going to be much cheaper. But I'd be interested to compare the performance you get from using pools to GHC's implementation of mallocForeignPtr, which also sidesteps malloc/free to get decent performance. Compared to mallocForeignPtr, there are some qualitative differences which might make one or the other more convenient in some situations: - mallocForeignPtr provides garbage-collected storage. No need to free the storage later. - However, you do need to use withForeignPtr to get access to the object. This can be quite annoying. So, assuming the performance is roughly the same (I'm guessing that using pools may be slightly faster than mallocForeignPtr, but not significantly), do people see any other compelling reasons to be using pools? GHC's runtime has an Arena abstraction which is an implementation of pool-style memory management, and could be used to implement Pools, BTW. Cheers, Simon ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: Proposal: Pooled memory management
Simon Marlow wrote: [...] So we might well ask what useful new functionality is provided by a pool-style memory manager. Well, I guess that about 90% of the hierarchical libraries don't provide any new functionality, but nevertheless they provide useful abstractions. The same holds for my proposal. One feature is performance: [...] That's a nice bonus, but I didn't have that in mind. So, assuming the performance is roughly the same [..], do people see any other compelling reasons to be using pools? I see the same compelling reasons as the ones for using foldl, sum, sequence, mapM, when, allocArray, with, ... :-) GHC's runtime has an Arena abstraction which is an implementation of pool-style memory management, and could be used to implement Pools, BTW. Nice, I wasn't aware of that. Cheers, S. ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
Re: Proposal: Pooled memory management
On 20-Jan-2003, Simon Marlow [EMAIL PROTECTED] wrote: So, assuming the performance is roughly the same (I'm guessing that using pools may be slightly faster than mallocForeignPtr, but not significantly) I am not yet convinced that this is a reasonable assumption if you are considering a wide range of Haskell implementations. Some Haskell implementations might use conservative collectors, and others (e.g. for .NET or JVM) may use collectors which are tuned for imperative/OO applications. On such implementations there might well be significant performance advantages to using pool-based manual memory management rather than garbage collection. In particular I suspect that pool-based manual memory management operations can be O(1) in some situations when non-copying GC'd allocation would be O(live data). I have observed significant overall speed-ups (e.g. for the Pseudoknot benchmark, about a factor of two) from using pool-like memory management rather than the Boehm collector. -- Fergus Henderson [EMAIL PROTECTED] | I have always known that the pursuit The University of Melbourne | of excellence is a lethal habit WWW: http://www.cs.mu.oz.au/~fjh | -- the last words of T. S. Garp. ___ FFI mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/ffi
