Re: Re[2]: [Haskell-cafe] bytestring vs. uvector
On Fri, 2009-03-13 at 20:30 +0300, Bulat Ziganshin wrote: > Hello Don, > > Friday, March 13, 2009, 8:08:57 PM, you wrote: > > >> What is the reason why you have decided to use unpinned arrays > >> (ByteArray#) instead of pinned arrays (Foreign.Ptr)? > > > They prevent heap fragmentation (and in general are faster). > > you probably mean faster alloc/gc operations, everything else should > be the same Right. Access times are the same. Both are just pointers internally. It is just the allocation time, GC time and extra memory use and lower cache utilisation caused by heap fragmentation. For big arrays it doesn't make much difference. Big ByteArray# allocations get pinned anyway. For small ones, like strings I expect the difference is much more noticeable, though I have not measured it. Using ByteArray# also means we can use ST/runST rather than IO/unsafePerformIO. In general we should prefer heap allocated byte arrays and ST unless we really really want to always use pinned allocations to interact with C libs easily. Duncan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: Re[2]: [Haskell-cafe] bytestring vs. uvector
On Tue, 2009-03-10 at 23:55 +0300, Bulat Ziganshin wrote: > starting with 6.6, ForeignArray access is no-op, so we can just use > obvious Ptr operations (via Storable class) to get unboxed arrays fast > access. so, no more need for those special ByteArray# access operations > > but Array library still old, so any effort based on its spources, got > the same restrictions > > also, ByteArray# may be unpinned, but afaik, this isn't really > important - it can be coerced to Ptr for the period of one operation Actually the fact that they may be unpinned is really important for small allocations like short strings. Pinned allocations are slow and lead to heap fragmentation. Duncan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re[2]: [Haskell-cafe] bytestring vs. uvector
Hello Don, Friday, March 13, 2009, 8:08:57 PM, you wrote: >> What is the reason why you have decided to use unpinned arrays >> (ByteArray#) instead of pinned arrays (Foreign.Ptr)? > They prevent heap fragmentation (and in general are faster). you probably mean faster alloc/gc operations, everything else should be the same -- Best regards, Bulatmailto:bulat.zigans...@gmail.com ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re[2]: [Haskell-cafe] bytestring vs. uvector
Hello Don, Wednesday, March 11, 2009, 12:48:35 AM, you wrote: >> unfortunately, Array library unboxed arrays still aren't based on any >> Unboxable *class* > Hmm. Aren't all the array library types based on MArray and IArray? > So I can define my own say, new STUArray element type by writing an instance > of > MArray for it. Like so: yes, you can, just this definition duplicates too much code -- Best regards, Bulatmailto:bulat.zigans...@gmail.com ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re[2]: [Haskell-cafe] bytestring vs. uvector
Hello Xiao-Yong, Wednesday, March 11, 2009, 12:28:45 AM, you wrote: > It goes beyond my current knowledge, now. How do you define > a custom data type as an instance of UA or Storable? just look at existing instances. basically, for complex data type, you just use instances for its basic types, plus you need to calculate offset of second and following fields (using sizeOf in Storable class) -- Best regards, Bulatmailto:bulat.zigans...@gmail.com ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re[2]: [Haskell-cafe] bytestring vs. uvector
Hello Don, Wednesday, March 11, 2009, 12:12:07 AM, you wrote: > Right, so my point stands: there's no difference now. If you can write a > Storable instance, you can write a UA et al instance. yes, if there is some class provided for this and not just hard-coded 4 or so base types > And GHC 6.6 was released what, 11 October 2006? So this has been the > case for a long time. unfortunately, Array library unboxed arrays still aren't based on any Unboxable *class* -- Best regards, Bulatmailto:bulat.zigans...@gmail.com ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re[2]: [Haskell-cafe] bytestring vs. uvector
Hello Xiao-Yong, Tuesday, March 10, 2009, 11:52:50 PM, you wrote: > So it's me who understand it wrong. If I want some high > performance array with elements of custom data type, I'm > stuck with Array, anyway? ForeignArray will be the best here. just make you type instance of Storable. if you need *movable* arrays, you will have to implement your own array datatype or use ArrayRef library and declare your type as instance of Unboxed -- Best regards, Bulatmailto:bulat.zigans...@gmail.com ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re[2]: [Haskell-cafe] bytestring vs. uvector
Hello Don, Tuesday, March 10, 2009, 11:01:31 PM, you wrote: >> if uavector use ghc's built-in unboxed array operations (as >> Data.Array.Unboxed does) then it's necessarily bounded to types >> supported by those operations > And what is Storable limited to? > Ultimately they're all limited to the primops for reading and writing, the full story: ghc up to 6.6 has slow access to ForeignArrays, as you may recall therefore, those primitives was added. ByteArra# plus those primitives was the only way to have unboxed arrays with fast access starting with 6.6, ForeignArray access is no-op, so we can just use obvious Ptr operations (via Storable class) to get unboxed arrays fast access. so, no more need for those special ByteArray# access operations but Array library still old, so any effort based on its spources, got the same restrictions also, ByteArray# may be unpinned, but afaik, this isn't really important - it can be coerced to Ptr for the period of one operation -- Best regards, Bulatmailto:bulat.zigans...@gmail.com ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re[2]: [Haskell-cafe] bytestring vs. uvector
Hello Don, Tuesday, March 10, 2009, 10:40:30 PM, you wrote: >> I think uvector only works with certain types that can be >> unboxed, while storablevector works with all types that >> instantiate Foreign.Storable.Storable. I don't know about >> vector. From the description of vector, I have the > That's interesting. I'd expect Storable and UA to have the same set of > inhabitants. Is there any difference? if uavector use ghc's built-in unboxed array operations (as Data.Array.Unboxed does) then it's necessarily bounded to types supported by those operations btw, i have extended this set in my own library by making a class which has implementation using those operations but may be extended further. OTOH, nowadays when ForeignArrays becomes as fast as built-in ones, i don't see any reasons for these built-in operations at all -- Best regards, Bulatmailto:bulat.zigans...@gmail.com ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
