Re: [Haskell-cafe] Using lenses
Simon, On Thu, 2013-10-03 at 08:07 +, Simon Peyton-Jones wrote: If you are using the lens library yourself, could you spare a few minutes to tell me how you are using it? I'm not a heavy 'lens'-user (yet), and this might not be the most pretty use-case from a theoretic point of view, but anyway: I use 'lens' in Kontiki [1], my implementation of Raft [2], a fairly recent consensus protocol for distributed systems (think Paxos multi-Paxos, Zookeeper,...). The Raft protocol description is fairly imperative (When in state S, if you receive a message M, broadcast message M'(M) to all other nodes, append entry E(M) to your log, and update the state to S'(S)), so in order to keep the code close to this description, I created a TransitionT monad transformer (where the underlying monad should, in some cases, provide access to the persisted log of the system, most likely requiring IO access, but this is abstracted over), which is basically RWST. The R environment gives access to system configuration (e.g. the set of nodes (or at least their names/IDs) part of the cluster), W is used to output/list Commands to be executed as part of a state transition, S gives access to the current state, and a transition should yield a new state (or the old one if e.g. a message is received which should be ignored in the current state). See [3] for an overview of the types involved. Finally, after this long introduction, the 'lens' part: thanks to the 'lens' operators, and my types involved in the TransitionT monad (specifically the R environment and S state) being lens'ified, accessing R S becomes fairly unobtrusive and more imperative-looking (which makes sense when using RWS). As an example, in [4], you see some parts of the current (Candidate) state being accessed: currentTerm - use cCurrentTerm votes - use cVotes Not using 'lens', this would be something (OTOH) like currentTerm - fmap _cCurrentTerm get which is less familiar from an imperative POV. Then also updating the current state 'in-place' (not for real, obviously): cVotes %= Set.insert sender And accessing the configuration (R) environment, e.g.: nodeId - view configNodeId See other handler functions in that module, or in the Follower and Leader modules in the same directory, for more examples. Regards, Nicolas [1] https://github.com/NicolasT/kontiki [2] https://ramcloud.stanford.edu/wiki/download/attachments/11370504/raft.pdf [3] https://github.com/NicolasT/kontiki/blob/master/src/Network/Kontiki/Types.hs [4] https://github.com/NicolasT/kontiki/blob/master/src/Network/Kontiki/Raft/Candidate.hs#L32 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Proposal: Partitionable goes somewhere + containers instances
I'd think partition :: t - Either t (t, t) might be more suited then... Nicolas On Sep 29, 2013 1:21 AM, Ryan Newton rrnew...@gmail.com wrote: subject change On Sun, Sep 29, 2013 at 3:31 AM, Mike Izbicki m...@izbicki.me wrote: I've got a Partitionable class that I've been using for this purpose: https://github.com/mikeizbicki/ConstraintKinds/blob/master/src/Control/ConstraintKinds/Partitionable.hs Mike -- Neat, that's a cool library! Edward -- ideally, where in the standard libraries should the Partitionable comonoid go? Btw, I'm not sure what the ideal return type for comappend is, given that it needs to be able to bottom out. Mike, our partition function's list return type seems more reasonable. Or maybe something simple would be this: *class Partitionable t where* * partition :: t - Maybe (t,t)* That is, at some point its not worth splitting and returns Nothing, and you'd better be able to deal with the 't' directly. So what I really want is for the *containers package to please get some kind of Partitionable instances! * Johan others, I would be happy to provide a patch if the class can be agreed on. This is important because currently the balanced tree structure of Data.Set/Map is an *amazing and beneficial property* that is *not* exposed at all through the API. For example, it would be great to have a parallel traverse_ for Maps and Sets in the Par monad. The particular impetus is that our new and enhanced Par monad makes extensive use of Maps and Sets, both the pure, balanced ones, and lockfree/inplace ones based on concurrent skip lists: http://www.cs.indiana.edu/~rrnewton/haddock/lvish/ Alternatively, it would be ok if there were a Data.Map.Internal module that exposed the Bin/Tip, but I assume people would rather have a clean Partitionable instance... Best, -Ryan On Sun, Sep 29, 2013 at 3:31 AM, Mike Izbicki m...@izbicki.me wrote: I've got a Partitionable class that I've been using for this purpose: https://github.com/mikeizbicki/ConstraintKinds/blob/master/src/Control/ConstraintKinds/Partitionable.hs The function called parallel in the HLearn library will automatically parallelize any homomorphism from a Partionable to a Monoid. I specifically use that to parallelize machine learning algorithms. I have two thoughts for better abstractions: 1) This Partitionable class is essentially a comonoid. By reversing the arrows of mappend, we get: comappend :: a - (a,a) By itself, this works well if the number of processors you have is a power of two, but it needs some more fanciness to get things balanced properly for other numbers of processors. I bet there's another algebraic structure that would capture these other cases, but I'm not sure what it is. 2) I'm working with parallelizing tree structures right now (kd-trees, cover trees, oct-trees, etc.). The real problem is not splitting the number of data points equally (this is easy), but splitting the amount of work equally. Some points take longer to process than others, and this cannot be determined in advance. Therefore, an equal split of the data points can result in one processor getting 25% of the work load, and the second processor getting 75%. Some sort of lazy Partitionable class that was aware of processor loads and didn't split data points until they were needed would be ideal for this scenario. On Sat, Sep 28, 2013 at 6:46 PM, adam vogt vogt.a...@gmail.com wrote: On Sat, Sep 28, 2013 at 1:09 PM, Ryan Newton rrnew...@gmail.com wrote: Hi all, We all know and love Data.Foldable and are familiar with left folds and right folds. But what you want in a parallel program is a balanced fold over a tree. Fortunately, many of our datatypes (Sets, Maps) actually ARE balanced trees. Hmm, but how do we expose that? Hi Ryan, At least for Data.Map, the Foldable instance seems to have a reasonably balanced fold called fold (or foldMap): fold t = go t where go (Bin _ _ v l r) = go l `mappend` (v `mappend` go r) This doesn't seem to be guaranteed though. For example ghc's derived instance writes the foldr only, so fold would be right-associated for a: data T a = B (T a) (T a) | L a deriving (Foldable) Regards, Adam ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Bytestring map/zipWith rationale
I did use that a couple of times (`xor`ing 2 ByteStrings together), and was surprised by the omission back then, but IIRC (can't validate now), there's a specialised zipWith (as proposed) in the module (with some other name, obviously), which is not exported, but used when you 'pack' the result of 'zipWith' when the result is '[Word8]'... You might want to look into that. Nicolas On Sep 12, 2013 8:11 PM, John Lato jwl...@gmail.com wrote: Carter: we don't have both. We have one function from each category. My guess is nobody's ever really needed a really fast zipWith :: (Word8-Word8-Word8) - ByteString - ByteString - ByteString; that's the only reason I can think of for its omission. On Thu, Sep 12, 2013 at 10:45 AM, Carter Schonwald carter.schonw...@gmail.com wrote: Scott: benchmark the two and you'll see why we have both :-) On Thursday, September 12, 2013, Scott Lawrence wrote: On Thu, 12 Sep 2013, Tom Ellis wrote: On Thu, Sep 12, 2013 at 09:21:20AM -0400, Scott Lawrence wrote: Something's always bothered me about map and zipWith for ByteString. Why is it map :: (Word8 - Word8) - ByteString - ByteString but zipWith :: (Word8 - Word8 - a) - ByteString - ByteString - [a] Well, what if you wanted to zipWith a function of type Word8 - Word8 - Foo instead of Word8 - Word8 - Word8? Then I would do what I do with map, and call `unpack` first. Either of the two options is usable: map :: (Word8 - Word8) - ByteString - ByteString zipWith :: (Word8 - Word8 - Word8) - ByteString - ByteString - ByteString (or) map :: (Word8 - a) - ByteString - [a] zipWith :: (Word8 - Word8 - a) - ByteString - ByteString - [a] I just don't understand why we have one from each. -- Scott Lawrence __**_ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/**mailman/listinfo/haskell-cafehttp://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Traversals of monomorphic containers (was: Re: mapM_ for bytestring)
# Redirected to haskell-cafe On Sun, 2013-09-01 at 14:58 +0400, Artyom Kazak wrote: Would this be an appropriate place to propose adding mapM_ (and then possibly mapM) to bytestring library? Was it suggested before? If yes, why was it rejected? This got me wondering: there are several type-classes useful for polymorphic container types, e.g. Functor, Foldable Traversable which all apply to some type of kind (* - *). Are there related things for monomorphic containers, like ByteString, Text or some newtype'd Vector with fixed element type, e.g. class MFunctor f a where mfmap :: (a - a) - f - f instance MFunctor ByteString Word8 where mfmap = ByteString.map or (maybe even better) class MFunctor f where type Elem mfmap :: (Elem - Elem) - f - f instance MFunctor ByteString where type Elem = Word8 mfmap = ByteString.map and similar for other classes. Nicolas ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Proposal: Generic conditions for 'if' and 'case'
On Sun, 2013-09-01 at 15:51 -0700, Wvv wrote:
I think it is an old idea, but nevertheless.
Now we have next functions:
if (a :: Bool) then x else y
case b of
a1 :: Bool - x1
a2 :: Bool - x2
...
Let we have generic conditions for 'if' and 'case':
class Boolean a where
toBool :: a - Bool
instance Boolean Bool where
toBool = id
instance Boolean [a] where
toBool [] = False
toBool _ = True
instance Boolean (Maybe a) where
toBool Nothing = False
toBool _ = True
instance Boolean Int where
toBool 0 = False
toBool _ = True
if' (a :: Boolean b) then x else y
case' d of
a1 :: Boolean b1 - x1
a2 :: Boolean b2 - x2
...
It is very easy to implement to desugar:
if' a then ... == if toBool ( a ) then ...
I wasn't at my computer when I sent my previous reply, so here's a more
full-fledged answer:
This is possible using the RebindableSyntax extension. Make sure to read
the documentation of the extension before using it, it might have some
unexpected implications.
Be careful when using this scheme as well... I'd think lots of
Haskell'ers would frown upon this kind of implicit conversions (they
remind me of Python and its __nonzero__ stuff).
Here's an example implementing your proposal:
{-# LANGUAGE RebindableSyntax #-}
import Prelude
class Boolean a where
toBool :: a - Bool
instance Boolean Bool where
toBool = id
instance Boolean [a] where
toBool = not . null
instance Boolean (Maybe a) where
toBool = maybe False (const True)
instance Boolean Int where
toBool = (/= 0)
ifThenElse :: Boolean a = a - b - b - b
ifThenElse i t e = case toBool i of
True - t
False - e
main :: IO ()
main = do
test False
test ([] :: [Int])
test [1]
test (Nothing :: Maybe Int)
test (Just 1 :: Maybe Int)
test (0 :: Int)
test (1 :: Int)
{- test 'c' fails to type-check: no instance Boolean Char defined!
-}
where
test v = putStrLn $ show v ++ is ++ (if v then true else
false)
which outputs
False is false
[] is false
[1] is true
Nothing is false
Just 1 is true
0 is false
1 is true
Using RebindableSyntax, 'if I then T else E' is rewritten into
'ifThenElse I T E' by the compiler, for whatever 'ifThenElse' is in
scope.
Nicolas
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Proposal: Generic conditions for 'if' and 'case'
I didn't test it, but you might want to look into the 'rebindable syntax' extension and its 'ifThenElse' feature. Nicolas On Sep 2, 2013 12:51 AM, Wvv vite...@rambler.ru wrote: I think it is an old idea, but nevertheless. Now we have next functions: if (a :: Bool) then x else y case b of a1 :: Bool - x1 a2 :: Bool - x2 ... Let we have generic conditions for 'if' and 'case': class Boolean a where toBool :: a - Bool instance Boolean Bool where toBool = id instance Boolean [a] where toBool [] = False toBool _ = True instance Boolean (Maybe a) where toBool Nothing = False toBool _ = True instance Boolean Int where toBool 0 = False toBool _ = True if' (a :: Boolean b) then x else y case' d of a1 :: Boolean b1 - x1 a2 :: Boolean b2 - x2 ... It is very easy to implement to desugar: if' a then ... == if toBool ( a ) then ... -- View this message in context: http://haskell.1045720.n5.nabble.com/Proposal-Generic-conditions-for-if-and-case-tp5735366.html Sent from the Haskell - Haskell-Cafe mailing list archive at Nabble.com. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] TypeLits Typeable
Hello Cafe,
I was playing around with TypeLits in combination with Typeable (using
GHC 7.7.7.20130812 FWIW), but was surprised to find Symbols aren't
Typeable, and as such the following doesn't work. Is this intentional,
or am I missing something?
Thanks,
Nicolas
{-# LANGUAGE DataKinds,
KindSignatures,
DeriveFunctor,
DeriveDataTypeable #-}
module Main where
import Data.Typeable
import GHC.TypeLits
data NoSymbol n a b = NoSymbol a b
deriving (Typeable)
data WithSymbol (n :: Symbol) a b = WithSymbol a b
deriving (Typeable)
data Sym
deriving (Typeable)
main :: IO ()
main = do
print $ typeOf (undefined :: NoSymbol Sym Int Int)
let d = undefined :: WithSymbol sym Int Int
{-
print $ typeOf d
No instance for (Typeable Symbol sym)
arising from a use of ‛typeOf’
-}
return ()
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Yet Another Forkable Class
On Fri, 2013-08-23 at 08:06 +, o...@okmij.org wrote: It will arbitrarily pick the first match in the former and fail to compile in the latter case. Of course we can have duplicate layers. In that case, the dynamically closest handler wins -- which sounds about right (think of reset in delimited control). Did anyone ever consider using type-level literals (strings) to 'name' effects (or transformer layers when using monad stacks)? A stupid example (OTOH) could be updateStats :: (Member (State min Int) r, Member (State max Int) r) = Int - Eff r () updateStats i = do min - askMin max - askMax when (i min) $ putMin i when (i max) $ putMax i where askMin :: Member (State min Int) r = Eff r Int askMin = ask putMax :: Member (State max Int) r = Int - Eff r () putMax = put -- askMax, putMin accordingly Using constraint synonyms/ConstraintKinds (e.g. type StateMax r = Member (State max Int) r) might reduce some notation overhead. Just a thought. Nicolas ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Stuck on design problem
All, Since I'm stuck on a coding problem and can't figure out how to proceed, I decided to call for help. I'm unable to get some code to typecheck, so maybe I'm missing something obvious on the implementation side, or more likely the design is completely wrong. Here's the idea: I have some code which uses some log of Entry a values (for some polymorphic type 'a'), and provides actions which output Command a values which should be interpreted by some wrapper code, e.g. adding some new entries to the log. These actions are implemented in a custom WriterT transformer (in this demo code, the original is more complex) to output Commands, while the inner monad should give access to the log entries (this could be in-memory like in the example, or stored on disk so using IO). You can find a trimmed-down version at https://gist.github.com/NicolasT/4230251f4f87f110d197 This doesn't type-check, and I'm not sure how to proceed. Am I taking a wrong approach? Do I need a different design? Thanks, Nicolas ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Failure deriving MonadRWS when using a type-family for the State part
All,
The following code results in a compilation error (I tried GHC 7.4.1 a
7.7.20130430 build):
{-# LANGUAGE TypeFamilies,
GeneralizedNewtypeDeriving,
StandaloneDeriving #-}
module Main where
import Control.Applicative
import Control.Monad.RWS
data C = C
data E = E
data S1 = S1 Int
type family I a :: *
type instance I S1 = Int
newtype T a s m r = T { unT :: RWST C [E] (I s) m r }
deriving ( Functor
, Applicative
, Monad
, MonadReader C
, MonadWriter [E]
, MonadState (I s)
, MonadRWS C [E] (I s)
, MonadTrans
)
Error:
No instance for (MonadState (I s) (T a s m))
arising from the 'deriving' clause of a data type declaration
Possible fix:
use a standalone 'deriving instance' declaration,
so you can specify the instance context yourself
When deriving the instance for (MonadRWS C [E] (I s) (T a s m))
Commenting out the MonadRWS line from the derivings list (i.e. the line
pointed at by the error) works as expected. I was somehow unable to get
a suitable standalone-deriving clause working, so didn't test that.
Is this expected?
Regards,
Nicolas
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Stuck on design problem
Miguel, On Mon, 2013-05-20 at 16:38 +0400, Miguel Mitrofanov wrote: :t runMemLog (runTransitionT $ demo 1) runMemLog (runTransitionT $ demo 1) :: MonadLog (MemLog a) () = Log a - ((), [Command ()]) That means, that foo, if you manage to compile it, would have type MonadLog (MemLog a) () = ((), [Command ()]). That means that in each call for foo it would be used as something of the type ((), [Command ()]), therefore giving no indication of which a should be used. The fact that there is only one instance of MonadLog (MemLog a) () - namely, the one with a = () - doesn't matter, as there could be others in the future. That's why GHC is complaining. So, you have to provide such indication by youself, replacing IntMap.empty with (IntMap.empty :: Log ()). Thanks for the pointer. I tried using an explicit type-annotation myself before as well, but that failed, and I found the root-cause: in my original code, the type-signature of runTransitionT was all wrong, instead of runTransitionT :: TransitionT a m a - m (a, [Command a]) this was intended to be runTransitionT :: TransitionT a m r - m (r, [Command a]) After making this change, and using an explicit type-signature for the empty IntMap as you suggested, things work as expected, see https://gist.github.com/NicolasT/4230251f4f87f110d197/revisions In my original code, this didn't help out somehow (some other things at play there as well), but another suggestion on this list did. Thanks for the help! Nicolas 20.05.2013, 15:36, Nicolas Trangez nico...@incubaid.com: All, Since I'm stuck on a coding problem and can't figure out how to proceed, I decided to call for help. I'm unable to get some code to typecheck, so maybe I'm missing something obvious on the implementation side, or more likely the design is completely wrong. Here's the idea: I have some code which uses some log of Entry a values (for some polymorphic type 'a'), and provides actions which output Command a values which should be interpreted by some wrapper code, e.g. adding some new entries to the log. These actions are implemented in a custom WriterT transformer (in this demo code, the original is more complex) to output Commands, while the inner monad should give access to the log entries (this could be in-memory like in the example, or stored on disk so using IO). You can find a trimmed-down version at https://gist.github.com/NicolasT/4230251f4f87f110d197 This doesn't type-check, and I'm not sure how to proceed. Am I taking a wrong approach? Do I need a different design? Thanks, Nicolas ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Stuck on design problem
Adam, On Mon, 2013-05-20 at 13:37 +0100, Adam Gundry wrote: Hi Nicolas, Your design doesn't look too unreasonable, though I haven't looked at in detail. I do have a quick observation regarding the implementation that I hope might help. Your gist defines class MonadLog m a where getEntry :: Index - m (Entry a) instance MonadLog (MemLog a) a and then you hit the error No instance for (MonadLog (MemLog a0) ()) which is a tell-tale ambiguity problem. This is a common issue with multi-parameter type classes. GHC cannot determine the existential variable a0, because it will not commit to the instance for MemLog in case a more specific instance turns up. One option is to turn on GADTs or TypeFamilies and write instance a ~ a' = MonadLog (MemLog a) a' which will allow the instance to match and generate the easily solved constraint a0 ~ (). You may need to do the same for other instances. Alternatively, you could add a functional dependency class MonadLog m a | m - a or use a type family: class MonadLog' m where type Element m getEntry' :: Index - m (Entry (Element m)) instance MonadLog' (MemLog a) where type Element (MemLog a) = a getEntry' i = flip (IntMap.!) i `fmap` ask Thanks a bunch. Using FunDeps, things work as expected, also in my more complicated original code. I chose the FunDeps approach because it seems the least intrusive (no need to clutter instances). Thanks a bunch for all help, this list is certainly one of the things which makes writing Haskell code a very rewarding experience. Thanks, Nicolas ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Failure deriving MonadRWS when using a type-family for the State part
The mistake might be on my side, since I expected the following to work
(but it doesn't, most likely for good reason, I didn't read any
TypeFamilies papers yet):
{-# LANGUAGE GeneralizedNewtypeDeriving, TypeFamilies #-}
module Main where
import Control.Monad.State
data S = S T
data T = T
type family F s :: *
type instance F S = T
newtype MT s m r = MT { unMT :: StateT (F s) m r }
deriving (Monad, MonadState (F s))
-- foo :: Monad m = MT S m T
-- foo = get
{-
Could not deduce (MonadState T (MT S m))
arising from a use of `get'
from the context (Monad m)
bound by the type signature for foo :: Monad m = MT S m T
at tf2.hs:17:1-9
Possible fix:
add (MonadState T (MT S m)) to the context of
the type signature for foo :: Monad m = MT S m T
or add an instance declaration for (MonadState T (MT S m))
In the expression: get
In an equation for `foo': foo = get
while GHCi says, as expected:
λ :i MT
newtype MT s m r = MT {unMT :: StateT (F s) m r}
-- Defined at tf2.hs:13:9
instance Monad m = Monad (MT s m) -- Defined at tf2.hs:14:13
instance Monad m = MonadState (F s) (MT s m)
-- Defined at tf2.hs:14:20
and
λ :t undefined :: F S
undefined :: F S :: T
-}
Nicolas
On Tue, 2013-05-21 at 00:08 +0200, Nicolas Trangez wrote:
All,
The following code results in a compilation error (I tried GHC 7.4.1 a
7.7.20130430 build):
{-# LANGUAGE TypeFamilies,
GeneralizedNewtypeDeriving,
StandaloneDeriving #-}
module Main where
import Control.Applicative
import Control.Monad.RWS
data C = C
data E = E
data S1 = S1 Int
type family I a :: *
type instance I S1 = Int
newtype T a s m r = T { unT :: RWST C [E] (I s) m r }
deriving ( Functor
, Applicative
, Monad
, MonadReader C
, MonadWriter [E]
, MonadState (I s)
, MonadRWS C [E] (I s)
, MonadTrans
)
Error:
No instance for (MonadState (I s) (T a s m))
arising from the 'deriving' clause of a data type declaration
Possible fix:
use a standalone 'deriving instance' declaration,
so you can specify the instance context yourself
When deriving the instance for (MonadRWS C [E] (I s) (T a s m))
Commenting out the MonadRWS line from the derivings list (i.e. the line
pointed at by the error) works as expected. I was somehow unable to get
a suitable standalone-deriving clause working, so didn't test that.
Is this expected?
Regards,
Nicolas
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Backward compatibility
On Fri, 2013-05-03 at 10:40 -0700, Hilco Wijbenga wrote: On 3 May 2013 09:44, Niklas Hambüchen m...@nh2.me wrote: While I certainly enjoy the discussion, how about addressing one of the original problems: On 02/05/13 13:27, Adrian May wrote: I just tried to use Flippi. It broke because of the syntax change so I tried WASH. I couldn't even install it because of the syntax change. I just fixed that in https://github.com/nh2/flippi (note that I have never seen this code before nor even known about it). https://github.com/nh2/flippi/commit/5e2fa93f82b4123d0d5b486209c3b722c4c1313d Had to delete 5 imports and convert one time value. Took me around 3 minutes. It seems fair to say that Haskell's designers lean more to evolution than maintaining backward compatibility. This reminds me of Go (the programming language). The approach chosen by Go's designers was to create a tool (gofix) that would automatically fix one's code to comply with the latest standard. See [http://talks.golang.org/2012/splash.article#TOC_17.] (yes, include that last period). Given the apparent simplicity of the changes needed to keep one's Haskell code up to snuff and the strong typing inherent in Haskell code, would it not be possible to create something similar? If there is a tool that moves (most of) one's code from Haskell version n to n+1 then making breaking changes would be even less of an issue. Just an idea, I have no clue about its feasibility... I mentioned the same on #haskell today. Something like Coccinelle (http://coccinelle.lip6.fr) semantic patches could be really useful to automate (some) API language changes. Somewhat like (but better than) the Python '2to3' tool. I think some message about a GSoC project regarding an AST-based refactoring tool was posted to this list. That might be a useful building block for such tool? Imagine one day whenever a new HP release is made, GitHub pull-requests are created automatically based on a automatically-generated patch verified through a compilationtest-cycle on TravisCI for GH-hosted packages published on Hackage. Nicolas ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] [extension]syntactic sugar for maps
On Wed, 2013-03-27 at 21:30 +0200, Răzvan Rotaru wrote: I am terribly missing some syntactic sugar for maps (associative data structures) in Haskell. I find myself using them more than any other data structure, and I think there is no big deal in adding some sugar for this to the language. I could not find out whether such an extension is beeing discussed. If not, I would like to propose and extension. Any help and suggestions are very welcome here. Thanks. http://hackage.haskell.org/trac/ghc/wiki/OverloadedLists comes to mind. Nicolas ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Streaming bytes and performance
On Tue, 2013-03-19 at 20:32 +, Don Stewart wrote:
Oh, I forgot the technique of inlining the lazy bytestring chunks, and
processing each chunk seperately.
$ time ./fast
4166680
./fast 1.25s user 0.07s system 99% cpu 1.325 total
Essentially inline Lazy.foldlChunks and specializes is (the inliner should
really get that).
And now we have a nice unboxed inner loop, which llvm might spot:
$ ghc -O2 -funbox-strict-fields fast.hs --make -fllvm
$ time ./fast
4166680
./fast 1.07s user 0.06s system 98% cpu *1.146 total*
So about 8x faster. Waiting for some non-lazy bytestring benchmarks... :)
You could try something like this using Conduit:
{-# LANGUAGE BangPatterns #-}
module Main (main) where
import Data.Conduit
import qualified Data.Conduit.List as L
import qualified Data.Conduit.Binary as B
import qualified Data.ByteString.Char8 as BS8
main :: IO ()
main = print = runResourceT (
B.sourceFile filename $$ L.fold (\(!a) (!b) - a + BS8.count ' ' b)
(0 :: Int))
where
filename = ...
Nicolas
{-# LANGUAGE BangPatterns #-}
import Data.ByteString.Internal
import Data.ByteString.Unsafe
import qualified Data.ByteString.Char8 as S
import qualified Data.ByteString.Lazy.Char8 as L
import qualified Data.ByteString.Lazy.Internal as L
import System.IO.Posix.MMap.Lazy
main = do
f - unsafeMMapFile test.txt
print . new 0 $ L.toChunks f
new :: Int - [ByteString] - Int
new i [] = i
new i (x:xs) = new (add i x) xs
-- jump into the fast path
{-# INLINE add #-}
add :: Int - ByteString - Int
add !i !s | S.null s = i
| isSpace' x = add (i+1) xs
| otherwise = add i xs
where T x xs = uncons s
data T = T !Char ByteString
uncons s = T (w2c (unsafeHead s)) (unsafeTail s)
isSpace' c = c == '\n'|| c == ' '
{-# INLINE isSpace' #-}
On Tue, Mar 19, 2013 at 7:36 PM, Don Stewart don...@gmail.com wrote:
Just for fun. Here's some improvements. about 6x faster.
I'd be interested to see what io-streams could do on this.
Using a 250M test file.
-- strict state monad and bang patterns on the uncons and accumulator
argument:
$ time ./A
4166680
./A 8.42s user 0.57s system 99% cpu 9.037 total
-- just write a loop
$ time ./A
4166680
./A 3.84s user 0.26s system 99% cpu 4.121 total
-- switch to Int
$ time ./A
4166680
./A 1.89s user 0.23s system 99% cpu 2.134 total
-- custom isSpace function
$ time ./A
4166680
./A 1.56s user 0.24s system 99% cpu 1.808 total
-- mmap IO
$ time ./A
4166680
./A 1.54s user 0.09s system 99% cpu 1.636 total
Here's the final program:
{-# LANGUAGE BangPatterns #-}
import qualified Data.ByteStringas S
import qualified Data.ByteString.Lazy.Char8 as L
import System.IO.Posix.MMap.Lazy
main = do
f - unsafeMMapFile test.txt
print $ go 0 f
where
go :: Int - L.ByteString - Int
go !a !s = case L.uncons s of
Nothing - a
Just (x,xs) | isSpaceChar8 x - go (a+1) xs
| otherwise - go a xs
isSpaceChar8 c = c == '\n'|| c == ' '
{-# INLINE isSpaceChar8 #-}
On Mon, Mar 18, 2013 at 8:53 AM, Konstantin Litvinenko
to.darkan...@gmail.com wrote:
Hi All!
I tune my toy project for performance and hit the wall on simple, in
imperative world, task. Here is the code that model what I'm trying to
achieve
import qualified Data.ByteString.Lazy as L
import Data.Word8(isSpace)
import Data.Word
import Control.Monad.State
type Stream = State L.ByteString
get_byte :: Stream (Maybe Word8)
get_byte = do
s - get
case L.uncons s of
Nothing - return Nothing
Just (x, xs) - put xs return (Just x)
main = do
f - L.readFile test.txt
let r = evalState count_spaces f
print r
where
count_spaces = go 0
where
go a = do
x - get_byte
case x of
Just x' - if isSpace x' then go (a + 1) else go a
Nothing - return a
It takes the file and count spaces, in imperative way, consuming bytes
one by one. The problem is: How to rewrite this to get rid of constant
allocation of state but still working with stream of bytes? I can rewrite
this as one-liner L.foldl, but that doesn't help me in any way to optimize
my toy project where all algorithms build upon consuming stream of bytes.
PS. My main lang is C++ over 10 years and I only learn Haskell :)
__**_
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/**mailman/listinfo/haskell-cafehttp://www.haskell.org/mailman/listinfo/haskell-cafe
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
___
Re: [Haskell-cafe] The state of binary (de)serialization
On Thu, 2013-02-28 at 01:22 -0800, wren ng thornton wrote:
On 2/27/13 2:17 AM, Vincent Hanquez wrote:
Two major problems of lazy bytestrings is that:
* you can't pass it to a C bindings easily.
* doing IO with it without rewriting the chunks, can sometimes (depending
how the lazy bytestring has been produced) result in a serious
degradation of
performance calling syscalls on arbitrary and small chunks (e.g.
socket's 'send').
If you're on a POSIX system, you can always make use of
unix-bytestring[1]. In particular, for lazy ByteStrings the function you
want is System.Posix.IO.ByteString.Lazy.fdWritev which performs a single
syscall to write all the chunks, and without manually concatenating them.
[1] http://hackage.haskell.org/package/unix-bytestring
FWIW: since I saw Network.Socket.ByteString.Lazy while browsing the
'network' docs I used it ('sendAll'), and when strace'ing my demo-app (a
bad habit of mine) to my pleasant surprise (although it shouldn't have
been much of a surprise given the high-quality of Haskell unix IO
libraries) I noticed it was using writev for the bytestring chunks.
So using plain 'network' (at least the version I'm using, don't know
when this was introduced) should be sufficient to get efficient LBS
socket IO handling.
Nicolas
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] The state of binary (de)serialization
On Mon, 2013-02-25 at 11:59 -0800, Johan Tibell wrote: On Mon, Feb 25, 2013 at 4:30 AM, Nicolas Trangez nico...@incubaid.comwrote: - cereal supports chunk-based 'partial' parsing (runGetPartial). It looks like support for this is introduced in recent versions of 'binary' as well (runGetIncremental) Yes. Binary now support an incremental interface. We intend to make sure binary has all the same functionality as cereal. We'd like to move away from having two packages if possible and since binary has the larger installed user base we're trying to make that the go-to package. This will certainly make things more obvious (and maybe ready for HP inclusion?). - cereal can output a strict bytestring (runPut) or a lazy one (runPutLazy), whilst binary only outputs lazy ones (runPut) The lazy one is more general and you can use toStrict (from bytestring) to get a strict ByteString from a lazy one, without loss of performance. Sure. Turned out I was using lazy bs' anyway so switched to 'binary' for deserialization. - Next to binary and cereal, there's bytestring's Builder interface for serialization, and Simon Meier's blaze-binary prototype Simon's builder (originally developed in blaze-binary) has been merged into the bytestring package. In the future binary will just re-export that builder. I was referring to https://github.com/meiersi/blaze-binary Overall: what's the advised future-proof strategy of handling binary (de)serialization? Use binary or the builder from bytestring whenever you can. Since the builder in bytestring was recently added you might have to fall back to blaze-builder if you believe your users can't rely on the latest version of bytestring. I switched to Builder for serialization. It seems to create 'more strict' lazy bytestrings than the cereal based code (as in: cereal seems to create a new Chunk whenever appending a lazy bytestring, whilst Builder concats them into a single chunk, at least for the short strings I've been using). The Monoidal interface feels very natural, maybe even more natural than the Monad interface of PutM in binary/cereal: instance Argument a = Argument [a] where put l = word32LE cnt s where (cnt, s) = foldr (\e (c, m) - (c + 1, put e m)) (0, mempty) l Thanks, Nicolas ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] The state of binary (de)serialization
On Wed, 2013-02-27 at 07:49 +0100, Vincent Hanquez wrote: On Mon, Feb 25, 2013 at 01:30:40PM +0100, Nicolas Trangez wrote: ... I've been looking at the same thing lately, and i've been quite surprised, to say the least, by the usual go-to packages (cereal, binary). Performance wise this is hard to summarize, but if you serialize something small and have a easy to compute size (e.g. fixed size structure), i would advise against using any kind of builder structure (builder,cereal,binary), and go directly at the Storable level, if performance need to be on-par other languages. My initial interpretation is that the builder initial cost is quite high, and only get amortized if the number of operations is quite high (and have less bytestrings). So if you have many structures encoded in one encoding operation it's probably ok-ish. I've made the following benchmark when i was doing my experiments, that shows basic serialization of bytestring-y data structures: * bclass is a simple function that use bytestring concat or append * bclass+io is a simple function that use mutable bytestring + poke to create the bytestring * cereal is cereal's encode function * binary is binary's encode function * builder is bytestring's builder. * simple bytestring of constant size: sz * n bytestrings of same size: n*sz * n bytestrings of different size: sz+sz2+.. * n bytestrings plus a w32 prefixed size: len+n*sz Obviously, caveat emptor: http://tab.snarc.org/others/benchmark-bytestring-serialization.html Let me know if anyone want the source file. These are some really interesting (and very consistent) results, thanks! I guess I should do some benchmarking myself and maybe change some thing around (heck, now I'm using Builder to serialize constants :-P). It might be worth to share these benchmarks with the 'binary' and 'bytestring'/'blaze-builder' maintainers? Nicolas ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] The state of binary (de)serialization
All,
In order to implement some network protocol clients recently, I needed
binary serialization of commands and deserialization of responses
('Command - ByteString' and 'ByteString - Response' functions,
preferably for both strict as well as lazy ByteStrings).
My go-to packages have always been 'binary' and 'cereal', but I was
wondering about the current (and future) state/goals:
- cereal supports chunk-based 'partial' parsing (runGetPartial). It
looks like support for this is introduced in recent versions of 'binary'
as well (runGetIncremental)
- cereal can output a strict bytestring (runPut) or a lazy one
(runPutLazy), whilst binary only outputs lazy ones (runPut)
- Next to binary and cereal, there's bytestring's Builder interface for
serialization, and Simon Meier's blaze-binary prototype
There are some blog posts and comments out there about merging cereal
and binary, is this what's the goal/going on (cfr runGetIncremental)?
In my use-case I think using Builder instead of binary/cereal's PutM
monad shouldn't be a major problem. Is this advisable performance-wise?
Overall: what's the advised future-proof strategy of handling binary
(de)serialization?
Thanks,
Nicolas
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] performance question
On Wed, 2013-02-13 at 08:39 -0800, David Thomas wrote: One way in which regexps are foreign to Haskell-think is that, if they break, they generally break at run-time. This could be ameliorated with template haskell Care to elaborate on the ameliorate using TH part? I figure regexes would be mostly used to parse some runtime-provided string, so how could compile-TH provide any help? Nicolas ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Layer on a layer of record syntax in the type synonym?
On Fri, 2012-12-21 at 04:36 -0900, Christopher Howard wrote:
Using a simple type I gave earlier from my monadic type question...
code:
data Socket3 a b c = Socket3 a b c
deriving (Show)
Is it possible somehow to layer on record syntax onto a synonym of the type?
The idea would be something like this...
code:
type SpaceShip =
Socket3 { engine :: Last Engine
, hull :: Last Hull
, guns :: [Guns]
}
...purely for the convenience. But this doesn't seem to work with type
as it assumes you are referring to already made constructors, and
evidently newtype only allows use of a single record. I could wrap it
in a normal data declaration but that would add an extra layer of
complexity I think.
Although this 'Socket3' data type which all of a sudden should be
aliased as 'SpaceShip' feels/looks really strange (are you sure that's
the right way to reach whatever the goal is?), you could use lenses:
import Control.Lens
data Socket3 a b c = Socket3 a b c
deriving (Show)
data Last a = Last a deriving Show
data Engine = Engine deriving Show
data Hull = Hull deriving Show
data Gun = Gun deriving Show
type SpaceShip = Socket3 (Last Engine) (Last Hull) [Gun]
engine :: Simple Lens SpaceShip (Last Engine)
engine = lens get lset
where
get (Socket3 a _ _) = a
lset (Socket3 _ b c) a' = Socket3 a' b c
hull :: Simple Lens SpaceShip (Last Hull)
hull = lens get lset
where
get (Socket3 _ b _ ) = b
lset (Socket3 a _ c) b' = Socket3 a b' c
guns :: Simple Lens SpaceShip [Gun]
guns = lens get lset
where
get (Socket3 _ _ c) = c
lset (Socket3 a b _) = Socket3 a b
main :: IO ()
main = do
print $ s0 ^. engine
print $ s0 ^. guns
let s1 = guns .~ [Gun, Gun] $ s0
print s1
print $ s1 ^. guns
where
s0 :: SpaceShip
s0 = Socket3 (Last Engine) (Last Hull) []
(I'm no Lens expert so maybe there are better ways than manually
creating these Lens instances, or make them shorter/abstract something
out)
Nicolas
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] (L)GPL libraries Haskell/GHC (was: Re: ANNOUNCE: tie-knot library)
Note: IANAL On Tue, 2012-12-11 at 17:45 -0800, David Thomas wrote: On Tue, Dec 11, 2012 at 5:35 PM, Brandon Allbery allber...@gmail.comwrote: (Oddly enough, GPL is not the only open source license.) There was no implication to the contrary. It was stated that BSD is a *weaker* license - this is true in the sense that it has fewer requirements (in particular, no copyleft) - and that strong copyleft licenses such as the GPL should be preferred as they do more to bolster the free software community. You can disagree with this claim (there are arguments both ways - delving into them is not my point here) but please try not to bring in straw men. Actually the library is made available under the LGPL-3 license, according to its README, not the GPL (although the latter is implicit, of course). In the Haskell world this does have a different effect compared to when one uses the LGPL for, say, a C library though, since (at least for now) GHC uses/defaults to static linking, which IIRC (though IANAL) turns the LGPL into GPL, so this has a severe impact for application authors. This might be something people aren't aware of when releasing Haskell libraries using the LGPL. I tend to use the LGPL myself for most library-style projects, and do so as well for Haskell code (although I'm aware of the drawbacks), but I'm perfectly fine with people linking the libs statically as long as they comply to the license as if they were using dynamic loading. If anyone knows some standard license which boils down to obligations like LGPL but OK for static linking as well, please let me know. Nicolas ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] ANN: zeromq3-conduit, Conduit bindings for zeromq3-haskell
Hi, I published zeromq3-conduit, a small library which facilitates using zeromq3-haskell, a binding for ZeroMQ 3.x, in a Conduit-based application. The 'examples' folder in the source repository contains ports of the zeromq3-haskell examples. The library also contains a module which might ease using plain zeromq3-haskell: it lifts some of the IO actions into MonadIO and provides a ReaderT which carries a Context around, so sockets can be created without passing this around explicitly. Package is on Hackage [1] and Github [2]. Comments/feedback/pull requests/... welcome! Nicolas [1] http://hackage.haskell.org/package/zeromq3-conduit [2] https://github.com/NicolasT/zeromq3-conduit/ ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Conduit and pipelined protocol processing using a threadpool
On Wed, 2012-11-28 at 09:17 +0200, Michael Snoyman wrote: On Tue, Nov 27, 2012 at 7:25 PM, Nicolas Trangez nico...@incubaid.com wrote: Michael, On Tue, 2012-11-27 at 17:14 +0200, Michael Snoyman wrote: I think the stm-conduit package[1] may be helpful for this use case. Each time you get a new command, you can fork a thread and give it the TBMChan to write to, and you can use sourceTBMChan to get a source to send to the client. That's +- what I had in mind. I did find stm-conduit before and did try to get the thing working using it, but these attempts failed. I attached an example which might clarify what I intend to do. I'm aware it contains several potential bugs (leaking threads etc), but that's beside the question ;-) If only I could figure out what to put on the 3 lines of comment I left in there... Thanks for your help, Nicolas The issue is that you're trying to put everything into a single Conduit, which forces reading and writing to occur in a single thread of execution. Since you want your writing to be triggered by a separate event (data being available on the Chan), you're running into limitations. The reason network-conduit provides a Source for the incoming data and a Sink for outgoing data is specifically to address your use case. You want to take the data from the Source and put it into the Chan in one thread, and take the data from the other Chan and put it into the Sink in a separate thread. Something like: myApp appdata = do chan1 - ... chan2 - ... replicateM_ 5 $ forkIO $ worker chan1 chan2 forkIO $ appSource appdata $$ sinkTBMChan chan1 sourceTBMChan chan2 $$ appSink appdata You'll also want to make sure to close chan1 and chan2 to make sure that your threads stop running. Thanks, I +- figured that out last night. Only thing left is managing the handshake before forking off the workers, if possible (otherwise things become very messy IMHO), but ResumableSources etc might be of use here. Maybe there's a library somewhere in here... Thanks a bunch, Nicolas ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Conduit and pipelined protocol processing using a threadpool
All, I've written a library to implement servers for some protocol using Conduit (I'll announce more details later). The protocol supports pipelining, i.e. a client can send a 'command' which contains some opaque 'handle' chosen by the client, the server processes this command, then returns some reply which contains this handle. The client is free to send other commands before receiving a reply for any previous request, and the server can process these commands in any order, sequential or concurrently. The library is based on network-conduit's Application style [1], as such now I write code like (OTOH) application :: AppData IO - IO () application client = appSource client $= handler $$ appSink client where handler = do negotiateResult - MyLib.negotiate liftIO $ validateNegotiateResult negotiateResult MyLib.sendInformation 123 loop loop = do command - MyLib.getCommand case command of CommandA handle arg - do result - liftIO $ doComplexProcessingA arg MyLib.sendReply handle result loop Disconnect - return () This approach handles commands in-order, sequentially. Since command processing can involve quite some IO operations to disk or network, I've been trying to support pipelining on the server-side, but as of now I was unable to get things working. The idea would be to have a pool of worker threads, which receive work items from some channel, then return any result on some other channel, which should then be returned to the client. This means inside loop I would have 2 sources: commands coming from the client (using 'MyLib.getCommand :: MonadIO m = Pipe ByteString ByteString o u m Command'), as well as command results coming from the worker threads through the result channel. Whenever the first source produces something, it should be pushed onto the work queue, and whenever the second on yields some result it should be sent to the client using 'MyLib.sendReply :: Monad m = Handle - Result - Pipe l i ByteString u m ()' I've been fighting this for a while and haven't managed to get something sensible working. Maybe the design of my library is flawed, or maybe I'm approaching the problem incorrectly, or ... Has this ever been done before, or would anyone have some pointers how to tackle this? Thanks, Nicolas [1] http://hackage.haskell.org/packages/archive/network-conduit/0.6.1.1/doc/html/Data-Conduit-Network.html#g:2 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Conduit and pipelined protocol processing using a threadpool
Michael,
On Tue, 2012-11-27 at 17:14 +0200, Michael Snoyman wrote:
I think the stm-conduit package[1] may be helpful for this use case.
Each time you get a new command, you can fork a thread and give it the
TBMChan to write to, and you can use sourceTBMChan to get a source to
send to the client.
That's +- what I had in mind. I did find stm-conduit before and did try
to get the thing working using it, but these attempts failed.
I attached an example which might clarify what I intend to do. I'm aware
it contains several potential bugs (leaking threads etc), but that's
beside the question ;-)
If only I could figure out what to put on the 3 lines of comment I left
in there...
Thanks for your help,
Nicolas
{-# LANGUAGE Rank2Types #-}
module Main where
import Data.Conduit
import qualified Data.Conduit.List as CL
import Data.Conduit.TMChan
import Control.Applicative
import Control.Concurrent (forkIO)
import Control.Concurrent.STM (atomically)
import Control.Monad (forM_)
import Control.Monad.IO.Class (MonadIO, liftIO)
data Command = Add Int Int
| Disconnect
deriving (Show)
data Reply = Result Int
deriving (Show)
application :: MonadIO m = GConduit Int m String
application = do
-- Create input and output channels to/from worker threads
(chanIn, chanOut) - liftIO $ (,) $ newTBMChanIO 10 * newTBMChanIO 10
-- Spawn some worker threads
liftIO $ forM_ [0..5] $ \i - forkIO $ processCommands i chanIn chanOut
-- How to make
-- sourceTBMChan chanOut
-- something of which all produced values are yield'ed by this Conduit?
loop chanIn
where
-- Loop retrieves one command from our source and pushes it to the
-- worker threads input channel, then loops
loop :: MonadIO m = TBMChan Command - GConduit Int m String
loop chan = do
liftIO $ putStrLn Enter loop
cmd - getCommand
liftIO $ do
putStrLn $ Got command: ++ show cmd
atomically $ writeTBMChan chan cmd
case cmd of
Disconnect - return ()
_ - loop chan
-- getCommand fetches and parses a single command from our source
getCommand :: Monad m = GSink Int m Command
getCommand = do
v - await
case v of
Nothing - return Disconnect
Just i - return $ Add i 1
-- processCommands reads commands from a given input channel, processes
-- them, and pushes the result to a given output channel
processCommands :: Int - TBMChan Command - TBMChan Reply - IO ()
processCommands i chanIn chanOut = do
putStrLn $ Enter processCommands ++ show i
cmd - atomically $ readTBMChan chanIn
putStrLn $ show i ++ read command: ++ show cmd
case cmd of
Nothing - return ()
Just (Add a b) - do
atomically $ writeTBMChan chanOut (Result (a + b))
putStrLn $ show i ++ pushed result
processCommands i chanIn chanOut
Just Disconnect - return ()
main :: IO ()
main = do
res - CL.sourceList [1..20] $= application $$ CL.consume
putStrLn $ Result: ++ show res
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Custom Enum instance with non-consecutive indices
All, I've been working on a server implementation of an existing networking protocol. The protocol uses magic constants in some places (e.g. to tag message types), as well as bitfields, or a combination of both packed in a single value. I created data types for both the identifiers as well as the bitfield masks, e.g. import Data.Bits data MessageType = Message1 | Message2 | Message5 data MessageFlag = Flag1 | Flag2 Since I need to be able to get a numeric representation of them, I thought making a custom Enum instance would make sense: instance Enum MessageType where fromEnum a = case a of Message1 - 1 Message2 - 2 Message5 - 5 toEnum n | n == 1 = Message1 | n == 2 = Message2 | n == 5 = Message5 instance Enum MessageFlag where fromEnum a = case a of Flag1 - 1 `shiftL` 0 Flag2 - 1 `shiftL` 1 toEnum n | n == 1 `shiftL` 0 = Flag1 | n == 1 `shiftL` 1 = Flag2 This is not a complete definition (not to mention non-exhaustive pattern matches), so I was wondering what the best practices are to extend this so these instances are 'correct'? I've been trying several approaches, but all of them failed on code like getFlags :: Int - [MessageFlag] getFlags i = filter (\v - (i .. fromEnum v) /= 0) [Flag1 ..] Unless I hard-code all options in the last list, of course, but this is obviously not the intention. Any help or pointers (including This is not how Enum is intended to be used) would be appreciated! Thanks, Nicolas ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Custom Enum instance with non-consecutive indices
On Sat, 2012-11-17 at 16:52 +0200, Roman Cheplyaka wrote: Hi Nicolas, The simplest approach would be to use the standard (derived) Enum instance that would be used for enumerations (like [Flag1..]), and have your own functions to convert to/from magic constants. Sure, but that kind-of defeats the purpose... fromEnum/toEnum would then return some nonsensical integer value. Nicolas ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Custom Enum instance with non-consecutive indices
On Sat, 2012-11-17 at 16:27 +0100, Herbert Valerio Riedel wrote: what do you hope to gain from making your flags type an instance of the Enum class? Among others, the ability to list all flags using [Flag1 ..] How is this handled for libraries wrapping C libs which use some bitset structure? Nicolas ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] performance issues with popCount
On Fri, 2012-09-07 at 15:55 -0700, Johan Tibell wrote: On Fri, Sep 7, 2012 at 4:54 AM, Nicolas Trangez nico...@incubaid.com wrote: On Thu, 2012-09-06 at 12:07 -0700, Johan Tibell wrote: Have a look at the popCount implementation for e.g. Int, which are written in C and called through the FFI: https://github.com/ghc/packages-ghc-prim/blob/master/cbits/popcnt.c Out of interest: isn't this compiled into the popCnt# primop (and popcnt instruction on SSE4.2)? It's the other way around the popCnt# primop is compiled into either calls to these C functions or into the popcnt instruction, if -msse4.2 is given. Yes, indeed. I was referring to Data.Bits.popCount, but looking back this was indeed completely non-obvious. My bad. I recently noticed Data.IntSet also contains a fairly basic bitcount implementation [1]. Is this kept as-is for a reason, instead of using popCount from Data.Bits? I don't think so, except that we want to support the last 3 released versions of GHC so we need to have a fallback if Data.Bits.popCount isn't defined. I might look into creating a (CPP-based) patch then. Nicolas ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] performance issues with popCount
On Thu, 2012-09-06 at 12:07 -0700, Johan Tibell wrote: Have a look at the popCount implementation for e.g. Int, which are written in C and called through the FFI: https://github.com/ghc/packages-ghc-prim/blob/master/cbits/popcnt.c Out of interest: isn't this compiled into the popCnt# primop (and popcnt instruction on SSE4.2)? I recently noticed Data.IntSet also contains a fairly basic bitcount implementation [1]. Is this kept as-is for a reason, instead of using popCount from Data.Bits? Nicolas [1] https://github.com/ghc/packages-containers/blob/master/Data/IntSet/Base.hs#L1459 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Vector: blit'ing a source into a destination
All, For some code I need to alter an (unboxed or storable) vector by blit'ing (copying) data from another (smaller) vector into it, overwriting the existing data. I couldn't find a function for this in the vector API, so I hacked something up which cuts the jobs (see below), but it feels really ugly/dirty. Any ideas about a better way to tackle this, and wouldn't this be a useful addition to the standard API? Thanks, Nicolas import Data.Vector.Unboxed (Unbox, Vector) import qualified Data.Vector.Unboxed as UV import qualified Data.Vector.Unboxed.Mutable as MUV blit :: Unbox a = Vector a - Vector a - Int - Vector a blit src dest offset = UV.modify act dest where act dest' = do let slice = MUV.slice offset len dest' src' - UV.unsafeThaw src MUV.copy slice src' len = UV.length src main :: IO () main = do let v = UV.fromList [0 :: Int .. 10] b = UV.fromList [20, 30, 40] putStrLn $ v = ++ show v putStrLn $ b = ++ show b putStrLn $ blit b v 2 = ++ show (blit b v 2) -- v = fromList [0,1,2,3,4,5,6,7,8,9,10] -- b = fromList [20,30,40] -- blit b v 2 = fromList [0,1,20,30,40,5,6,7,8,9,10] ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Memory corruption issues when using newAlignedPinnedByteArray, GC kicking in?
All, While working on my vector-simd library, I noticed somehow memory I'm using gets corrupted/overwritten. I reworked this into a test case, and would love to get some help on how to fix this. Previously I used some custom FFI calls to C to allocate aligned memory, which yields correct results, but this has a significant (+- 10x) performance impact on my benchmarks. Later on I discovered the newAlignedPinnedByteArray# function, and wrote some code using this. Here's what I did in the test case: I created an MVector instance, with the exact same implementation as vector's Data.Vector.Storable.Mutable.MVector instance, except for basicUnsafeNew where I pass one more argument to mallocVector [1]. I also use 3 different versions of mallocVector (depending on compile-time flags): mallocVectorOrig [2]: This is the upstream version, discarding the integer argument I added. Then here's my first attempt, very similar to the implementation of mallocPlainForeignPtrBytes [3] at [4] using GHC.* libraries. Finally there's something similar at [5] which uses the 'primitive' library. The test case creates vectors of increasing size, then checks whether they contain the expected values. For the default implementation this works correctly. For both others it fails at some random size, and the values stored in the vector are not exactly what they should be. I don't understand what's going on here. I suspect I lack a reference (or something along those lines) so GC kicks in, or maybe the buffer gets relocated, whilst it shouldn't. Basically I'd need something like GHC.ForeignPtr.mallocPlainAlignedForeignPtrBytes :: Int - Int - IO (ForeignPtr a) Thanks, Nicolas [1] https://gist.github.com/3084806#LC37 [2] https://gist.github.com/3084806#LC119 [3] http://hackage.haskell.org/packages/archive/base/latest/doc/html/src/GHC-ForeignPtr.html [4] https://gist.github.com/3084806#LC100 [5] https://gist.github.com/3084806#LC81 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] vector-simd: some code available, and some questions
On Sun, 2012-07-08 at 20:49 +1000, Reiner Pope wrote: I've not been following this thread very closely, but it seems like what you're trying to do may be related to Geoffrey Mainland's work on SIMD support in GHC. See [1] for his SIMD-enabled version of the vector library. He's also written some blog posts about this [2]. Thanks. I knew about this before (did at least some research ;-)), yet I think the scope is somewhat different (assuming my playground work has any scope at all...). I think Geoffreys work goes much further than what I'm trying to accomplish: it's not my intention to write SIMD-based code at a high level (which is, obviously, a great thing, but beyond my capabilities as of now I'm afraid). What I'd like to achieve is to be able to write SIMD code at the C level (or have SIMD code generated somehow through LLVM/Clang(/Poly?), ISPC or others), and call into this from Haskell, providing some guarantees w.r.t. alignment (and maybe length-in-bytes as well?) of the passed vector pointers (which you don't get when using plain Storable vectors as far as I could find/see). Consider it a very short-term stop-gap solution until the tools are in place to write high-level code which yields the same performance as a more low-level (assembly-level) implementation. Nicolas ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] vector-simd: some code available, and some questions
On Sun, 2012-07-08 at 10:27 +0200, Gábor Lehel wrote: On Sun, Jul 8, 2012 at 3:05 AM, Nicolas Trangez nico...@incubaid.com wrote: I implemented the inductive alignment calculation over One and Twice (good idea, and easy to do), but I don't get the thing about superclasses. I've been trying several approaches (including definitions based on forall and other trickery I never used before), but didn't get things to work, at least: the compiler always said I'd need UndecidableInstances, and that sounds scary... Care to elaborate? All I meant was class (Alignment n, Alignment a) = AlignedToAtLeast n a but I got a bit ahead of myself, because that rules out the instance on tuples. Maybe that's one of the things I ran into, can't remember OTOH. (I suppose you *could* write some kind of Alignment instance for them, taking their minimum or something, but that's getting a bit too subversive for me). Heh, it already feels evil-but-in-a-good-and-powerful-way to me right now ;-) Thanks for all your input! The alternative, if you want both Alignment as a superclass and the ability to constrain multiple types at once, is to have the above, remove the instance on tuples, and instead something like: class (AlignedToAtLeast n a, AlignedToAtLeast n b) = AlignedToAtLeast2 n a b instance (AlignedToAtLeast n a, AlignedToAtLeast n b) = AlignedToAtLeast2 n a b class (AlignedToAtLeast n a, AlignedToAtLeast n b, AlignedToAtLeast n c) = AlignedToAtLeast3 n a b c instance (AlignedToAtLeast n a, AlignedToAtLeast n b, AlignedToAtLeast n c) = AlignedToAtLeast3 n a b c (feel free to think of better names!) unsafeXorSSE42 :: (Storable a, SV.AlignedToAtLeast3 SV.A16 o1 o2 o3) = SV.Vector o1 a - SV.Vector o2 a - SV.Vector o3 a Implemented in [1]. Code says unsafeXorSSE42 :: (Storable a, SV.AlignedToAtLeast3 SV.A16 o1 o2 o3) = SV.Vector o1 a - SV.Vector o2 a - SV.Vector o3 a ghci says unsafeXorSSE42 :: (AlignedToAtLeast (Data.Vector.SIMD.Mutable.Twice A8) o2, AlignedToAtLeast (Data.Vector.SIMD.Mutable.Twice A8) o1, AlignedToAtLeast (Data.Vector.SIMD.Mutable.Twice A8) o3, Foreign.Storable.Storable a) = Vector o1 a - Vector o2 a - Vector o3 a which is the same thing, so should do. That will require UndecidableInstances, but all that means is that GHC can't prove to itself that instance checking will terminate. So you could end up getting the compiler into an infinite loop (or in practice, to exceed its recursion limit). But it doesn't allow anything unsafe to happen at runtime, and there's plenty of perfectly good instances which terminate even if GHC can't prove it. I see. I had some rather unsafe/undecidable-by-a-developer things in mind, but I guess I should read up on some of the language extensions GHC provides. Thanks, Nicolas [1] https://github.com/NicolasT/vector-simd/commit/8f934891c9630a96ce009fafa7f6ba70df306d4f ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] vector-simd: some code available, and some questions
All,
After my message of yesterday [1] I got down to it and implemented
something along those lines. I created a playground repository
containing the code at [2]. Initial benchmark results at [3]. More about
the benchmark at the end of this email.
First some questions and requests for help:
- I'm stuck with a typing issue related to 'sizeOf' calculation at [4].
I tried a couple of things, but wasn't able to figure out how to fix it.
- I'm using unsafePerformIO at [5], yet I'm not certain it's OK to do
so. Are there better (safer/performant/...) ways to get this working?
- Currently Alignment phantom types (e.g. A8 and A16) are not related to
each other: a function (like Data.Vector.SIMD.Algorithms.unsafeXorSSE42)
can have this signature:
unsafeXorSSE42 :: Storable a = SV.Vector SV.A16 a - SV.Vector SV.A16 a
- SV.Vector SV.A16 a
Yet, imaging I'd have an SV.Vector SV.A32 Word8 vector at hand, the
function should accept it as well (a 32-byte aligned vector is also
16-byte aligned). Is there any way to encode this at the type level?
That's about it :-)
As of now, I only implemented a couple of the vector API functions (the
ones required to execute my benchmark). Adding the others should be
trivial.
The benchmark works with Data.Vector.{Unboxed|Storable}.Vector (UV and
SV) vectors of Word8 values, as well as my custom
Data.Vector.SIMD.Vector type (MV) using 16-byte alignment (MV.Vector
MV.A16 Word8).
benchUV, benchSV and benchMV all take 2 pre-calculated Word8 vectors of
given size (1024 and 4096) and xor them pairwise into the result using
zipWith xor. benchMVA takes 2 suitable MV vectors and xor's them into
a third using a rather simple and unoptimized C implementation using
SSE4.2 intrinsics [6]. This could be enhanced quite a bit (I guess using
the prim calling convention, FFI overhead can be reduced as well).
Currently, only vectors of a multiple of 32 bytes are supported (mostly
because of laziness on my part).
As you can see, the zipWith Data.Vector.SIMD implementation is slightly
slower than the Data.Vector.Storable based one. I didn't perform much
profiling yet, but I suspect allocation and ForeignPtr creation is to
blame, this seems to be highly optimized in
GHC.ForeignPtr.mallocPlainForeignPtrBytes as used by
Data.Vector.Storable.
Thanks for any input,
Nicolas
[1] http://www.haskell.org/pipermail/haskell-cafe/2012-July/102167.html
[2] https://github.com/NicolasT/vector-simd/
[3] http://linode2.nicolast.be/files/vector-simd-xor1.html
[4]
https://github.com/NicolasT/vector-simd/blob/master/src/Data/Vector/SIMD/Algorithms.hs#L46
[5]
https://github.com/NicolasT/vector-simd/blob/master/src/Data/Vector/SIMD/Algorithms.hs#L43
[6]
https://github.com/NicolasT/vector-simd/blob/master/cbits/vector-simd.c#L47
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] vector-simd: some code available, and some questions
On Sat, 2012-07-07 at 21:59 +0200, Gábor Lehel wrote: An alternative solution is to encode all of the alignments in unary, which is more general; if they're all going to be a power of two you can store just the logarithm: data One data Twice n -- not practical to call it Double :) class AlignedToAtLeast n a instance AlignedToAtLeast One One instance AlignedToAtLeast One (Twice a) instance AlignedToAtLeast n a = AlignedToAtLeast (Twice n) (Twice a) type A1 = One type A4 = Twice (Twice A1) type A8 = Twice A4 type A16 = Twice A8 type A32 = Twice A16 and you can apply the same private class thing from above if you want. Very ingenious, thanks! I pushed this into [1], although export lists of all modules most likely will need some love once things get into shape. This also allows functions to become more general: unsafeXorSSE42 :: (Storable a, SV.AlignedToAtLeast SV.A16 o1, SV.Alignment o1, SV.AlignedToAtLeast SV.A16 o2, SV.Alignment o2, SV.AlignedToAtLeast SV.A16 o3, SV.Alignment o3) = SV.Vector o1 a - SV.Vector o2 a - SV.Vector o3 a I wonder whether GHC's upcoming type-level numerals could be useful in this situation as well. Nicolas [1] https://github.com/NicolasT/vector-simd/commit/a4f13745eb24d87a3628af13109f3e1d8232c925 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] vector-simd: some code available, and some questions
On Sat, 2012-07-07 at 21:13 +0200, Nicolas Trangez wrote: As you can see, the zipWith Data.Vector.SIMD implementation is slightly slower than the Data.Vector.Storable based one. I didn't perform much profiling yet, but I suspect allocation and ForeignPtr creation is to blame, this seems to be highly optimized in GHC.ForeignPtr.mallocPlainForeignPtrBytes as used by Data.Vector.Storable. I got the MV benchmark on-par with SV by reworking the allocation mechanism: no more FFI involved, but based on GHC.Exts.newAlignedPinnedByteArray# and some other trickery, see [1]. This could still be improved a little by using PlainPtr, but this is not exported by GHC.ForeignPtr. This did have a pretty big performance-impact on the SIMD-based benchmark, compare [2] to the old one [3]. I have no clue why the 4096 case now only uses twice the time of the 1024 one, unlike the expected 4x (+- as before). Nicolas [1] https://github.com/NicolasT/vector-simd/commit/5ec539167254435ef4e7d308706dcafae09504d2 [2] http://linode2.nicolast.be/files/vector-simd-xor2.html [3] http://linode2.nicolast.be/files/vector-simd-xor1.html ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] vector-simd: some code available, and some questions
On Sun, 2012-07-08 at 01:40 +0200, Gábor Lehel wrote: unsafeXorSSE42 :: (Storable a, SV.AlignedToAtLeast SV.A16 o1, SV.Alignment o1, SV.AlignedToAtLeast SV.A16 o2, SV.Alignment o2, SV.AlignedToAtLeast SV.A16 o3, SV.Alignment o3) = SV.Vector o1 a - SV.Vector o2 a - SV.Vector o3 a I wonder if you could get that a bit shorter... I suppose you could write: instance (AlignedToAtLeast n a, AlignedToAtLeast n b) = AlignedToAtLeast n (a, b) instance (AlignedToAtLeast n a, AlignedToAtLeast n b, AlignedToAtLeast n c) = AlignedToAtLeast n (a, b, c) ...and so on... Once again, nifty! And implemented in [1]. though it feels a little strange semantically (relating a tuple to a scalar), but I don't see what harm can come of it. And then you can just write SV.AlignedToAtLeast SV.A16 (o1, o2, o3) in signatures. You can also make (Alignment n, Alignment a) a superclass constraint of AlignedToAtLeast, and write instances for Alignment inductively on One and Twice, and then you don't have to write Alignment o1 etc. separately either. So the signature would be just: unsafeXorSSE42 :: (Storable a, SV.AlignedToAtLeast SV.A16 (o1, o2, o3)) = SV.Vector o1 a - SV.Vector o2 a - SV.Vector o3 a which is friendlier. I implemented the inductive alignment calculation over One and Twice (good idea, and easy to do), but I don't get the thing about superclasses. I've been trying several approaches (including definitions based on forall and other trickery I never used before), but didn't get things to work, at least: the compiler always said I'd need UndecidableInstances, and that sounds scary... Care to elaborate? Thanks! Nicolas [1] https://github.com/NicolasT/vector-simd/commit/aedf25460b410e04a3d103befea59ebcb3903fdc ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] vector, alignment and SIMD through FFI
Hello Cafe, Recently I've been playing with the implementation of an algorithm, for which we already have highly-optimized implementations available (in plain C/C++ as well as OCaml with calls to C through FFI). The algorithm works on buffers/arrays/vectors/whatever you want to call it, which needs to be combined in certain ways. This can be highly optimized by using SIMD instructions (like the ones provides by several SSE versions). I'd like to get a to Haskell version which is comparable in efficiency as the existing versions, whilst remaining as 'functional' as possible. I don't mind jumping into some low-level C glue and FFI (using ccall or custom primops), but this should be limited. Currently I have something working (still highly unoptimized) using (unboxed) vectors from the vector package, using mutable versions within a well-contained ST environment in some places. One hot zone of the current version is combining several vectors, and the performance of this operation could be greatly improved by using SIMD instructions. There's one catch though: when using these, memory should be aligned on certain boundaries (16 byte in this case). First and foremost, to be able to pass my vectors to some C functions, I should change my code into using Storable vectors (which should be fine, I guess I can expect similar performance characteristics?). I couldn't find any information on alignment guarantees of these vectors though... Which is how I get to my question: are there any such guarantees? If not, are there any pointers to how to proceed with this? I guess tracking alignment at the type level should be the goal, you can find some code trying to explain my reasoning up to now at the end of this email. I have some issues with this: - I'd need to re-implement almost all vector operations, which seems stupid. - It doesn't actually work right now ;-) - It'd be nice to be able to encode 'compatible' alignment: as an example, a 16 byte aligned buffer is also 8 byte aligned... I hope the above explains somewhat my goal. Any thoughts help on this would be very welcome! Thanks, Nicolas module Data.Vector.SIMD ( -- ... ) where import qualified Data.Vector.Storable as SV import Foreign.Storable (Storable, sizeOf) import Foreign.Ptr (Ptr, FunPtr) import Foreign.ForeignPtr (ForeignPtr, newForeignPtr) import System.IO.Unsafe (unsafePerformIO) class Alignment a where alignment :: a - Int data A8Byte instance Alignment A8Byte where alignment _ = 8 data A16Byte instance Alignment A16Byte where alignment _ = 16 newtype Alignment a = SIMDVector a b = V (SV.Vector b) replicate :: (Alignment a, Storable b) = a - Int - b - SIMDVector a b replicate a n b = V v where ptr = unsafePerformIO $ do v - _mm_malloc n (alignment a) -- memset etc return v v = SV.unsafeFromForeignPtr0 ptr n -- These are 2 _stub versions of the procedures since xmminstr.h (or mm_malloc.h -- when using GCC) contains them as inline procedures which are not available -- as-is in a library. There should be some C module which exports -- _mm_malloc_stub and _mm_free_stub, which simply includes xmminstr.h and calls -- the underlying procedures. foreign import ccall _mm_malloc_stub _mm_malloc_stub :: Int - Int - IO (Ptr a) foreign import ccall _mm_free_stub _mm_free_stub :: FunPtr (Ptr a - IO ()) _mm_malloc :: Storable a = Int - Int - IO (ForeignPtr a) _mm_malloc s l = do -- This fails: -- Ambiguous type variable `a0' in the constraint: -- (Storable a0) arising from a use of `sizeOf' -- v - c_mm_malloc (s * sizeOf (undefined :: a)) l newForeignPtr _mm_free_stub undefined -- This allocates a 16 byte aligned output buffer, takes 2 existing ones and -- calls some FFI function to perform some magic. -- The implementation could run inside ST, if the FFI import (which e.g. works -- on a mutable buffer and returns IO ()) is lifted into ST using unsafeIOToST mySIMDFun :: SIMDVector A16Byte a - SIMDVector A16Byte a - SIMDVector A16Byte a mySIMDFun a b = undefined ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] vector, alignment and SIMD through FFI
On Fri, 2012-07-06 at 13:43 -0700, Thomas DuBuisson wrote: On Fri, Jul 6, 2012 at 1:06 PM, Nicolas Trangez nico...@incubaid.com wrote: -- This fails: -- Ambiguous type variable `a0' in the constraint: -- (Storable a0) arising from a use of `sizeOf' Here you can either tie a type knot using proxy types or you can use the scoped type variable language extension. Guess I'll have to do some reading ;-) Thanks. Perhaps I'm missing something specific to your use, but for the alignment issue you should be OK just calling allocBytes or one of its variants. I made some noise about this a bit ago and it resulted in some extra words in the report under mallocBytes: The block of memory is sufficiently aligned for any of the basic foreign types that fits into a memory block of the allocated size. Which I'm pretty sure GHC did, and still does, follow. Hmh... as far as I could find, mallocBytes basically does what malloc(3) does, which is 8-byte alignment if I'm not mistaken on my x86-64 Linux system. I could use those and the over-allocate-and-offset tricks, but... that's ugly unless strictly necessary ;-) Normally posix_memalign or memalign or valloc or _mm_malloc should provide what I need as-is. Except, when using those and vector's unsafeFromForeignPtr0, all I get is a Vector a, which no longer has any alignment information in the type, so I can't write a function which only accepts N-aligned vectors. As a result, I'd need to be very careful only to pass aligned vectors to it (checking manually), add code to handle pre/post-alignment bytes in my SIMD functions (slow and stupid), or live with it and let my application crash at random. I found some work by Oleg Kiselyov and Chung-chieh Shan at [1] which might be related, yet as of now I feel like that's too general for my purpose (e.g. I don't see how to integrate it with vector). Thanks, Nicolas [1] http://okmij.org/ftp/Haskell/types.html#ls-resources ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Long-running request/response protocol server using enumerator/iterator/iterIO/pipes/conduits/...
Hello Cafe, Some time ago I tried to implement a network service using iteratee (or enumerator, can't remember), but gave up in the end. More recently I wanted to create something similar (a similar protocol), but failed again. So I'm looking for some example code or something similar (Google only helped slightly). First of all, I don't care which API/library to use, I guess for my purpose all of enumerator, iteratee, iterIO, pipes, conduits,... are OK, so all feedback is welcome. Here's the catch. Most examples out there implement some server which accepts a single client request, interprets it, creates a response, returns this, and closes the connection (or something alike, think HTTP). The protocol I'd like to implement is different: it's long-running using repeated requests responses on a single client connection. Basically, a client connects and sends some data to the server (where the length of this data is encoded in the header). Now the server reads parses this (binary) data, sets up some initial state for this client connection (e.g. opening a file handle), and returns a reply. Now the client can send another request, server parses/interprets it using the connection state, sends a reply, and so on. Might sound easy (and actually it's pretty easy in most other languages I know, including an OCaml implementation), yet I fail to figure out how to get this done using some enumerator-style library. Thanks for any help, I'll most likely write up something if I get things working for future reference. Nicolas ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Long-running request/response protocol server using enumerator/iterator/iterIO/pipes/conduits/...
On Tue, 2012-06-26 at 21:32 +0200, Christopher Done wrote: On 26 June 2012 21:22, Nicolas Trangez nico...@incubaid.com wrote: Might sound easy (and actually it's pretty easy in most other languages I know, including an OCaml implementation), yet I fail to figure out how to get this done using some enumerator-style library. Well, it's easy in Haskell, too. Just use the standard libraries. Sure, that could work. If you want to mess around with these still-in-research iteratees and eumerators for the composability then go for it, but when it's hard or weird, you can't really blame that on the language. :-) Make no mistake, I'm not blaming anything except my own inability to figure this out ;-) Thanks, Nicolas ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Long-running request/response protocol server using enumerator/iterator/iterIO/pipes/conduits/...
On Tue, 2012-06-26 at 22:39 +0300, Michael Snoyman wrote: I've run into those kinds of problems in the past as well. In general, interleaving of data streams can be difficult with enumerator. That's the reason I added connect-and-resume to conduit. I use the technique in warp[1], which in fact *does* support multiple request/response pairs due to connection keep-alive. But the code base isn't the easiest introduction to the technique. If there's interest, I'll try to put together a blog post on using connect-and-resume to solve this kind of problem. Thank you, Michael. I thought about HTTP keep-alive as well, but felt reluctant to start by looking at a 'large' codebase like warp... Anyway, what you point to seems reasonable to interpret, I should be able to write something similar based on this (even though I never used Conduits/ResourceT before). Thanks! Nicolas ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] haskell tcp server
On Thu, 2012-01-19 at 13:12 +, Alexander V Vershilov wrote: Hello. I'm interested if there exists some library like warp but only for tcp. Or maybe some web page with skeleton for such server or with some variants, eg. concurent/single-threaded. I know that each service can have it's specific properties and realization, but there are many common things left. If it matter — my main task is to create server for protobuf protocol, maybe for such a task there are already created solutions. You could take a look at https://github.com/jamwt/haskell-scalable-server Nicolas ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] __GLASGOW_HASKELL__ for GHC 7.4
On Tue, 2012-01-17 at 15:35 +0400, Eugene Kirpichov wrote: Hi, I'm fixing a build error in a package that depends on the RTS API, which changed in 7.4, using #if __GLASGOW_HASKELL__ = 740. However, build log shows that __GLASGOW_HASKELL__ is still 704, not 740 as I'd expect. Is this a bug? I don't think so. I recently came across [1]: ''' Stable Releases Stable branches are numbered x.y, where y is even. Releases on the stable branch x.y are numbered x.y.z, where z (= 1) is the patchlevel number. Patchlevels are bug-fix releases only, and never change the programmer interface to any system-supplied code. However, if you install a new patchlevel over an old one you will need to recompile any code that was compiled against the old libraries. The value of __GLASGOW_HASKELL__ (see Section 4.11.3, “Options affecting the C pre-processor”) for a major release x.y.z is the integer xyy (if y is a single digit, then a leading zero is added, so for example in version 6.8.2 of GHC we would have __GLASGOW_HASKELL__==608). ''' As such, release 7.4.0 would yield __GLASGOW_HASKELL__ 704. Nicolas [1] http://www.haskell.org/ghc/docs/7.0.4/html/users_guide/version-numbering.html ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] State Machine Composition
On Wed, 2011-12-21 at 18:34 +, Daniel Waterworth wrote: I made this simple state machine combinator library today. I think it works as a simple example of a good use for GADTs. https://gist.github.com/1507107 Any way to do something along the lines of type StateChange a = SC a a handleChange :: Monad m = StateChange ConnectionState - m () handleChange (SC Closed Opening) = return () handleChange (SC Opening Closed) = return () handleChange (SC Closed Open) = return () where the last line would yield a type checking error at compile time, and the compiler can warn about handleChange not being exhaustive (for the valid state change steps)? Nicolas ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
