Re: [Haskell-cafe] How to implement the mean function
What compiler errors are you getting? -deech On Fri, Jul 1, 2011 at 12:55 AM, Ruohao Li liruo...@gmail.com wrote: Hi guys, I just started learning some Haskell. I want to implement a mean function to compute the mean of a list. The signature of the function is: mean :: (Num a, Fractional b) = [a] - b But when I implement this simple function, the compiler keep whining at me on type errors. I know this is wrong: mean xs = sum xs / length xs But how to get it right? Thanks. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] How to implement the mean function
For mean xs = sum xs / length xs, I got the following: test.hs:8:10: No instance for (Fractional Int) arising from a use of `/' at test.hs:8:10-27 Possible fix: add an instance declaration for (Fractional Int) In the expression: sum xs / length xs In the definition of `mean': mean xs = sum xs / length xs test.hs:8:10: Couldn't match expected type `b' against inferred type `Int' `b' is a rigid type variable bound by the type signature for `mean' at test.hs:7:27 In the expression: sum xs / length xs In the definition of `mean': mean xs = sum xs / length xs test.hs:8:19: Couldn't match expected type `a' against inferred type `Int' `a' is a rigid type variable bound by the type signature for `mean' at test.hs:7:13 In the second argument of `(/)', namely `length xs' In the expression: sum xs / length xs In the definition of `mean': mean xs = sum xs / length xs On Fri, Jul 1, 2011 at 2:00 PM, aditya siram aditya.si...@gmail.com wrote: What compiler errors are you getting? -deech On Fri, Jul 1, 2011 at 12:55 AM, Ruohao Li liruo...@gmail.com wrote: Hi guys, I just started learning some Haskell. I want to implement a mean function to compute the mean of a list. The signature of the function is: mean :: (Num a, Fractional b) = [a] - b But when I implement this simple function, the compiler keep whining at me on type errors. I know this is wrong: mean xs = sum xs / length xs But how to get it right? Thanks. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] How to implement the mean function
(/) operates on a Fractional instance... but length returns an Int, which is not a Fractional. You can convert the Int to a Fractional instance: mean xs = sum xs / fromIntegral (length xs) or try an integer division: mean xs = sum xs `div` length xs -n On Thu, Jun 30, 2011 at 10:55 PM, Ruohao Li liruo...@gmail.com wrote: Hi guys, I just started learning some Haskell. I want to implement a mean function to compute the mean of a list. The signature of the function is: mean :: (Num a, Fractional b) = [a] - b But when I implement this simple function, the compiler keep whining at me on type errors. I know this is wrong: mean xs = sum xs / length xs But how to get it right? Thanks. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] How to implement the mean function
For mean xs = sum xs / fromIntegral (length xs), I got the following: test.hs:8:10: Could not deduce (Fractional a) from the context (Num a, Fractional b) arising from a use of `/' at test.hs:8:10-42 Possible fix: add (Fractional a) to the context of the type signature for `mean' In the expression: sum xs / fromIntegral (length xs) In the definition of `mean': mean xs = sum xs / fromIntegral (length xs) test.hs:8:10: Couldn't match expected type `b' against inferred type `a' `b' is a rigid type variable bound by the type signature for `mean' at test.hs:7:27 `a' is a rigid type variable bound by the type signature for `mean' at test.hs:7:13 In the expression: sum xs / fromIntegral (length xs) In the definition of `mean': mean xs = sum xs / fromIntegral (length xs) And the div way will do integer division, which is not what I want. On Fri, Jul 1, 2011 at 2:07 PM, Nathan Howell nathan.d.how...@gmail.comwrote: (/) operates on a Fractional instance... but length returns an Int, which is not a Fractional. You can convert the Int to a Fractional instance: mean xs = sum xs / fromIntegral (length xs) or try an integer division: mean xs = sum xs `div` length xs -n On Thu, Jun 30, 2011 at 10:55 PM, Ruohao Li liruo...@gmail.com wrote: Hi guys, I just started learning some Haskell. I want to implement a mean function to compute the mean of a list. The signature of the function is: mean :: (Num a, Fractional b) = [a] - b But when I implement this simple function, the compiler keep whining at me on type errors. I know this is wrong: mean xs = sum xs / length xs But how to get it right? Thanks. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] How to implement the mean function
Additionally, this SO question[0] is nearly identical, and provides a little more elaboration. [0]:http://stackoverflow.com/questions/2376981/haskell-types-frustrating-a-simple-average-function On Jul 1, 2011, at 2:07 AM, Ruohao Li wrote: For mean xs = sum xs / length xs, I got the following: test.hs:8:10: No instance for (Fractional Int) arising from a use of `/' at test.hs:8:10-27 Possible fix: add an instance declaration for (Fractional Int) In the expression: sum xs / length xs In the definition of `mean': mean xs = sum xs / length xs test.hs:8:10: Couldn't match expected type `b' against inferred type `Int' `b' is a rigid type variable bound by the type signature for `mean' at test.hs:7:27 In the expression: sum xs / length xs In the definition of `mean': mean xs = sum xs / length xs test.hs:8:19: Couldn't match expected type `a' against inferred type `Int' `a' is a rigid type variable bound by the type signature for `mean' at test.hs:7:13 In the second argument of `(/)', namely `length xs' In the expression: sum xs / length xs In the definition of `mean': mean xs = sum xs / length xs On Fri, Jul 1, 2011 at 2:00 PM, aditya siram aditya.si...@gmail.com wrote: What compiler errors are you getting? -deech On Fri, Jul 1, 2011 at 12:55 AM, Ruohao Li liruo...@gmail.com wrote: Hi guys, I just started learning some Haskell. I want to implement a mean function to compute the mean of a list. The signature of the function is: mean :: (Num a, Fractional b) = [a] - b But when I implement this simple function, the compiler keep whining at me on type errors. I know this is wrong: mean xs = sum xs / length xs But how to get it right? Thanks. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe Computer Science is no more about computers than astronomy is about telescopes. -- Edsger Dijkstra 398E692F.asc Description: application/apple-msg-attachment PGP.sig Description: This is a digitally signed message part ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] How to implement the mean function
The problem is that you need to convert (length xs) to a Num, then return a Fractional. On Fri, Jul 1, 2011 at 2:07 PM, Nathan Howell nathan.d.how...@gmail.com wrote: (/) operates on a Fractional instance... but length returns an Int, which is not a Fractional. You can convert the Int to a Fractional instance: mean xs = sum xs / fromIntegral (length xs) or try an integer division: mean xs = sum xs `div` length xs -n On Thu, Jun 30, 2011 at 10:55 PM, Ruohao Li liruo...@gmail.com wrote: Hi guys, I just started learning some Haskell. I want to implement a mean function to compute the mean of a list. The signature of the function is: mean :: (Num a, Fractional b) = [a] - b But when I implement this simple function, the compiler keep whining at me on type errors. I know this is wrong: mean xs = sum xs / length xs But how to get it right? Thanks. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Patterns for processing large but finite streams
At Fri, 1 Jul 2011 09:39:32 +0400, Eugene Kirpichov wrote: Hi, I'm rewriting timeplot to avoid holding the whole input in memory, and naturally a problem arises: How to represent large but finite streams and functions that process them, returning other streams or some kinds of aggregate values? Examples: * Adjacent differences of a stream of numbers * Given a stream of numbers with times, split it into buckets by time of given width and produce a stream of (bucket, 50%,75% and 90% quantiles in this bucket) * Sum a stream of numbers Is this, perhaps, what comonads are for? Or iteratees? Sounds like a good job for iteratees. Summing a stream of numbers is just an Iteratee. Transcoding a stream into another stream is a job for an Inum (Iteratee-enumerator) or enumeratee, depending on which package's nomenclature you use. You have three implementations to choose from: - http://hackage.haskell.org/package/iteratee(original) - http://hackage.haskell.org/package/enumerator (John Milikin's re-write) - http://hackage.haskell.org/package/iterIO (my 3rd-generation attempt) David ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Patterns for processing large but finite streams
Eugene Kirpichov wrote: I'm rewriting timeplot to avoid holding the whole input in memory, and naturally a problem arises: How to represent large but finite streams and functions that process them, returning other streams or some kinds of aggregate values? Examples: * Adjacent differences of a stream of numbers * Given a stream of numbers with times, split it into buckets by time of given width and produce a stream of (bucket, 50%,75% and 90% quantiles in this bucket) * Sum a stream of numbers Is this, perhaps, what comonads are for? Or iteratees? Plain old lazy lists? Best regards, Heinrich Apfelmus -- http://apfelmus.nfshost.com ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Patterns for processing large but finite streams
Plain old lazy lists do not allow me to combine multiple concurrent computations, e.g. I cannot define average from sum and length. 2011/7/1 Heinrich Apfelmus apfel...@quantentunnel.de: Eugene Kirpichov wrote: I'm rewriting timeplot to avoid holding the whole input in memory, and naturally a problem arises: How to represent large but finite streams and functions that process them, returning other streams or some kinds of aggregate values? Examples: * Adjacent differences of a stream of numbers * Given a stream of numbers with times, split it into buckets by time of given width and produce a stream of (bucket, 50%,75% and 90% quantiles in this bucket) * Sum a stream of numbers Is this, perhaps, what comonads are for? Or iteratees? Plain old lazy lists? Best regards, Heinrich Apfelmus -- http://apfelmus.nfshost.com ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe -- Eugene Kirpichov Principal Engineer, Mirantis Inc. http://www.mirantis.com/ Editor, http://fprog.ru/ ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] How to implement the mean function
Thanks for the SO link, change the Num a constraint to Real a and using realToFrac then it just works. On Fri, Jul 1, 2011 at 2:11 PM, Jack Henahan jhena...@uvm.edu wrote: Additionally, this SO question[0] is nearly identical, and provides a little more elaboration. [0]: http://stackoverflow.com/questions/2376981/haskell-types-frustrating-a-simple-average-function On Jul 1, 2011, at 2:07 AM, Ruohao Li wrote: For mean xs = sum xs / length xs, I got the following: test.hs:8:10: No instance for (Fractional Int) arising from a use of `/' at test.hs:8:10-27 Possible fix: add an instance declaration for (Fractional Int) In the expression: sum xs / length xs In the definition of `mean': mean xs = sum xs / length xs test.hs:8:10: Couldn't match expected type `b' against inferred type `Int' `b' is a rigid type variable bound by the type signature for `mean' at test.hs:7:27 In the expression: sum xs / length xs In the definition of `mean': mean xs = sum xs / length xs test.hs:8:19: Couldn't match expected type `a' against inferred type `Int' `a' is a rigid type variable bound by the type signature for `mean' at test.hs:7:13 In the second argument of `(/)', namely `length xs' In the expression: sum xs / length xs In the definition of `mean': mean xs = sum xs / length xs On Fri, Jul 1, 2011 at 2:00 PM, aditya siram aditya.si...@gmail.com wrote: What compiler errors are you getting? -deech On Fri, Jul 1, 2011 at 12:55 AM, Ruohao Li liruo...@gmail.com wrote: Hi guys, I just started learning some Haskell. I want to implement a mean function to compute the mean of a list. The signature of the function is: mean :: (Num a, Fractional b) = [a] - b But when I implement this simple function, the compiler keep whining at me on type errors. I know this is wrong: mean xs = sum xs / length xs But how to get it right? Thanks. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe Computer Science is no more about computers than astronomy is about telescopes. -- Edsger Dijkstra ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] compare iteratee with python's yield
I just read several tutorials on iteratee, i find that iteratee is similar to python's generator, both allow streamlined data processing. For example, i can implement enumFile and printChunks in python like this: EOF = None def enum_file(bufsize, filename): with open(filename) as input: while True: data = input.read(bufsize) if not data: break yield data yield EOF def print_chunks(print_empty, generator): for chunk in generator: if chunk==EOF: print 'EOF' return if len(chunk)==0 and not print_empty: continue print chunk print_chunks(True, enum_file(2, data)) But i find iteratee far more complicated than python's generator, is that because iteratee can do something python's generator can't, or i simply need to be more familar with functional programming style. -- http://www.yi-programmer.com/blog/ ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Patterns for processing large but finite streams
Sure you can. runningAverage :: Int - [Double] - [Double] runningAverage n xs = let chunk = take n xs in (sum chunk / length chunk) : runningAverage (tail xs) Lazy lists are absolutely ideal for this purpose. Regards, Malcolm On 1 Jul 2011, at 07:33, Eugene Kirpichov wrote: Plain old lazy lists do not allow me to combine multiple concurrent computations, e.g. I cannot define average from sum and length. 2011/7/1 Heinrich Apfelmus apfel...@quantentunnel.de: Eugene Kirpichov wrote: I'm rewriting timeplot to avoid holding the whole input in memory, and naturally a problem arises: How to represent large but finite streams and functions that process them, returning other streams or some kinds of aggregate values? Examples: * Adjacent differences of a stream of numbers * Given a stream of numbers with times, split it into buckets by time of given width and produce a stream of (bucket, 50%,75% and 90% quantiles in this bucket) * Sum a stream of numbers Is this, perhaps, what comonads are for? Or iteratees? Plain old lazy lists? Best regards, Heinrich Apfelmus -- http://apfelmus.nfshost.com ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe -- Eugene Kirpichov Principal Engineer, Mirantis Inc. http://www.mirantis.com/ Editor, http://fprog.ru/ ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Patterns for processing large but finite streams
I meant the average of the whole list - given a sumS and lengthS (S for Stream), write meanS as something like liftS2 (/) sumS lengthS. Or is that possible with lazy lists too? (looks like arrows actually - which arrow is appropriate here?) 2011/7/1 Malcolm Wallace malcolm.wall...@me.com: Sure you can. runningAverage :: Int - [Double] - [Double] runningAverage n xs = let chunk = take n xs in (sum chunk / length chunk) : runningAverage (tail xs) Lazy lists are absolutely ideal for this purpose. Regards, Malcolm On 1 Jul 2011, at 07:33, Eugene Kirpichov wrote: Plain old lazy lists do not allow me to combine multiple concurrent computations, e.g. I cannot define average from sum and length. 2011/7/1 Heinrich Apfelmus apfel...@quantentunnel.de: Eugene Kirpichov wrote: I'm rewriting timeplot to avoid holding the whole input in memory, and naturally a problem arises: How to represent large but finite streams and functions that process them, returning other streams or some kinds of aggregate values? Examples: * Adjacent differences of a stream of numbers * Given a stream of numbers with times, split it into buckets by time of given width and produce a stream of (bucket, 50%,75% and 90% quantiles in this bucket) * Sum a stream of numbers Is this, perhaps, what comonads are for? Or iteratees? Plain old lazy lists? Best regards, Heinrich Apfelmus -- http://apfelmus.nfshost.com ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe -- Eugene Kirpichov Principal Engineer, Mirantis Inc. http://www.mirantis.com/ Editor, http://fprog.ru/ ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe -- Eugene Kirpichov Principal Engineer, Mirantis Inc. http://www.mirantis.com/ Editor, http://fprog.ru/ ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Patterns for processing large but finite streams
On Fri, Jul 1, 2011 at 12:21 PM, Eugene Kirpichov ekirpic...@gmail.com wrote: I meant the average of the whole list - given a sumS and lengthS (S for Stream), write meanS as something like liftS2 (/) sumS lengthS. Or is that possible with lazy lists too? Sure you can. Sum, length and mean could be calculated as left fold. If you need to calculate more that one statistic at time you can combine accumulators sum = foldl (+) 0 length = foldl (\n _ - n+1) 0 data Mean Double Int mean = foldl (\(Mean m n) x - Mean (m + (x - m) / fromIntegral (n+1)) (n+1)) (Mean 0 0) AFAIU iteratees basically use same technique. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Patterns for processing large but finite streams
Alexey, your definition of mean does not look like liftS2 (/) sum length - you have to manually fuse these computations. I'm asking for a formalism that does this fusion automatically (and guaranteedly). 2011/7/1 Alexey Khudyakov alexey.sklad...@gmail.com: On Fri, Jul 1, 2011 at 12:21 PM, Eugene Kirpichov ekirpic...@gmail.com wrote: I meant the average of the whole list - given a sumS and lengthS (S for Stream), write meanS as something like liftS2 (/) sumS lengthS. Or is that possible with lazy lists too? Sure you can. Sum, length and mean could be calculated as left fold. If you need to calculate more that one statistic at time you can combine accumulators sum = foldl (+) 0 length = foldl (\n _ - n+1) 0 data Mean Double Int mean = foldl (\(Mean m n) x - Mean (m + (x - m) / fromIntegral (n+1)) (n+1)) (Mean 0 0) AFAIU iteratees basically use same technique. -- Eugene Kirpichov Principal Engineer, Mirantis Inc. http://www.mirantis.com/ Editor, http://fprog.ru/ ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Patterns for processing large but finite streams
On Fri, Jul 1, 2011 at 12:54 PM, Eugene Kirpichov ekirpic...@gmail.com wrote: Alexey, your definition of mean does not look like liftS2 (/) sum length - you have to manually fuse these computations. Well it was fused for numerical stability I'm asking for a formalism that does this fusion automatically (and guaranteedly). Joining accumulators is quite straightforward. So is joining of initial state. Just creating a joinAcc :: (acc1 - x - acc1) - (acc2 - x - acc2) - (acc1,acc2) - x - (acc1,acc2) joinAcc f1 f2 (s1,s2) x = (f1 s1 x, f2 s2 x) Still you have to handle them separately. sum' = foldl (+) 0 len = foldl (\n _ - n+1) 0 sumLen = foldl (joinAcc (+) (\n _ - n+1)) (0,0) There is more regular approach but it only works with statistics. (function which do not depend on order of elements in the sample) For every statistics monoid for its evaluation could be constructed. For example sum: newtype Sum a = Sum a instance Num a = Monoid (Sum a) where mempty = Sum 0 mappend (Sum a) (Sum b) = Sum (a+b) Composition of these monoids becomes trivial. Just use I pursued this approach in monoid-statistics[1] package. It's reasonably well documented [1] http://hackage.haskell.org/package/monoid-statistics ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Patterns for processing large but finite streams
Eugene Kirpichov wrote: Plain old lazy lists do not allow me to combine multiple concurrent computations, e.g. I cannot define average from sum and length. I meant the average of the whole list - given a sumS and lengthS (S for Stream), write meanS as something like liftS2 (/) sumS lengthS. Or is that possible with lazy lists too? (looks like arrows actually - which arrow is appropriate here?) That's a very good point. Just to clarify for everyone: Eugene wants to write the function average almost *literally* as average xs = sum xs / length xs but he wants the functions sum and length to fuse, so that the input stream xs is *not* shared as a whole. I have thought about this problem for a while actually and have observed the following: 1) You are not looking for a representation of streams, but for a representation of *functions* on streams. The essence of a function on streams is its case analysis of the input. Hence, the simplest solution is to make the case analysis explicit: data StringTo a = CaseOf a (Char - StringTo a) -- function on a stream (here: String) interpret :: StringTo a - (String - a) interpret (CaseOf nil cons) [] = nil interpret (CaseOf nil cons) (x:xs) = interpret (cons x) xs instance Applicative StringTo where pure a = CaseOf a (const $ pure a) (CaseOf nil1 cons1) * (CaseOf nil2 cons2) = CaseOf (nil1 $ nil2) (\c - cons1 c * cons2 c) length = go 0 where go n = CaseOf n (\_ - go $! n+1) average = liftA2 (/) sum length In other words, if you reify case .. of expression , you will be able to fuse them. 2) If Haskell were to support some kind of evaluation under the lambda (partial evaluation, head normal form instead of weak head normal form), it would be unnecessary to make the case expressions implicit. Rather, the applicative instance could be written as follows instance Applicative ((-) String) where pure a = const a f * x = \cs - case cs of [] - f [] $ x [] (c:cs) - let f' cs = f (c:cs) -- partial evaluation on this x' cs = x (c:cs) in f' `partialseq` x' `partialseq` (f' * x') cs We could simply write average = liftA2 (/) sum length and everything would magically fuse. 3) John Hughes has already thought about this problem in his PhD thesis. :) (but it is not available for download on the internet, unfortunately. :( ). His solution was a SYNCHLIST primitive in conjunction with some sort of parallelism PAR. Basically, the SYNCHLIST primitive only allows simultaneous access to the input stream and the parallelism is used to make that simultaneity happen. Best regards, Heinrich Apfelmus -- http://apfelmus.nfshost.com ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Patterns for processing large but finite streams
Thanks but I'm afraid that's still not quite what I'm looking for; guess I'll have to define my desire by my implementation - so once it's ready I'll show the result to cafe :) 2011/7/1 Alexey Khudyakov alexey.sklad...@gmail.com: On Fri, Jul 1, 2011 at 12:54 PM, Eugene Kirpichov ekirpic...@gmail.com wrote: Alexey, your definition of mean does not look like liftS2 (/) sum length - you have to manually fuse these computations. Well it was fused for numerical stability I'm asking for a formalism that does this fusion automatically (and guaranteedly). Joining accumulators is quite straightforward. So is joining of initial state. Just creating a joinAcc :: (acc1 - x - acc1) - (acc2 - x - acc2) - (acc1,acc2) - x - (acc1,acc2) joinAcc f1 f2 (s1,s2) x = (f1 s1 x, f2 s2 x) Still you have to handle them separately. sum' = foldl (+) 0 len = foldl (\n _ - n+1) 0 sumLen = foldl (joinAcc (+) (\n _ - n+1)) (0,0) There is more regular approach but it only works with statistics. (function which do not depend on order of elements in the sample) For every statistics monoid for its evaluation could be constructed. For example sum: newtype Sum a = Sum a instance Num a = Monoid (Sum a) where mempty = Sum 0 mappend (Sum a) (Sum b) = Sum (a+b) Composition of these monoids becomes trivial. Just use I pursued this approach in monoid-statistics[1] package. It's reasonably well documented [1] http://hackage.haskell.org/package/monoid-statistics -- Eugene Kirpichov Principal Engineer, Mirantis Inc. http://www.mirantis.com/ Editor, http://fprog.ru/ ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Patterns for processing large but finite streams
From: Eugene Kirpichov ekirpic...@gmail.com Subject: [Haskell-cafe] Patterns for processing large but finite streams To: Haskell Cafe haskell-cafe@haskell.org Message-ID: banlktikdsvq2wv4wjr+qmuvksoav0kt...@mail.gmail.com Content-Type: text/plain; charset=ISO-8859-1 Hi, I'm rewriting timeplot to avoid holding the whole input in memory, and naturally a problem arises: How to represent large but finite streams and functions that process them, returning other streams or some kinds of aggregate values? Examples: * Adjacent differences of a stream of numbers * Given a stream of numbers with times, split it into buckets by time of given width and produce a stream of (bucket, 50%,75% and 90% quantiles in this bucket) * Sum a stream of numbers Is this, perhaps, what comonads are for? Or iteratees? This is exactly what iteratees are for. Specifically, enumeratees are stream transformers, iteratees are stream consumers, and enumerators are stream producers. Consider adjacent differences: Given the stream [a, b, c, d, e ], you want to produce [b-a, c-b, d-c, e-d, ...] This is a stream transformer, so you need an enumeratee. Using iteratee, there are at least two obvious ways to produce it: 1) High-level, but probably not as good performance. Use Data.Iteratee.ListLike.roll import Data.Iteratee as I import Control.Applicative diff [x,y] = y-x diff [x]= 0 diffs = convStream (map diff $ roll 2 1) 2) somewhat explicit, probably better performance import qualified Data.ListLike as LL e' iter = do h - I.head unfoldConvStream f h iter where f lastEl = do c - getChunk if LL.null c then return (lastEl, LL.empty) else do let h = LL.head c t = LL.tail c return (LL.last c, LL.cons (h-lastEl) (LL.zipWith (-) t (LL.init c))) either of these can be run by using an enumerator: *Main enumPure1Chunk [1..10] (joinI $ e stream2list) = run [1,1,1,1,1,1,1,1,1,0] *Main let e'2 = e' :: Enumeratee [Int] [Int] IO a *Main enumPure1Chunk [1..10] (joinI $ e'2 stream2list) = run [1,1,1,1,1,1,1,1,1] I should optimize 'roll', it wouldn't be hard. Summing is easy; iteratee has Data.Iteratee.ListLike.sum built-in, but you could also use a fold. enumPure1Chunk is only useful for small amounts of data, but iteratee packages provide enumerators over files, handles, etc., as well as mechanisms by which you can create your own enumerators. The e' enumeratee is really just a model; I'd probably write one specific to whichever type of stream I wanted to work with. This one assumes a cheap 'cons', for example. For producing a stream of buckets, if the times are ordered it would be simple to do with Data.Iteratee.ListLike.breakE. If the times aren't ordered, I would probably use 'group' instead to collect a set number of samples. In my view, the biggest difference between iteratee and enumerator is the stream abstraction. Iteratee provides I.Iteratee s m a where 's' is the stream type, e.g. [Int], String, ByteString, Vector Word8, etc. Although the library only processes a chunk at a time (where a chunk is a subsection of the stream), the type is that of the whole stream. Enumerator instead provides E.Iteratee s m a here, 's' is the type of the chunk. Enumerator treats the stream as having type [s]. The implementations are different too, but ideally that would be hidden from most users. Although the iteratee implementation allows you to use = and $, whereas enumerator sometimes requires you to use == and $$. I think IterIO mostly follows the same type as Enumerator, although the implementation is again quite different. John ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] compare iteratee with python's yield
yi huang yi.codepla...@gmail.com wrote: I just read several tutorials on iteratee, i find that iteratee is similar to python's generator, both allow streamlined data processing. For example, i can implement enumFile and printChunks in python like this: EOF = None def enum_file(bufsize, filename): with open(filename) as input: while True: data = input.read(bufsize) if not data: break yield data yield EOF def print_chunks(print_empty, generator): for chunk in generator: if chunk==EOF: print 'EOF' return if len(chunk)==0 and not print_empty: continue print chunk print_chunks(True, enum_file(2, data)) But i find iteratee far more complicated than python's generator, is that because iteratee can do something python's generator can't, or i simply need to be more familar with functional programming style. I don't know Python very well, but I suspect that its generators are really a sort of coroutines. Iteratees are also coroutines, but their architecture is quite different. The difference is that conceptually an iteratee does not know about its input. In Python the generator stops to wait for the iterator to request more input: The consumer talks to the producer. This control flow is turned inside out in iteratees, where the iteratee stops to wait for the enumerator to provide more input. The producer talks to the consumer in iteratees. This is a conceptual difference, so what's the advantage? The main advantage of iteratees, compared to generators, can be seen in a statically typed language such as Haskell. Let's say that instead of printing the input lines your consumer would instead just calculate its length and return it. Let's call this consumer 'length'. If you would translate generators to Haskell, you would find that your 'length' consumer would suddenly include a MonadIO constraint, even though it doesn't need it. With iteratees' inversion of control only the part which needs the MonadIO constraint really has it. An enumerator is really a function from an iteratee to an iteratee. It converts an arbitrary iteratee to an iteratee with additional input, adding the constraints necessary to fulfill this task. While with the generator concept you apply a producer to a consumer, with iteratees you apply a consumer to an producer. Hope that helps. Greets, Ertugrul -- nightmare = unsafePerformIO (getWrongWife = sex) http://ertes.de/ ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Patterns for processing large but finite streams
After the list discussion, I'm surprised nobody mentioned Max Rabkin/Conal Elliott's blog posts on folds and zips. http://squing.blogspot.com/2008/11/beautiful-folding.html http://conal.net/blog/posts/enhancing-a-zip/ They develop a formalism for zipping functions on lists. Iteratee's `zip` set of functions are somewhat similar, but not quite the same. Specifically they still require multiple traversals of the data, but only over a bounded portion of it, so they're much more efficient. Of course you could combine the above patterns with iteratees by creating functions as above, then just running them with a 'fold'. John L. From: Eugene Kirpichov ekirpic...@gmail.com Thanks but I'm afraid that's still not quite what I'm looking for; guess I'll have to define my desire by my implementation - so once it's ready I'll show the result to cafe :) 2011/7/1 Alexey Khudyakov alexey.sklad...@gmail.com: On Fri, Jul 1, 2011 at 12:54 PM, Eugene Kirpichov ekirpic...@gmail.com wrote: Alexey, your definition of mean does not look like liftS2 (/) sum length - you have to manually fuse these computations. Well it was fused for numerical stability I'm asking for a formalism that does this fusion automatically (and guaranteedly). Joining accumulators is quite straightforward. So is joining of initial state. Just creating a joinAcc :: (acc1 - x - acc1) - (acc2 - x - acc2) - (acc1,acc2) - x - (acc1,acc2) joinAcc f1 f2 (s1,s2) x = (f1 s1 x, f2 s2 x) Still you have to handle them separately. sum' = foldl (+) 0 len ?= foldl (\n _ - n+1) 0 sumLen = foldl (joinAcc (+) (\n _ - n+1)) (0,0) There is more regular approach but it only works with statistics. (function which do not depend on order of elements in the sample) For every statistics monoid for its evaluation could be constructed. For example sum: newtype Sum a = Sum a instance Num a = Monoid (Sum a) where ? mempty = Sum 0 ? mappend (Sum a) (Sum b) = Sum (a+b) Composition of these monoids becomes trivial. Just use I pursued this approach in monoid-statistics[1] package. It's reasonably well documented ?[1] http://hackage.haskell.org/package/monoid-statistics ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Patterns for processing large but finite streams
Hi, You're right, reifying stream processing functions seems indeed the way to go - and that looks even more like arrows :) I thought of something like this: data SP i o = Yield [o] (Maybe (Maybe i - SP i o)) Scalar functions like sum and length are just SP's that return a single item in the output stream. sum :: (Num a) = SP a a sum = sum' 0 where sum' s = Yield [] $ Just $ maybe (Yield [s] Nothing) (sum' . (s+)) Adjacent differences would be like liftA2 (-) input laggedInput laggedInput would be like: laggedInput :: SP i i laggedInput = li Nothing where li maybePrev = Yield (maybe2list maybePrev) $ Just $ maybe empty (li . Just) Looks like this can be made into an instance of Arrow and can be composed etc. 2011/7/1 Heinrich Apfelmus apfel...@quantentunnel.de: Eugene Kirpichov wrote: Plain old lazy lists do not allow me to combine multiple concurrent computations, e.g. I cannot define average from sum and length. I meant the average of the whole list - given a sumS and lengthS (S for Stream), write meanS as something like liftS2 (/) sumS lengthS. Or is that possible with lazy lists too? (looks like arrows actually - which arrow is appropriate here?) That's a very good point. Just to clarify for everyone: Eugene wants to write the function average almost *literally* as average xs = sum xs / length xs but he wants the functions sum and length to fuse, so that the input stream xs is *not* shared as a whole. I have thought about this problem for a while actually and have observed the following: 1) You are not looking for a representation of streams, but for a representation of *functions* on streams. The essence of a function on streams is its case analysis of the input. Hence, the simplest solution is to make the case analysis explicit: data StringTo a = CaseOf a (Char - StringTo a) -- function on a stream (here: String) interpret :: StringTo a - (String - a) interpret (CaseOf nil cons) [] = nil interpret (CaseOf nil cons) (x:xs) = interpret (cons x) xs instance Applicative StringTo where pure a = CaseOf a (const $ pure a) (CaseOf nil1 cons1) * (CaseOf nil2 cons2) = CaseOf (nil1 $ nil2) (\c - cons1 c * cons2 c) length = go 0 where go n = CaseOf n (\_ - go $! n+1) average = liftA2 (/) sum length In other words, if you reify case .. of expression , you will be able to fuse them. 2) If Haskell were to support some kind of evaluation under the lambda (partial evaluation, head normal form instead of weak head normal form), it would be unnecessary to make the case expressions implicit. Rather, the applicative instance could be written as follows instance Applicative ((-) String) where pure a = const a f * x = \cs - case cs of [] - f [] $ x [] (c:cs) - let f' cs = f (c:cs) -- partial evaluation on this x' cs = x (c:cs) in f' `partialseq` x' `partialseq` (f' * x') cs We could simply write average = liftA2 (/) sum length and everything would magically fuse. 3) John Hughes has already thought about this problem in his PhD thesis. :) (but it is not available for download on the internet, unfortunately. :( ). His solution was a SYNCHLIST primitive in conjunction with some sort of parallelism PAR. Basically, the SYNCHLIST primitive only allows simultaneous access to the input stream and the parallelism is used to make that simultaneity happen. Best regards, Heinrich Apfelmus -- http://apfelmus.nfshost.com ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe -- Eugene Kirpichov Principal Engineer, Mirantis Inc. http://www.mirantis.com/ Editor, http://fprog.ru/ ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Printing the empty list.
Figuring out how to tell what type ghci is defaulting to was an interesting exercise. The sum [] trick seemed cool, so I tried a variant: Prelude let f xs = const xs $ show xs Prelude f [] [] Prelude :t it it :: [()] -- ryan On Thu, Jun 30, 2011 at 6:44 PM, Ivan Lazar Miljenovic ivan.miljeno...@gmail.com wrote: On 1 July 2011 11:35, Brent Yorgey byor...@seas.upenn.edu wrote: On Fri, Jul 01, 2011 at 09:05:05AM +1000, Ivan Lazar Miljenovic wrote: On 1 July 2011 08:58, Joshua Ball joshbb...@gmail.com wrote: GHCi seems to be clever about some things: If I try to print the empty list in ghci, I encounter no problems: Prelude [] [] Prelude show [] [] Prelude print [] [] Even though the type of the list is clearly unknown, it must be picking SOME type. (why does it print [] instead of )? Type defaulting: if you don't specify a type, then ghci makes it [Integer]. In this case I'm pretty sure it is [()] since there is only a Show constraint. If there were a Num constraint it would pick Integer. Yeah, I forgot about () -- Ivan Lazar Miljenovic ivan.miljeno...@gmail.com IvanMiljenovic.wordpress.com ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] NLP libraries and tools?
Hi, Please advise on NLP libraries similar to Natural Language Toolkit ( www.nltk.org) First of all I need: - tools to construct 'bag of words' ( http://en.wikipedia.org/wiki/Bag_of_words_model), which is a list of words in the article. - tools to prune common words, such as prepositions and conjunctions, as well as extremely rare words, such as the ones with typos. - stemming tools - Naive Bayes classifier - SVM classifier - k-means clustering Thanks! ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Patterns for processing large but finite streams
Eugene Kirpichov ekirpic...@gmail.com writes: 2011/7/1 Heinrich Apfelmus apfel...@quantentunnel.de: Eugene Kirpichov wrote: I'm rewriting timeplot to avoid holding the whole input in memory, and naturally a problem arises: Plain old lazy lists? Heretic! :-) I generally have written a bunch of programs that do things that way, and I think it works pretty well with a couple of caveats: 1. Make sure you collect data into strict data structures. Dangerous operations are addition and anything involving Data.Map. And use foldl'. 2. If you plan on working on multiple files, extra care might be needed to close them, or you'll run out of file descriptors. As long as you avoid these pitfalls, the advantage is very clean and simple code. Plain old lazy lists do not allow me to combine multiple concurrent computations, e.g. I cannot define average from sum and length. Yes, this is clunky. I'm not aware of any good solution. -k -- If I haven't seen further, it is by standing in the footprints of giants ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Patterns for processing large but finite streams
This sound exactly like what attribute grammars, like the system developed at Utrecht University [1], are useful for. Erik [1] http://www.cs.uu.nl/wiki/HUT/AttributeGrammarSystem On Fri, Jul 1, 2011 at 10:54, Eugene Kirpichov ekirpic...@gmail.com wrote: Alexey, your definition of mean does not look like liftS2 (/) sum length - you have to manually fuse these computations. I'm asking for a formalism that does this fusion automatically (and guaranteedly). 2011/7/1 Alexey Khudyakov alexey.sklad...@gmail.com: On Fri, Jul 1, 2011 at 12:21 PM, Eugene Kirpichov ekirpic...@gmail.com wrote: I meant the average of the whole list - given a sumS and lengthS (S for Stream), write meanS as something like liftS2 (/) sumS lengthS. Or is that possible with lazy lists too? Sure you can. Sum, length and mean could be calculated as left fold. If you need to calculate more that one statistic at time you can combine accumulators sum = foldl (+) 0 length = foldl (\n _ - n+1) 0 data Mean Double Int mean = foldl (\(Mean m n) x - Mean (m + (x - m) / fromIntegral (n+1)) (n+1)) (Mean 0 0) AFAIU iteratees basically use same technique. -- Eugene Kirpichov Principal Engineer, Mirantis Inc. http://www.mirantis.com/ Editor, http://fprog.ru/ ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Is there Functor instance of Enumerator'
Say i want to compose Enumerator ByteString m b and Iteratee Builder m b, so I try to transform the enum to Enumerator Builder m b, providing function ByteString - Builder. It's like implement a Functor instance for Enumerator. But i failed, there are no way to make type system happy. Am I right that there is no Functor instance of Enumerator? -- http://www.yi-programmer.com/blog/ ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Twitter API?
Hi, Does anybody work with any Haskell Twitter API library? In particular supporting OAuth authentication? Thanks! ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Twitter API?
On Fri, Jul 1, 2011 at 5:56 PM, Dmitri O.Kondratiev doko...@gmail.com wrote: Hi, Does anybody work with any Haskell Twitter API library? In particular supporting OAuth authentication? Thanks! Hi, Horimi Ishii added support for OAuth/Twitter to the authenticate[1] package. Michael [1] http://hackage.haskell.org/package/authenticate ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Twitter API?
On Fri, Jul 1, 2011 at 6:59 PM, Michael Snoyman mich...@snoyman.com wrote: On Fri, Jul 1, 2011 at 5:56 PM, Dmitri O.Kondratiev doko...@gmail.com wrote: Hi, Does anybody work with any Haskell Twitter API library? In particular supporting OAuth authentication? Thanks! Hi, Horimi Ishii added support for OAuth/Twitter to the authenticate[1] package. Michael [1] http://hackage.haskell.org/package/authenticate Thanks! It would be nice to have end-to-end example of using some Twitter scenario. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] NLP libraries and tools?
On Fri, Jul 1, 2011 at 3:31 AM, Dmitri O.Kondratiev doko...@gmail.com wrote: Hi, Please advise on NLP libraries similar to Natural Language Toolkit There is a (slowly?) growing NLP community for haskell over at: http://projects.haskell.org/nlp/ The nlp mailing list may be a better place to ask for details. To the best of my knowledge, most of the NLTK / OpenNLP capabilities have yet to be implemented/ported to Haskell, but there are some packages to take a look at on Hackage. First of all I need: - tools to construct 'bag of words' (http://en.wikipedia.org/wiki/Bag_of_words_model), which is a list of words in the article. This is trivially implemented if you have a natural language tokenizer you're happy with. Toktok might be worth looking at: http://hackage.haskell.org/package/toktok but I *think* it takes a pretty simple view of tokens (assume it is the tokenizer I've been using within the GF). Eric Kow (?) has a tokenizer implementation, which I can't seem to find at the moment - if I recall correctly, it is also very simple, but it would be a great place to implement a more complex tokenizer :) - tools to prune common words, such as prepositions and conjunctions, as well as extremely rare words, such as the ones with typos. I'm not sure what you mean by 'prune'. Are you looking for a stopword list to remove irrelevant / confusing words from something like a search query? (that's not hard to do with a stemmer and a set) - stemming tools There is an implementation of the porter stemmer on Hackage: - http://hackage.haskell.org/package/porter - Naive Bayes classifier I'm not aware of a general-purpose bayesian classifier lib. for haskell, but it *would* be great to have :) There are probably some general-purpose statistical packages that I'm unaware of that offer a larger set of capabilities... - SVM classifier There are a few of these. Take a look at the AI category on hackage: - http://hackage.haskell.org/packages/archive/pkg-list.html#cat:ai --Rogan - k-means clustering ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Is there Functor instance of Enumerator'
On Fri, Jul 1, 2011 at 7:38 AM, yi huang yi.codepla...@gmail.com wrote: Say i want to compose Enumerator ByteString m b and Iteratee Builder m b, so I try to transform the enum to Enumerator Builder m b, providing function ByteString - Builder. It's like implement a Functor instance for Enumerator. But i failed, there are no way to make type system happy. Am I right that there is no Functor instance of Enumerator? I'm pretty sure there is one, but it's a functor over b. So something like: instance Functor (Enumerator a m) I think you want to use an Enumeratee: enumerateeByteStringBuilder :: Enumeratee ByteString Builder m b Check out map :: (a - b) - Enumeratee a b m c http://hackage.haskell.org/packages/archive/enumerator/0.4.0.1/doc/html/Data-Enumerator.html#v:map ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Haskell and Databases
I am studying Haskell database libraries. In particular I am looking at haskelldb and the Takusen Database.Enumerator library. In haskelldb, there are very good ideas aiming to represent database objects as (typeable) Haskell objects, instead of embedding SQL in plain strings. This is really an interesting approach that overcomes the hiatus between the database there and the processing logic here. Unfortunately, the library cannot deal with some obvious issues related to this concept; conflicts arise, for instance, when Haskell keywords are used as names of attributes or tables or when the same name is used for an attribute and for a table (which is perfectly legal in SQL). Also, haskelldb cannot cope with peculiarities of popular database tables, such as ISAM tables - which are case sensitive. Another, more conceptual issue, is the lack of database cursors. Database programs, usually, do not just issue isolated SQL statements, but implement a processing logic with nested queries and DML statements. A DB library should provide control structures to ease this kind of processing. A library that tackles this issue, is the Takusen Database.Enumerator, again, a library with very strong ideas. Unfortunately, in Takusen there is nothing like the haskelldb DSL. Concerning, cursors, Takusen uses iterators to stream query results. This makes a lot of sense. The iterators, however, cannot be used to implement the nesting of queries. The user has to provide an additional action which is responsible for all the cursor-related logic. The tools provided to deal with cursors resemble very much the instructions found in other database-oriented languages, such as PL/SQL. This leads to verbose code where cursor control is implemented in a quite imperative way. The user, however, should be concerned with getting her queries right, not with the technical details of fetching from cursors. In summary, what I miss is a database library that - integrates haskelldb approach with - a functional-style cursor concept Is anybody working on something like this? Thanks, Tobias ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Haskell and Databases
There is something that bothers me with that text, I can't get to grasp what it is... 2011/7/1 Tobias Schoofs tobias.scho...@gmx.net ** I am studying Haskell database libraries. In particular I am looking at haskelldb and the Takusen Database.Enumerator library. In haskelldb, there are very good ideas aiming to represent database objects as (typeable) Haskell objects, instead of embedding SQL in plain strings. This is really an interesting approach that overcomes the hiatus between the database there and the processing logic here. Unfortunately, the library cannot deal with some obvious issues related to this concept; conflicts arise, for instance, when Haskell keywords are used as names of attributes or tables or when the same name is used for an attribute and for a table (which is perfectly legal in SQL). Also, haskelldb cannot cope with peculiarities of popular database tables, such as ISAM tables - which are case sensitive. Another, more conceptual issue, is the lack of database cursors. Database programs, usually, do not just issue isolated SQL statements, but implement a processing logic with nested queries and DML statements. A DB library should provide control structures to ease this kind of processing. A library that tackles this issue, is the Takusen Database.Enumerator, again, a library with very strong ideas. Unfortunately, in Takusen there is nothing like the haskelldb DSL. Concerning, cursors, Takusen uses iterators to stream query results. This makes a lot of sense. The iterators, however, cannot be used to implement the nesting of queries. The user has to provide an additional action which is responsible for all the cursor-related logic. The tools provided to deal with cursors resemble very much the instructions found in other database-oriented languages, such as PL/SQL. This leads to verbose code where cursor control is implemented in a quite imperative way. The user, however, should be concerned with getting her queries right, not with the technical details of fetching from cursors. In summary, what I miss is a database library that - integrates haskelldb approach with - a functional-style cursor concept Is anybody working on something like this? Thanks, Tobias ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Haskell and Databases
On 1 July 2011 20:51, Yves P limestr...@gmail.com wrote: There is something that bothers me with that text, I can't get to grasp what it is... It's bigger than Godzilla? ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Haskell and Databases
'Courier New, 18pt' considered harmful? On Jul 1, 2011, at 3:08 PM, Christopher Done wrote: On 1 July 2011 20:51, Yves P limestr...@gmail.com wrote: There is something that bothers me with that text, I can't get to grasp what it is... It's bigger than Godzilla? ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe Computer Science is no more about computers than astronomy is about telescopes. -- Edsger Dijkstra 398E692F.asc Description: application/apple-msg-attachment PGP.sig Description: This is a digitally signed message part ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Haskell and Databases
HTML emails considered harmful. On 1 Jul 2011, at 23:17, Jack Henahan wrote: 'Courier New, 18pt' considered harmful? On Jul 1, 2011, at 3:08 PM, Christopher Done wrote: On 1 July 2011 20:51, Yves P limestr...@gmail.com wrote: There is something that bothers me with that text, I can't get to grasp what it is... It's bigger than Godzilla? ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe Computer Science is no more about computers than astronomy is about telescopes. -- Edsger Dijkstra 398E692F.asc ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Haskell and Databases
Hi Tobias, Have you seen DSH [1]? You might also be interested in Persistent [2], but it sounds like it has different goals than what you are after. [1] http://hackage.haskell.org/package/DSH [2] http://www.yesodweb.com/book/persistent ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] NLP libraries and tools?
On Fri, Jul 1, 2011 at 9:34 PM, Rogan Creswick cresw...@gmail.com wrote: On Fri, Jul 1, 2011 at 3:31 AM, Dmitri O.Kondratiev doko...@gmail.com wrote: First of all I need: ... - tools to construct 'bag of words' (http://en.wikipedia.org/wiki/Bag_of_words_model), which is a list of words in the article. This is trivially implemented if you have a natural language tokenizer you're happy with. Toktok might be worth looking at: http://hackage.haskell.org/package/toktok but I *think* it takes a pretty simple view of tokens (assume it is the tokenizer I've been using within the GF). Unfortunately 'cabal install' fails with toktok: Building toktok-0.5... [1 of 7] Compiling Toktok.Stack ( Toktok/Stack.hs, dist/build/Toktok/Stack.o ) [2 of 7] Compiling Toktok.Sandhi( Toktok/Sandhi.hs, dist/build/Toktok/Sandhi.o ) [3 of 7] Compiling Toktok.Trie ( Toktok/Trie.hs, dist/build/Toktok/Trie.o ) [4 of 7] Compiling Toktok.Lattice ( Toktok/Lattice.hs, dist/build/Toktok/Lattice.o ) [5 of 7] Compiling Toktok.Transducer ( Toktok/Transducer.hs, dist/build/Toktok/Transducer.o ) [6 of 7] Compiling Toktok.Lexer ( Toktok/Lexer.hs, dist/build/Toktok/Lexer.o ) [7 of 7] Compiling Toktok ( Toktok.hs, dist/build/Toktok.o ) Registering toktok-0.5... [1 of 1] Compiling Main ( Main.hs, dist/build/toktok/toktok-tmp/Main.o ) Linking dist/build/toktok/toktok ... [1 of 1] Compiling Main ( tools/ExtractLexicon.hs, dist/build/gf-extract-lexicon/gf-ex\ tract-lexicon-tmp/Main.o ) tools/ExtractLexicon.hs:5:35: Module `PGF' does not export `getLexicon' cabal: Error: some packages failed to install: toktok-0.5 failed during the building phase. The exception was: ExitFailure 1 Any ideas how to solve this? ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] NLP libraries and tools?
On Fri, Jul 1, 2011 at 12:38 PM, Dmitri O.Kondratiev doko...@gmail.com wrote: On Fri, Jul 1, 2011 at 9:34 PM, Rogan Creswick cresw...@gmail.com wrote: On Fri, Jul 1, 2011 at 3:31 AM, Dmitri O.Kondratiev doko...@gmail.com wrote: First of all I need: Unfortunately 'cabal install' fails with toktok: tools/ExtractLexicon.hs:5:35: Module `PGF' does not export `getLexicon' cabal: Error: some packages failed to install: toktok-0.5 failed during the building phase. The exception was: ExitFailure 1 Oh, right - I ran into this problem too, and forgot about it (I should have reported a bug...) I think this fails because of (relatively) recent changes in GF, which isn't constrained to specific versions in the toktok cabal file... --Rogan Any ideas how to solve this? ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Parsing cabal files to calculate average number of dependencies
Athas on #haskell wondered how many dependencies the average Haskell
package had. I commented that it seemed like some fairly simple
scripting to find out, and as these things tend to go, I wound up
doing a complete solution myself.
First, we get most/all of Hackage locally to examine, as tarballs:
for package in `cabal list | grep '\*' | tr -d '\*'`; do cabal
fetch $package; done
Then we cd .cabal/packages/hackage.haskell.org
Now we can run a command which extracts the .cabal file from each
tarball to standard output:
find . -name *.tar.gz -exec tar --wildcards *.cabal -Oxf {} \;
We could grep for 'build-depends' or something, but that gives
unreliable dirty results. (80k items, resulting in a hard to believe
87k total deps and an average of 27 deps.) So instead, we use the
Cabal library and write a program to parse Cabal files spit out the
dependencies, and we feed each .cabal into that:
find . -name *.tar.gz -exec sh -c 'tar --wildcards *.cabal
-Oxf {} | runhaskell ~/deps.hs' \;
And what is deps.hs? Turns out to be surprisingly easy to parse a
String, extract the Library and Executable AST, and grab the
[Dependency] field, and then print it out (code is not particularly
clean):
import Distribution.Package
import Distribution.PackageDescription
import Distribution.PackageDescription.Parse
main :: IO ()
main = do cbl - getContents
let desc = parsePackageDescription cbl
case desc of
ParseFailed _ - return ()
ParseOk _ d - putStr $ unlines $ map show $ map
(\(Dependency x _) - x) $ extractDeps d
extractDeps :: GenericPackageDescription - [Dependency]
extractDeps d = ldeps ++ edeps
where ldeps = case (condLibrary d) of
Nothing - []
Just c - condTreeConstraints c
edeps = concat $ map (condTreeConstraints . snd) $ condExecutables d
So what are the results? (The output of one run is attached.) I get
18,134 dependencies, having run on 3,137 files, or 5.8 dependencies
per package.
--
gwern
http://www.gwern.net
deps.txt.gz
Description: GNU Zip compressed data
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Haskell and Databases
Sorry for the awful message format in the first try! I am studying Haskell database libraries. In particular I am looking at haskelldb and the Takusen Database.Enumerator library. In haskelldb, there are very good ideas aiming to represent database objects as (typeable) Haskell objects, instead of embedding SQL in plain strings. This is really an interesting approach that overcomes the hiatus between the database there and the processing logic here. Unfortunately, the library cannot deal with some obvious issues related to this concept; conflicts arise, for instance, when Haskell keywords are used as names of attributes or tables or when the same name is used for an attribute and for a table (which is perfectly legal in SQL). Also, haskelldb cannot cope with peculiarities of popular database tables, such as ISAM tables - which are case sensitive. Another, more conceptual issue, is the lack of database cursors. Database programs, usually, do not just issue isolated SQL statements, but implement a processing logic with nested queries and DML statements. A DB library should provide control structures to ease this kind of processing. A library that tackles this issue, is the Takusen Database.Enumerator, again, a library with very strong ideas. Unfortunately, in Takusen there is nothing like the haskelldb DSL. Concerning, cursors, Takusen uses iterators to stream query results. This makes a lot of sense. The iterators, however, cannot be used to implement the nesting of queries. The user has to provide an additional action which is responsible for all the cursor-related logic. The tools provided to deal with cursors resemble very much the instructions found in other database-oriented languages, such as PL/SQL. This leads to verbose code where cursor control is implemented in a quite imperative way. The user, however, should be concerned with getting her queries right, not with the technical details of fetching from cursors. In summary, what I miss is a database library that - integrates haskelldb approach with - a functional-style cursor concept Is anybody working on something like this? Thanks, Tobias ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Parsing cabal files to calculate average number of dependencies
On Fri, Jul 1, 2011 at 4:49 PM, L Corbijn aspergesoe...@gmail.com wrote: Is this including or exluding 'or'-ed dependency lists like http://hackage.haskell.org/package/hugs2yc ? Excluding, it seems. When I run the script on that tarball: $ tar --wildcards *.cabal -Oxf `find . -name *.tar.gz | g hugs2yc` | runhaskell /home/gwern/deps.hs PackageName mtl PackageName uniplate PackageName yhccore PackageName ycextra PackageName parsec PackageName directory PackageName filepath No version of base or containers appears. (mtl appears in both branches and also the general build-depends list.) -- gwern http://www.gwern.net ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Parsing cabal files to calculate average number of dependencies
On Fri, Jul 1, 2011 at 1:43 PM, Gwern Branwen gwe...@gmail.com wrote:
Athas on #haskell wondered how many dependencies the average Haskell
package had. I commented that it seemed like some fairly simple
scripting to find out, and as these things tend to go, I wound up
doing a complete solution myself.
First, we get most/all of Hackage locally to examine, as tarballs:
for package in `cabal list | grep '\*' | tr -d '\*'`; do cabal
fetch $package; done
I think the index tarball has all the info you need, and would be
faster to retrieve / process, if you or anyone else needs to get the
.cabal files again:
http://hackage.haskell.org/packages/archive/00-index.tar.gz (2.2mb)
The set of the latest package sdists is also available:
http://hackage.haskell.org/cgi-bin/hackage-scripts/archive.tar (~150mb)
--Rogan
Then we cd .cabal/packages/hackage.haskell.org
Now we can run a command which extracts the .cabal file from each
tarball to standard output:
find . -name *.tar.gz -exec tar --wildcards *.cabal -Oxf {} \;
We could grep for 'build-depends' or something, but that gives
unreliable dirty results. (80k items, resulting in a hard to believe
87k total deps and an average of 27 deps.) So instead, we use the
Cabal library and write a program to parse Cabal files spit out the
dependencies, and we feed each .cabal into that:
find . -name *.tar.gz -exec sh -c 'tar --wildcards *.cabal
-Oxf {} | runhaskell ~/deps.hs' \;
And what is deps.hs? Turns out to be surprisingly easy to parse a
String, extract the Library and Executable AST, and grab the
[Dependency] field, and then print it out (code is not particularly
clean):
import Distribution.Package
import Distribution.PackageDescription
import Distribution.PackageDescription.Parse
main :: IO ()
main = do cbl - getContents
let desc = parsePackageDescription cbl
case desc of
ParseFailed _ - return ()
ParseOk _ d - putStr $ unlines $ map show $ map
(\(Dependency x _) - x) $ extractDeps d
extractDeps :: GenericPackageDescription - [Dependency]
extractDeps d = ldeps ++ edeps
where ldeps = case (condLibrary d) of
Nothing - []
Just c - condTreeConstraints c
edeps = concat $ map (condTreeConstraints . snd) $ condExecutables d
So what are the results? (The output of one run is attached.) I get
18,134 dependencies, having run on 3,137 files, or 5.8 dependencies
per package.
--
gwern
http://www.gwern.net
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] ANNOUNCE: Hs2lib-0.4.8
What is it? A preprocessor and library which allow you to create dynamic libs from arbitrary annotated Haskell programs with one click. It also allows you to use the generated lib in C, C++ and C# just by including the generated header files. At a minimum it can be considered the inverse of c2hs. Where to get it? You can get it from Hackage (http://hackage.haskell.org/package/Hs2lib) or by using cabal (cabal install Hs2lib). Documentation, Mailing List, Source, etc === Go to http://mistuke.wordpress.com/category/hs2lib/ for information. What's New? = - Currently Supported: * Generates Marshaling information for arbitrary data types. Types with kind other then * have to be fully applied. (e.g. Maybe String). A specialized Marshaling instance is then generated for each of these instances. * Generates FFI exports for any function you mark, with FFI compatible type * Properly supports marshaling of lists. [Int] * Supports Ptr, FunPtr and StablePtr. Which introduces the possibility to have caches/sessions. * Supports Callbacks via Higher order functions (everything is auto generated from the function type) * Re-exports existing exports found in source file * Honors existing Storable instances found for types it needs * Avoids unsafePerformIO as much as possible, except for in one specific instance. * Generates Initialization functions for you * Hides unnecessary exports using a DEF file * Allows you to override default type conversions (e.g. String – CWString) * Provides helper includes for C, C++ and C# (placed into wherever cabal places extra includes. %AppData%\cabal\Hs2lib-0.4.8 for windows) And more Details = A more detailed documentation will follow in the next few weeks. Why 0.4.8? -- This project has been in development for a very long time. Nearly two years on and off. It was developed mainly to facilitate the creation of Visual Haskell 2010. This is just the first public release of this tool. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Parsing cabal files to calculate average number of dependencies
On Fri, Jul 1, 2011 at 5:23 PM, Rogan Creswick cresw...@gmail.com wrote: I think the index tarball has all the info you need, and would be faster to retrieve / process, if you or anyone else needs to get the .cabal files again: http://hackage.haskell.org/packages/archive/00-index.tar.gz (2.2mb) Looking at it, the index tarball contains the .cabal files for all versions known to Hackage, which isn't necessarily the interesting set of cabal files - I'm usually more interested in just the cabal files of the latest version of every package. No doubt there's a scripting solution (loop over the untarred directory of packages, and take the lexically last cabal file?), but it was easier to just exploit cabal fetch's behavior of fetching only the latest version and work with those tarballs. -- gwern http://www.gwern.net ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] NLP libraries and tools?
On Fri, Jul 1, 2011 at 11:58 PM, Rogan Creswick cresw...@gmail.com wrote: On Fri, Jul 1, 2011 at 12:38 PM, Dmitri O.Kondratiev doko...@gmail.com wrote: On Fri, Jul 1, 2011 at 9:34 PM, Rogan Creswick cresw...@gmail.com wrote: On Fri, Jul 1, 2011 at 3:31 AM, Dmitri O.Kondratiev doko...@gmail.com wrote: First of all I need: Unfortunately 'cabal install' fails with toktok: tools/ExtractLexicon.hs:5:35: Module `PGF' does not export `getLexicon' cabal: Error: some packages failed to install: toktok-0.5 failed during the building phase. The exception was: ExitFailure 1 Oh, right - I ran into this problem too, and forgot about it (I should have reported a bug...) I think this fails because of (relatively) recent changes in GF, which isn't constrained to specific versions in the toktok cabal file... --Rogan Any other then 'toktok' Haskell word tokenizer that compiles and works? I need something like: http://nltk.googlecode.com/svn/trunk/doc/api/nltk.tokenize.regexp.WordPunctTokenizer-class.html Thanks! ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] NLP libraries and tools?
On Fri, Jul 1, 2011 at 2:52 PM, Dmitri O.Kondratiev doko...@gmail.com wrote: Any other then 'toktok' Haskell word tokenizer that compiles and works? I need something like: http://nltk.googlecode.com/svn/trunk/doc/api/nltk.tokenize.regexp.WordPunctTokenizer-class.html I don't think this exists out of the box, but since it appears to be a basic regex tokenizer, you could use Data.List.Split to create one. (or one of the regex libraries may be able to do this more simply). If you go the Data.List.Split route, I suspect you'll want to create a Splitter based on the whenElt Splitter: http://hackage.haskell.org/packages/archive/split/0.1.1/doc/html/Data-List-Split.html#v:whenElt which takes a function from an element to a bool. (which you can implement however you wish, possibly with a regular expression, although it will have to be pure.) If you want something like a maxent tokenizer, then you're currently out of luck :( (as far as I know). --Rogan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] ANNOUNCE: Hs2lib-0.4.8
This is interesting, From what I can see, you can automatically mashall haskell datatypes into C structs. Are there portable libraries that do that ? Or do you do this by yourself ? 2011/7/1 Phyx loneti...@gmail.com What is it? A preprocessor and library which allow you to create dynamic libs from arbitrary annotated Haskell programs with one click. It also allows you to use the generated lib in C, C++ and C# just by including the generated header files. At a minimum it can be considered the inverse of c2hs. Where to get it? You can get it from Hackage (http://hackage.haskell.org/package/Hs2lib) or by using cabal (cabal install Hs2lib). Documentation, Mailing List, Source, etc === Go to http://mistuke.wordpress.com/category/hs2lib/ for information. What's New? = - Currently Supported: * Generates Marshaling information for arbitrary data types. Types with kind other then * have to be fully applied. (e.g. Maybe String). A specialized Marshaling instance is then generated for each of these instances. * Generates FFI exports for any function you mark, with FFI compatible type * Properly supports marshaling of lists. [Int] * Supports Ptr, FunPtr and StablePtr. Which introduces the possibility to have caches/sessions. * Supports Callbacks via Higher order functions (everything is auto generated from the function type) * Re-exports existing exports found in source file * Honors existing Storable instances found for types it needs * Avoids unsafePerformIO as much as possible, except for in one specific instance. * Generates Initialization functions for you * Hides unnecessary exports using a DEF file * Allows you to override default type conversions (e.g. String – CWString) * Provides helper includes for C, C++ and C# (placed into wherever cabal places extra includes. %AppData%\cabal\Hs2lib-0.4.8 for windows) And more Details = A more detailed documentation will follow in the next few weeks. Why 0.4.8? -- This project has been in development for a very long time. Nearly two years on and off. It was developed mainly to facilitate the creation of Visual Haskell 2010. This is just the first public release of this tool. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Parsing cabal files to calculate average number of dependencies
Another thing you can do along the same lines is generate a script to download all the repos from packages which declare repos. Some ugly code: import Data.Maybe (fromJust) import Distribution.PackageDescription import Distribution.PackageDescription.Parse import Control.Monad (unless) main :: IO () main = do cbl - getContents let desc = parsePackageDescription cbl case desc of ParseFailed _ - return () ParseOk _ d - do let repos = repoPair $ extractHead $ extractRepos d let cmd = concatMap shellify repos unless (null cmd) $ putStrLn cmd shellify :: (RepoType, String) - String shellify (rt,url) = case rt of Darcs - darcs get ++ url Git - git clone ++ url SVN - svn clone ++ url CVS - cvs co ++ url Mercurial - hg clone ++ url _ - repoPair :: [SourceRepo] - [(RepoType, String)] repoPair = map (\x - (fromJust $ repoType x, fromJust $ repoLocation x)) extractHead :: [SourceRepo] - [SourceRepo] extractHead rs = filter (\x - isnothing x ishead x) rs where ishead sr = case repoKind sr of RepoHead - True _ - False isnothing ss = case repoType ss of Nothing - False Just _ - case repoLocation ss of Nothing - False Just _ - True extractRepos :: GenericPackageDescription - [SourceRepo] extractRepos = sourceRepos . packageDescription This generates results (with the same find command and setup as previously) like: ... git clone git://gitorious.org/maximus/mandulia.git darcs get http://darcs.cielonegro.org/HsOpenSSL/ darcs get http://darcs.cielonegro.org/HsOpenSSL/ hg clone https://bitbucket.org/bos/text-icugit clone https://github.com/bos/text-icu darcs get http://code.haskell.org/Graphalyze darcs get http://code.haskell.org/~roelvandijk/code/base-unicode-symbols git clone git://github.com/roelvandijk/base-unicode-symbols.git darcs get http://code.haskell.org/~basvandijk/code/regions git clone https://github.com/skogsbaer/xmlgen git clone git://github.com/tanakh/HongoDB.git darcs get http://repos.mornfall.net/shellish darcs get http://patch-tag.com/r/Saizan/syb-with-class/ git clone git://github.com/ekmett/eq.git git clone git://github.com/ekmett/data-lens-fd.git git clone git://github.com/ekmett/streams.git git clone git://github.com/alanz/hjsmin.git darcs get http://patch-tag.com/r/byorgey/diagrams-lib ... -- gwern http://www.gwern.net/haskell/Archiving%20GitHub ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Lambdabot plugin system description
Hey all, I'm curious if there are any papers or anything else describing the plugin system in Lambdabot. If not I'll dig through the code, but my Haskell isn't yet that strong so a higher level introduction would be very helpful. Thanks, Rich ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Lambdabot plugin system description
In fact there is! Plugging Haskell In. André Pang, Don Stewart, Sean Seefried, and Manuel M. T. Chakravarty. In Proceedings of the ACM SIGPLAN Workshop on Haskell, pages 10-21. ACM Press, 2004 http://www.cse.unsw.edu.au/~dons/papers/hs-plugins.pdf And Dynamic Applications From the Ground Up. Don Stewart and Manuel M. T. Chakravarty. In Proceedings of the ACM SIGPLAN Workshop on Haskell, pages 27-38. ACM Press, 2005 http://www.cse.unsw.edu.au/~dons/papers/yi.pdf -- Don On Fri, Jul 1, 2011 at 7:43 PM, Richard Wallace rwall...@thewallacepack.net wrote: Hey all, I'm curious if there are any papers or anything else describing the plugin system in Lambdabot. If not I'll dig through the code, but my Haskell isn't yet that strong so a higher level introduction would be very helpful. Thanks, Rich ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] compare iteratee with python's yield
On Fri, Jul 1, 2011 at 6:01 AM, Ertugrul Soeylemez e...@ertes.de wrote: I don't know Python very well, but I suspect that its generators are really a sort of coroutines. Iteratees are also coroutines, but their architecture is quite different. Python generators were originally a sort of heavily restricted coroutine mostly used to implement corecursive sequences, i.e. what we use lazy lists for Haskell. As Python allows arbitrary side-effects, this makes them pretty directly equivalent to using lazy I/O in Haskell. They were later extended to be more like full coroutines, allowing them to both generate and consume data. I imagine that something akin to iteratees could be built on top of the coroutine-style extended generators, but it would likely be more reliant on the programmer not using things incorrectly and the benefit of the whole idea is unclear in this context (for the reasons outlined in the rest of your message). - C. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Lambdabot plugin system description
Awesome, thanks! That looks really useful. On Fri, Jul 1, 2011 at 7:45 PM, Don Stewart don...@gmail.com wrote: In fact there is! Plugging Haskell In. André Pang, Don Stewart, Sean Seefried, and Manuel M. T. Chakravarty. In Proceedings of the ACM SIGPLAN Workshop on Haskell, pages 10-21. ACM Press, 2004 http://www.cse.unsw.edu.au/~dons/papers/hs-plugins.pdf And Dynamic Applications From the Ground Up. Don Stewart and Manuel M. T. Chakravarty. In Proceedings of the ACM SIGPLAN Workshop on Haskell, pages 27-38. ACM Press, 2005 http://www.cse.unsw.edu.au/~dons/papers/yi.pdf -- Don On Fri, Jul 1, 2011 at 7:43 PM, Richard Wallace rwall...@thewallacepack.net wrote: Hey all, I'm curious if there are any papers or anything else describing the plugin system in Lambdabot. If not I'll dig through the code, but my Haskell isn't yet that strong so a higher level introduction would be very helpful. Thanks, Rich ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
