[Haskell-cafe] attoparsec double precision, quickCheck and aeson
The double parser provided by Data.Attoparsec.ByteString.Char8 looses precision around the 13-15th decimal place (http://hackage.haskell.org/packages/archive/attoparsec/0.10.2.0/doc/html/Data-Attoparsec-ByteString-Char8.html#v:double). Unfortunately this reeks havoc with my attempts to write a quickCheck test that validates print-read equivalence for a program that uses Aeson. I have tried compensating for this round-off error in my quickCheck generator using a function like this: roundDouble :: Double - Double roundDouble d = let Right v = A8.parseOnly A8.double (C.pack (show d)) in v which helps in many cases, but for some the parsing seems bi-stable, alternating between two imprecise double values and causing the test to fail. I was wondering if anyone could suggest a better work-around for this problem, or explain why Attoparsec's double parser can't be isomorphic to haskell's. Thanks, Warren ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] attoparsec double precision, quickCheck and aeson
On Jun 5, 2012, at 9:38 AM, Johan Tibell wrote: You want to perform your test as d1 - d2 epsilon What's the best way to do this though, since aeson's Value type already provides instance Eq? I guess I can write my own suite of equality comparisons, but these aeson values are the leaves of an extensive grammar and it seems a shame to mimic everything that Eq does (without the benefits of deriving) just to compare doubles differently. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] attoparsec double precision, quickCheck and aeson
On Jun 5, 2012, at 9:57 AM, Johan Tibell wrote: I don't think applying == to something that contains floating point values at the leaves makes much sense. You want some approxEq function that uses approximate equality on floating point value or you want to equality function that ignores the floating point values. Probably not the answer you like, but I don't know how to define Eq in a robust way for types that include floating point values. I buy that in general for comparing floats (those that result from arithmetic operations), but this is a case where attoparsec's parser is munging the value. I would like to have a law that says parse . print == id ... which is why this seems more like a bug than the usual floating point concerns. This law seems to hold for haskell's double parser: quickCheck (\d - read (show d) == d) ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] attoparsec double precision, quickCheck and aeson
On Jun 5, 2012, at 10:51 AM, Bryan O'Sullivan wrote: On Tue, Jun 5, 2012 at 9:12 AM, Warren Harris warrensomeb...@gmail.com wrote: which helps in many cases, but for some the parsing seems bi-stable, alternating between two imprecise double values and causing the test to fail. I was wondering if anyone could suggest a better work-around for this problem, or explain why Attoparsec's double parser can't be isomorphic to haskell's. If you need the full precision, use rational instead. The double parser is there because parsing floating point numbers is often a bottleneck, and double intentionally trades speed for precision. I'm actually using Aeson and the parser it provides, so I don't really have the option of using rational.___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] using ResourceT with MVars
I would like to use LevelDB in my code, and the HEAD version of leveldb-haskell
now uses runResourceT to manage open db connections and iterators. I would also
like to run multiple threads, and coordinate them for transaction-like
behavior, i.e. read and writing data without interruption. LevelDB doesn't
support transactions itself, so the alternative would seem to be using MVars
for mutual exclusion.
With the 0.0.3 version of leveldb-haskell, the get and put operations were in
the IO monad, which made it trivial to write something like this:
withMVar state $ \db - do
maybeValue - get db rdOpts key
put db wrOpts key $ maybe init incr maybeValue
Now with the HEAD version, the get and put operations have types:
get :: MonadResource m = DB - ReadOptions - ByteString - m (Maybe
ByteString)
put :: MonadResource m = DB - WriteOptions - ByteString - ByteString - m ()
I don't think it makes sense in general to lift the withMVar into MonadResource
and use runResourceT to run its body in IO. This would not satisfy the
intention behind using ResourceT -- resource allocated inside the withMVar body
would be deallocated when the withMVar scope is exited, rather than being
managed by the outermost ResourceT. (Although, in this case we aren't
allocating any resources, so maybe it's a moot point for my simple example.)
I also don't think it makes sense to use takeMVar and putMVar instead of
withMVars as this would subvert the exception handling that withMVars provides.
And my attempts to define a withMVarR (in the ResourceT IO monad) also seem to
require runResourceT:
withMVarR :: MVar a - (a - ResourceT IO b) - ResourceT IO b
withMVarR m io = do
-- mask $ \restore - do
a - lift $ takeMVar m
b - {-restore-} (io a) `onExceptionR` (lift $ putMVar m a)
lift $ putMVar m a
return b
onExceptionR :: ResourceT IO a - ResourceT IO b - ResourceT IO a
onExceptionR a b = do
runResourceT $ transResourceT (\a - transResourceT (onException a) b) a
Maybe I need to create a HasMVar class analogous to HasRef?
(http://hackage.haskell.org/packages/archive/conduit/0.0.0/doc/html/Control-Monad-Trans-Resource.html#t:HasRef)
I'd appreciate any suggestions on how to make these fit together.
Warren___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] using ResourceT with MVars
On May 2, 2012, at 2:06 AM, Michael Snoyman wrote: I don't really know the details of LevelDB, but if the question is how do I run MVar operations in ResourceT, yes the answer would be lifted-base[1]. Since ResourceT is an instance of MonadBaseControl, you can use any of the functions in Control.Concurrent.MVar.Lifted. Perfect. I didn't know about lifted-base. Many thanks, Warren ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] prettyprint with IO
I wrote a parsec parser that does symbols lookups during the parsing process (ParsecT String Store IO a). Now I'd like to write a pretty printer that does the reverse. Unfortunately there doesn't appear to be a transformer version of Text.PrettyPrint.HughesPJ. Can anyone suggest a way to do this? Thanks, Warren ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] prettyprint with IO
Antoine, Thanks for the suggestions. No sooner did I send my message than I came to the same conclusion of creating a monadic version of the combinators to simplify the migration. It actually worked out fairly well -- most of the code ported over to the monadic version unaltered. The only exception is with literal lists used by combinators such as 'sep', e.g. sep [...]. This has to become sep $ sequence [...] in order to convert the argument into the expected monad. I put my code up on github https://github.com/warrenharris/pretty/blob/master/src/Text/PrettyPrint/Reader.hs. If you can take a look, I'd appreciate your suggestions. Warren On Apr 12, 2012, at 6:22 PM, Antoine Latter wrote: Hi Warren, On Thu, Apr 12, 2012 at 6:31 PM, Warren Harris warrensomeb...@gmail.com wrote: I wrote a parsec parser that does symbols lookups during the parsing process (ParsecT String Store IO a). Now I'd like to write a pretty printer that does the reverse. Unfortunately there doesn't appear to be a transformer version of Text.PrettyPrint.HughesPJ. Can anyone suggest a way to do this? Thanks, It seems like the opposite would be a function of type 'a - Store - IO Doc'. Maybe a function of type 'a - ReaderT Store IO Doc' could be easier to work with. If you go this route you could write a lifted versions of (), (+), hcat etc. An example: () :: Applicative m = m Doc - m Doc - m Doc I haven't tried any of this, so I'm not sure if you would get any big win over just using the first suggestion (a function of type 'a - Store - IO Doc') and using the stock combinators and threading the store around by hand. But do let me know if something works out. Antoine ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] haskell-platform vs macports
I assume that many haskell users out there on macs who are also users of macports, and I bet they've hit this same issue that I've hit numerous times. The problem is that there are 2 incompatible versions of libiconv -- one that ships with the mac, and one that's built with macports and that many macports-compiled libraries depend on. Work-arounds have been documented in numerous places (notably here: http://blog.omega-prime.co.uk/?p=96), but if you are trying to link with a macports-compiled libraries that depends on /opt/local/lib/libiconv.2.dylib, your only alternative seems to be building ghc from source in order to avoid the incompatibility. I hit this problem while trying to build a foreign-function interface to libarchive. So I built ghc from scratch -- in fact I built the whole haskell-platform. This was relatively painless, and fixed the libiconv problem. However, it brings me to the real point of my message, which is that the version of haskell-platform available via macports is way out of date (2009.2.0.2 http://www.macports.org/ports.php?by=namesubstr=haskell-platform). I'm wondering whether the haskell-platform team has considered maintaining a macports version of the haskell-platform for us mac users in addition to the binary install that's available now. This would avoid the incompatibilities such as this nagging one with libiconv. Perhaps it's just a matter of maintaining template versions of the port files? Warren Undefined symbols for architecture x86_64: _locale_charset, referenced from: _localeEncoding in libHSbase-4.3.1.0.a(PrelIOUtils.o) _iconv_close, referenced from: _hs_iconv_close in libHSbase-4.3.1.0.a(iconv.o) (maybe you meant: _hs_iconv_close) _iconv, referenced from: _hs_iconv in libHSbase-4.3.1.0.a(iconv.o) (maybe you meant: _hs_iconv, _hs_iconv_open , _hs_iconv_close ) _iconv_open, referenced from: _hs_iconv_open in libHSbase-4.3.1.0.a(iconv.o) (maybe you meant: _hs_iconv_open) ld: symbol(s) not found for architecture x86_64 collect2: ld returned 1 exit status cabal: Error: some packages failed to install: Libarchive-0.1 failed during the building phase. The exception was: ExitFailure 1 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] haskell-platform vs macports
Thomas, Thanks for the recommendation. I tried installing with homebrew, and it went fairly smoothly. Only 12 minutes to build haskell-platform, as opposed to the 11 hours to run port upgrade outdated yesterday! I did have to get help on one thing though. Although the mac ships with libarchive.dylib, it doesn't have archive.h. In order to install that, the following steps are necessary, since the libarchive package is otherwise blacklisted: brew tap homebrew/dupes brew install `brew --prefix`/Library/Formula/libarchive.rb Warren On Mar 22, 2012, at 3:31 PM, Thomas Schilling wrote: If you're not otherwise attached to MacPorts, you might want to check out Homebrew [1]. Its integration with the rest of OS X is generally more smoothly and I haven't come across any missing packages yet. [1]: http://mxcl.github.com/homebrew/ On 22 March 2012 16:34, Warren Harris warrensomeb...@gmail.com wrote: I assume that many haskell users out there on macs who are also users of macports, and I bet they've hit this same issue that I've hit numerous times. The problem is that there are 2 incompatible versions of libiconv -- one that ships with the mac, and one that's built with macports and that many macports-compiled libraries depend on. Work-arounds have been documented in numerous places (notably here: http://blog.omega-prime.co.uk/?p=96), but if you are trying to link with a macports-compiled libraries that depends on /opt/local/lib/libiconv.2.dylib, your only alternative seems to be building ghc from source in order to avoid the incompatibility. I hit this problem while trying to build a foreign-function interface to libarchive. So I built ghc from scratch -- in fact I built the whole haskell-platform. This was relatively painless, and fixed the libiconv problem. However, it brings me to the real point of my message, which is that the version of haskell-platform available via macports is way out of date (2009.2.0.2 http://www.macports.org/ports.php?by=namesubstr=haskell-platform). I'm wondering whether the haskell-platform team has considered maintaining a macports version of the haskell-platform for us mac users in addition to the binary install that's available now. This would avoid the incompatibilities such as this nagging one with libiconv. Perhaps it's just a matter of maintaining template versions of the port files? Warren Undefined symbols for architecture x86_64: _locale_charset, referenced from: _localeEncoding in libHSbase-4.3.1.0.a(PrelIOUtils.o) _iconv_close, referenced from: _hs_iconv_close in libHSbase-4.3.1.0.a(iconv.o) (maybe you meant: _hs_iconv_close) _iconv, referenced from: _hs_iconv in libHSbase-4.3.1.0.a(iconv.o) (maybe you meant: _hs_iconv, _hs_iconv_open , _hs_iconv_close ) _iconv_open, referenced from: _hs_iconv_open in libHSbase-4.3.1.0.a(iconv.o) (maybe you meant: _hs_iconv_open) ld: symbol(s) not found for architecture x86_64 collect2: ld returned 1 exit status cabal: Error: some packages failed to install: Libarchive-0.1 failed during the building phase. The exception was: ExitFailure 1 ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe -- Push the envelope. Watch it bend. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] erratic behavior for System.Time.diffClockTimes
I'm trying out GHC-7.0.3-x86_64 for Mac OS X and see what seems to be a bug in
System.Time.diffClockTimes. The TimeDiff tdPicosec field returns values that
seem to jump around erratically:
test = do
startTime - System.Time.getClockTime
endTime - System.Time.getClockTime
let dt = diffClockTimes endTime startTime
putStrLn $ show dt
Prelude Test test
TimeDiff {tdYear = 0, tdMonth = 0, tdDay = 0, tdHour = 0, tdMin = 0, tdSec = 0,
tdPicosec = 0}
Prelude Test test
TimeDiff {tdYear = 0, tdMonth = 0, tdDay = 0, tdHour = 0, tdMin = 0, tdSec = 0,
tdPicosec = 300}
Prelude Test test
TimeDiff {tdYear = 0, tdMonth = 0, tdDay = 0, tdHour = 0, tdMin = 0, tdSec = 0,
tdPicosec = -332901224613832315000}
Prelude Test test
TimeDiff {tdYear = 0, tdMonth = 0, tdDay = 0, tdHour = 0, tdMin = 0, tdSec = 0,
tdPicosec = 300}
Prelude Test test
TimeDiff {tdYear = 0, tdMonth = 0, tdDay = 0, tdHour = 0, tdMin = 0, tdSec = 0,
tdPicosec = -332901224613832285100}
Prelude Test
I can file a bug, unless I'm overlooking something here.
Warren
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] timely shutdown of timer threads
Interesting. I hadn't thought of this solution. You're forking the timer to yet a third thread so that if it continues waiting beyond the checkpoint thread shutdown it doesn't really matter. I guess that works as long as the main thread doesn't wait for all other threads to terminate before terminating the app. It still seems to me that haskell is lacking when it comes to operations that can wait for multiple conditions. Warren On Feb 1, 2011, at 6:25 PM, Albert Y. C. Lai wrote: On 11-02-01 02:58 PM, Warren Harris wrote: I have an application that forks a thread to run an activity on a timer. (The activity happens to be Berkeley DB checkpointing, but that's actually beside the point here.) The problem is that when the application wants to quit, I would like my main thread to be able to tell the timer thread to shut down in a timely way. However, I don't see a primitive in haskell that allows me to both wait for a timeout, or a notification. (If I were to do this in java, I would use wait/notify.) Use an MVar for signalling; use a two-valued data type to represent time-to-work or time-to-die. For extra correctness, use a second MVar to be notified that the child thread is really done --- because otherwise there would be the race condition of the child thread still in the middle of critical I/O when the program quits. import Control.Concurrent import Control.Exception(finally) data DoOrDie = Do | Die main = do v - newEmptyMVar finale - newEmptyMVar forkIO (timeloop v `finally` putMVar finale ()) putStrLn press enter to quit getLine putMVar v Die takeMVar finale timeloop v = run where run = do forkIO (threadDelay 1500 putMVar v Do) m - takeMVar v case m of Do - putStrLn checkpoint run Die - putStrLn checkmate ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] timely shutdown of timer threads
On Feb 2, 2011, at 2:02 PM, Johan Tibell wrote: On Wed, Feb 2, 2011 at 10:42 PM, Warren Harris warrensomeb...@gmail.com wrote: Interesting. I hadn't thought of this solution. You're forking the timer to yet a third thread so that if it continues waiting beyond the checkpoint thread shutdown it doesn't really matter. I guess that works as long as the main thread doesn't wait for all other threads to terminate before terminating the app. It still seems to me that haskell is lacking when it comes to operations that can wait for multiple conditions. I think we can make waiting for both I/O activity and timers at the same time using the I/O manager. I will need to do this for my planned timeout support in the network package. I could see wanting to wait for an MVar, timeout, IO activity, STM channels (anything that could block), all in the same event handler. (Rather like the old WaitForMultipleObjects on Windows.) This would also require a way to test each condition without blocking (MVar has this with tryTakeMVar, but it doesn't look like they all support it.) Warren ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] timely shutdown of timer threads
I have an application that forks a thread to run an activity on a timer. (The activity happens to be Berkeley DB checkpointing, but that's actually beside the point here.) The problem is that when the application wants to quit, I would like my main thread to be able to tell the timer thread to shut down in a timely way. However, I don't see a primitive in haskell that allows me to both wait for a timeout, or a notification. (If I were to do this in java, I would use wait/notify.) Here's the code I wrote to attempt this shutdown procedure: import Control.Exception import Database.Berkeley.Db import Data.IORef import System.Posix.Unistd ... closeEnv :: Env - IO () closeEnv env = do threadId - readIORef (envCheckpointThread env) case threadId of Just tid - killThread tid Nothing - return () dbEnv_close [] (envDbEnv env) startCheckpointing :: Env - IO ThreadId startCheckpointing env = do forkOS run where run = catch checkpoint handler checkpoint = checkpoint1 checkpoint checkpoint1 = unblock $ do let dbenv = envDbEnv env _ - sleep checkpointInterval putStrLn # checkpoint dbEnv_txn_checkpoint [] dbenv 0 0 handler ThreadKilled = return () handler exn = throw exn However, there are several problems here: First, the checkpointInterval is 15 sec, so it takes at least 15 seconds for the thread to wake up and be interrupted. Second, it isn't clear to me whether the killThread call should interrupt the sleep or not. In practice, several checkpoints occur before the ThreadKilled message is delivered, and this can take up to 2 minutes to shut down. Is there a better way to do this? Suggestions would be greatly appreciated, Warren ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] dph question
I trying to learn a bit about data parallel haskell, and started from the wiki page here: http://www.haskell.org/haskellwiki/GHC/Data_Parallel_Haskell. Two questions: The examples express the dot product as: dotp_double xs ys = sumP [:x * http://haskell.org/ghc/docs/latest/html/libraries/base/Prelude.html#v:. y | x - xs | y - ys:] Unless I'm missing something, shouldn't this actually be: dotp_double xs ys = sumP [:x * http://haskell.org/ghc/docs/latest/html/libraries/base/Prelude.html#v:. y | x - xs, y - ys:] Second, when I run Main with the prescribed 1 element array, everything seems to work quite nicely. The task takes about 2 seconds on my 4 processor x86_64, and threadscope shows all processors nicely utilized. However, when bumping this to 10 elements, rather than taking 10x longer as I expected, the process never terminates. During one run I even lost control of my machine and needed to do a hard reset. Are there known limits to the array sizes that can be handled with dph, or can someone suggest what might be going wrong here? Thanks, Warren ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] dph question
Got it - thanks. Any idea about the run-away process problem? Thanks,
Warren
On Fri, Oct 15, 2010 at 9:32 AM, Daniel Fischer daniel.is.fisc...@web.dewrote:
On Friday 15 October 2010 14:59:18, Warren Harris wrote:
I trying to learn a bit about data parallel haskell, and started from
the wiki page here:
http://www.haskell.org/haskellwiki/GHC/Data_Parallel_Haskell. Two
questions:
The examples express the dot product as:
dotp_double xs ys = sumP [:x *
http://haskell.org/ghc/docs/latest/html/libraries/base/Prelude.html#v:.
y | x - xs | y - ys:]
Unless I'm missing something, shouldn't this actually be:
dotp_double xs ys = sumP [:x *
http://haskell.org/ghc/docs/latest/html/libraries/base/Prelude.html#v:.
y | x - xs, y - ys:]
No, it's supposed to be a parallel list comprehension, the dot product is
sum $ zipWith (*) xs ys
and the
{ blah x y | x - xs | y - ys }
syntax (where {, } stand in for [, ] in parallel list comprehensions and
for [:, :] in parallel array comprehensions) means
{ blah x y | (x,y) - zip xs ys }
Second, when I run Main with the prescribed 1 element array,
everything seems to work quite nicely. The task takes about 2 seconds on
my 4 processor x86_64, and threadscope shows all processors nicely
utilized. However, when bumping this to 10 elements, rather than
taking 10x longer as I expected, the process never terminates. During
one run I even lost control of my machine and needed to do a hard reset.
Are there known limits to the array sizes that can be handled with dph,
or can someone suggest what might be going wrong here? Thanks,
Warren
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Build problems (hsp, trhsx, ultimately Happstack)
I thought I'd install happstack to give it a try, but immediately hit the following build problem: $ cabal install happstack Resolving dependencies... cabal: cannot configure HJScript-0.5.0. It requires hsx =0.7.0 For the dependency on hsx =0.7.0 there are these packages: hsx-0.7.0. However none of them are available. hsx-0.7.0 was excluded because happstack-0.4.1 requires hsx =0.5.5 0.6 Any suggestions? I'm on mac os 10.5.8 with ghc 6.12.1. Thx, Warren ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Build problems (hsp, trhsx, ultimately Happstack)
On May 1, 2010, at 12:08 AM, Alexander Solla wrote: I think that if you run this, you will satisfy all the dependencies: cabal install hjscript0.5.0 happstack Thank you. That worked. I had a similar problem installing the happstack-tutorial, but found that I could work around it with this: cabal install Crypto4.2.1 Warren___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Build problems (hsp, trhsx, ultimately Happstack)
On May 1, 2010, at 1:28 AM, Ivan Lazar Miljenovic wrote: Warren Harris warrensomeb...@gmail.com writes: On May 1, 2010, at 12:08 AM, Alexander Solla wrote: I think that if you run this, you will satisfy all the dependencies: cabal install hjscript0.5.0 happstack Thank you. That worked. I had a similar problem installing the happstack-tutorial, but found that I could work around it with this: cabal install Crypto4.2.1 And this, people, is why bounded dependencies in cabal files are so desirable. Uh... actually I was a little too quick in sending my last message. I only built crypto, not the tutorial. Trying again I hit this: $ cabal install Crypto4.2.1 happs-tutorial Resolving dependencies... Configuring containers-0.2.0.1... Preprocessing library containers-0.2.0.1... Building containers-0.2.0.1... Data/IntMap.hs:182:7: Could not find module `Data.Data': It is a member of the hidden package `base'. Perhaps you need to add `base' to the build-depends in your .cabal file. Use -v to see a list of the files searched for. cabal: Error: some packages failed to install: HSH-2.0.3 depends on containers-0.2.0.1 which failed to install. HStringTemplate-0.5 depends on containers-0.2.0.1 which failed to install. HStringTemplateHelpers-0.0.14 depends on containers-0.2.0.1 which failed to install. HaXml-1.13.3 depends on containers-0.2.0.1 which failed to install. MissingH-1.0.3 depends on containers-0.2.0.1 which failed to install. PBKDF2-0.3 depends on containers-0.2.0.1 which failed to install. SMTPClient-1.0.2 depends on containers-0.2.0.1 which failed to install. binary-0.5.0.2 depends on containers-0.2.0.1 which failed to install. containers-0.2.0.1 failed during the building phase. The exception was: ExitFailure 1 deepseq-1.1.0.0 depends on containers-0.2.0.1 which failed to install. happs-tutorial-0.9.3 depends on containers-0.2.0.1 which failed to install. happstack-0.4.1 depends on containers-0.2.0.1 which failed to install. happstack-data-0.4.1 depends on containers-0.2.0.1 which failed to install. happstack-helpers-0.49 depends on containers-0.2.0.1 which failed to install. happstack-ixset-0.4.1 depends on containers-0.2.0.1 which failed to install. happstack-server-0.4.1 depends on containers-0.2.0.1 which failed to install. happstack-state-0.4.1 depends on containers-0.2.0.1 which failed to install. happstack-util-0.4.1 depends on containers-0.2.0.1 which failed to install. hslogger-1.0.10 depends on containers-0.2.0.1 which failed to install. parallel-2.2.0.1 depends on containers-0.2.0.1 which failed to install. pureMD5-1.0.0.3 depends on containers-0.2.0.1 which failed to install. regex-base-0.93.1 depends on containers-0.2.0.1 which failed to install. regex-compat-0.92 depends on containers-0.2.0.1 which failed to install. regex-posix-0.94.1 depends on containers-0.2.0.1 which failed to install. strict-concurrency-0.2.3 depends on containers-0.2.0.1 which failed to install. syb-with-class-0.6.1 depends on containers-0.2.0.1 which failed to install. template-haskell-2.4.0.0 depends on containers-0.2.0.1 which failed to install. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Build problems (hsp, trhsx, ultimately Happstack)
On May 1, 2010, at 3:39 AM, Daniel Fischer wrote:
Try
$ cabal install --constraint=Crypto4.2.1 --
constraint=HJScript0.5
happs-tutorial
This had the same problem building containers:
Building containers-0.2.0.1...
Data/IntMap.hs:182:7:
Could not find module `Data.Data':
It is a member of the hidden package `base'.
Perhaps you need to add `base' to the build-depends in
your .cabal file.
Use -v to see a list of the files searched for.
cabal: Error: some packages failed to install:
On May 1, 2010, at 6:24 AM, Jeremy Shaw wrote:
I would recommend:
darcs get --lazy http://patch-tag.com/r/mae/happstack
cd happstack
chmod +x bin/build-install-all.sh
./bin/build-install-all.sh
This should install the latest version of happstack from darcs which
resolves most install problems. I intend to release it any minute
now, but I am waiting for a patch to hsp.cabal to be applied.
This had a different problem building happstack itself:
Building happstack-0.4.3...
trhsx: Error at SrcLoc {srcFilename = src/HSP/IdentityT.hs, srcLine
= 73, srcColumn = 1}:
Illegal instance declaration
cabal: Error: some packages failed to install:
happstack-0.4.3 failed during the building phase. The exception was:
ExitFailure 1
I can't see anything that looks obviously wrong with line 73 of that
file, but haven't studied it in detail.
Warren
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] haskell platform questions
FWIW, downloading the haskell-platform-2010.1.0.0 tarball and building it on my 10.5.8 system (with ghc 6.12.1 installed from the dmg) worked just fine. Didn't take too long either. Unfortunately I don't see any telltale linker options in the build logs. Warren On Mar 26, 2010, at 7:38 AM, Gregory Collins wrote: Malcolm Wallace malcolm.wall...@cs.york.ac.uk writes: It's a known issue, and it's mine. If you (naively) just expect to link on Snow Leopard without passing any special backwards- compatibility flags, and have things work on Leopard, well, Apple has news for you. gcc -mmacox-version-min=10.5.8 ? Something like that, on both gcc and ld; I think you also need a -isysroot /blah.sdk GCC option (unless that's only for Cocoa- specific stuff, I'm not sure). I've been dreading it but I think I'm going to have to dig out my Leopard CD and run it in a VM. G -- Gregory Collins g...@gregorycollins.net ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] haskell platform questions
BTW, I started to try the macports method (thinking that maybe building on my machine would resolve the linker problem), but the package up there seems to be the old one: $ port info haskell-platform haskell-platform @2009.2.0.2 (devel, haskell) Description: This is the the Haskell Platform: a single, standard Haskell distribution for every system. The Haskell Platform is a blessed library and tool suite for Haskell distilled from Hackage. Homepage: http://hackage.haskell.org/platform/ Runtime Dependencies: ghc, hs-platform-cgi, hs-platform-fgl, hs- platform-editline, hs-platform-GLUT, hs-platform-haskell-src, hs- platform-html, hs-platform-HUnit, hs-platform-mtl, hs-platform-network, hs-platform-OpenGL, hs-platform-parallel, hs- platform-parsec, hs-platform-QuickCheck, hs-platform-regex-base, hs- platform-regex-compat, hs-platform-regex-posix, hs-platform-stm, hs-platform-time, hs-platform-xhtml, hs- platform-zlib, hs-platform-HTTP, hs-platform-alex, hs-platform-happy, hs-platform-cabal Platforms:darwin License: unknown Maintainers: gwri...@macports.org Maybe the dependencies haven't changed? Also, not a big deal, but the instructions for the dmg version here http://hackage.haskell.org/platform/new/mac.html page say to follow the instructions, but there aren't any packaged with it. I mistakenly ran the Haskell Platform 2010.1.0.0 installer first thinking that maybe it would install ghc as a dependency, but this crashed. Then I uninstalled everything, installed GHC-6.12.1- i386.pkg, then Haskell Platform 2010.1.0.0, and rebooted... and then experienced the cabal problem. BTW, there's some description of linker flags here: http://discussions.apple.com/thread.jspa?threadID=2151112 but the developer claims 10.5.8 solved his problems (that's what I'm on). Warren On Mar 22, 2010, at 7:45 AM, Gregory Collins wrote: wren ng thornton w...@freegeek.org writes: I'm still on 10.5.8. I don't have cabal-install installed yet, but I just installed GHC-6.12.1/HP-2010.1.0.0. I can verify that ghci works fine so far. I'll check out cabal-install in the next couple days. If there is an issue here it'd be with the binaries that ship with the platform, not GHC; can you check /usr/local/bin/cabal to see if yours has the same issue? I'm betting I probably have to re-link the binaries with some magic don't break on Leopard linker flag. G -- Gregory Collins g...@gregorycollins.net ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] haskell platform questions
I downloaded the new haskell-platform-2010.1.0.0-i386.dmg today... ran the uninstaller, ghc installer and the platform installer. When I run ghci, it seems to work fine, but when I try cabal, I get this crash: $ cabal --version dyld: unknown required load command 0x8022 Trace/BPT trap Any suggestions? (I'm on 10.5.8). Maybe I'll try the macport install process instead. Thanks for your help, Warren On Mar 18, 2010, at 9:26 AM, John Velman wrote: I'm also on Mac Leopard. I tried installing ghc 6.12 with Haskell Platform 2009.2.9.2-i386.dmg (ghc 6.10.4) for some reason, and ran into a bunch of problems (problems to me, anyway). I ended up uninstalling 6.12 and reinstalling haskell platform. Uninstall is easy, there is an uninstaller script in /Library/Frameworks/GHC.framework/Tools. (strangely, not in ..Frameworks/HaskellPaltform.framework). I'm still using Leopard, but would like to move to Snow Leopard once I get a few things out of the way. But I see that there are (or were) some tricks in getting ghc to work on OS X 10.6, apparently. These seem to be well documented, but, I'd rather spend the time on my own projects. Best, John V. On Wed, Mar 17, 2010 at 09:07:03PM -0700, Warren Harris wrote: I installed haskell platform some time ago, and now I'm wondering what version I have. How do I find out? Also, when the new one comes along on the 21st, is there a way to upgrade? Or if I must first uninstall the one I have now, how do I uninstall it? Is it recommended to periodically upgrade packages that came with the platform (cabal upgrade), or is it recommended that they be left alone to avoid dependency incompatibilities. Similar question for ghc itself -- can/should it be upgraded in the context of haskell platform? (I was hoping to try Leksah, but it dies without ghc 6.12.1. I seem to have 6.10.4.) Apologies in advance if this is all documented somewhere, but I couldn't find it on the haskell platform site/trac. BTW, I'm on Mac/Leopard -- love the fact that it didn't take hours to build everything! Warren ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] haskell platform questions
I installed haskell platform some time ago, and now I'm wondering what version I have. How do I find out? Also, when the new one comes along on the 21st, is there a way to upgrade? Or if I must first uninstall the one I have now, how do I uninstall it? Is it recommended to periodically upgrade packages that came with the platform (cabal upgrade), or is it recommended that they be left alone to avoid dependency incompatibilities. Similar question for ghc itself -- can/should it be upgraded in the context of haskell platform? (I was hoping to try Leksah, but it dies without ghc 6.12.1. I seem to have 6.10.4.) Apologies in advance if this is all documented somewhere, but I couldn't find it on the haskell platform site/trac. BTW, I'm on Mac/ Leopard -- love the fact that it didn't take hours to build everything! Warren ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] buildExpressionParser prefix recursion
I was wondering why Parsec's buildExpressionParser doesn't allow prefix expressions to be handled recursively, e.g. given a prefix ! operation, it seems that !!a could parse without requiring parentheses (!(!a)). Is there an easy way to extend it? (I have a rich expression grammar I'd like to parse which includes several prefix forms, and it would be a shame to have to abandon buildExpressionParser since it is so concise and convenient.) Thanks, Warren ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Time Typeable Instances
Hi John, I just stumbled on this issue while trying to compile turbinado with the haskell platform / ghc 6.10.4. I got past it by manually editing back in the time definitions, but just wondering if there was an official resolution. Thanks, Warren On Oct 13, 2009, at 6:48 AM, John Goerzen wrote: Hugo Gomes wrote: The Glorious Glasgow Haskell Compilation System, version 6.10.4 with old-time-1.0.0.2 and time-1.1.2.4 This is a standard haskell platform on a windows xp. Cabal install didn't work complaining about missing instances of typeable for posix time and other datatypes, yet, after removing the macros (thus, adding those instances), hdbc and convertible compiled and installed fine. Removing the macros might be a bit overkill, probably finetuning them so that they add only the necessary instances for typeable in ghc 610 might be a better solution. I'm going to CC haskell-cafe on this because I am confused. Hugo, can you confirm what version of convertible you have? Back on May 21, I started a thread [1] on haskell-cafe complaining that GHC 6.10.3 included a newer time that included instances of Typeable for NominalDiffTime and UTCTime. This broke my code, which had manually defined instances for these types, as they were needed. Things got complicated, as only time's minor version number got incremented (x.x.x.Y) [2]. Cabal can't test against that version number. I wanted my code to work with old and new versions of GHC. Since testing against the version of time was impossible, I did the next best thing: tested against the version of GHC. #if __GLASGOW_HASKELL__ = 610 -- instances added in GHC 6.10.3 #else instance Typeable NominalDiffTime where typeOf _ = mkTypeName NominalDiffTime instance Typeable UTCTime where typeOf _ = mkTypeName UTCTime #endif Also, in the .cabal, there is a build-depends on time=1.1.2.4. Now, that would break for GHC 6.10.1 and 6.10.2 users, but will work for 6.10.3 and above, or 6.8 and below. Or so I thought. Now I'm getting complaints from people using 6.10.4 saying that there are now missing instances of Typeable with time 1.1.2.4. 1) Did the Typeable instances get dropped again from time? 2) What exactly should I do so this library compiles on GHC 6.8 and 6.10.x? I'm looking at the darcs repo for time and don't see the instances ever getting dropped. [1] http://osdir.com/ml/haskell-cafe@haskell.org/2009-05/msg00982.html [2] http://osdir.com/ml/haskell-cafe@haskell.org/2009-05/msg00985.html later addendum so it appears that what's happening here is that GHC 6.10.3 extralibs included time 1.1.3, but then haskell-platform standardized on 1.1.2.4. This is pretty annoying -- that haskell-platform would standardize on a version older than what shipped with a GHC release -- but I guess I can work around it by restricting my build-dep to be time 1.1.3 and re-adding the instances. Does this sound right? On Tue, Oct 13, 2009 at 2:51 AM, John Goerzen jgoer...@complete.org mailto:jgoer...@complete.org wrote: Hugo Gomes wrote: Hi, convertible and hdbc packages fail to compile in my standard instalation of the haskell platform. I think this has to do with those if ghc = 610 macros on the typeable instances for some datatypes. I removed them and now they work fine... What version of GHC and time do you have? -- John ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] mapping large structures into memory
I've dabbled in haskell, but am by no means an expert. I was hoping someone here could help me settle this debate so that we can more seriously consider haskell for a next version of an application we're building I would like to understand better what its capabilities are for directly mapping and managing memory. For instance, I would like mmap many large files into memory and mutate their internals directly... without needing to reallocate them (or chunks of them) in the haskell heap, and without resorting to a byte-array and byte-offset representation. Furthermore, I might also like to map intrinsic haskell data structures into this mmap'd memory such that standard library functions can manipulate them (perhaps in a purely functional way, e.g. treating them as haskell arrays of smaller foreign structures). I understand that the foreign function interface has the ability to marshall/unmarshall C structs, but I'm unsure of the memory implications of using this mechanism. Our application has a very large footprint, and reallocating some or all of these mapped files is a non- starter. Thanks, Warren ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] mapping large structures into memory
On Sep 25, 2009, at 12:14 PM, Don Stewart wrote: It is entirely possible to use mmap to map structures into memory. Thanks to the foreign function interface, there are well-defined semantics for calling to and from C. The key questions would be: * what is the type and representation of the data you wish to map * what operations on them Right... my question relates more to how well the intrinsic type system integrates with foreign/mapped structures. For instance, I wouldn't want to create my own foreign arrays, and have to replicate all sorts of library code that only works on haskell's intrinsic arrays. I'm assuming here that all this mapped data is self-contained, and doesn't point to heap-allocated structures, although that's a related question -- is it possible to inform the gc about heap pointers stored (temporarily) in these structures (and later identify them in order to swizzle them out when flushing the mapped file to disk). Warren ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Re: Haskell Logo write-in candidate
Hi Jon, I agree with much of your rant, and would agree that the logo is probably the least interesting about haskell, but I think that it's worth spending a little time to spiffy up haskell's image from a marketing perspective. Although I downplayed much of my design decisions by focusing on the logo's t-shirt potential, I just wanted to say that a lot of thought did go into the design aspects of what I sent out. A logo needs to be a crisp graphic, needs to draw people in who don't yet understand (pure lazy fun-- huh? or what's with that Amtrak symbol?) and convey a deeper meaning to those who do understand (the interplay of the monad and lambda). I think many of the logos convey some or all of these points, although many also fall short. This is all off in the realm of marketing psychology, which is a far cry from programming language design, but important in the overall product perception nonetheless. The other thing about this logo design that is so great is the community process that's creating it. It's the open source process in a nutshell -- the brightest minds playing off each other to build something bigger than the sum of the parts. So even if the new logo ends up looking like something that rolled down hill collecting rubbish, the story behind it will be brilliant -- like a family photo reflecting who we are and how we do things here. I hesitated in sending my write-in candidate in the first place because I didn't want to derail the process that's underway, but did in the end because I thought I saw something that was a little bigger than some of the parts here, and thought that others might be encouraged if they saw it too. Now at the risk of further muddling things, I'll just say that I like your idea of focusing on the :: symbol, and just wanted to provide my interpretation: inline: Fairbairn.png I think that's not bad either, although I think it loses a little of the distinction and intrigue of Pollard's lovely monad/lambda symbol with its curved edges. I'd be happy to wear this one too though... actually, given what I said above, I'll be happy to wear any of them! Warren On Mar 20, 2009, at 2:17 AM, Jon Fairbairn wrote: Warren Harris warrensomeb...@gmail.com writes: After spending a bit of time trying to decide how to vote, I ended up deciding that my favorite would be a hybrid of several of the designs (#9 #49 FalconNL, and #50 George Pollard). It's probably too late to include this in the voting, but here it is nonetheless: That's quite nice, but the = lambda thing looks too busy to me. What surprises me is that none(?) of the candidates makes use of the type symbol. I'd like to see a version something like yours, but with :: instead of =/lambda ::Haskell means of type Haskell, which is what we want people's programmes to be. Colour it interestingly and choose a good font and there you are. The interestingly coloured :: on its own would make a reasonable choice for a badge (eg for a favicon). * * * semi-rant warning: This whole badge/logo business seems to me to be an excellent example of Parkinson's law of triviality (choosing the colour of the bikeshed). We have a large (too large) number of variations on relatively few themes and a really sophisticated voting system, but no very clear idea of what they're for and no explanation (such as my of type Haskell above) of why the candidates are the way they are. I didn't join in much to the earlier discussion because I thought things would work out to something sensible in the end, but it doesn't look like that happened. Work out what the problem is before putting the solution up for election! I agree that the current badge is horrid (it looks like something that rolled down a hill and collected some rubbish on the way), but in the absence of a reasoned replacement, the first step would simply be to get rid of it. Designing these things isn't trivial, and while many of the candidates are quite good pieces of art, a badge needs to be more than that. Not that professional designers do better in general; only a few of them are any good at it -- the rest rely on most people not knowing pretty from appropriate and just rake in the cash. -- Jón Fairbairn jon.fairba...@cl.cam.ac.uk ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Haskell Logo write-in candidate
After spending a bit of time trying to decide how to vote, I ended up deciding that my favorite would be a hybrid of several of the designs (#9 #49 FalconNL, and #50 George Pollard). It's probably too late to include this in the voting, but here it is nonetheless: inline: HaskellHybrid.png (My primary criterion is would I wear this t-shirt?) :-) Warren___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] teaching functional programming at work
Thank you all for your responses on this. These ideas/materials are very helpful. In particular, the John Harrison book looks excellent (http://www.cl.cam.ac.uk/teaching/Lectures/funprog-jrh-1996/index.html ) -- a comprehensive and understandable introduction to all the concepts I had in mind. (I'm not sure what my audience's attention span will be though.) The WhyFP paper is also very good, although doesn't go into monads or types, it does a good job of illustrating modularity/composability. I'll send an update if this study group gets off the ground, to let you all know how it worked out. Warren___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] teaching functional programming at work
I am seeking suggestions from the haskell cafe for teaching functional programming concepts to colleagues at work. I'm currently working on a project using ocaml and functional programming techniques, and am a lone ranger at my workplace when it comes to this sort of thing (we are primarily a python shop). However there seems to be a growing curiosity in functional programming, and I think there's a lot to be learned from from it whether one chooses a functional language for their next big project or not. But I'm a practitioner, not an academic or lecturer, and although I find the idea of helping my colleagues understand these concepts to be an exciting prospect, I'm not really sure where to start in terms of materials or overall direction. My sense is to form a study group (perhaps going through Real World Haskell together), but I'm a little afraid of assembling people together for a now what? experience. So my first question is whether this is even a good idea? Some things I think would be useful to get across are: - fp is more than just an exercise in avoiding assignment statements -- why a smart programmer should care? - reading knowledge of ocaml and/or haskell (even syntax, precedence and infix operators can be an initial stumbling block) - core concepts: type classes, proper tail recursion, higher-order functions, folds, combinators, monads, monad transformers, arrows - evidence that concise expression can lead to less bugs / better maintainability (hard to prove, but perhaps a sense can be conveyed) - relationship between types / theorems / programs / proofs and why this is important for the future of software Regarding this last point, my own sense has been that by leveraging a powerful type system such as found in ocaml or haskell has allowed me to achieve things I never would have been able to without it (or at least would have ended up with a much less elegant implementation and much more debugging) but many seem to find types to be either an impedance to getting things done quickly, or at best irrelevant. I know this is for the most part a religious war, but I would like my colleagues to arrive at their opinion from an informed perspective, which means learning that there's more to typeful programming than what is presented by java or c++. Finally, any comments on how to make the learning experience fun, engaging, and a positive experience would be greatly appreciated. Warren ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] streaming translation using monads
On Nov 18, 2008, at 6:53 PM, Brandon S. Allbery KF8NH wrote: On 2008 Nov 18, at 21:23, Warren Harris wrote: However, each of the clauses is actually an output routine to send the expression that it denotes to a remote server, and a parser for receiving the results. Since a clause is really a pair of operations, it doesn't seem possible to formulate a monad that will compose all the output routines together and compose all the input routines together in one shot. (Note that the variables in the above code (v1, v2) represent inputs to be received from the remote server -- all outputs are packaged into the clause expressions themselves and are effectively literals.) Have you considered using arrows instead? I'm not that familiar with arrows, but have just looked at some papers by Hughes and Paterson. It appears that unlike monads, the right-hand side of the arrow operator is not a function, and as such could be used to define a pairwise stream sequencing operator for my particular case. Are there any specific references that show something like this, e.g. for implementing network protocols? (I'm still trying to get my head around the basics of arrows.) Thanks, Warren ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] streaming translation using monads
On Nov 19, 2008, at 9:11 AM, Justin Bailey wrote: On Tue, Nov 18, 2008 at 6:23 PM, Warren Harris [EMAIL PROTECTED] wrote: I am working on a query language translator, and although I feel that a monadic formulation would work well for this application, I've stumbled on a number of questions and difficulties that I thought the knowledgeable people here might be able to help me with. HaskellDB takes a similar approach. It's Query monad allows you to build queries which are then translated to SQL by a runQuery function. Could your bind operation collect the 'input' expressions and then output them all at once via a runTranslation function? Thanks for pointing this out. I had looked at Leigen Meijer's paper a while back which describes this, and in one branch of my code taken a very similar approach by using an abstract datatype with phantom types to represent the target language. However, I had concluded (perhaps incorrectly) that the technique was not amenable to streaming the results back to the client without first constructing an in-memory list with a universal type representing the values returned. Now perhaps the in-memory list part was a bad conclusion since the queries can be decorated with translation functions capable of streaming the results out to another channel. However, the use of a universal type for the values would still seem to be required since there is no way to implement type-indexed values when the queries themselves are expressed as an abstract datatype rather than as functions. Am I overlooking something? As an aside, maybe I've been unduly concerned with wrapping the raw values in a universal type since I've found that in most cases for my target language I must deal with the possibility of null results, and must wrap most values in a Maybe type anyway. So I must pay the allocation cost in most cases... and with phantom types perhaps I can avoid the possibility of dynamic type errors due to unexpected data or ill-formed target queries. Do you have to do in-place translation? Not sure I follow. Warren ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] streaming translation using monads
On Nov 19, 2008, at 12:19 PM, Justin Bailey wrote: On Wed, Nov 19, 2008 at 11:50 AM, Warren Harris [EMAIL PROTECTED] wrote: Now perhaps the in-memory list part was a bad conclusion since the queries can be decorated with translation functions capable of streaming the results out to another channel. However, the use of a universal type for the values would still seem to be required since there is no way to implement type-indexed values when the queries themselves are expressed as an abstract datatype rather than as functions. Am I overlooking something? If the type of the input determines the type of the output, and the type of the input can be determined statically, then I think you are overlooking this technique. But if your application requires that each input expression be sent to the remote client before type information can be determined, then you are right. In the case of HaskellDB, each operator/term in the Query monad enriches the type information available. Of course, that enrichment happens at compile time and occurs via type inference. For example, assuming two tables T1 and T2, with columns T1col and T2col, this expression: simpleQuery = do t1 - table T1 t2 - table T2 restrict ( .. some expression ...) project (t1 ! T1col1 # t2 ! T2col1) Gives a type similar to Query (RecCons T1Col1 Int (RecCons T2 Col2 Int RecNil)). RecCons and RecNil are types that allow a list to be built at the type level. The list carries the column names and types in the resulting projection. The type information is used to ensure queries only refer to columns that exist, datatypes are compared sensibly, etc. If in your case you can build that kind of structure based purely on the input language operators/terms, then it seems you could build the entire output expression in one shot. But again, if input and output have to be interleaved then I think you are stuck. What you describe is very similar to my current implementation, except I'm using a continuation function to receive the results input from the remote server rather than creating a list. The continuation's type (a curried function) is analogous to a list at the type level, if I understand correctly. However, since what I would like to do is stream the results in from one server and out to another (translating as I go), I really need some way to hook my translation in between each primitive field read from the input. So inside the body of the 'project' call you've given above, I might like to say something like: v1 - t1 ! T1col1; v2 - t2 ! T2col1; send client (trans v1 v2); return () However, since this input-receiving expression is now expressed as monad, it doesn't seem possible to use it to simultaneously formulate the output expressions that must be sent to the remote server to cause T1col1 and T2col1 to be requested. In my particular application, the remote server not only handles simple directives to retrieve sequences of primitives like T1col1, T2col1, but also higher-order formulations of them as lists and tuples. So the server may be sent a request like (T1col1, (list (T2col1, T2col2))) meaning request the tuple of T1col1 followed by a (possibly long) sequence of T2col1, T2col2 pairs and I want to be able to translate the incoming result stream without holding the entire thing in memory. Obviously I can do this by having one operation to generate the outbound query from a target language expression (implemented with an algebraic datatype), and another operation to handle the results based on the same expression, but it seems like some sort of formulation should be possible that allows both operations to be expressed simultaneously. In fact, I get that with the continuation-oriented combinators I mentioned earlier, but in practice I've found them to be very hard to work with. Just to make it more concrete, the above example might be expressed with my continuation-based combinators as: T1col1 == (\ k t1 - send client (trans1 t1); k) ++ list (T2col1 ++ T2col2 == (\ k t2c1 t2c2 - send client (trans2 t2c1 t2c2); k))) + + ... Here, the first function receives the value read from T1col1, translates it and send it to the originator of the request. This function is substituted for the an overall continuation -- receives the overall continuation as its first argument, k, and returns it at the end without passing any values to it. This effectively consumes the value t1. The second function is similar, and is called for each pair received from the list. The combinator '++', primitive directives like T1col1, and higher-order directives like 'list' are responsible for dealing with the overall expression grammar (i/o of delimiters, etc) and folding the overall continuation through the results as they are received. Sorry for being so long-winded here... I really appreciate your input. Warren
[Haskell-cafe] streaming translation using monads
I am working on a query language translator, and although I feel that a monadic formulation would work well for this application, I've stumbled on a number of questions and difficulties that I thought the knowledgeable people here might be able to help me with. As a translator, there's a source language and a target language, and both of these are specified as a grammar of nested expressions. My first thought was to formulate the target language as an embedded language where each clause expression is represented as a monad operation, so that the bind operator can join the pieces together, e.g.: (clause1, clause2 ...) could be specified as an embedded language as: clause1 = \ v1 - clause2 = \ v2 - ... However, each of the clauses is actually an output routine to send the expression that it denotes to a remote server, and a parser for receiving the results. Since a clause is really a pair of operations, it doesn't seem possible to formulate a monad that will compose all the output routines together and compose all the input routines together in one shot. (Note that the variables in the above code (v1, v2) represent inputs to be received from the remote server -- all outputs are packaged into the clause expressions themselves and are effectively literals.) A naive formulation of a monad to implement the above as output - input v might appear to work, but has the ill-effect of interleaving the output and input statements for each clause rather than composing something that can send the entire request, and then receive the entire result. This forces me to use output * input v as the type of each clause expression, but now it is not possible to write a monad with a bind operation that will compose pieces in terms of input variables. Instead I have had to resort to using a set of combinators that thread a continuation function through each clause and accumulate inputs as they are received: clause1 == (\ k v1 - k (trans1 v1)) ++ clause2 == (\ k v2 - k (trans2 v2)) ++ ... This threading is necessary in that I want to stream the translation back to the client requesting the translation rather than building up the (possibly large) results in memory. This formulation has proven to be quite cumbersome in practice, as the resulting continuation types reflect the depth-first traversal of the nested query expressions, and type errors can be quite unintuitive. (It is quite interesting though that each continuation/transformation function can receive not only receive the input from the immediately preceding clause, but from any of the preceding clauses, and also return more or fewer results. However getting anything wrong can be very problematic in that it can lead to either downstream *or* upstream errors depending on how the clauses are composed into an overall query expression.) An alternative to all this would be to use an algebraic datatype to specify the target language (with separate routines for the output and input operations), but that would appear to require another sum type to express the values to be received. I'd like to avoid that if possible since the projection of those values back into my program could lead to dynamic type errors, and also causes seemingly needless memory allocations. There must be another technique for this sort of streaming translation out there... I welcome any suggestions you might have! Warren Harris ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
