[Haskell-cafe] serialize an unknown type
Hi the list! I have a simple question, how can I serialize/deserialize a structure like this: data InputChoice c deriving Typeable data Event a where InputChoice :: (Eq c, Show c) => [c] -> c -> Event (InputChoice c) (...) I'd like that the values of type "c" get serialized to a String... That's the easy part, but for deserializing, oops! Cheers, Corentin ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Blocking IO & FIFOs
2012/10/20 Donn Cave : > Quoth Jason Dusek , > ... >> For my application, it's important to be able to run multiple >> queries against the same Bash session. Waiting for Bash to shut >> down is thus not a viable way to finalize the response. > > You could redirect to disk files and also use a pipe to wait for exit. > > I suppose you redirect I/O for individual queries by applying shell > redirections to the command? So like this, > >uname -a > /tmp/cmd1 2> /tmp/cmd2 7> /tmp/exitpipe > > ... then read from /tmp/exitpipe, ignore empty result and read command > outputs from the disk files. If I could somehow arrange to detect EOF when /tmp/exitpipe is closed, then I might as well redirect 1 and 2 to FIFOs and wait for them to EOF, collecting the output. However, all of my experiments suggest that there is simply no way in Haskell to detect the closing of the write end of a FIFO. With `openFileBlocking', one can detect when a FIFO is *opened*; but not when it is closed. -- Jason Dusek pgp // solidsnack // C1EBC57DC55144F35460C8DF1FD4C6C1FED18A2B ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Blocking IO & FIFOs
Quoth Jason Dusek , ... > For my application, it's important to be able to run multiple > queries against the same Bash session. Waiting for Bash to shut > down is thus not a viable way to finalize the response. You could redirect to disk files and also use a pipe to wait for exit. I suppose you redirect I/O for individual queries by applying shell redirections to the command? So like this, uname -a > /tmp/cmd1 2> /tmp/cmd2 7> /tmp/exitpipe ... then read from /tmp/exitpipe, ignore empty result and read command outputs from the disk files. Donn ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Blocking IO & FIFOs
2012/10/20 Wilfried van Asten : > Perhaps an interleaving solution as in process-conduit is > still viable: > > - Check if one or both of the fifo's are still ready (Based >on your statement about the reading end receiving EOF >hIsEOF should work here). If both fifos are done the query >is finished so break the loop. Alas, checking for EOF does not work. I mentioned this in passing in my prior email; the code was somewhat involved and I have deleted it. Here is a simple example of something that does not work as expected: In the first terminal: :; mkfifo fifo :; ghci -- :m + GHC.IO.Handle.FD System.IO -- do { h <- openFileBlocking "fifo" ReadMode ; hGetContents h } In the second terminal, *after* doing everything in the first terminal: :; cat > fifo < type some characters here > ^D Notice that the characters appear in the first terminal, as the output of hGetContents. Sending ^D to end cat does not register any effect in GHCi; hGetContents dutifully waits and you can in fact run cat on the FIFO again to send more characters to the same instances of hGetContents. This would seem to be due to non-blocking IO, deep in the IO manager. > - Check if some output is available on oh. If so read some of >it. Repeat. > > - Otherwise check if some output is available on eh. If so >read some of it. Repeat loop > > I also see you don't do anything with the std_out and std_err > pipes of bash as given by runInteractiveProcess. These could > also cause a problem even when the FIFO's are working > correctly. Replace these by handles to the null file or let > the output be dumped on the parent's std_in and std_out > (StdStream Inherit). I would prefer to leave them be, since they're passed in from the caller, who nominally owns them. If you mean that I should close them in `start', well, that would make it hard to debug this stuff; and if I simply tie them to the parent's file descriptors, it will make it hard to deal with more than a few CoBashes at one time while testing. Using cat to read the FIFOs and allowing Haskell to read from cat does work, actually. https://gist.github.com/3923673 Shell really is such a nice language for tying together processes. -- Jason Dusek pgp // solidsnack // C1EBC57DC55144F35460C8DF1FD4C6C1FED18A2B {-# LANGUAGE OverloadedStrings , ScopedTypeVariables , ParallelListComp , TupleSections #-} module CoBash where import Control.Applicative import Control.Concurrent import Control.Concurrent.MVar import Control.Exception import Control.Monad import Data.Bits import Data.ByteString.Char8 (ByteString) import qualified Data.ByteString.Char8 as Bytes import Data.Maybe import Data.Monoid import qualified GHC.IO.Handle.FD import System.IO import System.IO.Error import System.Process import System.Posix.ByteString import System.IO.Temp import qualified Text.ShellEscape as Esc start :: IO (Handle, Handle, Handle, ProcessHandle) start = runInteractiveProcess "bash" [] Nothing (Just []) query :: (Handle, Handle, Handle, ProcessHandle) -> ByteString -> IO (ByteString, ByteString) query (i, _, _, _) query = withFIFOs query' where query' ofo efo = do Bytes.hPut i cmd hFlush i [ob, eb] <- backgroundReadFIFOs [ofo, efo] return (ob, eb) where cmd = Bytes.unlines ["{", query, "} 1>" <> ofo <> " 2>" <> efo] shutdown :: (Handle, Handle, Handle, ProcessHandle) -> IO () shutdown (i, _, _, p) = () <$ hClose i <* waitForProcess p openFIFO path = GHC.IO.Handle.FD.openFileBlocking (Bytes.unpack path) ReadMode -- | Run an IO action with two FIFOs in scope, which will removed after it -- completes. withFIFOs :: (RawFilePath -> RawFilePath -> IO a) -> IO a withFIFOs m = withSystemTempDirectory "cobash." m' where m' = (uncurry m =<<) . mk . Bytes.pack mk d = (o, e) <$ (createNamedPipe o mode >> createNamedPipe e mode) where (o, e) = (d <> "/o", d <> "/e") mode = ownerReadMode .|. ownerWriteMode .|. namedPipeMode drainFIFO :: ByteString -> IO ByteString drainFIFO path = do (i, o, e, p) <- bash ["-c", "exec cat <"<>(Bytes.unpack path)] hClose i hClose e Bytes.hGetContents o <* waitForProcess p backgroundReadFIFOs theFIFOs = do cells <- sequence (newEmptyMVar <$ theFIFOs) sequence_ [ forkIO (drainFIFO p >>= putMVar c) | p <- theFIFOs | c <- cells ] sequence (takeMVar <$> cells) bash args = runInteractiveProcess "bash" args Nothing (Just []) ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Blocking IO & FIFOs
Perhaps an interleaving solution as in process-conduit is still viable: - Check if one or both of the fifo's are still ready (Based on your statement about the reading end receiving EOF hIsEOF should work here). If both fifos are done the query is finished so break the loop. - Check if some output is available on oh. If so read some of it. Repeat. - Otherwise check if some output is available on eh. If so read some of it. Repeat loop I also see you don't do anything with the std_out and std_err pipes of bash as given by runInteractiveProcess. These could also cause a problem even when the FIFO's are working correctly. Replace these by handles to the null file or let the output be dumped on the parent's std_in and std_out (StdStream Inherit). On Sat, Oct 20, 2012 at 4:00 PM, Jason Dusek wrote: > For my application, it's important to be able to run multiple > queries against the same Bash session. Waiting for Bash to shut > down is thus not a viable way to finalize the response. > > Perhaps I can spawn two cats with their outputs connected to the > FIFOs and wait for them to terminate. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Blocking IO & FIFOs
2012/10/20 Asten, W.G.G. van (Wilfried, Student M-CSC) : > Would you be happy with a solution like this: > > - First create two handles to two files in the tmp directory > - Then use StdStream's UseHandle to redirect std_err and std_out > (using CreatePipe for std_in) to these files > - Then write your query to the Handle for std_in > - waitForProcess > - Collect std_out and std_err from the temporary files. > > If that is not satisfactory you may want to check out conduit-process > (http://hackage.haskell.org/packages/archive/process-conduit/0.5.0.2/doc/html/src/Data-Conduit-Process.html#conduitProcess) > that also does some interaction with a process and interleaves std_in > and std_out. It should not be to hard to combine std_err into that > concept. I also faced this same problem and in one case solved it by > using a temporary file to hold my content which was also deleted > afterwards (This immediately prevented the content building up in > memory). For my application, it's important to be able to run multiple queries against the same Bash session. Waiting for Bash to shut down is thus not a viable way to finalize the response. Perhaps I can spawn two cats with their outputs connected to the FIFOs and wait for them to terminate. -- Jason Dusek pgp // solidsnack // C1EBC57DC55144F35460C8DF1FD4C6C1FED18A2B ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Blocking IO & FIFOs
Hi all, I am developing a coroutine-like interface to Bash. http://hpaste.org/76523 The idea is that one can send multiple "queries" to an interpreter and then collect the results of each query. How do we know when Bash is done with each query? Waiting for "no more output" seems ambiguous; so the way CoBash works is: * Each query gets a tmp dir with two named pipes in it. * The query is wrapped in redirections to the pipes. * The pipes are removed when the query completes. This does work, sort of: +Prelude> :load CoBash.hs [1 of 1] Compiling CoBash ( CoBash.hs, interpreted ) Ok, modules loaded: CoBash. *CoBash> tuple@(i,o,e,p) <- start e :: Handle i :: Handle o :: Handle p :: ProcessHandle tuple :: (Handle, Handle, Handle, ProcessHandle) *CoBash> query tuple "for n in {1..4}; do sleep 1; echo $n; done" ("1\n2\n3\n4\n","") it :: (ByteString, ByteString) I say sort of because it is quite brittle. Many commands do not return at all, for example: *CoBash> query tuple "uname -a" The way I retrieve the output from the FIFOs seems dangerous: (,) <$> Bytes.hGetContents oh <*> Bytes.hGetContents eh Surely, the FIFO for STDERR can not be read from until the FIFO for STDOUT is finished; but if there is a great deal of error output then the process will fill the FIFO's buffer and get stuck. If we switch the order of the reads, the "for n in ..." example above blocks: (,) <$> Bytes.hGetContents eh <*> Bytes.hGetContents oh I have tried a few different ways to read from the two handles concurrently; for example, by giving each thread an MVar to put the contents in, or by using hGetNonBlocking on a list of handles in a loop. Using the latter method, I never get EOF; it just collects empty strings forever. For comparison's sake, the expect behaviour with FIFOs is: In the first terminal: :; mkfifo fifo :; cat > fifo a b c d ^D :; In the second terminal: :; cat < fifo a b c d :; Here I "open" the FIFO for reading with < while opening it for writing with >. As long as the writer writes, the reader reads; when the writer closes the write end of the pipe, the reader receives EOF. Trying to duplicate the read behaviour in Haskell, using hGetContents from GHCi while using cat to write to the FIFO, doesn't work; which seems a little bogus. There have a been a few past threads about FIFOs and their troublesome interaction with Haskell's async-by-default IO style. To switch to System.Posix for IO -- and deal with Ptr Word8, in order to handle binary data -- seems like an awful step down. -- Jason Dusek pgp // solidsnack // C1EBC57DC55144F35460C8DF1FD4C6C1FED18A2B {-# LANGUAGE OverloadedStrings , ScopedTypeVariables , ParallelListComp , TupleSections #-} module CoBash where import Control.Applicative import Control.Concurrent import Control.Concurrent.MVar import Control.Exception import Control.Monad import Data.Bits import Data.ByteString.Char8 (ByteString) import qualified Data.ByteString.Char8 as Bytes import Data.Maybe import Data.Monoid import qualified GHC.IO.Handle.FD import System.IO import System.IO.Error import System.Process import System.Posix.ByteString import System.IO.Temp import qualified Text.ShellEscape as Esc start :: IO (Handle, Handle, Handle, ProcessHandle) start = runInteractiveProcess "bash" [] Nothing (Just []) query :: (Handle, Handle, Handle, ProcessHandle) -> ByteString -> IO (ByteString, ByteString) query (i, _, _, _) query = withFIFOs query' where query' ofo efo = do Bytes.hPut i cmd hFlush i [oh, eh] <- mapM openFIFO [ofo, efo] (,) <$> Bytes.hGetContents oh <*> Bytes.hGetContents eh -- Works. -- (,) <$> Bytes.hGetContents eh <*> Bytes.hGetContents oh -- Blocks. where cmd = Bytes.unlines ["{", query, "} 1>" <> ofo <> " 2>" <> efo] shutdown :: (Handle, Handle, Handle, ProcessHandle) -> IO () shutdown (i, _, _, p) = () <$ hClose i <* waitForProcess p openFIFO path = GHC.IO.Handle.FD.openFileBlocking (Bytes.unpack path) ReadMode -- | Run an IO action with two FIFOs in scope, which will removed after it -- completes. withFIFOs :: (RawFilePath -> RawFilePath -> IO a) -> IO a withFIFOs m = withSystemTempDirectory "cobash." m' where m' = (uncurry m =<<) . mk . Bytes.pack mk d = (o, e) <$ (createNamedPipe o mode >> createNamedPipe e mode) where (o, e) = (d <> "/o", d <> "/e") mode = ownerReadMode .|. ownerWriteMode .|. namedPipeMode ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Monad.Reader #21 call for copy
Call for Copy: The Monad.Reader - Issue 21 Whether you're an established academic or have only just started learning Haskell, if you have something to say, please consider writing an article for The Monad.Reader! The submission deadline for Issue 21 will be: **Tuesday, January 1** The Monad.Reader The Monad.Reader is a electronic magazine about all things Haskell. It is less formal than journal, but somehow more enduring than a wiki- page. There have been a wide variety of articles: exciting code fragments, intriguing puzzles, book reviews, tutorials, and even half-baked research ideas. Submission Details ~~ Get in touch with me if you intend to submit something -- the sooner you let me know what you're up to, the better. Please submit articles for the next issue to me by e-mail (ezy...@mit.edu). Articles should be written according to the guidelines available from http://themonadreader.wordpress.com/contributing/ Please submit your article in PDF, together with any source files you used. The sources will be released together with the magazine under a BSD license. If you would like to submit an article, but have trouble with LaTeX please let me know and we'll work something out. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe