Re: [Haskell-cafe] forkProcess, forkIO, and multithreaded runtime

2012-10-15 Thread Joey Hess
Michael Snoyman wrote: I think I have a misunderstanding of how forkProcess should be working. Ultimately this relates to some bugs in the development version of keter, but I've found some behavior in a simple test program which I wouldn't have expected either, which may or may not be related.

Re: [Haskell-cafe] strange hangs with -threaded runtime (now with test case)

2012-07-18 Thread Joey Hess
Just following up to my problem, I was seeing lots of hangs in various places in my program when it was built with the threaded runtime. I eventually tracked every single hang back to calls to MissingH's System.Cmd.Utils, including pipeFrom, pipeTo, pipeBoth, and pOpen. I was at this point

Re: [Haskell-cafe] strange hangs with -threaded runtime

2012-07-14 Thread Joey Hess
Joey Adams wrote: Off the top of my head, I'm not aware of any (non-Windows-related) gotchas with using -threaded. However, some functions in MissingH (e.g. in System.Cmd.Utils) call forkProcess. To quote the documentation of forkProcess [1]: --- forkProcess comes with a giant warning:

Re: [Haskell-cafe] strange hangs with -threaded runtime (now with test case)

2012-07-14 Thread Joey Hess
I've found a minimal test case that seems to demonstrate a bug in either MissingH or ghc's threaded runtime. Or I'm doing something stupid in fewer lines of code than usual. ;) When built with the threaded runtime, after a short while it hangs in hGetContents. import System.Cmd import System.IO

Re: [Haskell-cafe] strange hangs with -threaded runtime (now with test case)

2012-07-14 Thread Joey Hess
Antoine Latter wrote: Well, hPipeFrom does indeed call forkProcess internally. I don't fully understand when it is and is not safe to use 'forkProcess' with the threaded runtime of GHC. Which version of GHC are you using? I've reproduced the problem with 7.4.2, and 7.4.1. Just tried

Re: [Haskell-cafe] strange hangs with -threaded runtime (now with test case)

2012-07-14 Thread Joey Hess
I've isolated the hang further to hPipeFrom's use of System.Log.Logger. Commenting out calls to logging functions makes it reliably run ok. See the LINE OF DEATH in the attached updated test case. ... And it turns out that System.Log.Logger uses a MVar to hold the logger information. So I think

[Haskell-cafe] strange hangs with -threaded runtime

2012-07-13 Thread Joey Hess
I have a rather large program that works fine with ghc's default runtime, but now I'd like to build it with -threaded, and this causes it to hang in a way I have not been able to pin down to anything specific. It hangs at different points each time it's run; running it in strace will consistently

[Haskell-cafe] [Word8] - FilePath using ghc's file system encoding

2012-06-20 Thread Joey Hess
I found myself writing the code below today, and thought I'd write to see if there's a better way, perhaps one that doesn't use unsafePerformIO. A sample use case for this is a program that reads a FilePath from a Handle, and then operates on that file on disk. GHC uses a special encoding for

Re: [Haskell-cafe] building ghc on arch linux ARM?

2012-04-10 Thread Joey Hess
Joachim Breitner wrote: Most of these architectures do not have a native code generator (so they are compiled via C) and are unregisterized, i.e. GHC knows nothing about their registers. Both cause a performance penalty; I don’t know numbers. I assume this is what Joey refers to. But maybe

Re: [Haskell-cafe] building ghc on arch linux ARM?

2012-04-08 Thread Joey Hess
Thomas DuBuisson wrote: On Sun, Apr 8, 2012 at 4:03 PM, Francesco Mazzoli f...@mazzo.li wrote: No, it is not possible to build GHC without GHC. Building GHC on ARM is going to be extremely tricky (I'm not sure anyone has ever done it). I used to use an unregistered build of GHC built by

Re: [Haskell-cafe] Why so many strings in Network.URI, System.Posix and similar libraries?

2012-03-12 Thread Joey Hess
Jason Dusek wrote: :info System.Posix.Env.getEnvironment System.Posix.Env.getEnvironment :: IO [(String, String)] -- Defined in System.Posix.Env But there is no law that environment variables must be made of characters: The recent ghc release provides

Re: [Haskell] [Haskell-cafe] ANNOUNCE: system-filepath 0.4.5 and system-fileio 0.3.4

2012-02-06 Thread Joey Hess
John Millikin wrote: That was my understanding also, then QuickCheck found a counter-example. It turns out that there are cases where a valid path cannot be roundtripped in the GHC 7.2 encoding. The issue is that [238,189,178] decodes to 0xEF72, which is within the 0xEF00-0xEFFF range that

Re: [Haskell-cafe] ANNOUNCE: system-filepath 0.4.5 and system-fileio 0.3.4

2012-02-06 Thread Joey Hess
John Millikin wrote: That was my understanding also, then QuickCheck found a counter-example. It turns out that there are cases where a valid path cannot be roundtripped in the GHC 7.2 encoding. The issue is that [238,189,178] decodes to 0xEF72, which is within the 0xEF00-0xEFFF range that

Re: [Haskell-cafe] ANNOUNCE: system-filepath 0.4.5 and system-fileio 0.3.4

2012-02-06 Thread Joey Hess
John Millikin wrote: Perhaps I'm misunderstanding, but the definition of 'withFilePath' you provided is definitely locale-dependent. Unless getFileSystemEncoding is constant? I think/hope it's locale dependent, but undecodable bytes are remapped, so as long as the system's locale doesn't

Re: [Haskell] [Haskell-cafe] ANNOUNCE: system-filepath 0.4.5 and system-fileio 0.3.4

2012-02-05 Thread Joey Hess
John Millikin wrote: In GHC 7.2 and later, file path handling in the platform libraries was changed to treat all paths as text (encoded according to locale). This does not work well on POSIX systems, because POSIX paths are byte sequences. There is no guarantee that any particular path will

Re: [Haskell-cafe] ANNOUNCE: system-filepath 0.4.5 and system-fileio 0.3.4

2012-02-05 Thread Joey Hess
John Millikin wrote: In GHC 7.2 and later, file path handling in the platform libraries was changed to treat all paths as text (encoded according to locale). This does not work well on POSIX systems, because POSIX paths are byte sequences. There is no guarantee that any particular path will

Re: [Haskell-cafe] space leak when repeatedly calling Control.Monad.State.Strict.modify

2012-01-30 Thread Joey Hess
Yves Parès wrote: Have you tried to compile your code with optimisations? I guess GHC's strictness analysis would find strict evaluation is better here. The original code I saw this happen to the wild was built with -O2. I didn't try building the test case with optimisations. -- see shy jo

[Haskell-cafe] space leak when repeatedly calling Control.Monad.State.Strict.modify

2012-01-29 Thread Joey Hess
The attached test case quickly chews up hundreds of MB of memory. If modified to call work' instead, it runs in constant space. Somehow the value repeatedly read in from the file and stored in the state is leaking. Can anyone help me understand why? (ghc 7.0.4) -- see shy jo {-# LANGUAGE

Re: [Haskell-cafe] space leak when repeatedly calling Control.Monad.State.Strict.modify

2012-01-29 Thread Joey Hess
Claude Heiland-Allen wrote: Control.Monad.State.Strict is strict in the actions, but the state itself is still lazy, so you end up building a huge thunk in the state containing all the updates that ever took place to the initial state. Using this should fix it: modify' :: MonadState s m

Re: [Haskell-cafe] where to put general-purpose utility functions

2012-01-23 Thread Joey Hess
David Fox wrote: I try to create a workflow for this sort of thing. I create a package with a name like set-extra, with one module Data.Set.Extra and an alternative Data.Set module that exports both the old Data.Set and the symbols in Data.Set.Extra. Then I email the maintainers of the

[Haskell-cafe] where to put general-purpose utility functions

2012-01-21 Thread Joey Hess
I'm finding a rather unusual problem as I write haskell.. Unlike every other language I've used, large portions of my haskell code are turning out to be general-purpose, reusable code. Fully 20% of the haskell code I've written for git-annex is general purpose. Now, I came out of a decade of perl

Re: [Haskell-cafe] ANNOUNCE: monad-control-0.3

2011-12-06 Thread Joey Hess
Bas van Dijk wrote: You can use the following: {-# LANGUAGE GeneralizedNewtypeDeriving, TypeFamilies, MultiParamTypeClasses #-} import Control.Applicative import Control.Monad import Control.Monad.Base import Control.Monad.Trans.Class import Control.Monad.Trans.Control import

Re: [Haskell-cafe] ANNOUNCE: monad-control-0.3

2011-12-06 Thread Joey Hess
Bas van Dijk wrote: On 6 December 2011 09:12, Bas van Dijk v.dijk@gmail.com wrote: instance MonadBaseControl IO Annex where    newtype StM Annex a = StAnnex (StM (StateT AnnexState IO) a)    liftBaseWith f = Annex $ liftBaseWith $ \runInIO -                       f $ liftM StAnnex .

Re: [Haskell-cafe] ANNOUNCE: monad-control-0.3

2011-12-05 Thread Joey Hess
I'm trying to convert from 0.2 to 0.3, but in way over my head. {-# LANGUAGE GeneralizedNewtypeDeriving #-} newtype Annex a = Annex { runAnnex :: StateT AnnexState IO a } deriving ( Monad, MonadIO, -- MonadControlIO

Re: [Haskell-cafe] ANNOUNCE: monad-control-0.3

2011-12-05 Thread Joey Hess
Michael Snoyman wrote: I just spent a fair amount of time yesterday upgrading packages to monad-control 0.3. What I had to do was add in the MonadTransControl and MonadBaseControl instances manually. It's actually not too difficult; just copy out the instance for StateT and make a few

Re: [Haskell-cafe] Fwd: Data.Time

2011-07-02 Thread Joey Hess
Yitzchak Gale wrote: The API for parsing and rendering time in Data.Time is based on the standard API for that in C - like the libraries in most languages. It's pretty standard stuff. I'm sure it can be improved upon though. If you have a useful alternative time parsing library, please

[Haskell-cafe] unicode filenames advice

2011-02-11 Thread Joey Hess
I've been trying to deal with how my haskell program handles unicode filenames. Been dealing with problems like those described here: http://hackage.haskell.org/trac/ghc/ticket/3307 Or, simply demonstrated by feeding unicode to getLine = readFile My approach currently is to carefully identify

[Haskell-cafe] ghc -O2 and HUnit weirdness

2011-01-06 Thread Joey Hess
Suppose I'm playing with HUnit for nearly the first time. I might write a test program like this, to see what a test failure looks like in practice: import Test.HUnit import Test.HUnit.Tools main = runVerboseTests $ TestList [ TestCase $ assertBool should fail False , TestCase $