Re: recompilation on --main-is
Will be fixed in 7.4.2: http://hackage.haskell.org/trac/ghc/ticket/5878 Cheers, Simon On 16/02/2012 05:10, Evan Laforge wrote: Thanks for bringing this up. I've been having the same problem and was thinking there was something wrong with my system. I have 3 main files and one util that they all import. Every build recompiles every file, even if there were no changes. On Feb 15, 2012 8:39 PM, Conrad Parker con...@metadecks.org mailto:con...@metadecks.org wrote: Hi, We have a project with around 200 Haskell source files, and around 20 executables which import these (as well as importing third-party libraries). We first build all the non-main files using a makefile generated with ghc -M. We then build the executables using a separate invocation of ghc --make --main-is foo.hs, for each foo.hs. This worked fine with ghc-7.2.1; each final ghc --make would simply link the pre-built object files -- but with ghc-7.4.1 all these files are recompiled for each target executable. As a result our full build takes around 10-20x longer with ghc-7.4.1 than ghc-7.2.1. Looking at compiler/iface/FlagChecker.hs, it seems that the --main-is flag is used as an input to the recompilation checker. It would make sense that --main-is should force recompilation of the particular file that exports 'main', but it also forces recompilation of its dependencies. Is this a bug or is there a good reason to recompile everything? Conrad. ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org mailto:Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Removal of #include HsFFI.h from template-hsc.h breaks largefile support on 32bit Linux
On 15/02/2012 12:31, Eugene Crosser wrote: Hello all, I am new here, but I want to report what I suspect may be a problem. I ran into it while using some third-party package from hackage on a 32bit Linux with ghc 7.4.1. I discovered that off_t fields in the .hsc files in the package where interpreted as 32bit words. I suspected that 64bit offsets should be used because even 32bit Linux has largefile support with 64bit offsets. I found that earlier versions of hsc2hs included HsFFI.h into the generated C code, and HsFFI.h in turn indirectly includes ghcautoconf.h which has #define _FILE_OFFSET_BITS 64 in it. So, if I build the .hsc files like this: hsc2hs -i HsFFI.h filename.hsc then off_t is 64bit and 64bit file manipulation syscalls are used. I did not check it but I think that earlier versions of hsc2hs where creating largefile-aware version of the code by default, because HsFFI.h was included in the code by default. This is a simple test program: checktypes.hsc -- run like this: hsc2hs checktypes.hsc runhaskell checktypes.hs module Main where #includesys/types.h main = do putStrLn $ show (#size off_t) $ hsc2hs checktypes.hsc runhaskell checktypes.hs 4 $ hsc2hs -i HsFFI.h checktypes.hsc runhaskell checktypes.hs 8 As I understand, this situation means that while the ghc itself and haskell programs compiled by it are largefile-capable, any third party modules that contain .hsc files are not. If I am right, this is probably not a good thing. Please can some guru take a look at this issue? Guru at your service :-) We discovered this during the 7.4 cycle: http://hackage.haskell.org/trac/ghc/ticket/2897#comment:12 Packages that were relying on `HsFFI.h` to define `_FILE_OFFSET_BITS` should no longer do this, instead they should use an appropriate autoconf script or some other method. See the `unix` package for an example of how to do this. It was really a mistake that it worked before. Cheers, Simon See also: http://www.haskell.org/pipermail/glasgow-haskell-users/2009-February/016606.html http://www.haskell.org/pipermail/cvs-ghc/2011-September/065848.html Thanks, Eugene ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: recent changes to hsc2hs break wrapping c++
On 15/02/2012 19:51, Evan Laforge wrote: On Wed, Feb 15, 2012 at 11:17 AM, Eugene Crossercros...@average.org wrote: On 02/15/2012 09:59 PM, Evan Laforge wrote: Unfortunately, the result is I (apparently) can't use it now. Here's how that happens: The change was to remove the dependency on HsFFI.h. Evan, *if* including HsFFI.h is the only thing you need, you might try to add -i HsFFI.h to your hsc2hs rule. Well, the main thing is that the C++ compiler chokes on the C stub file created. So as long as it creates and tries to compile that with the compiler given in '-c' then it'll crash. I guess another solution would be a separate '-c' flag for the compiler used for the stub :) The problem still stands because we cannot expect every module author to do the same. I think it's not too big a burden since for ghc it's just -I$(ghc --print-libdir)/include, and Duncan makes a good case for requiring it. I'm not sure why you're using the C++ compiler from hsc2hs, and I don't think that is guaranteed to work. But maybe we can fix it if there's a legitimate reason to want it. Cheers, Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Removal of #include HsFFI.h from template-hsc.h breaks largefile support on 32bit Linux
Hello Simon, thanks for your attention :) On 02/16/2012 04:25 PM, Simon Marlow wrote: I found that earlier versions of hsc2hs included HsFFI.h into the [...] As I understand, this situation means that while the ghc itself and haskell programs compiled by it are largefile-capable, any third party modules that contain .hsc files are not. If I am right, this is probably not a good thing. We discovered this during the 7.4 cycle: http://hackage.haskell.org/trac/ghc/ticket/2897#comment:12 Packages that were relying on `HsFFI.h` to define `_FILE_OFFSET_BITS` should no longer do this, instead they should use an appropriate autoconf script or some other method. See the `unix` package for an example of how to do this. It was really a mistake that it worked before. But that means that the C build environment has to be constructed independently for each module (that needs it), and consequently is not guaranteed to match the compiler's environment. Would it be better (more consistent) to propagate GHC's (or other compiler's) environment by default, along the lines of the comment #16? To cite Duncan, each Haskell implementation has its own C environment, and hsc2hs must use that same environment or it will produce incorrect results. Just a thought, and, as I said, I am not really qualified to argue... Eugene signature.asc Description: OpenPGP digital signature ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Removal of #include HsFFI.h from template-hsc.h breaks largefile support on 32bit Linux
I have similar issues to this in jhc due to its pervasive caching of compilation results. Basically I must keep track of any potentially ABI-changing flags and ensure they are consistently passed to every compilation unit and include them in the signature hash along with the file contents. I make sure to always pass said flags on the command line to all the tools, as in -D_FILE_OFFSET_BITS=64 gets passed to both gcc and hsc2hs rather than relying on a config file. John ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Help me grok addFinalizer
When I read the docs for System.Mem.Weak, it all seems to make sense. But then this code doesn't run as I expect it to when I turn on -threaded: http://hpaste.org/63832 (Expected/actual output are listed in the paste.) I've tried this on 7.4.1 and 7.0.4 with the same results. Can someone enlighten me? Cheers, Mike Craig ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Help me grok addFinalizer
On Thu, Feb 16, 2012 at 2:04 PM, Michael Craig mks...@gmail.com wrote: When I read the docs for System.Mem.Weak, it all seems to make sense. But then this code doesn't run as I expect it to when I turn on -threaded: http://hpaste.org/63832 (Expected/actual output are listed in the paste.) I've tried this on 7.4.1 and 7.0.4 with the same results. Can someone enlighten me? First off, I'm pretty sure finalizers won't run until the data they were associated with has been GCd, and GHC doesn't do GCs unless there is allocation - threadDelay doesn't allocate much, I imagine. Also, from the docs: A weak pointer may also have a finalizer of type IO (); if it does, then the finalizer will be run at most once, at a time after the key has become unreachable by the program (dead). The storage manager attempts to run the finalizer(s) for an object soon after the object dies, but promptness is not guaranteed. It is not guaranteed that a finalizer will eventually run, and no attempt is made to run outstanding finalizers when the program exits. Therefore finalizers should not be relied on to clean up resources - other methods (eg. exception handlers) should be employed, possibly in addition to finalisers. Antoine ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Help me grok addFinalizer
Quoting Antoine Latter aslat...@gmail.com: On Thu, Feb 16, 2012 at 2:04 PM, Michael Craig mks...@gmail.com wrote: When I read the docs for System.Mem.Weak, it all seems to make sense. But then this code doesn't run as I expect it to when I turn on -threaded: http://hpaste.org/63832 (Expected/actual output are listed in the paste.) I've tried this on 7.4.1 and 7.0.4 with the same results. Can someone enlighten me? First off, I'm pretty sure finalizers won't run until the data they were associated with has been GCd, and GHC doesn't do GCs unless there is allocation - threadDelay doesn't allocate much, I imagine. This seems to be an explanation of why a finalizer might run later than expected (or not run at all). But his paste shows that it runs *earlier* than what he expected. ~d ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Help me grok addFinalizer
This seems to be an explanation of why a finalizer might run later than expected (or not run at all). But his paste shows that it runs *earlier* than what he expected. What he said. 64-bit GHC on OS X gives me this: ... However, it's a different story when `-O2` is specified: ... You're right. I was compiling with cabal and had -O turned on without knowing it. So this looks like an optimization bug? Mike Craig On Thu, Feb 16, 2012 at 3:55 PM, Austin Seipp mad@gmail.com wrote: 64-bit GHC on OS X gives me this: $ ghc -fforce-recomp -threaded finalizer [1 of 1] Compiling Main ( finalizer.hs, finalizer.o ) Linking finalizer ... $ ./finalizer waiting ... done! waiting ... running finalizer done! However, it's a different story when `-O2` is specified: $ ghc -O2 -fforce-recomp -threaded finalizer [1 of 1] Compiling Main ( finalizer.hs, finalizer.o ) Linking finalizer ... $ ./finalizer waiting ... running finalizer done! waiting ... done! This smells like a bug. The stranger thing is that the GC will run the finalizer, but it doesn't reclaim the object? I'd think `readIORef` going after an invalidated pointer the GC reclaimed would almost certainly crash. On Thu, Feb 16, 2012 at 2:04 PM, Michael Craig mks...@gmail.com wrote: When I read the docs for System.Mem.Weak, it all seems to make sense. But then this code doesn't run as I expect it to when I turn on -threaded: http://hpaste.org/63832 (Expected/actual output are listed in the paste.) I've tried this on 7.4.1 and 7.0.4 with the same results. Can someone enlighten me? Cheers, Mike Craig ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users -- Regards, Austin ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Help me grok addFinalizer
On Thu, Feb 16, 2012 at 02:55:13PM -0600, Austin Seipp wrote:
64-bit GHC on OS X gives me this:
$ ghc -fforce-recomp -threaded finalizer
[1 of 1] Compiling Main ( finalizer.hs, finalizer.o )
Linking finalizer ...
$ ./finalizer
waiting ...
done!
waiting ...
running finalizer
done!
However, it's a different story when `-O2` is specified:
$ ghc -O2 -fforce-recomp -threaded finalizer
[1 of 1] Compiling Main ( finalizer.hs, finalizer.o )
Linking finalizer ...
$ ./finalizer
waiting ...
running finalizer
done!
waiting ...
done!
This smells like a bug. The stranger thing is that the GC will run the
finalizer, but it doesn't reclaim the object? I'd think `readIORef`
going after an invalidated pointer the GC reclaimed would almost
certainly crash.
The finalizer is attached to the Thing, not the IORef. I haven't
checked, but I assume that ioref gets inlined, so effectively (ioref x)
is evaluated early. If you change it to
readIORef (ioref' x) = \ix - ix `seq` return ()
and define
{-# NOINLINE ioref' #-}
ioref' :: Thing - IORef Bool
ioref' = ioref
then you'll get the sort of output you expect.
Finalizers are tricky things, especially when combined with some of
GHC's optimisations.
Thanks
Ian
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Help me grok addFinalizer
I haven't checked, but ...
I checked, and your solution works. In the context of a larger program,
getting NOLINE pragmas in all the right places would be challenging,
wouldn't it?
I found a bug report on the GHC Trac [1] in which Simon explains the
importance of evaluating the thunk before calling addFinalizer. (Otherwise
the finalizer is added to the thunk.) This works:
newThing :: IO Thing
newThing = do
x - Thing `fmap` newIORef True
return $ unsafePerformIO ( do
x' - evaluate x
addFinalizer x' $ putStrLn running finalizer ) `seq` x
If anyone can show me how to get rid of unsafePerformIO in there, that'd be
great. Tried a few things to no avail.
Finalizers are tricky things, especially when combined with some of
GHC's optimisations.
No kidding!
[1] http://hackage.haskell.org/trac/ghc/ticket/5365
Mike Craig
On Thu, Feb 16, 2012 at 4:15 PM, Ian Lynagh ig...@earth.li wrote:
On Thu, Feb 16, 2012 at 02:55:13PM -0600, Austin Seipp wrote:
64-bit GHC on OS X gives me this:
$ ghc -fforce-recomp -threaded finalizer
[1 of 1] Compiling Main ( finalizer.hs, finalizer.o )
Linking finalizer ...
$ ./finalizer
waiting ...
done!
waiting ...
running finalizer
done!
However, it's a different story when `-O2` is specified:
$ ghc -O2 -fforce-recomp -threaded finalizer
[1 of 1] Compiling Main ( finalizer.hs, finalizer.o )
Linking finalizer ...
$ ./finalizer
waiting ...
running finalizer
done!
waiting ...
done!
This smells like a bug. The stranger thing is that the GC will run the
finalizer, but it doesn't reclaim the object? I'd think `readIORef`
going after an invalidated pointer the GC reclaimed would almost
certainly crash.
The finalizer is attached to the Thing, not the IORef. I haven't
checked, but I assume that ioref gets inlined, so effectively (ioref x)
is evaluated early. If you change it to
readIORef (ioref' x) = \ix - ix `seq` return ()
and define
{-# NOINLINE ioref' #-}
ioref' :: Thing - IORef Bool
ioref' = ioref
then you'll get the sort of output you expect.
Finalizers are tricky things, especially when combined with some of
GHC's optimisations.
Thanks
Ian
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Unpack primitive types by default in data
Hi all, I've been thinking about this some more and I think we should definitely unpack primitive types (e.g. Int, Word, Float, Double, Char) by default. The worry is that reboxing will cost us, but I realized today that at least one other language, Java, does this already today and even though it hurts performance in some cases, it seems to be a win on average. In Java all primitive fields get auto-boxed/unboxed when stored in polymorphic fields (e.g. in a HashMap which stores keys and fields as Object pointers.) This seems analogous to our case, except we might also unbox when calling lazy functions. Here's an idea of how to test this hypothesis: 1. Get a bunch of benchmarks. 2. Change GHC to make UNPACK a no-op for primitive types (as library authors have already worked around the lack of unpacking by using this pragma.) 3. Run the benchmarks. 4. Change GHC to always unpack primitive types (regardless of the presence of an UNPACK pragma.) 5. Run the benchmarks. 6. Compare the results. Number (1) might be what's keeping us back right now, as we feel that we don't have a good benchmark set. I suggest we try with nofib first and see if there's a different and then move on to e.g. the shootout benchmarks. I imagine that ignoring UNPACK pragmas selectively wouldn't be too hard. Where the relevant code? Cheers, Johan ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Unpack primitive types by default in data
On Thu, Feb 16, 2012 at 4:25 PM, Johan Tibell johan.tib...@gmail.com wrote: The worry is that reboxing will cost us, but I realized today that at least one other language, Java, does this already today and even though it hurts performance in some cases, it seems to be a win on average. In Java all primitive fields get auto-boxed/unboxed when stored in polymorphic fields (e.g. in a HashMap which stores keys and fields as Object pointers.) This seems analogous to our case, except we might also unbox when calling lazy functions. By the way, I'd like to keep the two sources of reboxing separate. Users cannot do much about the reboxing that occurs due to using polymorphic data structures [1], such as Data.Map. However, the user can do something about reboxing due to lazy functions, namely making those functions stricter [2]. 1. In theory the user could create a cut-n-paste copy of the data structure and specialize it to a particular type, but I think we all agree that would be unfortunate (not to say that it cannot be justified in extreme cases.) 2. Although there might be cases when this isn't possible either, as in the case of non-specialized, non-inlined polymorphic functions. -- Johan ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Unpack primitive types by default in data
FWIW jhc has always unboxed everything smaller or equal to the size of a pointer unconditionally. It's all about the cache performance. John On Thu, Feb 16, 2012 at 4:25 PM, Johan Tibell johan.tib...@gmail.com wrote: Hi all, I've been thinking about this some more and I think we should definitely unpack primitive types (e.g. Int, Word, Float, Double, Char) by default. The worry is that reboxing will cost us, but I realized today that at least one other language, Java, does this already today and even though it hurts performance in some cases, it seems to be a win on average. In Java all primitive fields get auto-boxed/unboxed when stored in polymorphic fields (e.g. in a HashMap which stores keys and fields as Object pointers.) This seems analogous to our case, except we might also unbox when calling lazy functions. Here's an idea of how to test this hypothesis: 1. Get a bunch of benchmarks. 2. Change GHC to make UNPACK a no-op for primitive types (as library authors have already worked around the lack of unpacking by using this pragma.) 3. Run the benchmarks. 4. Change GHC to always unpack primitive types (regardless of the presence of an UNPACK pragma.) 5. Run the benchmarks. 6. Compare the results. Number (1) might be what's keeping us back right now, as we feel that we don't have a good benchmark set. I suggest we try with nofib first and see if there's a different and then move on to e.g. the shootout benchmarks. I imagine that ignoring UNPACK pragmas selectively wouldn't be too hard. Where the relevant code? Cheers, Johan ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Unpack primitive types by default in data
On Thu, Feb 16, 2012 at 4:42 PM, Johan Tibell johan.tib...@gmail.com wrote: 1. In theory the user could create a cut-n-paste copy of the data structure and specialize it to a particular type, but I think we all agree that would be unfortunate (not to say that it cannot be justified in extreme cases.) I thought data families could help here, as in, you could do a SPECIALIZE on a data type and the compiler will turn it into a data family behind the scenes and specialize everything to said type. though, that wouldn't do things like turn a Data.Map into a Data.IntMap... John ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Unpack primitive types by default in data
Hi Johan, Sounds like it's definitely worth playing with. I would hesitate to use the shootout benchmarks though, simply because anything there is already going to be unpacked to the hilt. How difficult do you think it would be to implement this in GHC? Cheers, Alex On 17/02/2012, at 11:25, Johan Tibell wrote: Hi all, I've been thinking about this some more and I think we should definitely unpack primitive types (e.g. Int, Word, Float, Double, Char) by default. The worry is that reboxing will cost us, but I realized today that at least one other language, Java, does this already today and even though it hurts performance in some cases, it seems to be a win on average. In Java all primitive fields get auto-boxed/unboxed when stored in polymorphic fields (e.g. in a HashMap which stores keys and fields as Object pointers.) This seems analogous to our case, except we might also unbox when calling lazy functions. Here's an idea of how to test this hypothesis: 1. Get a bunch of benchmarks. 2. Change GHC to make UNPACK a no-op for primitive types (as library authors have already worked around the lack of unpacking by using this pragma.) 3. Run the benchmarks. 4. Change GHC to always unpack primitive types (regardless of the presence of an UNPACK pragma.) 5. Run the benchmarks. 6. Compare the results. Number (1) might be what's keeping us back right now, as we feel that we don't have a good benchmark set. I suggest we try with nofib first and see if there's a different and then move on to e.g. the shootout benchmarks. I imagine that ignoring UNPACK pragmas selectively wouldn't be too hard. Where the relevant code? Cheers, Johan ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Unpack primitive types by default in data
Couldn't you measure it by changing base to use data Int = I# {-# UNPACK
#-} Int# and see what happens?
On Thursday, February 16, 2012, Alex Mason wrote:
Hi Johan,
Sounds like it's definitely worth playing with. I would hesitate to use
the shootout benchmarks though, simply because anything there is already
going to be unpacked to the hilt.
How difficult do you think it would be to implement this in GHC?
Cheers,
Alex
On 17/02/2012, at 11:25, Johan Tibell wrote:
Hi all,
I've been thinking about this some more and I think we should
definitely unpack primitive types (e.g. Int, Word, Float, Double,
Char) by default.
The worry is that reboxing will cost us, but I realized today that at
least one other language, Java, does this already today and even
though it hurts performance in some cases, it seems to be a win on
average. In Java all primitive fields get auto-boxed/unboxed when
stored in polymorphic fields (e.g. in a HashMap which stores keys and
fields as Object pointers.) This seems analogous to our case, except
we might also unbox when calling lazy functions.
Here's an idea of how to test this hypothesis:
1. Get a bunch of benchmarks.
2. Change GHC to make UNPACK a no-op for primitive types (as library
authors have already worked around the lack of unpacking by using this
pragma.)
3. Run the benchmarks.
4. Change GHC to always unpack primitive types (regardless of the
presence of an UNPACK pragma.)
5. Run the benchmarks.
6. Compare the results.
Number (1) might be what's keeping us back right now, as we feel that
we don't have a good benchmark set. I suggest we try with nofib first
and see if there's a different and then move on to e.g. the shootout
benchmarks.
I imagine that ignoring UNPACK pragmas selectively wouldn't be too
hard. Where the relevant code?
Cheers,
Johan
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org javascript:;
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org javascript:;
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Unpack primitive types by default in data
On Thu, Feb 16, 2012 at 4:52 PM, Alex Mason axm...@gmail.com wrote: Sounds like it's definitely worth playing with. I would hesitate to use the shootout benchmarks though, simply because anything there is already going to be unpacked to the hilt. That was the point of number (2) above. By disabling their unpack pragmas through the compiler and measuring the performance, we can simulate a world where people don't write unpack pragmas and then see how much GHC could improve things in such a world. If the gain is large enough we can turn this optimization on and people can waste less time cluttering their code with unpack pragmas! Note that this proposal doesn't but you any new expressive powers. The intention is two-fold: decrease the amount of clutter in Haskell programs and have code written by beginner/intermediate level Haskellers perform better out-of-the-box. How difficult do you think it would be to implement this in GHC? Quite easy I think. All the difficulty is in getting good benchmarks. -- Johan ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Unpack primitive types by default in data
Welcome back to reality Don! ;)
On Thu, Feb 16, 2012 at 4:56 PM, Don Stewart don...@gmail.com wrote:
Couldn't you measure it by changing base to use data Int = I# {-# UNPACK #-}
Int# and see what happens?
I'm not sure what this would mean. Int# is already unpacked.
-- Johan
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Unpack primitive types by default in data
Of course, that makes sense. Do we already have a way to specify that a field should be lazy? I can imagine programs where we don't want this behaviour, and people would need an escape hatch (or their programs might run slower, or even not terminate). I know we've got lazy patterns with ~, do we have data Foo = Foo ~Int | Bar ~Double Alex On 17/02/2012, at 11:59, Johan Tibell wrote: On Thu, Feb 16, 2012 at 4:52 PM, Alex Mason axm...@gmail.com wrote: Sounds like it's definitely worth playing with. I would hesitate to use the shootout benchmarks though, simply because anything there is already going to be unpacked to the hilt. That was the point of number (2) above. By disabling their unpack pragmas through the compiler and measuring the performance, we can simulate a world where people don't write unpack pragmas and then see how much GHC could improve things in such a world. If the gain is large enough we can turn this optimization on and people can waste less time cluttering their code with unpack pragmas! Note that this proposal doesn't but you any new expressive powers. The intention is two-fold: decrease the amount of clutter in Haskell programs and have code written by beginner/intermediate level Haskellers perform better out-of-the-box. How difficult do you think it would be to implement this in GHC? Quite easy I think. All the difficulty is in getting good benchmarks. -- Johan ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Unpack primitive types by default in data
On Thu, Feb 16, 2012 at 5:03 PM, Alex Mason axm...@gmail.com wrote: Of course, that makes sense. Do we already have a way to specify that a field should be lazy? I can imagine programs where we don't want this behaviour, and people would need an escape hatch (or their programs might run slower, or even not terminate). I know we've got lazy patterns with ~, do we have data Foo = Foo ~Int | Bar ~Double This optimization would only apply for strict fields (otherwise it would be unsound.) There's already a NOUNPACK pragma in HEAD that lets you turn the optimization off. I think NOUNPACK was added both to support this feature and so one could turn off the effect of `-funbox-strict-fields` selectively. -- Johan ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Unpack primitive types by default in data
On Thu, Feb 16, 2012 at 4:25 PM, Johan Tibell johan.tib...@gmail.com wrote: I've been thinking about this some more and I think we should definitely unpack primitive types (e.g. Int, Word, Float, Double, Char) by default. Initially we could hide this feature behind a new -funbox-strict-primitive-fields, but I definitely think we should push towards making it the default as some of the benefit (especially for beginners) comes from that. -- Johan ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: recompilation on --main-is
On Thu, Feb 16, 2012 at 3:45 AM, Simon Marlow marlo...@gmail.com wrote: Will be fixed in 7.4.2: http://hackage.haskell.org/trac/ghc/ticket/5878 Nice, thanks for the quick fix! ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: parallelizing ghc
However, the GHC API doesn't provide a way to do this directly (I hadn't really thought about this when I suggested the idea before, sorry). The GHC API provides support for compiling multiple modules in the way that GHCi and --make work; each module is added to the HPT as it is compiled. But when compiling single modules, GHC doesn't normally use the HPT - interfaces for modules in the home package are normally demand-loaded in the same way as interfaces for package modules, and added to the PIT. The crucial difference between the HPT and the PIT is that the PIT supports demand-loading of interfaces, but the HPT is supposed to be populated in the right order by the compilation manager - home package modules are assumed to be present in the HPT when they are required. Yah, that's what I don't understand about HscEnv. The HPT doc says that in one-shot mode, the HPT is empty and even local modules are demand-cached in the ExternalPackageState (which the PIT belongs to). And the EPS doc itself reinforces that where it says in one-shot mode home-package modules accumulate in the external package state. So why not just ignore the HPT, and run multiple one-shot compiles, and let all the info accumulate in the PIT? A fair amount of work in GhcMake is concerned with trimming old data out of the HPT, I assume this is for ghci that wants to reload changed modules but keep unchanged ones. I don't actually care about that since I can assume the modules will be unchanged over one run. So I tried just calling compileFile multiple times in the same GhcMonad, assuming the mutable bits of the HscEnv get updated appropriately. Here are the results for a build of about 200 modules: with persistent server: no link: 3.30s user 1.60s system 12% cpu 38.323 total 3.50s user 1.66s system 13% cpu 38.368 total link: 21.66s user 4.13s system 35% cpu 1:11.62 total 21.59s user 4.54s system 38% cpu 1:08.13 total 21.82s user 4.70s system 35% cpu 1:14.56 total without server (ghc -c): no link: 109.25s user 19.90s system 240% cpu 53.750 total 109.11s user 19.23s system 243% cpu 52.794 total link: 128.10s user 21.66s system 201% cpu 1:14.29 total ghc --make (with linking since I can't turn that off): 42.57s user 5.83s system 74% cpu 1:05.15 total The 'user' is low for the server because it doesn't count time spent by the subprocesses on the other end of the socket, but excluding linking it looks like I can shave about 25% off compile time. Unfortunately it winds up being just about the same as ghc --make, so it seems too low. Perhaps I should be using the HPT? I'm also falling back to plain ghc for linking, maybe --make can link faster when it has everything cached? I guess it shouldn't, because it presumably just dispatches to ld. ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Unpack primitive types by default in data
Hi all, On Thu, Feb 16, 2012 at 4:59 PM, Johan Tibell johan.tib...@gmail.com wrote: On Thu, Feb 16, 2012 at 4:52 PM, Alex Mason axm...@gmail.com wrote: How difficult do you think it would be to implement this in GHC? Quite easy I think. All the difficulty is in getting good benchmarks. The computeRep function in compiler/basicTypes/DataCon.lhs seems to be the right place for this. -- Johan ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: recompilation on --main-is
Hi Simon, I've tested with current master (commit 6f4a073) and the link time is back down. In fact the compile time seems fast in general, and an unrelated ld error which I hadn't yet tracked down has disappeared too. Thanks for the quick patch! Conrad. On 16 February 2012 19:45, Simon Marlow marlo...@gmail.com wrote: Will be fixed in 7.4.2: http://hackage.haskell.org/trac/ghc/ticket/5878 Cheers, Simon On 16/02/2012 05:10, Evan Laforge wrote: Thanks for bringing this up. I've been having the same problem and was thinking there was something wrong with my system. I have 3 main files and one util that they all import. Every build recompiles every file, even if there were no changes. On Feb 15, 2012 8:39 PM, Conrad Parker con...@metadecks.org mailto:con...@metadecks.org wrote: Hi, We have a project with around 200 Haskell source files, and around 20 executables which import these (as well as importing third-party libraries). We first build all the non-main files using a makefile generated with ghc -M. We then build the executables using a separate invocation of ghc --make --main-is foo.hs, for each foo.hs. This worked fine with ghc-7.2.1; each final ghc --make would simply link the pre-built object files -- but with ghc-7.4.1 all these files are recompiled for each target executable. As a result our full build takes around 10-20x longer with ghc-7.4.1 than ghc-7.2.1. Looking at compiler/iface/FlagChecker.hs, it seems that the --main-is flag is used as an input to the recompilation checker. It would make sense that --main-is should force recompilation of the particular file that exports 'main', but it also forces recompilation of its dependencies. Is this a bug or is there a good reason to recompile everything? Conrad. ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org mailto:Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
