Re: profiling experience
On Wed, Dec 06, 2006 at 03:21:23PM +, Kirsten Chevalier wrote: On 12/6/06, Serge D. Mechveliani [EMAIL PROTECTED] wrote: What may consitute this strange CAF cost of 96% ? Kirsten Chevalier [EMAIL PROTECTED] wrote I didn't look at your code all that carefully, but did you build the GHC libraries with -prof -auto-all? (Not just -prof.) If you don't build the libraries with -auto-all, then cost centres won't get inserted for library functions, and if it's really a standard library function that's taking all of that time, the profiling report won't indicate that. I made ghc-6.6 from the official source in a standard way: ./configure ...; make; make install Does this presume that this also generates the .p.o GHC library versions for -prof -auto-all ? No; you must have built the profiling libraries (i.e., building the libraries with -prof), otherwise you wouldn't have been able to compile your code for profiling. But, building the profiling libraries in the standard way doesn't add -auto-all to the compile flags. So if you want to build the libraries with -auto-all, do: make EXTRA_HC_OPTS=-auto-all Thank you. I am trying this. One could ask, why the GHC installation does not apply -auto-all by default? but be careful! As I said in my previous message, adding cost centres disables some optimizations, so the profiling results you get from this may not be accurate with respect to what would happen if you ran your code after building it with -O and no profiling. In my example, -O makes the run 2 times faster, and the profiling time per-cents are shown the same as for -Onot. I suspect that this particular invilisible time eater will present independently on -O flag and on the cost centers. - Serge Mechveliani [EMAIL PROTECTED] ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
throwTo block statements considered harmful
Title: throwTo block statements considered harmful This is a short essay to prove that the current GHC concurrency implementation has a critical flaw. And this is on the wiki at: http://haskell.org/haskellwiki/GHC/Concurrency#throwTo_.26_block_statements_considered_harmful The key problem is, at least in the presence of block/unblock, that Exceptions are never reliably delivered. And this is not a theoretical case, but can cause a hang in something as innocuous as program A given below. Simon Marlow wrote: I think people are misunderstanding the nature of a safepoint. The safepoint is a point at which you are prepared to have exceptions delivered. This does not mean that they *will* be delivered, just that they can. If you need to *wait* for an asynchronous exception, then you shouldn't be using them at all. Right. If a thread mostly runs inside 'block' with the occasional safe point, then your exceptions are not really asynchronous, they're synchronous. In this case, I'd say a better solution is to have an explicit event queue, and instead of the safe point take an event from the queue. The action on receiving an event can be to raise an exception, if necessary. Cheers, Simon The implementation of asynchronous signals, as described by the paper Asynchronous exceptions in Haskell Simon Marlow, Simon Peyton Jones, Andy Moran and John Reppy, PLDI'01. is fatally inconsistent with the implementation in GHC 6.4 and GHC 6.6 today. The implemented semantics have strictly weaker guarantees and render programs using asynchronous expressions impossible to write correctly. The semantics in the paper were carefully designed to solve the problem laid out in the first sentence of the abstract: Asynchronous exceptions, such as timeouts, are important for robust, modular programs, but are extremely difficult to program with -- so much so that most programming languages either heavily restrict them or ban them altogether. And I believe the paper succeeded. The paper shows how to replace other languages pervasive and intrusive error catching and handling code with much cleaner, clearer, and often more correct code. The implementation in GHC has changed the behavior of throwTo from asynchronous (not-interruptible) to synchronous (interruptible?) as discussed in section 8 of the paper. This change, in and of itself, is not the fatal problem; as described in the paper a (forkIO (throwTo ...)) recovers the asynchronous behavior. The fatal change between the paper and GHC comes from not following section 7.2 as published. Section 7.2 Implementation of throwTo has two bullet point, and the second bullet point is (retyped, so typos are my own fault): As soon as a thread exits the scope of a 'block', and at regular intervals during execution inside 'unblock', it should check its queue of pending exceptions. If the queue is non-empty, the first exception from the queue should be raised. A test of GHC 6.6 shows that this is not the case. Test program A: loop = block (print alive) loop main = do tid - forkIO loop threadDelay 1 killThread tid Program A, compiled with (-threaded) on a single CPU machine never halts. It will print alive forever while the the main thread is blocked on killThread. This is wh As an aside, removing the threadDelay causes killThread to destroy the child before it can enter the block, thus showing the need to add forkBlockedIO or forkInheritIO to the library. This can be worked around using an MVar. Changing the definition of loop produces Test program B: loop = block (print alive) yield loop main = do tid - forkIO loop threadDelay 1 killThread tid This prints alive twice before the killThread succeeds. The paper demands that when the loop in Program A exits the scope of block (print a) that it check a queue of pending exceptions, see that it is non-empty, and raise the exception thrown by killThread. This can also be seen in Figure 5. Transition Rules for Asynchronous Exceptions, where killThread should use throwTo to create an in-flight exception and exiting the scope of block in the presence of this in-flight exception should raise the exception. The implementation in GHC sleeps the main thread at the killThread command, and it is awoken when the block is exited and to succeed in delivering the exception it must execute while the child is still in the unblocked state. But the child re-enters a blocked state too quickly in Program A, so killThread never succeeds. The change in Program B has the child yield when unblocked and this gives the main thread a change to succeed. This trick using yield to make a safepopint was suggested by Simon Marlow: The window in 'unblock (return ())' is tiny, I'm not really surprised if nothing ever gets through it. You might have more luck with 'unblock yield'. It has been said on this mailing list thread that needing yield to program concurrently is a bug in the user's code and should be replaced by other
Re: throwTo block statements considered harmful
The key problem is, at least in the presence of block/unblock, that Exceptions are never reliably delivered. Never? Even in a function which is in a blocking state? The implementation of asynchronous signals, as described by the paper Asynchronous exceptions in Haskell Simon Marlow, Simon Peyton Jones, Andy Moran and John Reppy, PLDI'01. is fatally inconsistent with the implementation in GHC 6.4 and GHC 6.6 today. Is it a goal of the GHC developers to offer an implementation of asynchronous signals which has the features and benefits described in the original paper? If the answer is no, then there are a couple points... A: That the current implementation works differently than the original paper is important to know, and the library documentation should be updated to clearly describe what the implementation does and does not do. B: Since you are programming in a language which doesn't offer the semantics of the original paper, and you can implement your algorithm using an event queue... you can go ahead an implement your algorithm with an event queue. The situation doesn't rise to the level of fatal :-) until you have an algorithm which you're not able to implement with the facilities provided by GHC. For example, if the implementation did not reliably deliver an asynchronous exception when a function was blocking, then that probably would be a *fatal* flaw, because then there'd be no way to break out of a blocking function. Otherwise, we're talking about convenience. You'd like asynchronous signals in GHC to offer the features and benefits described in the original paper, and that's a reasonable request to ask for, but it doesn't rise to the level of a fatal flaw. ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re[2]: throwTo block statements considered harmful
Hello Cat, Friday, December 8, 2006, 5:00:46 PM, you wrote: The implementation of asynchronous signals, as described by the paper Asynchronous exceptions in Haskell Simon Marlow, Simon Peyton Jones, Andy Moran and John Reppy, PLDI'01. is fatally inconsistent with the implementation in GHC 6.4 and GHC 6.6 today. Is it a goal of the GHC developers to offer an implementation of asynchronous signals which has the features and benefits described in the original paper? this paper is widely used to learn this GHC corner. so if it don't corresponds to reality, it will be great to update paper at least :) -- Best regards, Bulatmailto:[EMAIL PROTECTED] ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
RE: Unlifted type variables in GHC
| I want to make ByteArray# and MutableByteArray# parameterized over | their element types. ByteArray# would have kind # - #, and | MutableByteArray#, * - # - # . indexByteArray# would have the type | (in pseudo haskell) forall (e::#). ByteArray# e - Int# - e. Probably a bad idea. The point is that you could then write polymorphic functions that would apparently work both on values of type Int# and Double#... but they are of different sizes. Seg-fault city! This stuff is not impossible. Microsoft's CLR allows exactly this, and compiles fresh code for each instantiation of a function at a type of different size or pointer-hood than the ones already compiled. But GHC does not do that, and won't anytime soon. The paper I wrote with John Launchbury Unboxed types as first class citizens elaborates, I think. Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
RE: [Haskell] GHC Error question
| And why isn't C a b equivalent to C a b1? |forall a b . C a b = a - a | and |forall a b1 . C a b1 = a - a | look alpha convertible to me. You may say it's just common sense: a) I have a dictionary of type (C a b) provided by the caller b) I need a dictionary of type (C a b1) , where b1 is an as-yet-unknown type (a unification variable in the type inference system) c) There are no other constraints on b1 So, in view of (c), perhaps we can simply choose b1 to be b, and we're done! Sounds attractive. But consider this: f :: (Read a, Show a) = String - String f x = show (read x) From this I get a) I have a dictionary of type (Show a) provided by the caller b) I need a dictionary of type (Show a1), arising from the call to show c) There are no other constraints on a1 So perhaps I should choose a1 to be a, thereby resolving the ambiguity! Well, perhaps. Or I could choose a1 to be Int, or Bool. (A similar ambiguity exists in Norman's example, if there were instances for (C a Int) or (C a Bool).) There may be a heuristic that would help more programs to go through... but I prefer asking the programmer to make the desired behaviour explicit. Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
RE: bang patterns give fundamentally new capabilities?
| | Also, is there a way to do something similar but for 'lazy' rather than
| | 'seq'? I want something of type
| |
| | type World__ = State# RealWorld
| |
| | {-# NOINLINE newWorld__ #-}
| | newWorld__ :: a - World__
| | newWorld__ x = realWord# -- ???
| |
| | except that I need newWorld__ to be lazy in its first argument. I need
| | to convince the opimizer that the World__ newWorld__ is returning
| | depends on the argument passed to newWorld__.
|
| I don't understand what you meant here. The definition of newWorld__ that
you give is, of course,
| lazy in x.
|
| it is getting type 'Absent' assigned to it by the demand analysis, I
| want it to be lazy (and not strict)
Ah I think I understand now. You want a lazy primop
discard# :: a - ()
Now you can write
newWorld x = discard x `seq` realWorld#
The code generator treats (discard# x) as (), and
(case (discard# x) of () - e) as e.
It should be a primop so that this behaviour is not exposed too early. An
alternative would be to do the transformation in the core-to-STG step, but that
might be too early. Still easier would be to write
discard x = ()
{-# NOINLINE[0] discard #-}
to prevent it getting inlined until the final stages of the optmisier. The
trouble is that I have no idea of what it means to expose discard too early
is in your case.
Not hard to implement if you feel like doing so.
Simon
___
Glasgow-haskell-users mailing list
Glasgow-haskell-users@haskell.org
http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Unlifted type variables in GHC
On Fri, Dec 08, 2006 at 03:48:22PM +, Simon Peyton-Jones wrote: | I want to make ByteArray# and MutableByteArray# parameterized over | their element types. ByteArray# would have kind # - #, and | MutableByteArray#, * - # - # . indexByteArray# would have the type | (in pseudo haskell) forall (e::#). ByteArray# e - Int# - e. Probably a bad idea. The point is that you could then write polymorphic functions that would apparently work both on values of type Int# and Double#... but they are of different sizes. Seg-fault city! Right... which is why you'd need to make sure you never used those functions in a polymorphic way. The code generator should choke if it sees an unlifted type variable. I'm basically looking for a way to avoid enumerating every possible unlifted type in the ByteArray# operations. I'm probably not giving enough details for this to make any sense. I'm actually writing a new ghc backend and I have way more primitive types than the normal dozen or so (actually an infinite number since you can add new ones, I resurrected that foreign type code that was leftover from the .NET backend). I'll just try it and see how I make out. -Brian ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: throwTo block statements considered harmful
Program A and B got word wrapped by mistake...damn it. Program A loop = block (print alive) loop main = do tid - forkIO loop threadDelay 1 killThread tid the above print alive forever while killThread stays blocked. Program B loop = block (print alive) loop yield main = do tid - forkIO loop threadDelay 1 killThread tid the above prints alive about twice before killThread succeeds. ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: [Haskell] GHC Error question
There may be a heuristic that would help more programs to go through... but I prefer asking the programmer to make the desired behaviour explicit. Simon How can the user make this explicit? With the class C a b where op :: a - a instance C Int Int where op a = -a test d = op d example, I have been unable to figure out what type annotations I need to add to 'test', for exampl,e to define test to be for the C Int Int instance. Or is it simply impossible in GHC to give test a type signature and to use it as a useful function? Example ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: [Haskell] GHC Error question
Rene de Visser wrote: There may be a heuristic that would help more programs to go through... but I prefer asking the programmer to make the desired behaviour explicit. Simon How can the user make this explicit? With the class C a b where op :: a - a instance C Int Int where op a = -a test d = op d example, I have been unable to figure out what type annotations I need to add to 'test', for exampl,e to define test to be for the C Int Int instance. Or is it simply impossible in GHC to give test a type signature and to use it as a useful function? Example I can't quite do that but I have a trick: In GHC 6.6 with -fglasgow-exts this works: module Foo where data CInst a b class C a b where cInst :: CInst a b cInst = undefined op :: a - a op' :: CInst a b - a - a instance C Int Int where op a = -a op' _ a = (-a) intInst :: CInst Int Int intInst = cInst test x = (op' intInst) x And test is inferred to have type: *Foo :i test test :: Int - Int-- Defined at /tmp/test.hs:18:0 Essentially the trick used in this case is to reify the instance (C Int Int) as the value (undefined :: CInst Int Int). Then you can specify to op -- Chris ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: ghc-6.6-src-extralibs.tar.bz2
On Fri, Dec 08, 2006 at 08:55:28AM +0100, Sven Panne wrote: Am Donnerstag, 7. Dezember 2006 11:37 schrieb Christian Maeder: The archive http://www.haskell.org/ghc/dist/6.6/ghc-6.6-src-extralibs.tar.bz2 does not contain the files ControlPoint.hs and Domain.hs from directory libraries/OpenGL/Graphics/Rendering/OpenGL/GL/ If I see things correctly, the 6.6 extralibs contain the version 2.1 of the OpenGL package, i.e. the stuff currently in http://hackage.haskell.org/packages/unstable/OpenGL/, at least I hope so. :-/ These files are listed by the binary distribution http://www.haskell.org/ghc/dist/6.6/ghc-6.6-i386-unknown-linux.tar.bz2 This will probably have been made with whatever OpenGL was in darcs when the build was done (the binary distributions come from the nightly builds). The extralibs are not part of the GHC release, they are just sometimes bundled to make users' lives easier, so the GHC release is not tied to any particular version of the extralibs. To be honest, I don't fully understand the workflow for building the official GHC distributions currently. In former times, the whole tree, including libraries, had a CVS tag, so things were crystal-clear. GHC and the core libraries all have a 6.6 release tag. One problem we do have is that if you get a library (core, extralibs or otherwise) from darcs on two different days then you might get two different libraries with the same version number. We should possibly do something like having only odd second components (e.g. version 2.3 but not version 2.4) in darcs repos so we can at least spot these unstable version numbers. Then to do a release you'd push the three patches Version=2.4; tag 2.4; Version=2.5 all at once. Thanks Ian ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
