Re: thread blocked indefinitely in an MVar operation in unsafePerformIO
Hi Simon. On Mon, Jul 30, 2012 at 12:00 PM, Simon Marlow marlo...@gmail.com wrote: (...) Is it possible that the String you are passing to uLog contains unevaluated calls to uLog itself, which would thereby create a deadlock as uLog tries to take the MVar that is already being held by the same thread? Yes, that's right. I removed the recursive call to uLog and it worked, thanks. Greetings. (...) -- marcot http://marcot.eti.br/ ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
thread blocked indefinitely in an MVar operation in unsafePerformIO
Hi. I'm having a problem calling logM from hsLogger inside unsafePerformIO. I have described the problem in Haskell-cafe, so I'll avoid repeating it here: http://www.haskell.org/pipermail/haskell-cafe/2012-July/102545.html I've had this problem both with GHC 7.4.1 and 7.4.2. Do you have any suggestion? Greetings. -- marcot http://marcot.eti.br/ ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: thread blocked indefinitely in an MVar operation in unsafePerformIO
On 30/07/2012 15:30, Marco Túlio Gontijo e Silva wrote: Hi. I'm having a problem calling logM from hsLogger inside unsafePerformIO. I have described the problem in Haskell-cafe, so I'll avoid repeating it here: http://www.haskell.org/pipermail/haskell-cafe/2012-July/102545.html I've had this problem both with GHC 7.4.1 and 7.4.2. Do you have any suggestion? Is it possible that the String you are passing to uLog contains unevaluated calls to uLog itself, which would thereby create a deadlock as uLog tries to take the MVar that is already being held by the same thread? We once had this problem with hPutStr where if the argument string contained a call to trace, which is unsafePerformIO $ hPutStr, the result would be deadlock. Now hPutStr has to go to some trouble to evaluate the input string while not holding the lock. Cheers, Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Strange performance effects with unsafePerformIO
On 07/04/2011 13:51, Björn Peemöller wrote: Simon Marlow schrieb: Incidentally this will be faster with GHC 7.2, because we implemented chunked stacks, so unsafePerformIO never has to traverse more than 32k of stack (you can tweak the chunk size with an RTS option). This is still quite a lot of overhead, but at least it is bounded. The example above runs in 1.45s for me with current HEAD, and I gave up waiting with 7.0. Thank you all for your explanations, the blackholing indeed seems to be the cause for the slowdown. Is there any documentation available about the blackholing process? Unfortunately no. But the main point is that unsafePerformIO needs to traverse the stack down to the most recent thunk evaluation, so the bad case happens when you're in a non-tail-recursive loop with no intervening thunk evaluations. Maybe we can find a hint on how to change our code to avoid the problem. If you don't mind your unsafePerformIO being performed multiple times when running in parallel, then you can use unsafeDupablePerformIO to avoid the overhead. Apart from that, there's really no way around it. Cheers, Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Strange performance effects with unsafePerformIO
Simon Marlow schrieb: Incidentally this will be faster with GHC 7.2, because we implemented chunked stacks, so unsafePerformIO never has to traverse more than 32k of stack (you can tweak the chunk size with an RTS option). This is still quite a lot of overhead, but at least it is bounded. The example above runs in 1.45s for me with current HEAD, and I gave up waiting with 7.0. Thank you all for your explanations, the blackholing indeed seems to be the cause for the slowdown. Is there any documentation available about the blackholing process? Maybe we can find a hint on how to change our code to avoid the problem. Regards, Bjoern ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Strange performance effects with unsafePerformIO
2011/3/25 Thomas Schilling nomin...@googlemail.com: unsafePerformIO traverses the stack to perform blackholing. It could be that your code uses a deep stack and unsafePerformIO is repeatedly traversing it. Just a guess, though. Sounds reasonable. Here is a variant of the program without intermediate lists. import System.IO.Unsafe main = run (10^5) run 0 = return () run n = (unsafePerformIO . return) (run (n - 1)) return () I think it does not do much more than producing a large stack and (like the original program) is much faster if the unsafe-return combination or the final return (which probably prohibits tail-call optimization) is removed. Sebastian 2011/3/24 Björn Peemöller b...@informatik.uni-kiel.de: Hello, we have a strange performance behaviour when we use unsafePerformIO, at least with GHC 6.12.3 and 7.0.1. Please consider the example program following at the end of this post. Running the original code the execution time is about 26 seconds, while uncommenting one (or both) of the comments shrinks it to about 0.01 seconds on our machine. Is there an explanation for this effect? Regards, Bjoern -- --- module Main where import System.IO.Unsafe traverse [] = return () -- traverse (_:xs) = traverse xs traverse (_:xs) = traverse xs return () makeList 0 = [] -- makeList n = () : (makeList (n - 1)) makeList n = () : (unsafePerformIO . return) (makeList (n - 1)) main = traverse $ makeList (10^5) ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users -- Push the envelope. Watch it bend. ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Strange performance effects with unsafePerformIO
On 25/03/2011 08:56, Sebastian Fischer wrote: 2011/3/25 Thomas Schillingnomin...@googlemail.com: unsafePerformIO traverses the stack to perform blackholing. It could be that your code uses a deep stack and unsafePerformIO is repeatedly traversing it. Just a guess, though. Sounds reasonable. Here is a variant of the program without intermediate lists. import System.IO.Unsafe main = run (10^5) run 0 = return () run n = (unsafePerformIO . return) (run (n - 1)) return () I think it does not do much more than producing a large stack and (like the original program) is much faster if the unsafe-return combination or the final return (which probably prohibits tail-call optimization) is removed. Incidentally this will be faster with GHC 7.2, because we implemented chunked stacks, so unsafePerformIO never has to traverse more than 32k of stack (you can tweak the chunk size with an RTS option). This is still quite a lot of overhead, but at least it is bounded. The example above runs in 1.45s for me with current HEAD, and I gave up waiting with 7.0. Cheers, Simon Sebastian 2011/3/24 Björn Peemöllerb...@informatik.uni-kiel.de: Hello, we have a strange performance behaviour when we use unsafePerformIO, at least with GHC 6.12.3 and 7.0.1. Please consider the example program following at the end of this post. Running the original code the execution time is about 26 seconds, while uncommenting one (or both) of the comments shrinks it to about 0.01 seconds on our machine. Is there an explanation for this effect? Regards, Bjoern -- --- module Main where import System.IO.Unsafe traverse [] = return () -- traverse (_:xs) = traverse xs traverse (_:xs) = traverse xs return () makeList 0 = [] -- makeList n = () : (makeList (n - 1)) makeList n = () : (unsafePerformIO . return) (makeList (n - 1)) main = traverse $ makeList (10^5) ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users -- Push the envelope. Watch it bend. ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Strange performance effects with unsafePerformIO
Hello, we have a strange performance behaviour when we use unsafePerformIO, at least with GHC 6.12.3 and 7.0.1. Please consider the example program following at the end of this post. Running the original code the execution time is about 26 seconds, while uncommenting one (or both) of the comments shrinks it to about 0.01 seconds on our machine. Is there an explanation for this effect? Regards, Bjoern -- --- module Main where import System.IO.Unsafe traverse [] = return () -- traverse (_:xs) = traverse xs traverse (_:xs) = traverse xs return () makeList 0 = [] -- makeList n = () : (makeList (n - 1)) makeList n = () : (unsafePerformIO . return) (makeList (n - 1)) main = traverse $ makeList (10^5) ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Strange performance effects with unsafePerformIO
unsafePerformIO traverses the stack to perform blackholing. It could be that your code uses a deep stack and unsafePerformIO is repeatedly traversing it. Just a guess, though. 2011/3/24 Björn Peemöller b...@informatik.uni-kiel.de: Hello, we have a strange performance behaviour when we use unsafePerformIO, at least with GHC 6.12.3 and 7.0.1. Please consider the example program following at the end of this post. Running the original code the execution time is about 26 seconds, while uncommenting one (or both) of the comments shrinks it to about 0.01 seconds on our machine. Is there an explanation for this effect? Regards, Bjoern -- --- module Main where import System.IO.Unsafe traverse [] = return () -- traverse (_:xs) = traverse xs traverse (_:xs) = traverse xs return () makeList 0 = [] -- makeList n = () : (makeList (n - 1)) makeList n = () : (unsafePerformIO . return) (makeList (n - 1)) main = traverse $ makeList (10^5) ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users -- Push the envelope. Watch it bend. ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: [Haskell-cafe] Space leak with unsafePerformIO
On 30/06/2010 18:13, Yitzchak Gale wrote: Henning Thielemann wrote on Haskell Cafe: Attached is a program with a space leak... I have coded a simple 'map' function, once using unsafePerformIO and once without. UnsafePerformIO has a space leak in some circumstances. In the main program I demonstrate cases with and without space leak. Without space leak the program writes a file to the disk until it is full. Bertram Felgenhauer wrote: The program relies on the GC doing short-cut evaluation of record selectors to avoid a space leak... Use of suffix: Due to the call of Main.go1 this is *not* a record selector. It is compiled to an actual case expression, which to the garbage collector looks just like an ordinary thunk. A reference to Main.ds is kept around until the suffix is about to be processed and a memory leak ensues. The good news is that the problem is completely unrelated to unsafePerformIO (the presence of unsafePerformIO makes optimisations more difficult, but any pure function of sufficient complexity would have the same effect). Is there already a GHC Trac bug for this? This is it: http://hackage.haskell.org/trac/ghc/ticket/2607 Link to Bertram's original email, with detailed analysis of the GHC Core: http://www.haskell.org/pipermail/haskell-cafe/2010-June/079479.html Added to the ticket, thanks. Cheers, Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: [Haskell-cafe] Space leak with unsafePerformIO
Henning Thielemann wrote on Haskell Cafe: Attached is a program with a space leak... I have coded a simple 'map' function, once using unsafePerformIO and once without. UnsafePerformIO has a space leak in some circumstances. In the main program I demonstrate cases with and without space leak. Without space leak the program writes a file to the disk until it is full. Bertram Felgenhauer wrote: The program relies on the GC doing short-cut evaluation of record selectors to avoid a space leak... Use of suffix: Due to the call of Main.go1 this is *not* a record selector. It is compiled to an actual case expression, which to the garbage collector looks just like an ordinary thunk. A reference to Main.ds is kept around until the suffix is about to be processed and a memory leak ensues. The good news is that the problem is completely unrelated to unsafePerformIO (the presence of unsafePerformIO makes optimisations more difficult, but any pure function of sufficient complexity would have the same effect). Is there already a GHC Trac bug for this? Link to Bertram's original email, with detailed analysis of the GHC Core: http://www.haskell.org/pipermail/haskell-cafe/2010-June/079479.html Thanks, Yitz ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: What does unsafePerformIO do to the stack
Simon Marlow wrote: Perhaps there are some trivial examples of unshared thunks that we could spot, though. The sharing analysis in the interpreter is also very simple and inexpensive. But the gain is frequent. Maybe giving it a try would be worthwhile. Thanks again for all your answers! Bernd ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: What does unsafePerformIO do to the stack
Bernd Brassel wrote: There is another point that makes me wonder now. If the update frame for the recursive call is the problem then my solution with foreign C functions would produce a bad stack also. But this is not the case. The code looks now like this: sim [] = True sim (_:xs) = yags (sim xs) ref = cinitialize yags x = replace (C_Ref ref) x () And it is running within 0.15M of stack. Did your explanation also account for this phenomenon? Sorry to take that much time off you with this! Bernd The stack of update frames is not in itself a problem, because stack squeezing removes them leaving you with O(1) stack again. The problem with unsafePerformIO was that the duplication-protection was interfering with stack squeezing. Regarding sharing analysis, we did used to have an update analyser in GHC, but it was expensive to run and rarely gave worthwhile benefits, so eventually we dropped it. Perhaps there are some trivial examples of unshared thunks that we could spot, though. Cheers, Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: What does unsafePerformIO do to the stack
I have got around the problem by defining my unsafe actions by the foreign function interface. But I still think there is a bug concerning stack use in unsafePerformIO in ghc. And I also think that this bug potentially concerns every use of unsafe. Or did I just not get the point of your argument and my answer to it was still beside the point? Greetings Bernd ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: What does unsafePerformIO do to the stack
Bernd Brassel wrote: Thanks for your answer Simon. Simon Marlow wrote: Bernd Brassel wrote: Consider the following program: module Stack where import System.IO.Unsafe main = print (sim (replicate 1299959 ())) sim [] = True sim (_:xs) = goodStack (sim xs) goodStack x = fromJust (Just x) --no stack overflow badStack x = unsafePerformIO (return x) --stack overflow fromJust (Just x) = x goodStack == id, and GHC even with no optimisation will transform it into id, and inline it into sim. So with goodStack, sim ends up being tail-recursive. With badStack, sim is no longer tail recursive (unsafePerformIO is not inlined), so it runs out of stack. Simple! Is it really that simple? I guess that in a lazy language we have to look a bit closer to see what is tail recursive and what is not. If I understand you correctly, you say that if goodStack was not inlined you would have a bad stack as well, right? My apologies - I oversimplified things. You're quite right, it looks like the unsafePerformIO version ought to be tail-recursive too. It turns out to be rather subtle. If you replace unsafePerformIO by unsafeDupablePerformIO (from GHC.IOBase), then you do indeed get tail-recursion. But this is only due to a clever trick in the RTS: what happens is that (sim xs) is a thunk, so when evaluating it, even in a tail-recursive position, an update frame is pushed on the stack. The next recursive call to (sim xs) pushes another update frame on the stack, and so on. Since these update frames are all adjacent to one another, a trick known as stack squeezing can squash them down into a single frame, and this is what happens for unsafeDupablePerformIO. The ordinary unsafePerformIO is performing stack squeezing once per call, because the stack squeezing is done by the same code that does the duplicate computation check that unsafePerformIO needs to do. Stack squeezing doesn't look at the same bit of stack twice, so subsequent squeezings don't manage to remove any update frames. I've fixed this in my tree. It also needed some tweaks to the heuristic which decides whether to squeeze or not based on a cost/benefit tradeoff. So the upshot is: you can use unsafeDupablePerformIO right now, or you can wait until I've tested and committed this patch to get tail-recursion with unsafePerformIO. I've no idea how it works in Hugs, you'll have to ask them :) Cheers, Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: What does unsafePerformIO do to the stack
Simon Marlow wrote: So the upshot is: you can use unsafeDupablePerformIO right now, or you can wait until I've tested and committed this patch to get tail-recursion with unsafePerformIO. Wow, thank you for the detailed answer! That was really interesting. I've no idea how it works in Hugs, you'll have to ask them :) At least I can tell you how it is done in that other interpreter I mentioned. The reason that you can be better for that example is that the argument of badStack is not shared. No need to push an update frame for it. I am pretty sure that it is the same with the other compiler, both for Curry not for Haskell btw., but like you I am not sure with Hugs. If the ghc does not take into account what is statically not shared, I guess there is potential for a good optimization here which would go well beyond unsafe programs. Greetings and thanks! Bernd ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: What does unsafePerformIO do to the stack
There is another point that makes me wonder now. If the update frame for the recursive call is the problem then my solution with foreign C functions would produce a bad stack also. But this is not the case. The code looks now like this: sim [] = True sim (_:xs) = yags (sim xs) ref = cinitialize yags x = replace (C_Ref ref) x () And it is running within 0.15M of stack. Did your explanation also account for this phenomenon? Sorry to take that much time off you with this! Bernd ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: What does unsafePerformIO do to the stack
Bernd Brassel wrote: Consider the following program: module Stack where import System.IO.Unsafe main = print (sim (replicate 1299959 ())) sim [] = True sim (_:xs) = goodStack (sim xs) goodStack x = fromJust (Just x) --no stack overflow badStack x = unsafePerformIO (return x) --stack overflow fromJust (Just x) = x goodStack == id, and GHC even with no optimisation will transform it into id, and inline it into sim. So with goodStack, sim ends up being tail-recursive. With badStack, sim is no longer tail recursive (unsafePerformIO is not inlined), so it runs out of stack. Simple! Is this behaviour necessary? Is there any work around, e.g., employing the foreign function interface? There's unsafeInlinePerformIO (sometimes called inlinePerformIO), which is usable in certain cases, but be very careful. From Data.ByteString.Internal: {-# INLINE inlinePerformIO #-} inlinePerformIO :: IO a - a #if defined(__GLASGOW_HASKELL__) inlinePerformIO (IO m) = case m realWorld# of (# _, r #) - r #else inlinePerformIO = unsafePerformIO #endif But even this might not give you tail recursion, depending on the context. Cheers, Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: What does unsafePerformIO do to the stack
Thanks for your answer Simon. Simon Marlow wrote: Bernd Brassel wrote: Consider the following program: module Stack where import System.IO.Unsafe main = print (sim (replicate 1299959 ())) sim [] = True sim (_:xs) = goodStack (sim xs) goodStack x = fromJust (Just x) --no stack overflow badStack x = unsafePerformIO (return x) --stack overflow fromJust (Just x) = x goodStack == id, and GHC even with no optimisation will transform it into id, and inline it into sim. So with goodStack, sim ends up being tail-recursive. With badStack, sim is no longer tail recursive (unsafePerformIO is not inlined), so it runs out of stack. Simple! Is it really that simple? I guess that in a lazy language we have to look a bit closer to see what is tail recursive and what is not. If I understand you correctly, you say that if goodStack was not inlined you would have a bad stack as well, right? But look at what it would be doing. In a lazy language the call to sim would go to the heap and whatever goodStack does to the stack is already done before sim is restarted. And the same could be true with the unsafePerformIO-return combination. What is the reason to hold anything on the stack for this after the call to unsafe is finished? I have tried the example with badStack in one other compiler and two interpreters. None of them has any problem running the example. For one of the interpreters I could exactly measure that the stack is constant the whole time. And I know that no optimisation or inlining is going on for that interpreter. Just try the example with hugs. It easily runs through while replacing badStack with the function pushStack True = True immediately runs out of memory. (With this function, the example is indeed not tail recursive and your argument is valid.) So there is definitely something that unsafePerformIO does to the stack in the ghc that is special to that compiler. ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
What does unsafePerformIO do to the stack
Consider the following program: module Stack where import System.IO.Unsafe main = print (sim (replicate 1299959 ())) sim [] = True sim (_:xs) = goodStack (sim xs) goodStack x = fromJust (Just x) --no stack overflow badStack x = unsafePerformIO (return x) --stack overflow fromJust (Just x) = x I always thought that unsafePerformIO would do something similar as goodStack. i.e. be stack neutral: $ ghc --make -o stack Stack [1 of 1] Compiling Main ( Stack.hs, Stack.o ) Linking stack ... $ stack +RTS -K0.1M True But if you exchange goodStack with badStack, the picture changes unfortunately to: $ ghc --make -o stack Stack [1 of 1] Compiling Main ( Stack.hs, Stack.o ) Linking stack ... $ stack +RTS -K9.883647M Stack space overflow: current size 9883644 bytes. Use `+RTS -Ksize' to increase it. $ stack +RTS -K9.883648M True I am using: $ ghc --version The Glorious Glasgow Haskell Compilation System, version 6.8.2 Is this behaviour necessary? Is there any work around, e.g., employing the foreign function interface? Thanks for your time! Bernd ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Prevent optimization from tempering with unsafePerformIO
Hello to all who have given me hints an suggestions about this topic whether directly by mail or over the mailing list. I do now have a much more concrete idea of how to optimize my programs. Thanks for your time! Bernd ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Prevent optimization from tempering with unsafePerformIO
Hi Stefan! Thanks for your answer. Stefan O'Rear wrote: Might I suggest, that the problem is your use of unsafePerformIO? Yes you might. And you are surely right. But not using it in this way is not an alternative. why do you want to do this unsafely, instead of just using 'length'? unsafePerformIO is a very slow function, remember) The big picture is that we generate programs like this example in order to compile the functional logic language Curry down to Haskell. Basically, there are two ways to do this: a) write an interpreter for Curry in Haskell, e.g., by employing non-determinism monads b) extend Haskell by employing side effects Alternative a) is not really an issue for us. Doing it this way, all Curry programs will suffer in performance in such a magnitude that - in comparison - unsafePerformIO is super-fast. In addition, we already have interpreters for Curry which I do not assume a Haskell interpreter to outperform without 5 years worth of tuning. Alternative b) is the choice, because doing it this way, all deterministic, i.e., purely functional parts of Curry programs would take advantage of Haskell being a functional language. If then the logic parts are not so fast, e.g., because unsafePerformIO is slow, this does not matter so much. In comparison to other approaches (like Alternative a) and many other implementations of Curry) our slogan is: make functions fast and logic possible. Fast functions will outweigh the slowdown for logics. But to get functions fast employing optimization sounds like a good idea to me. But even without any optimization, our system can compare well to most other implementations for many applications. Thanks for your input! Bernd ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Prevent optimization from tempering with unsafePerformIO
Hi David, thank you! This is really useful information! I think it's the let floating (out) together with common subexpression elimination: ghc --make -O2 -no-recomp -fno-cse -o curry-no-cse curry.hs [1 of 1] Compiling Main ( curry.hs, curry.o ) Linking curry-no-cse ... ghc --make -O2 -no-recomp -fno-full-laziness -o curry-no-fll curry.hs [1 of 1] Compiling Main ( curry.hs, curry.o ) Linking curry-no-fll ... ghc --make -O2 -no-recomp -fno-full-laziness -fno-cse -o curry-no-cse-no-fll curry.hs [1 of 1] Compiling Main ( curry.hs, curry.o ) Linking curry-no-cse-no-fll ... ./curry-no-cse 3 possibilities: [True,False] 2 possibilities: [True,False] ./curry-no-fll 3 possibilities: [True,False] 2 possibilities: [True,False] ./curry-no-cse-no-fll 3 possibilities: [True,True,False] 2 possibilities: [True,False] I will try this on large scale Curry programs. I hope the remaining optimizations will still do some good. Do you think that there is a way to be more selective? I mean to select those parts of the program which can and which cannot be optimized? ps.: Maybe it is interesting to look at HasFuse [1] (somewhat outdated), but it exactly forbids both transformations [1] http://www.ki.informatik.uni-frankfurt.de/research/diamond/hasfuse/ Yes, that looks interesting, too. Are there plans to update it with the ghc? Thanks for your hints! Bernd ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Prevent optimization from tempering with unsafePerformIO
On 10/17/07, Bernd Brassel [EMAIL PROTECTED] wrote: why do you want to do this unsafely, instead of just using 'length'? unsafePerformIO is a very slow function, remember) The big picture is that we generate programs like this example in order to compile the functional logic language Curry down to Haskell. Basically, there are two ways to do this: a) write an interpreter for Curry in Haskell, e.g., by employing non-determinism monads b) extend Haskell by employing side effects Alternative a) is not really an issue for us. Doing it this way, all Curry programs will suffer in performance in such a magnitude that - in comparison - unsafePerformIO is super-fast. In addition, we already have interpreters for Curry which I do not assume a Haskell interpreter to outperform without 5 years worth of tuning. Alternative b) is the choice, because doing it this way, all deterministic, i.e., purely functional parts of Curry programs would take advantage of Haskell being a functional language. If then the logic parts are not so fast, e.g., because unsafePerformIO is slow, this does not matter so much. In comparison to other approaches (like Alternative a) and many other implementations of Curry) our slogan is: make functions fast and logic possible. Fast functions will outweigh the slowdown for logics. But to get functions fast employing optimization sounds like a good idea to me. But even without any optimization, our system can compare well to most other implementations for many applications. May I suggest a third route that has the advantages of both your approaches. The backside is of course that it takes a bit of work. My suggestion is to do an effect analysis of your curry programs to identify the purely functional parts and compile them directly to pure Haskell. The rest of the programs can run in a monad. This approach should be more robust than relying on unsafePerformIO. It is also very much in the spirit of your slogan. Just my 2 cents, /Josef ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Prevent optimization from tempering with unsafePerformIO
Hi Bernd, Bernd Brassel wrote: Hi David, thank you! This is really useful information! I think it's the let floating (out) together with common subexpression elimination: ghc --make -O2 -no-recomp -fno-cse -o curry-no-cse curry.hs [1 of 1] Compiling Main ( curry.hs, curry.o ) Linking curry-no-cse ... ghc --make -O2 -no-recomp -fno-full-laziness -o curry-no-fll curry.hs [1 of 1] Compiling Main ( curry.hs, curry.o ) Linking curry-no-fll ... ghc --make -O2 -no-recomp -fno-full-laziness -fno-cse -o curry-no-cse-no-fll curry.hs [1 of 1] Compiling Main ( curry.hs, curry.o ) Linking curry-no-cse-no-fll ... ./curry-no-cse 3 possibilities: [True,False] 2 possibilities: [True,False] ./curry-no-fll 3 possibilities: [True,False] 2 possibilities: [True,False] ./curry-no-cse-no-fll 3 possibilities: [True,True,False] 2 possibilities: [True,False] I will try this on large scale Curry programs. I hope the remaining optimizations will still do some good. Do you think that there is a way to be more selective? I mean to select those parts of the program which can and which cannot be optimized? I think this would require an analysis of the code during compilation (to split into pure parts and impure parts), but we did not investigate this. ps.: Maybe it is interesting to look at HasFuse [1] (somewhat outdated), but it exactly forbids both transformations [1] http://www.ki.informatik.uni-frankfurt.de/research/diamond/hasfuse/ Yes, that looks interesting, too. Are there plans to update it with the ghc? Honestly, we did nothing since 2004 on the code, but maybe we could port the changes to the current source of ghc. I remember that in an early version of ghc (I think 5) there was an option -O file which gave the user control over the optimizations with a script, this would probably a nice feature to have again... Regards, David Thanks for your hints! Bernd ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
RE: Prevent optimization from tempering with unsafePerformIO
I think you'll find it useful to be explicit about what you are relying on. For example I think that if your program contains two calls unsafePerformIO e1 unsafePerformIO e2 then * You don't care about the *order* in which these are performed * You do care that they are both performed separately, even if e1=e2. That is, you do not want CSE (common sub-expression) * I'm not sure whether you care if they are not performed at all. E.g. if you have let x = unsafePerformIO e in True then GHC will discard the binding for x. You can control this by adding data dependencies, so that the body of the 'let' depends on 'x'. * I'm not sure if you care whether they are performed once or many times. E.g. f = \x. x + unsafePerformIO (h z) If the (h z) doesn't mention 'x', GHC will transform this to v = unsafePerformIO (h z) f = \x. x+v So now the effect happens once only, rather than once per call of f. Again you can control this by adding an articifical dependency on x. Adding these artificial dependencies may amount to exactly the kind of effect analysis that Joesef suggests. Simon | -Original Message- | From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On | Behalf Of Bernd Brassel | Sent: 17 October 2007 09:02 | To: glasgow-haskell-users@haskell.org | Subject: Prevent optimization from tempering with unsafePerformIO | | Hi David, | | thank you! This is really useful information! | | I think it's the let floating (out) together with common subexpression | elimination: | |ghc --make -O2 -no-recomp -fno-cse -o curry-no-cse curry.hs | [1 of 1] Compiling Main ( curry.hs, curry.o ) | Linking curry-no-cse ... |ghc --make -O2 -no-recomp -fno-full-laziness -o curry-no-fll curry.hs | [1 of 1] Compiling Main ( curry.hs, curry.o ) | Linking curry-no-fll ... |ghc --make -O2 -no-recomp -fno-full-laziness -fno-cse -o | curry-no-cse-no-fll curry.hs | [1 of 1] Compiling Main ( curry.hs, curry.o ) | Linking curry-no-cse-no-fll ... |./curry-no-cse | 3 possibilities: [True,False] | 2 possibilities: [True,False] |./curry-no-fll | 3 possibilities: [True,False] | 2 possibilities: [True,False] |./curry-no-cse-no-fll | 3 possibilities: [True,True,False] | 2 possibilities: [True,False] | | I will try this on large scale Curry programs. I hope the remaining | optimizations will still do some good. Do you think that there is a way | to be more selective? I mean to select those parts of the program which | can and which cannot be optimized? | | ps.: Maybe it is interesting to look at HasFuse [1] (somewhat outdated), | but it exactly forbids both transformations | | [1] http://www.ki.informatik.uni-frankfurt.de/research/diamond/hasfuse/ | | Yes, that looks interesting, too. Are there plans to update it with the | ghc? | | Thanks for your hints! | Bernd | ___ | Glasgow-haskell-users mailing list | Glasgow-haskell-users@haskell.org | http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Prevent optimization from tempering with unsafePerformIO
May I suggest a third route that has the advantages of both your approaches. The backside is of course that it takes a bit of work. My suggestion is to do an effect analysis of your curry programs to identify the purely functional parts and compile them directly to pure Haskell. The rest of the programs can run in a monad. This approach should be more robust than relying on unsafePerformIO. It is also very much in the spirit of your slogan. Thank you for the suggestion. We were thinking along the lines of identifying the purely functional parts and compile them directly to pure Haskell as a possibility of optimization (with some code duplication). But if the rest of the programs can run in a monad the question is: How much of the example program I submitted would you classify as purely functional? Actually, most of it is purely functional apart from the (?) - operator. But I do not see a way to make use of this. If, however, the whole program is classified as non-functional then this is definitely not in the spirit of the slogan. One effect we like very much about our implementation is that all expressions which do not depend on non-deterministic values are shared across the whole search. Note, that this dependence is a dynamic property. If the code of programs like the example was monadic, I do not see how not to loose this property. Thanks again! Bernd ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Prevent optimization from tempering with unsafePerformIO
I think you'll find it useful to be explicit about what you are relying on. Yes, good point. For example I think that if your program contains two calls unsafePerformIO e1 unsafePerformIO e2 then * You don't care about the *order* in which these are performed Right, the order is not important. * You do care that they are both performed separately, even if e1=e2. That is, you do not want CSE (common sub-expression) Right. * I'm not sure whether you care if they are not performed at all. E.g. if you have let x = unsafePerformIO e in True I do not care whether or not the action is performed. Even more so, if the whole evaluation strategy is not lazy but speculatively eager or something the like, this does not change the semantics of our programs. * I'm not sure if you care whether they are performed once or many times. E.g. f = \x. x + unsafePerformIO (h z) If the (h z) doesn't mention 'x', GHC will transform this to v = unsafePerformIO (h z) f = \x. x+v So now the effect happens once only, rather than once per call of f. Again you can control this by adding an articifical dependency on x. This is a very good example. The effect has to happen once per call to f. So what do you mean by adding an artificial dependency? Would for this example something like the following be enough? f = \x. x + myEffect x myEffect _ = unsafePerformIO (h z) Adding these artificial dependencies may amount to exactly the kind of effect analysis that Joesef suggests. Yes such a determinism analysis is a good idea to optimize functional logic programs and we have already done work in that direction. But I would not like to turn those programs into a monad which we cannot prove to be deterministic. Your hint of adding artificial dependencies sounds more promising. The hints and suggestions I got in this thread are already very helpful, thanks for that! Bernd ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Prevent optimization from tempering with unsafePerformIO
Bernd Brassel wrote: * I'm not sure if you care whether they are performed once or many times. E.g. f = \x. x + unsafePerformIO (h z) If the (h z) doesn't mention 'x', GHC will transform this to v = unsafePerformIO (h z) f = \x. x+v So now the effect happens once only, rather than once per call of f. Again you can control this by adding an articifical dependency on x. This is a very good example. The effect has to happen once per call to f. So what do you mean by adding an artificial dependency? Would for this example something like the following be enough? f = \x. x + myEffect x myEffect _ = unsafePerformIO (h z) Adding these artificial dependencies may amount to exactly the kind of effect analysis that Joesef suggests. Yes such a determinism analysis is a good idea to optimize functional logic programs and we have already done work in that direction. But I would not like to turn those programs into a monad which we cannot prove to be deterministic. Your hint of adding artificial dependencies sounds more promising. perhaps unsafePerformIODependingOn :: dep - IO a - a {-# NOINLINE unsafePerformIODependingOn #-} --is that pragma enough to avoid strictness/unused-ness analysis? --use 'lazy'? compile the module this is defined in with -O0? --I don't think seq'ing dep is desired. unsafePerformIODependingOn dep a = unsafePerformIO a f = \x. x + unsafePerformIODependingOn (x) (h z) --use a tuple if there are multiple variables you depend on also, do you care if the same `unsafePerformIO` is repeatedly performed more than once for no reason? Isaac ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Prevent optimization from tempering with unsafePerformIO
Hi Neil, hi Don! Nice meeting you at ICFP by the way. Can you give a specific example of what you have tried to do, and how it failed? I have attached a short synopsis of what our Curry to Haskell conceptually does. I could explain what all the parts mean and why they are defined this way, if it is important. On first glance it looks as if we were doing unsafe things in the very worst way. But the invariants within the generated code clean up things again. E.g., the result of main does not at all depend on whether or not the program is evaluated eagerly or lazily. I hope it is okay that I did not add any no-inline pragmata or something like that. Unfortunately, I forgot all the things we have tried more than a year ago to make optimization work. But this is the way it should work: $ ghc --make -O0 -o curry-no-opt curry.hs [1 of 1] Compiling Main ( curry.hs, curry.o ) Linking curry-no-opt ... $ curry-no-opt 3 possibilities: [True,True,False] 2 possibilities: [True,False] and this is what happens after optimization: $ rm curry.hi curry.o $ ghc --make -O2 -o curry-opt curry.hs [1 of 1] Compiling Main ( curry.hs, curry.o ) Linking curry-opt ... $ curry-opt 3 possibilities: [True,False] 2 possibilities: [True,False] As the code is now that is no surprise. But how can I prevent this from happening by adding pragmata? Thanks a lot for your time! Bernd import Data.IORef import System.IO.Unsafe --- -- a generated data definition --- data Bool' = True' | False' | Or Int [Bool'] --- -- two generated functions --- ifThenElse :: Bool' - Bool' - Bool' - Bool' ifThenElse True' x y = x ifThenElse False' x y = y ifThenElse (Or i bs) x y = Or i (map (\b - ifThenElse b x y) bs) (|||) :: Bool' - Bool' - Bool' x ||| y = ifThenElse x x y -- -- the main goal to solve -- main = do putStr 3 possibilities: print (depthFirstSearch ((True' ? False') ||| (True' ? False'))) putStr 2 possibilities: print (depthFirstSearch (let x = True' ? False' in x ||| x)) - -- a synopsis of the Curry core - depthFirstSearch :: Bool' - [Bool] depthFirstSearch = dfs [] dfs :: [(Int,Int)] - Bool' - [Bool] dfs _ True' = [True] dfs _ False' = [False] dfs st (Or i bs) = maybe (concat (zipWith (choose st i) bs [0..])) (dfs st . (bs!!)) (lookup i st) choose :: [(Int,Int)] - Int - Bool' - Int - [Bool] choose st i b j = dfs ((i,j):st) b --- -- unsafe part --- (?) :: Bool' - Bool' - Bool' x ? y = Or (unsafeNextOrRef ()) [x,y] globalCounter :: IORef Int globalCounter = unsafePerformIO (newIORef 0) unsafeNextOrRef :: () - Int unsafeNextOrRef _ = unsafePerformIO $ do r - readIORef globalCounter writeIORef globalCounter (r+1) return r ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Prevent optimization from tempering with unsafePerformIO
Neil Mitchell schrieb: It varies by each piece of code, can you post a code fragment that gets optimised in the wrong way? Sorry for posting this twice! I have added the code to the other thread. Thanks! Bernd ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Prevent optimization from tempering with unsafePerformIO
Hi, I think it's the let floating (out) together with common subexpression elimination: ghc --make -O2 -no-recomp -fno-cse -o curry-no-cse curry.hs [1 of 1] Compiling Main ( curry.hs, curry.o ) Linking curry-no-cse ... ghc --make -O2 -no-recomp -fno-full-laziness -o curry-no-fll curry.hs [1 of 1] Compiling Main ( curry.hs, curry.o ) Linking curry-no-fll ... ghc --make -O2 -no-recomp -fno-full-laziness -fno-cse -o curry-no-cse-no-fll curry.hs [1 of 1] Compiling Main ( curry.hs, curry.o ) Linking curry-no-cse-no-fll ... ./curry-no-cse 3 possibilities: [True,False] 2 possibilities: [True,False] ./curry-no-fll 3 possibilities: [True,False] 2 possibilities: [True,False] ./curry-no-cse-no-fll 3 possibilities: [True,True,False] 2 possibilities: [True,False] Regards, David ps.: Maybe it is interesting to look at HasFuse [1] (somewhat outdated), but it exactly forbids both transformations [1] http://www.ki.informatik.uni-frankfurt.de/research/diamond/hasfuse/ Bernd Brassel wrote: Hi Neil, hi Don! Nice meeting you at ICFP by the way. Can you give a specific example of what you have tried to do, and how it failed? I have attached a short synopsis of what our Curry to Haskell conceptually does. I could explain what all the parts mean and why they are defined this way, if it is important. On first glance it looks as if we were doing unsafe things in the very worst way. But the invariants within the generated code clean up things again. E.g., the result of main does not at all depend on whether or not the program is evaluated eagerly or lazily. I hope it is okay that I did not add any no-inline pragmata or something like that. Unfortunately, I forgot all the things we have tried more than a year ago to make optimization work. But this is the way it should work: $ ghc --make -O0 -o curry-no-opt curry.hs [1 of 1] Compiling Main ( curry.hs, curry.o ) Linking curry-no-opt ... $ curry-no-opt 3 possibilities: [True,True,False] 2 possibilities: [True,False] and this is what happens after optimization: $ rm curry.hi curry.o $ ghc --make -O2 -o curry-opt curry.hs [1 of 1] Compiling Main ( curry.hs, curry.o ) Linking curry-opt ... $ curry-opt 3 possibilities: [True,False] 2 possibilities: [True,False] As the code is now that is no surprise. But how can I prevent this from happening by adding pragmata? Thanks a lot for your time! Bernd ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Prevent optimization from tempering with unsafePerformIO
On Tue, Oct 16, 2007 at 04:06:26PM +0200, Bernd Brassel wrote: Hi Neil, hi Don! Nice meeting you at ICFP by the way. Can you give a specific example of what you have tried to do, and how it failed? I have attached a short synopsis of what our Curry to Haskell conceptually does. I could explain what all the parts mean and why they are defined this way, if it is important. On first glance it looks as if we were doing unsafe things in the very worst way. But the invariants within the generated code clean up things again. E.g., the result of main does not at all depend on whether or not the program is evaluated eagerly or lazily. I hope it is okay that I did not add any no-inline pragmata or something like that. Unfortunately, I forgot all the things we have tried more than a year ago to make optimization work. Might I suggest, that the problem is your use of unsafePerformIO? If you use unsafePerformIO in accordance with the rules listed in the specification (which happens to be the FFI addendum), -O options will have no effect. (Not that what you are trying to do is achievable with correct use of unsafePerformIO; why do you want to do this unsafely, instead of just using 'length'? unsafePerformIO is a very slow function, remember) Stefan signature.asc Description: Digital signature ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Prevent optimization from tempering with unsafePerformIO
Hello, I am writing a compiler from the functional logic language Curry to Haskell. As the result is a real extension of Haskell, there has to be SOME side effect involved. Unfortunately, this seems to prevent me from using any of ghc's optimizations on the resulting code. I have tried for some time to invoke the no-inline pragmata, but they seem to have no effect at all. Recently at ICFP, Neil Mitchell told me that there definitely is a way to avoid this problem. But does anybody know how? Thanks for your help! Bernd Brassel ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: GHC -O2 and unsafePerformIO
Neil Mitchell wrote: Hi Thanks to dcoutts, I have now come up with an answer. I don't understand why it works now, but not before. I do remember than browsing either Core or STG is not a fun thing to do... p_System_IO_hGetChar h = trace i am here $ unsafePerformIO $ getCharIO h {-# NOINLINE getCharIO #-} getCharIO h = do c - getchar print c return $ if c == (-1) then 0 else chr_ c This is clearly a misuse of unsafePerformIO (as I'm sure you're aware). Just out of interest - what's the context? Cheers, Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: GHC -O2 and unsafePerformIO
Hi Simon, This is clearly a misuse of unsafePerformIO (as I'm sure you're aware). Just out of interest - what's the context? I am writing an optimiser for Yhc, doing whole-program optimisation, with the intention of keeping it fast and high performance. Since writing out Yhc bytecode would kill any performance benefits, I am writing back to Haskell to be compiled with GHC. So the basic sequence of steps is: compile Haskell to Yhc Core transform Yhc Core convert Yhc Core to Haskell compile Haskell to GHC The problem I'm currently overcoming is that Yhc inserts its own IO monad, which isn't the same as the GHC one. By the time I get to GHC Haskell, all of the Yhc monad is invisible to GHC, so I have to unsafePerformIO the getchar/putchar functions. With this as the primitive getchar, it seems to work for my particular example, but is clearly a bit fragile. All the examples I'm doing for now are wc -c, wc -l, so shouldn't stress the IO much more than getchar/putchar. Thanks Neil ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
GHC -O2 and unsafePerformIO
Hi I have a program (below) which when compiled with -O2 gives the result: H:\work\supero\charcounttype log.txt | diff.exe i am here 109 done The process tried to write to a nonexistent pipe. And when compiled with -O0 gives: H:\work\supero\charcounttype log.txt | diff i am here 109 i am here 111 done The process tried to write to a nonexistent pipe. It tries to read two characters, so I really do want two characters to appear. I have tried NOINLINE, but with no effect. Any suggestions? Thanks Neil --- -- The program import System.IO.Unsafe import System.IO import Data.Word import Debug.Trace main = p_System_IO_hGetChar 1 `seq` p_System_IO_hGetChar 2 `seq` putStrLn done {-# NOINLINE wrapIO #-} wrapIO x = unsafePerformIO (x = return) foreign import ccall stdio.h getchar getchar :: IO Word8 {-# NOINLINE p_System_IO_hGetChar #-} p_System_IO_hGetChar h = trace i am here $ wrapIO (getchar = \c - print c return (if c == (-1) then 0 else chr_ c)) chr_ = fromEnum ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: GHC -O2 and unsafePerformIO
Hello Neil, Wednesday, May 2, 2007, 7:00:05 PM, you wrote: {-# NOINLINE wrapIO #-} wrapIO x = unsafePerformIO (x = return) -fno-cse ? it's usual company for unsafePerformIO+NOINLINE :) -- Best regards, Bulatmailto:[EMAIL PROTECTED] ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: GHC -O2 and unsafePerformIO
Hi Bulat, Wednesday, May 2, 2007, 7:00:05 PM, you wrote: {-# NOINLINE wrapIO #-} wrapIO x = unsafePerformIO (x = return) -fno-cse ? it's usual company for unsafePerformIO+NOINLINE :) No luck, alas. A slightly tweaked version, which is slightly simpler and still gives the same behaviour is below. Thanks Neil -- main = p_System_IO_hGetChar undefined `seq` p_System_IO_hGetChar 12 `seq` putStrLn done foreign import ccall stdio.h getchar getchar :: IO Word8 {-# NOINLINE p_System_IO_hGetChar #-} p_System_IO_hGetChar h = trace i am here $ unsafePerformIO (getchar = \c - print c return (if c == (-1) then 0 else chr_ c)) chr_ = fromEnum ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: GHC -O2 and unsafePerformIO
Hi Thanks to dcoutts, I have now come up with an answer. I don't understand why it works now, but not before. I do remember than browsing either Core or STG is not a fun thing to do... p_System_IO_hGetChar h = trace i am here $ unsafePerformIO $ getCharIO h {-# NOINLINE getCharIO #-} getCharIO h = do c - getchar print c return $ if c == (-1) then 0 else chr_ c Thanks Neil On 5/2/07, Neil Mitchell [EMAIL PROTECTED] wrote: Hi Bulat, Wednesday, May 2, 2007, 7:00:05 PM, you wrote: {-# NOINLINE wrapIO #-} wrapIO x = unsafePerformIO (x = return) -fno-cse ? it's usual company for unsafePerformIO+NOINLINE :) No luck, alas. A slightly tweaked version, which is slightly simpler and still gives the same behaviour is below. Thanks Neil -- main = p_System_IO_hGetChar undefined `seq` p_System_IO_hGetChar 12 `seq` putStrLn done foreign import ccall stdio.h getchar getchar :: IO Word8 {-# NOINLINE p_System_IO_hGetChar #-} p_System_IO_hGetChar h = trace i am here $ unsafePerformIO (getchar = \c - print c return (if c == (-1) then 0 else chr_ c)) chr_ = fromEnum ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: GHC -O2 and unsafePerformIO
On Wed, 2007-05-02 at 16:33 +0100, Neil Mitchell wrote: Hi Thanks to dcoutts, I have now come up with an answer. I don't understand why it works now, but not before. main = p_System_IO_hGetChar 1 `seq` p_System_IO_hGetChar 2 `seq` putStrLn done This is fine (though note that p_System_IO_hGetChar 1 is a constant) the real problem is here: {-# NOINLINE p_System_IO_hGetChar #-} p_System_IO_hGetChar h = trace i am here $ unsafePerformIO $ getchar = \c - print c return (if c == (-1) then 0 else chr_ c) You've tried to trick ghc into always calling this by passing a dummy 'h' parameter. Then 'h' is never used in the body. Note however that the whole body of this function is a constant and so ghc can (and at -O2 does) float it out as a CAF. This means you get the side effects of p_System_IO_hGetChar at most once. The solution of course is to use a full data dependency like IO or ST uses. I do remember than browsing either Core or STG is not a fun thing to do... So yeah, we see the above CAFs and let floating by looking at the core. We could do with a prettier pretty printer for core, I agree. Duncan ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: unsafePerformIO safety.
I wouldn't like if unsafePerformIO could never be inlined, sometimes you want it inlined for performance. And not all uses are hacky, it's just that when I use it, then burden is on me to convince myself that it is safe. On Mar 6, 2007, at 23:56 , Neil Mitchell wrote: Hi On 3/6/07, Lennart Augustsson [EMAIL PROTECTED] wrote: Yeah, you really need {-# NOINLINE var #-} to make it reasonable safe. Couldn't GHC bake in knowledge about unsafePerformIO, and never inline it? It is a slightly hacky solution, but since unsafePerformIO is pretty much only used in hacks, I think its almost fitting. Thanks Neil ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: unsafePerformIO safety.
Lennart Augustsson wrote: I wouldn't like if unsafePerformIO could never be inlined, sometimes you want it inlined for performance. And not all uses are hacky, it's just that when I use it, then burden is on me to convince myself that it is safe. unsafePerformIO is currently not inlined - there's a NOINLINE pragma on its definition in GHC.IOBase, and some comments in there to explain why. You're right that sometimes you really want it to be inlined - this is why there's a function called inlinePerformIO in Data.ByteString.Base, for example. You'd better really know what you're doing before using that one, though :-) Cheers, Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: unsafePerformIO safety.
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Simon Marlow wrote: You're right that sometimes you really want it to be inlined - this is why there's a function called inlinePerformIO in Data.ByteString.Base, for example. You'd better really know what you're doing before using that one, though :-) yep, it should be generally available as unsafeInlineUnsafePerformIO (-: It's especially unsafe because... all of the IO might not be executed, due to lazy evaluation + optimization? Some of the IO might be run twice without other parts of it being so? Isaac -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.3 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFF7p9jHgcxvIWYTTURAqAKAKCu8dJ4rpyMcMH7ebPv0HfIUyGSgACgy0Jq naeTCZk/2xZraOrSJtyObm8= =oKKG -END PGP SIGNATURE- ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
unsafePerformIO safety.
I've noticed quite a few pages referencing constructs such as: var :: MVar ([Foo]) var = unsafePerformIO (newMVar ([])) and the likes. Is there a danger of different uses of 'var' getting new MVars instead of all sharing one. Having a reliable way to create a piece of global state would be very convenient. Dave ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: unsafePerformIO safety.
On Tue, 06 Mar 2007 12:03:05 -0800 David Brown [EMAIL PROTECTED] wrote: I've noticed quite a few pages referencing constructs such as: var :: MVar ([Foo]) var = unsafePerformIO (newMVar ([])) and the likes. Is there a danger of different uses of 'var' getting new MVars instead of all sharing one. Having a reliable way to create a piece of global state would be very convenient. This operation is unsafe by definition. I use it extensively, without problems. The unsafe in the name reminds you that there are situations for which the function is inappropriate, but all of my deployed commercial programs have functionality of this sort. Understand the risk, but don't hesitate to use it. Seth Kurtzberg Dave ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: unsafePerformIO safety.
Seth Kurtzberg wrote: On Tue, 06 Mar 2007 12:03:05 -0800 David Brown [EMAIL PROTECTED] wrote: I've noticed quite a few pages referencing constructs such as: var :: MVar ([Foo]) var = unsafePerformIO (newMVar ([])) and the likes. Is there a danger of different uses of 'var' getting new MVars instead of all sharing one. Having a reliable way to create a piece of global state would be very convenient. This operation is unsafe by definition. I use it extensively, without problems. The unsafe in the name reminds you that there are situations for which the function is inappropriate, but all of my deployed commercial programs have functionality of this sort. Understand the risk, but don't hesitate to use it. Do you do anything to keep 'var' from getting inlined? I can envision a case where the code would be inlined, and then each use would get a separate MVar. Thanks, Dave ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: unsafePerformIO safety.
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 David Brown wrote: I've noticed quite a few pages referencing constructs such as: var :: MVar ([Foo]) var = unsafePerformIO (newMVar ([])) and the likes. Is there a danger of different uses of 'var' getting new MVars instead of all sharing one. If I remember correctly, you should put {-# NOINLINE var #-} on a line just before that. Having a reliable way to create a piece of global state would be very convenient. This is well known and extensively discussed - unfortunately it is a complicated (or at least controversial) issue Isaac -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.3 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFF7fiFHgcxvIWYTTURAoMXAJ9SCryCX+daNLKrMIhWlMh/aJmVXwCghosx 6/lBweYnNslHLal57RAtX0Y= =2oIN -END PGP SIGNATURE- ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: unsafePerformIO safety.
Yeah, you really need {-# NOINLINE var #-} to make it reasonable safe. On Mar 6, 2007, at 23:18 , David Brown wrote: Seth Kurtzberg wrote: On Tue, 06 Mar 2007 12:03:05 -0800 David Brown [EMAIL PROTECTED] wrote: I've noticed quite a few pages referencing constructs such as: var :: MVar ([Foo]) var = unsafePerformIO (newMVar ([])) and the likes. Is there a danger of different uses of 'var' getting new MVars instead of all sharing one. Having a reliable way to create a piece of global state would be very convenient. This operation is unsafe by definition. I use it extensively, without problems. The unsafe in the name reminds you that there are situations for which the function is inappropriate, but all of my deployed commercial programs have functionality of this sort. Understand the risk, but don't hesitate to use it. Do you do anything to keep 'var' from getting inlined? I can envision a case where the code would be inlined, and then each use would get a separate MVar. Thanks, Dave ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: unsafePerformIO safety.
Hi On 3/6/07, Lennart Augustsson [EMAIL PROTECTED] wrote: Yeah, you really need {-# NOINLINE var #-} to make it reasonable safe. Couldn't GHC bake in knowledge about unsafePerformIO, and never inline it? It is a slightly hacky solution, but since unsafePerformIO is pretty much only used in hacks, I think its almost fitting. Thanks Neil ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: unsafePerformIO safety.
Neil Mitchell wrote: On 3/6/07, Lennart Augustsson [EMAIL PROTECTED] wrote: Yeah, you really need {-# NOINLINE var #-} to make it reasonable safe. Couldn't GHC bake in knowledge about unsafePerformIO, and never inline it? It is a slightly hacky solution, but since unsafePerformIO is pretty much only used in hacks, I think its almost fitting. It seems to be used a bit more than just as a hack. Many things that interface with the real world, but try to present lazy interfaces have to use it. Maintaining a shared state that doesn't have to be passed around to everything that uses it isn't really a hack. As an example, I would like to have something that performs logging, and is used by many clients. Without the unsafePerformIO, everything has to somehow find and pass around the state of this logging system, whereas with it, it can just allocate one when first needed. Non-strict semantics are kind of a new thing for me. I'm still trying to get a good grasp of what can be lazy, what should live in the IO Monad, and what should have a carefully crafted bridge between the two. Things like hGetContents have somewhat set a trend here. Dave ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
RE: [Haskell] crash unsafeperformio stm
[Redirecting to GHC users] A crash is bad, and the HEAD reports an error in a civilised way. It is indeed illegal to use atomically inside an unsafePerformIO. I've also committed a fix to the documentation of atomically to mention this point. In general, the documentation of STM is very thin -- we would welcome someone to improve it. Simon | -Original Message- | From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Johannes Goetz | Sent: 12 July 2006 04:50 | To: Haskell Mailing List | Subject: [Haskell] crash unsafeperformio stm | | hi folks, | | i'm using ghc6 (apt-get install ghc6) on debian31. as far is i | understand, the following program shouldn't crash. can somebody tell me | why it does? thanks a lot. johannes. | | module Main(main) where | | import Control.Concurrent.STM | import System.IO.Unsafe | | {-# NOINLINE a #-} | a :: TMVar Int | a = unsafePerformIO $ atomically $ newTMVar 123 | | main = do | atomically $ takeTMVar a = putTMVar a | | ghc --make -fno-cse Test.hs -o test | ./test | Segmentation fault | | ___ | Haskell mailing list | Haskell@haskell.org | http://www.haskell.org/mailman/listinfo/haskell ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: unsafePerformIO and NOINLINE Pragma
Am Dienstag, 13. September 2005 16:13 schrieb David Sabel: Hi, Hi! I want to analyse the laziness of a data structure. To check how many nodes are constructed I use a global counter. counter :: IORef Int counter = unsafePerformIO (newIORef 0) This counter is increased every time the constructor is called by redefining the constructor OBDD as follows. oBDD low var high = seq (unsafePerformIO (modifyIORef counter (+1))) (OBDD low var high) This works fine. When I compile with optimisations the counter is always set to one no matter how many nodes are constructed. I thought this would be caused by inlining. Therefore I have added two NOINLINE pragmata. {-# NOINLINE counter #-} {-# NOINLINE oBDD #-} Although the counter doesn't work. Is there another optimisation that can cause harm? Is there something wrong with the pragmata? Two comments: 1. There are other optimisations than inlining that can break sharing, e.g. common subexpression elimination, full-laziness-transformation. 2. I tested the following: {-# NOLINE counter #-} {-# INLINE oBDD #-} then the counter seems to work correct. In my opinion oBDD is an abstraction and can always be inlined. Sadly this solution doesn't work in my environment. The counter is always set to 2 while it should be something around 15.000. I think it's not worth going into detail of the transformations for me thus I will check the number of nodes without optimisations and the performance with optimisations. Thanks anyway. ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
unsafePerformIO and optimizations
Hello, http://haskell.org/ghc/docs/latest/html/libraries/base/ System.IO.Unsafe.html#v%3AunsafePerformIO talks about what optimizations you should disable if you apply unsafePerformIO to an action which contains side effects. These are: * inlining of functions which call unsafePerformIO * common subexpression elimination (on the module which contains the unsafePerformIO application, if I understand correctly) * let-floating Alas, the documentation is very terse, in my opinion, and I don't understand these things fully. First, what is the problem with inlining a function call? Say, I have a function f defined as follows: f :: Char - () f c = unsafePerformIO (putChar c) If inlining is allowed, the expression f '*' becomes unsafePerformIO (putChar '*'). Is this a problem? Or is inlining dangerous in other situations? On the other hand, I can see that inlining non-functions could cause harm. Say I have something like this: u :: () u = unsafePerformIO (putChar '*') Now, inlining u would transform (u,u) to (unsafePerformIO (putChar '*),unsafePerformIO (putChar '*')) which could result in putChar '*' being executed multiple times. So why does the library documentation only talk about disabling inlining of functions? I understand that common subexpression elimination could cause harm. It could transform the expression (unsafePerformIO (putChar '*),unsafePerformIO (putChar '*')) to let u = unsafePerformIO (putChar '*') in (u,u), couldn't it? This would result in the I/O action be performed at most once which is not what was intended. But couldn't the same thing also happen if I use the expression (f '*',f '*'), probably in a different module? Does this mean that I have to use -fno-cse not only in the module which contains the unsafePerformIO application but also in every other module which uses unsafePerformIO indirectly via f or functions using f? Does let-floating only have to be disabled in the module which uses unsafePerformIO? Say, I have f defined as above and have the following definition in a module different from the one f is defined in: g :: Int - () g n = f '*' Couldn't it be that the asterisk is output only once instead of every time, g is called? I would be thankful for any help. Best regards, Wolfgang ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
STM and unsafePerformIO
Hello All, I think I've read somewhere that STM doesn't like unsafePerformIO. However, I would like to use a global STM variable. Something like this: *module* Main *where* import GHC.Conc import System.IO.Unsafe tSid = unsafePerformIO (atomically (newTVar 0)) tickSessionID :: STM Int tickSessionID = *do* sid - readTVar tSid writeTVar tSid (sid + 1) return sid main = atomically tickSessionID But, when I try this, the evaluation of main causes a segmentation fault. Is there a workaround for this bug? Regards, Robert ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: STM and unsafePerformIO
Never mind, I can probably use an MVar to work around this... Robert Robert van Herk wrote: Hello All, I think I've read somewhere that STM doesn't like unsafePerformIO. However, I would like to use a global STM variable. Something like this: module Main where import GHC.Conc import System.IO.Unsafe tSid = unsafePerformIO (atomically (newTVar 0)) tickSessionID :: STM Int tickSessionID = do sid - readTVar tSid writeTVar tSid (sid + 1) return sid main = atomically tickSessionID But, when I try this, the evaluation of main causes a segmentation fault. Is there a workaround for this bug? Regards, Robert ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: STM and unsafePerformIO
On Wed, Aug 03, 2005 at 12:50:54PM +0200, Robert van Herk wrote: Hello All, I think I've read somewhere that STM doesn't like unsafePerformIO. However, I would like to use a global STM variable. Something like this: module Main where import GHC.Conc import System.IO.Unsafe tSid = unsafePerformIO (atomically (newTVar 0)) tickSessionID :: STM Int tickSessionID = do sid - readTVar tSid writeTVar tSid (sid + 1) return sid main = atomically tickSessionID But, when I try this, the evaluation of main causes a segmentation fault. Is there a workaround for this bug? Regards, Robert It probably dies not because of unsafePerformIO per se, but because STM doesn't understand nested transactions, and unsafePerformIO here results in a nested transaction. Using the following main works for me, as it forces both atomically's to be evaluated sequentially: main = tSid `seq` atomically tickSessionID See also http://haskell.org/pipermail/glasgow-haskell-users/2005-June/008615.html and http://sourceforge.net/tracker/index.php?func=detailaid=1235728group_id=8032atid=108032 Happy hacking, Remi P.S. Could you find out (and fix) what inserts those spurious *'s in your code? -- Nobody can be exactly like me. Even I have trouble doing it. ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
RE: STM and unsafePerformIO
On 03 August 2005 11:51, Robert van Herk wrote: I think I've read somewhere that STM doesn't like unsafePerformIO. However, I would like to use a global STM variable. Something like this: *module* Main *where* import GHC.Conc import System.IO.Unsafe tSid = unsafePerformIO (atomically (newTVar 0)) tickSessionID :: STM Int tickSessionID = *do* sid - readTVar tSid writeTVar tSid (sid + 1) return sid main = atomically tickSessionID But, when I try this, the evaluation of main causes a segmentation fault. Is there a workaround for this bug? You're ok as long as the global variable doesn't get evaluated inside another atomically. The problem is that nested atomically isn't supported; the RTS gets confused. For example, you could say main = do evaluate tSid ... to force evaluation of tSid before doing anything else. Cheers, Simon ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: STM and unsafePerformIO
Oh I see! Your sollutions are indeed a lot better than implementating all my code using MVars :-). Thanks, Robert Remi Turk wrote: On Wed, Aug 03, 2005 at 12:50:54PM +0200, Robert van Herk wrote: Hello All, I think I've read somewhere that STM doesn't like unsafePerformIO. However, I would like to use a global STM variable. Something like this: module Main where import GHC.Conc import System.IO.Unsafe tSid = unsafePerformIO (atomically (newTVar 0)) tickSessionID :: STM Int tickSessionID = do sid - readTVar tSid writeTVar tSid (sid + 1) return sid main = atomically tickSessionID But, when I try this, the evaluation of main causes a segmentation fault. Is there a workaround for this bug? Regards, Robert It probably dies not because of unsafePerformIO per se, but because STM doesn't understand nested transactions, and unsafePerformIO here results in a nested transaction. Using the following main works for me, as it forces both atomically's to be evaluated sequentially: main = tSid `seq` atomically tickSessionID See also http://haskell.org/pipermail/glasgow-haskell-users/2005-June/008615.html and http://sourceforge.net/tracker/index.php?func=detailaid=1235728group_id=8032atid=108032 Happy hacking, Remi P.S. Could you find out (and fix) what inserts those spurious *'s in your code? ___ Glasgow-haskell-users mailing list Glasgow-haskell-users@haskell.org http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: unsafePerformIO
Sven Panne wrote: Huh? I'm not sure what you mean exactly, but with the help of unsafePerformIO and a pragma for clever compilers you can simulate something like a global variable in Haskell. Here an excerpt from the GLUT menu handling module: {-# NOINLINE theMenuTable #-} theMenuTable :: IORef MenuTable theMenuTable = unsafePerformIO (newIORef emptyMenuTable) ... Definitely not something to be proud of, but quite handy from time to time. :-) Alternatives are passing the IORef to every function which needs it (but this would clutter up the GLUT API in the above case) or using the FFI for holding global data on the C side. GHC uses a similar hack internally, too, BTW. This is not just handy from time to time, but very frequently. I've just discovered it occurs about 100 times in 100K LOC that I am responsible for, so often that I have stopped feeling guilty about it. I don't really know what else I could do. One solution would be to use implicit parameters to pass a global state variable around, but this would require me to change the type of huge numbers of functions, since implicit parameters are still explicit in types. I don't remember having any actual bugs caused by unsafePerformIO. Well, only one, before I knew about putting NOINLINE in. ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
RE: unsafePerformIO and IORefs
Hal Daume III wrote: You can't. [...] Well, you can, but only for CAFs. This idiom/hack is used quite happily throughout GHC, HOpenGL, H/Direct, ... I think quite happily is a bit strong ;-) We'd much rather have a safe way to do what is really quite a reasonable thing. Cheers, Simon ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
unsafePerformIO and IORefs
I want to write something like type State a = IORef a newState :: a - State a newState v = unsafePerformIO (newIORef v) But I don't want the compileer to inline this nor to inline any application of this. {#NOINLINE newState#} But how can I stop this function to be inlined when applied for example : let x = newState 0 in {... code where x is used twice ...} How to be sure that x isn't inlined and that all occurences of x are pointing to the same memory place ? Best regards, Nicolas Oury ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: unsafePerformIO and IORefs
Hal Daume III wrote: You can't. [...] Well, you can, but only for CAFs. This idiom/hack is used quite happily throughout GHC, HOpenGL, H/Direct, ... Slightly modified stuff from GHC's sources: -- global variables in Haskell :-) global :: a - IORef a global a = unsafePerformIO (newIORef a) #define GLOBAL_VAR(name,value,ty) \ name :: IORef (ty) ; \ name = global (value) ; \ {-# NOINLINE name #-} -- examples GLOBAL_VAR(counter, 0, Int) GLOBAL_VAR(flag, False, Bool) Cheers, S. ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: unsafePerformIO and IORefs
On Mon, Nov 18, 2002 at 10:36:05AM -0800, Hal Daume III wrote: You can't. CSE (common subexpression elimination) will replace any occurances of 'newState 0' in a function body with the same value. In short: don't use upIO :) Sorry, cannot resist to pour a little salt onto the wound :) [232]% grep global ghc/compiler/utils/Util.lhs , global global :: a - IORef a global a = unsafePerformIO (newIORef a) [233]% ghc/compiler/HsVersions.h: [...] #ifdef __GLASGOW_HASKELL__ #define GLOBAL_VAR(name,value,ty) \ name = Util.global (value) :: IORef (ty); \ {-# NOINLINE name #-} #endif [237]% grep -r GLOBAL_VAR ghc/compiler | wc -l 90 Muahahah... ;-P Cheers, Michael ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Optimisation and unsafePerformIO
- Original Message - From: Albert Lai [EMAIL PROTECTED] To: [EMAIL PROTECTED] Sent: Wednesday, October 30, 2002 7:35 AM Subject: Re: Optimisation and unsafePerformIO David Sabel [EMAIL PROTECTED] writes: {-# NOINLINE b #-} b x = if even x then unsafePerformIO getChar else bot bot = bot main = do putChar (b 4) putChar (b 6) I am not a compiler implementer (or lawyer, for that matter :) But I propose this guess. First, both even 4 and even 6 get constant-folded to True; so b 4 and b 6 both become unsafePerformIO getChar. Then there is a common subexpression elimination. No! I used the option -fno-cse, what means that common supexpression elimination is turned off. ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Optimisation and unsafePerformIO
David Sabel [EMAIL PROTECTED] writes: {-# NOINLINE b #-} b x = if even x then unsafePerformIO getChar else bot bot = bot main = do putChar (b 4) putChar (b 6) I am not a compiler implementer (or lawyer, for that matter :) But I propose this guess. First, both even 4 and even 6 get constant-folded to True; so b 4 and b 6 both become unsafePerformIO getChar. Then there is a common subexpression elimination. ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
RE: Optimisation and unsafePerformIO
Consider the following program: - {-# NOINLINE b #-} b x = if even x then unsafePerformIO getChar else bot bot = bot main = do putChar (b 4) putChar (b 6) - when you compile the programm with the options: -O0 and execute the program you get: test ab (That's the input) ab (That's the ouput) when you compile the programm with the options: -O1 -fno-cse you get: test ab aa You are using unsafePerformIO in an unsafe way. The meaning of your program depends on whether the compiler implements full laziness or not, which is a decision left entirely up to the compiler implementor. If you want to write portable code, don't use unsafePerformIO in this way. What exactly is it you're trying to achieve? Perhaps we can suggest a better solution. Cheers, Simon ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: Optimisation and unsafePerformIO
Of course, I used unsafePerformIO in an unsafe way! I'm thinking about a way to make unsafePerformIO safe. Therefore the compiler can't do any transformation the ghc does and I want to locate these transformations. - Original Message - From: Simon Marlow [EMAIL PROTECTED] To: David Sabel [EMAIL PROTECTED]; [EMAIL PROTECTED] Sent: Monday, October 28, 2002 10:41 AM Subject: RE: Optimisation and unsafePerformIO Consider the following program: - {-# NOINLINE b #-} b x = if even x then unsafePerformIO getChar else bot bot = bot main = do putChar (b 4) putChar (b 6) - when you compile the programm with the options: -O0 and execute the program you get: test ab (That's the input) ab (That's the ouput) when you compile the programm with the options: -O1 -fno-cse you get: test ab aa You are using unsafePerformIO in an unsafe way. The meaning of your program depends on whether the compiler implements full laziness or not, which is a decision left entirely up to the compiler implementor. If you want to write portable code, don't use unsafePerformIO in this way. What exactly is it you're trying to achieve? Perhaps we can suggest a better solution. Cheers, Simon ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Optimisation and unsafePerformIO
Consider the following program: - {-# NOINLINE b #-} b x = if even x then unsafePerformIO getChar else bot bot = bot main = do putChar (b 4) putChar (b 6) - when you compile the programm with the options: -O0 and execute the program you get: test ab (That's the input) ab (That's the ouput) when you compile the programm with the options: -O1 -fno-cse you get: test ab aa my question is now: which transformation/optimisation is responsible for that, and is it possible to switch off this transformation? David Sabel ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: unsafePerformIO
- Original Message - From: Simon Marlow [EMAIL PROTECTED] Sent: Tuesday, September 24, 2002 2:58 PM Subject: RE: unsafePerformIO [...] As for sharing, we currently don't provide any guarnatees, although we should. It is currently the case that if you write a = unsafePerformIO (putStr hello) b = unsafePerformIO (putStr hello) and both a and b are evaluated, then you may get either one or two hellos on stdout. You can currently make things more deterministic by (a) adding NOINLINE pragmas for a and b, and (b) using the flag -fno-cse to disable common sub-expression elimination. Then you'll get exactly two instances of hello on stdout, although we won't guarantee that behaviour for ever. At some point we'll fix it so that unsafePerformIO applications are never duplicated or coalesced. Are there any (short) examples available where using of unsafePerformIO leads to unexpected behaviour, especially an example with the terms a and b from above? with best regards, David - JWGU Frankfurt ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
RE: unsafePerformIO
The actions performed by unsafePerformIO are simply done at some entirely unpredictable time, perhaps interleaved with actions on the main execution path. The formal semantics in the notes doesn't have a good way to express that because the purely-functional part is given a denotational semantics... yet unsafePerformIO can occur in the middle of that. So it's tiresome to characterise in theory, and hard to predict in practice. You should only use it when the state it is fiddling with is well partitioned. Simon | -Original Message- | From: David Sabel [mailto:[EMAIL PROTECTED]] | Sent: 20 September 2002 13:48 | To: Simon Peyton-Jones; [EMAIL PROTECTED] | Subject: unsafePerformIO | | In read your paper Tackling the Awkward Squad: monadic input / output, | concurrency, exceptions, and foreign-language calls in Haskell, and have | a question about unsafePerformIO. | | In your operational semantic of the IO-Monad you tell nothing about, how | 'unsafe' IO actions are performed, is there another paper / documentation | about | this available, or can you - or someone else - give me a review about that? | | David | JWGU Frankfurt | ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
RE: unsafePerformIO
Simon Peyton-Jones wrote: | The actions performed by unsafePerformIO are simply | done at some entirely unpredictable time, perhaps | interleaved with actions on the main execution path. But it is a fact that many of us have at least some idea of what happens under the hood when we use unsafePerformIO. This is also described in your paper Stretching the storage manager: weak pointers and stable names in Haskell. However, I for example have no idea what happens when unsafely executing something that throws exceptions, performs a forkIO, something that uses MVar's, etc. It would be nice to see what happens in such a case, and at least a document that informally describes what happens. This includes issues such as loss of sharing because of inlining, etc. unsafePerformIO is the best thing since sliced bread for someone who wants to add stuff to the compiler without changing the compiler. The usefulness/portability at the moment is limited since nobody dares to say what is going on. /Koen. ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
RE: unsafePerformIO
But it is a fact that many of us have at least some idea of what happens under the hood when we use unsafePerformIO. This is also described in your paper Stretching the storage manager: weak pointers and stable names in Haskell. However, I for example have no idea what happens when unsafely executing something that throws exceptions, performs a forkIO, something that uses MVar's, etc. Exceptions: an exception raised by the computation inside unsafePerformIO will be propagated to the enclosing expression, as per the normal semantics for imprecise exceptions. The exception raised by an application of unsafePerformIO may of course also be imprecise; consider 'unsafePerformIO $ (1/0 + error foo) `seq` return ()'. The behaviour of forkIO and other side-effecting operations inside unsafePerforIO can I think be explained by the following statement: if an expression unsafePerformIO e has been reduced to head normal form, then all the I/O computations in e have been performed. The exact timing of the evaluation is underspecified, just as any other expression in Haskell. However, if you're using unsafePerformIO with real side effects, then I suspect that what you're doing is highly dodgy and you should find another way to do it :-) Does that help? As for sharing, we currently don't provide any guarnatees, although we should. It is currently the case that if you write a = unsafePerformIO (putStr hello) b = unsafePerformIO (putStr hello) and both a and b are evaluated, then you may get either one or two hellos on stdout. You can currently make things more deterministic by (a) adding NOINLINE pragmas for a and b, and (b) using the flag -fno-cse to disable common sub-expression elimination. Then you'll get exactly two instances of hello on stdout, although we won't guarantee that behaviour for ever. At some point we'll fix it so that unsafePerformIO applications are never duplicated or coalesced. If you're interested in when unsafePerformIO is safe to use, see: http://www.haskell.org/pipermail/glasgow-haskell-users/2002-July/003683. html (and see my followups for an alternative view). Cheers, Simon ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
RE: unsafePerformIO
Koen Based on this info, how would you like to write the notes you would like to see on unsafePerformIO? The starting point is at http://haskell.cs.yale.edu/ghc/docs/latest/html/base/GHC.IOBase.html#uns afePerformIO If you write the notes, we'll check their veracity and add them. We'll do the formatting too! Simon | -Original Message- | From: Simon Marlow [mailto:[EMAIL PROTECTED]] | Sent: 24 September 2002 13:59 | To: Koen Claessen; [EMAIL PROTECTED] | Subject: RE: unsafePerformIO | | But it is a fact that many of us have at least some idea of | what happens under the hood when we use unsafePerformIO. | This is also described in your paper Stretching the storage | manager: weak pointers and stable names in Haskell. | | However, I for example have no idea what happens when | unsafely executing something that throws exceptions, | performs a forkIO, something that uses MVar's, etc. | | Exceptions: an exception raised by the computation inside | unsafePerformIO will be propagated to the enclosing expression, as per | the normal semantics for imprecise exceptions. The exception raised by | an application of unsafePerformIO may of course also be imprecise; | consider 'unsafePerformIO $ (1/0 + error foo) `seq` return ()'. | | The behaviour of forkIO and other side-effecting operations inside | unsafePerforIO can I think be explained by the following statement: if | an expression unsafePerformIO e has been reduced to head normal form, | then all the I/O computations in e have been performed. The exact | timing of the evaluation is underspecified, just as any other expression | in Haskell. However, if you're using unsafePerformIO with real side | effects, then I suspect that what you're doing is highly dodgy and you | should find another way to do it :-) | | Does that help? | | As for sharing, we currently don't provide any guarnatees, although we | should. It is currently the case that if you write | | a = unsafePerformIO (putStr hello) | b = unsafePerformIO (putStr hello) | | and both a and b are evaluated, then you may get either one or two | hellos on stdout. You can currently make things more deterministic by | (a) adding NOINLINE pragmas for a and b, and (b) using the flag -fno-cse | to disable common sub-expression elimination. Then you'll get exactly | two instances of hello on stdout, although we won't guarantee that | behaviour for ever. At some point we'll fix it so that unsafePerformIO | applications are never duplicated or coalesced. | | If you're interested in when unsafePerformIO is safe to use, see: | | http://www.haskell.org/pipermail/glasgow-haskell-users/2002-July/003683. | html | | (and see my followups for an alternative view). | | Cheers, | Simon | ___ | Glasgow-haskell-users mailing list | [EMAIL PROTECTED] | http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: unsafePerformIO
Koen Claessen [EMAIL PROTECTED] writes: However, I for example have no idea what happens when unsafely executing something that throws exceptions, performs a forkIO, something that uses MVar's, etc. I won't dare to try to characterize the difference exactly but you should expect very different behaviour between Hugs and GHC when using unsafePerformIO with threads and exceptions. The Hugs version of unsafePerformIO isn't intended to receive as much abuse as the GHC version whereas the GHC version has been (ab)used by large numbers of people. -- Alastair ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
unsafePerformIO
In read your paper Tackling the Awkward Squad: monadic input / output, concurrency, exceptions, and foreign-language calls in Haskell, and have a question about unsafePerformIO. In your operational semantic of the IO-Monad you tell nothing about, how 'unsafe' IO actions are performed, is there another paper / documentation about this available, or can you - or someone else - give me a review about that? David JWGU Frankfurt ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: unsafePerformIO
I'm not sure of a reference, the basic idea is this: IO a is represented by the pair (RealWorld, a) (unboxed, really but whatever) When an IO action is run (via main), a RealWorld state is provided by the compiler and passes it around. When you do unsafePerformIO, the compiler conjures up some value of type RealWorld to use. It is unsafe because it doesn't guarentee the relative order of IO actions. Of course, someone correct me if I'm mistaken. - Hal -- Hal Daume III Computer science is no more about computers| [EMAIL PROTECTED] than astronomy is about telescopes. -Dijkstra | www.isi.edu/~hdaume On Fri, 20 Sep 2002, David Sabel wrote: In read your paper Tackling the Awkward Squad: monadic input / output, concurrency, exceptions, and foreign-language calls in Haskell, and have a question about unsafePerformIO. In your operational semantic of the IO-Monad you tell nothing about, how 'unsafe' IO actions are performed, is there another paper / documentation about this available, or can you - or someone else - give me a review about that? David JWGU Frankfurt ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
RE: unsafePerformIO around FFI calls
Hal Daume [EMAIL PROTECTED] writes: I'm curious exactly what is safe and what is unsafe to wrap unsafePerformIO around when it comes to FFI calls. Here's a simple test: Could you imagine an alternative implementation of the same API in pure Haskell? (Don't consider efficiency or effort required to write the implementation, just whether it can be done.) If so, then it is ok to use unsafePerformIO and the ffi to implement the API instead. If it fails that test, it is incredibly unlikely that it is ok and a proof that it is ok is likely to be pretty complex - maybe worth a PLDI paper or some such. That's a nice succinct way to describe it. Another way, which boils down to the same thing but which is a little more concrete, is to ask: - Does the function's result depend only on the values of its arguments? (obviously only makes sense for a top-level IO function which you want to wrap in unsafePerformIO - for a non-top-level function or expression just replace 'arguments' with 'arguments and free variables'). Cheers, Simon ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: unsafePerformIO around FFI calls
That's a nice succinct way to describe it. Another way, which boils down to the same thing but which is a little more concrete, is to ask: - Does the function's result depend only on the values of its arguments? I have two problems with this alternative test: 1) It is sometimes slightly stricter test than necessary. Consider a hypothetical pair of C functions Foo toFoo(int); int fromFoo(Foo); which satisfy the property fromFoo(toFoo(x)) == x but such that the result of toFoo does not depend on its argument. (Perhaps toFoo allocates some memory in which to stores its result and returns a pointer to that memory.) The function's result does vary independently of its values so it fails your test. But if toFoo/fromFoo are the only functions on Foo, then we could obviously have implemented the same API in Haskell with the aid of newtype so it passes my test. 2) It fails to recognise the fact that IO actions have side effects. For example, the C library function 'free' always returns the same result (i.e., '()') but it's a bad idea to call free twice on the same argument. One could argue that the side effect is part of the result because IO actions return a modified world but only a long term functional programmer would think like that so it doesn't help the man in the street. (In fact, IIRC, the Concurrent Haskell semantics doesn't use the state-passing explanation so you don't even see side effects reflected as changes in the returned world state.) -- Alastair Reid [EMAIL PROTECTED] Reid Consulting (UK) Limited http://www.reid-consulting-uk.ltd.uk/alastair/ ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
RE: unsafePerformIO around FFI calls
That's a nice succinct way to describe it. Another way, which boils down to the same thing but which is a little more concrete, is to ask: - Does the function's result depend only on the values of its arguments? I have two problems with this alternative test: 1) It is sometimes slightly stricter test than necessary. Consider a hypothetical pair of C functions Foo toFoo(int); int fromFoo(Foo); which satisfy the property fromFoo(toFoo(x)) == x but such that the result of toFoo does not depend on its argument. (Perhaps toFoo allocates some memory in which to stores its result and returns a pointer to that memory.) The function's result does vary independently of its values so it fails your test. But if toFoo/fromFoo are the only functions on Foo, then we could obviously have implemented the same API in Haskell with the aid of newtype so it passes my test. Ok, if we're going to nit pick :) When talking about equality you have to restrict that to observable equivalence in the context of whatever abstract types you're dealing with. If a function always returns observably equivalent results when given observably equivalent arguments, then it is safe. For example, toFoo would not be safe if you can test for equality between pointers. But if all you can do is convert it back using fromFoo, then you're ok. 2) It fails to recognise the fact that IO actions have side effects. Well, side effects are usually only visible in the IO monad so that's ok. In your free() example, the result is either () or _|_, so the function does depend on more than just the value of the argument. I think the definition holds (but only just). For example, we normally consider a function which uses some temporary allocation on the C heap as safe to use from unsafePerformIO. But its side effect is visible from the IO monad by inspecting the C heap pointer. Cheers, Simon ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
unsafePerformIO around FFI calls
I'm curious exactly what is safe and what is unsafe to wrap unsafePerformIO around when it comes to FFI calls. If there's a general discussion of this somewhere and someone could send me a pointer that would be another acceptable solution. I googled for unsafePerformIO FFI but nothing relevant turned up. Anyway, here's the issue at hand. I have a function which takes a value and an IO action, allocates some memory, put the value in there, executes the IO action (which is an FFI function), gets the result, deallocates the memory and then returns the rest. For instance, we have an FFI function from LAPack which computes, say, eigenvalues. Because this is from fortran and in fortran everything is by reference, the function is an IO action of (simplified) type: maxEigenvalue :: HMatrix - IO Double implemented using FFI. We also have the CPS function to make the HMatrix: withMatrix :: [[Double]] - (HMatrix - IO Double) - IO Double which looks basically like (I'm on vacation so I don't have the real code right now): withHMatrix m action = do memory - allocate memory hm - write m into memory result - action hm deallocate memory return result (it's slightly more complex because i need to make it safe in the case that action throws an exception and the memory is still deallocated before the exception is re-raised.) i can then wrap these together and say: compEig :: [[Double]] - IO Double compEig m = withHMatrix m maxEigenvalue so the question is: Under what circumstances is it safe to make instead: compEig' :: [[Double]] - Double compEig' m = unsafePerformIO $ withHMatrix m maxEigenvalue ??? Thanks! - Hal p.s., Please continue to CC Carl as this issue came up in conversations with him -- Hal Daume III Computer science is no more about computers| [EMAIL PROTECTED] than astronomy is about telescopes. -Dijkstra | www.isi.edu/~hdaume ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
Re: unsafePerformIO around FFI calls
Hal Daume [EMAIL PROTECTED] writes: I'm curious exactly what is safe and what is unsafe to wrap unsafePerformIO around when it comes to FFI calls. Here's a simple test: Could you imagine an alternative implementation of the same API in pure Haskell? (Don't consider efficiency or effort required to write the implementation, just whether it can be done.) If so, then it is ok to use unsafePerformIO and the ffi to implement the API instead. If it fails that test, it is incredibly unlikely that it is ok and a proof that it is ok is likely to be pretty complex - maybe worth a PLDI paper or some such. -- Alastair Reid [EMAIL PROTECTED] Reid Consulting (UK) Limited http://www.reid-consulting-uk.ltd.uk/alastair/ ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
RE: unIO vs. unsafePerformIO
What's the difference between unIO and unsafePerformIO? And why is the former safe? (I would like to apply the same questions to unsafeIOToST and ioToST) unIO is an internal function used in GHC's libraries, it is not for external consumption. GHC represents the IO type using newtype, and the unIO function just removes the newtype constructor. It doesn't actually perform the IO action like unsafePerformIO does. Cheers, Simon ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users
unIO vs. unsafePerformIO
What's the difference between unIO and unsafePerformIO? And why is the former safe? (I would like to apply the same questions to unsafeIOToST and ioToST) Thanks, -- Andre ___ Glasgow-haskell-users mailing list [EMAIL PROTECTED] http://www.haskell.org/mailman/listinfo/glasgow-haskell-users