[Haskell-cafe] createProcess interferes with sockets?
I've had a strange bug that's baffled me for a long time. I finally got serious about tracking it down, and managed to reduce it to a small program that exhibits the unexpected behaviour, namely that a createProcess seems to block writing to and closing a socket. Here's the example program: --- import Control.Monad import qualified Network import qualified System.Environment as Environment import qualified System.IO as IO import qualified System.Process as Process main :: IO () main = Network.withSocketsDo $ do args - Environment.getArgs case args of [server] - server [client] - client _ - error $ show args server :: IO () server = do socket - Network.listenOn port forever $ do putStrLn accept (hdl, _host, _port) - Network.accept socket msg - IO.hGetLine hdl putStrLn $ from client: ++ show msg sleep putStrLn send response IO.hPutStr hdl response IO.hClose hdl client :: IO () client = do hdl - Network.connectTo localhost port IO.hPutStr hdl hi from client\n IO.hFlush hdl resp - IO.hGetContents hdl print resp port = Network.UnixSocket port sleep = Process.createProcess (Process.proc sleep [5]) --- You can test with: % ghc --make Test.hs rm -f port ./Test server then on another window: % ./Test client Since the createProcess is async (it doesn't wait on the pid), I expect the response to come back to the client immediately. And the server immediately says send response then accept, so it doesn't get stuck on the sleep. But the client waits 5 seconds before displaying response, so evidently even though the handle has already been written to and closed from the server, the client waits until the subprocess (of the server!) completes before getting the response. Comment out the sleep line and everything is fast. Can anyone else repro this? I'm guessing it has to do with the ghc IO manager, and the fork implied by a createProcess is causing it the socket close to block, but I haven't dug any deeper yet, in case this is a know issue, or I'm just doing something wrong. This is OS X 10.8.3. thanks! ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Fwd: GSoC Project Proposal: Markdown support for Haddock
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 09/04/13 21:50, Johan Tibell wrote: On Tue, Apr 9, 2013 at 12:40 PM, Joe Nash joen...@blackvine.co.uk wrote: I would be interested in discussing this project with a potential mentor if one happens to be reading. I'm a second year Computer Science student at the University of Nottingham, very interested in doing a haskell.org SoC project. Normally I would volunteer to mentor any project I propose, but my schedule doesn't allow for it this summer. Perhaps Mark Lentczner is interested. He did some Haddock work before. I'm happy to provide input on what I think the feature should be about (in fact, I will probably write a blog post about it, like I do every year before GSoC). -- Johan ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe Have you written the post? I'm interested in applying with this as a project and it would be nice to be able to have a read beforehand. - -- Mateusz K. -BEGIN PGP SIGNATURE- Version: GnuPG v2.0.19 (GNU/Linux) iQIcBAEBAgAGBQJRc/bJAAoJEM1mucMq2pqXzO4P/jET1Khe/H+rOWe7HVdGrLbq x+fPETfhhFZGatRtsKm2HaO8CgLOAyZcV21x5gCRgdoNhAQb9cXeYKRRKEaQYIit joiOGLtn5t45RYt+7d/C3/45CpjaukI2BJ819lLnFHNTebiGEi3UhANzdp6W2uJJ K34tucRxChUL+Th7RzgyvkkgdiVbA8/arDhTCHLX1mQ1DQVm3T0y7M4GxzAc6OqV bb15QA0tH/3/rKGnVc3y+po7EHs+oO5H4XKGvibbdNH0fz8M0XzxnK8oZ0SU2eYE PyELmfI3z7jlKVhnOy+LPw+7+mMP2Ju/lFCWB4S+aotEjPFb4VszUCPwrvqxQsrB nZ2Ekt76IBV58och/lPFG138C1pFwVG+HVE3JMRnWPC271meTiyMtMz9ZMEkDRVL Hq9Djb7IUxgp0G8Z676U4jFH+PzmVVxXi39JIO05aTSxfrJAlYbpRUX5sizCiMv0 xi6osV8PQHurMXIm04CpRFcrqpdiMo9jpIpZyntBTEmJAiIZR7sTIYoqdaI0UmNl bQGdr1fqm5EZm8Iz8x99BQjJrbGrL6UT0qTFmaWJ+o8HPu5zW+ekbWa7G8yMCiq+ tTnN4kH0uVZwwzG5tcM70RUg2bsg3bJSMgfE8Qf340d9lfALeMyPiGoQ+o5CDJ1l hJXAjpXFaP8tWPyW5Kld =syxC -END PGP SIGNATURE- ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] createProcess interferes with sockets?
Quoth Evan Laforge qdun...@gmail.com,
sleep = Process.createProcess (Process.proc sleep [5])
sleep = Process.createProcess
((Process.proc sleep [5]) {Process.close_fds = True})
- Because the client uses buffered I/O (hGetContents in this case, but
hGet-anything would be the same), it doesn't see the server response
until a) buffer full, or b) end of file (server closes connection.)
- The server does close the connection, but after the sleep process
has forked off with a copy of the connection fd. If it doesn't close
that fd explicitly, it holds it open until process exit (5 seconds.)
Donn
___
Haskell-Cafe mailing list
Haskell-Cafe@haskell.org
http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Two GHC-related GSoC Proposals
Hi, I'm interested in participating in the GSoC by improving GHC with one of these two features: 1) Implement native support for compiling modules in parallel (see #910http://hackage.haskell.org/trac/ghc/ticket/910). This will involve making the compilation pipeline thread-safe, implementing the logic for building modules in parallel (with an emphasis on keeping compiler output deterministic), and lots of testing and benchmarking. Being able to seamlessly build modules in parallel will shorten the time it takes to recompile a project and will therefore improve the life of every GHC user. 2) Improve existing constant folding, strength reduction and peephole optimizations on arithmetic and logical expressions, and optionally implement a core-to-core pass for optimizing nested comparisons (relevant tickets include #2132 http://hackage.haskell.org/trac/ghc/ticket/2132, #5615 http://hackage.haskell.org/trac/ghc/ticket/5615,#4101http://hackage.haskell.org/trac/ghc/ticket/4101). GHC currently performs some of these simplifications (via its BuiltinRule framework), but there is a lot of room for improvement. For instance, the core for this snippet is essentially identical to the Haskell source: foo :: Int - Int - Int - Int foo a b c = 10*((b+7+a+12+b+9)+4) + 5*(a+7+b+12+a+9) + 7 + b + c Yet the RHS is actually equivalent to 20*a + 26*b + c + 467 And: bar :: Int - Int - Int bar a b = a + b - a - b -- the RHS is should be optimized away to 0 Other optimizations include: multiplication and division by powers of two should be converted to shifts; multiple plusAddr calls with constant operands should be coalesced into a single plusAddr call; floating point functions should be constant folded, etc.. GHC should be able to perform all these algebraic simplifications. Of course, emphasis should be placed on the correctness of such transformations. A flag for performing unsafe optimizations like assuming floating point arithmetic is associative and distributive should be added. This proposal will benefit anybody writing or using numerically intensive code. I'm wondering what the community thinks of these projects. Which project is a better fit for GSoC, or are both a good fit? Is a mentor willing to supervise one of these projects? Thanks for your time. Patrick (A little about myself: I'm a Mathematics student in the US, and I've been programming in Haskell for about 3.5 years. Having a keen interest in Haskell and compilers, I began studying the GHC source about 1 year ago and I've since gotten a good understanding of its internals, contributing a few patches along the way.) ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Two GHC-related GSoC Proposals
Hi, I think the first proposal may be a bit too much for a GSoC, depending on how much you actually are familiar with the code base. If you can write down everything that needs to be done, in a detailed way (I mean a *lot* of details), for each of these steps, and if you sincerely consider all of these stuffs can be done in 3 months, yeah, that would be great! For example, making the compilation pipeline thread safe may end up being trickier than expected if it's not studied properly before making a proposal. The latter is a good idea, and a good proposal would ideally include some estimation on how it can impact some benchmarks/projects. It looks much less like a trap and if you propose enough improvements that it can fill a whole GSoC, considering how big the impact can be, yes, this would of course be a good idea. On Sun, Apr 21, 2013 at 6:20 PM, Patrick Palka patr...@parcs.ath.cx wrote: Hi, I'm interested in participating in the GSoC by improving GHC with one of these two features: 1) Implement native support for compiling modules in parallel (see #910http://hackage.haskell.org/trac/ghc/ticket/910). This will involve making the compilation pipeline thread-safe, implementing the logic for building modules in parallel (with an emphasis on keeping compiler output deterministic), and lots of testing and benchmarking. Being able to seamlessly build modules in parallel will shorten the time it takes to recompile a project and will therefore improve the life of every GHC user. 2) Improve existing constant folding, strength reduction and peephole optimizations on arithmetic and logical expressions, and optionally implement a core-to-core pass for optimizing nested comparisons (relevant tickets include #2132 http://hackage.haskell.org/trac/ghc/ticket/2132, #5615 http://hackage.haskell.org/trac/ghc/ticket/5615,#4101http://hackage.haskell.org/trac/ghc/ticket/4101). GHC currently performs some of these simplifications (via its BuiltinRule framework), but there is a lot of room for improvement. For instance, the core for this snippet is essentially identical to the Haskell source: foo :: Int - Int - Int - Int foo a b c = 10*((b+7+a+12+b+9)+4) + 5*(a+7+b+12+a+9) + 7 + b + c Yet the RHS is actually equivalent to 20*a + 26*b + c + 467 And: bar :: Int - Int - Int bar a b = a + b - a - b -- the RHS is should be optimized away to 0 Other optimizations include: multiplication and division by powers of two should be converted to shifts; multiple plusAddr calls with constant operands should be coalesced into a single plusAddr call; floating point functions should be constant folded, etc.. GHC should be able to perform all these algebraic simplifications. Of course, emphasis should be placed on the correctness of such transformations. A flag for performing unsafe optimizations like assuming floating point arithmetic is associative and distributive should be added. This proposal will benefit anybody writing or using numerically intensive code. I'm wondering what the community thinks of these projects. Which project is a better fit for GSoC, or are both a good fit? Is a mentor willing to supervise one of these projects? Thanks for your time. Patrick (A little about myself: I'm a Mathematics student in the US, and I've been programming in Haskell for about 3.5 years. Having a keen interest in Haskell and compilers, I began studying the GHC source about 1 year ago and I've since gotten a good understanding of its internals, contributing a few patches along the way.) ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe -- Alp Mestanogullari ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Implementing an embedded language that threads a global structure.
Hi, I've been working on this problem for some time and I was hoping for some feedback. I'm trying to implement an embedded language that can thread a global structure through the evaluation of a term. A mini version of this language only includes three functions, lam, app and ext (lambda, apply and extension respectively). The only difference between this and other embedded languages is that terms embedded in an extension have access to a global variable that is invisible to the terms themselves. The writer of a term in this language can only access the global variable by applying an extension. The writer extensions insures that the effect of evaluation on the global variable is commutative and insures that the effect of the global variable on terms is constant regardless of the order of evaluation. For example the global variable could be a map with some sort of lazy processing. An extension that queries the map for an element might cause a change in the state of the map and return some information about an element, but as long as the change in state are commutative and the same information is returned about elements regardless of the order of evaluation then the concept is safe. My motivation for experimenting with such a language comes from a belief that certain complicated programs would be easier to express in such a language and that there might be very useful code transformations that could eventually be applied to such a language that would be possible because of the constraints on the behavior of extensions. However I have yet to properly implement a language in Haskell and so I decided to post what I have and look for comments. Basically the language takes a common form of embedded languages; a lambda expression such as \x y-x y would have the form: lam (\x - lam (\y - app x y)) however in order to evaluate an expression the need to also apply it to a global term: lam (\x - lam (\y - app x y)) global Ideally the result of evaluating this would be a (global', \x y- x y), a modified version of global paired with the term. So the type of the term: lam (\x - lam (\y - app x y)) is something like g - (g, (a - b) - a - b) where g is the type of global. From that came the first idea of the types of the evaluator functions: lam :: ( (g-(g,a)) - (g-(g,b)) )- (g-(g, a-b)) app :: (g-(g,a-b)) - (g-(g, a)) - (g-(g,b)) with some additional parenthesis for sake of clarity. Each sub-term has the type g-(g,t) any function can be included as an extension as long as the function has the type: (g-(g,a)) - (g-(g,b)) This seems to work well except that it seems to be impossible (as far as I've tried) to construct a definition for the lam function that both fits this type and properly passes the global through the entire evaluation. For example it's easy to define app like this: app :: (g-(g,a-b)) - (g-(g,a)) - (g-(g,b)) app = (\ eA eB - \ g0 - let (gB, fB) = eB g0 in let (gA, fA) = eA gB in (gA, fA fB) ) but so far the definition of lam has eluded me. This is the closest I've come: lam :: ((g-(g,a)) - (g-(g,b))) - (g-(g,a-b)) lam = (\ f - \ g0 - (g0, \x - snd $ (f (\ gv-(gv,x))) g0 )) This fits the type but I fear this definition does not return the properly modified version of global. I tried some other iterations of this idea. In one effort the first argument of a function is extracted into the type of a term. So a term of type a-b has the type (g-a-(g,b)). And a term of type a has the type (g-((g, a)) such that: lam2 :: ((g-(g,a)) - (g-(g,b))) - (g-a-(g,b)) lam2 = (\ f - \ g x - (f (\ gv - (gv, x))) g) app2 :: (g-a-(g,b)) - (g-(g,a)) - (g-(g,b)) app2 = (\ eA eB - \ g0 - let (gB, fB) = eB g0 in eA gB fB ) This seems like a step in the right direction unfortunately because the return type of lam is now (g-a-(g,b)) and not (g-(g-c)) we cannot type the term lam (\x - lam (\y - app x y)). In other words this language can only express unary and nullary functions. Just to help clarify this for myself I created to addition functions lam2Helper and app2Helper: lam2Helper :: (g-a-(g,b)) - (g-(g,a-b)) lam2Helper = (\f - \g - (g, \a - snd $ f g a)) app2Helper :: (g-(g,a-b)) - (g-a-(g,b)) app2Helper = (\f - \g a- let (g1, f1) = f g in (g1, f1 a)) these allow me to create the term: lam2 (\x - lam2Helper $ lam2 (\y - app2 (app2Helper x) y)) but just like the original definition of lam from my first try, there is no way to construct lam2Helper without improperly passing the global as I've done in my attempt. Finally on my third try I attempted to make every term have type (g-a-(g,b) by allowing embedded nullary functions to have the type (g-()-(g,t)) lam3 :: ((g-()-(g,a)) - (g-()-(g,b))) - (g-a-(g,b)) lam3 = (\ f - \ g x - (f (\ gv () - (gv, x))) g () ) app3 :: (g-a-(g,b)) - (g-()-(g,a)) - (g-()-(g,b)) app3 = (\ eA eB - \ gz () - let (gB, fB) = eB gz
Re: [Haskell-cafe] Two GHC-related GSoC Proposals
Hey Patrick, both are excellent ideas for work that would be really valuable for the community! (independent of whether or not they can be made into GSOC sided chunks ) --- I'm actually hoping to invest some time this summer investigating improving the numerics optimization story in ghc. This is in large part because I'm writing LOTs of numeric codes in haskell presently (hopefully on track to make some available to the community ). That said, its not entirely obvious (at least to me) what a tractable focused GSOC sized subset of the numerics optimization improvement would be, and that would have to also be a subset that has real performance impact and doesn't benefit from eg using -fllvm rather than -fasm . - For helping pave the way to better parallel builds, what are some self contained units of work on ghc (or related work on cabal) that might be tractable over a summer? If you can put together a clear roadmap of work chunks that are tractable over the course of the summer, I'd favor choosing that work, especially if you can give a clear outline of the plan per chunk and how to evaluate the success of each unit of work. basically: while both are high value projects, helping improve the parallel build tooling (whether in performance or correctness or both!) has a more obvious scope of community impact, and if you can layout a clear plan of work that GHC folks agree with and seems doable, i'd favor that project :) hope this feedback helps you sort out project ideas cheers -Carter On Sun, Apr 21, 2013 at 12:20 PM, Patrick Palka patr...@parcs.ath.cxwrote: Hi, I'm interested in participating in the GSoC by improving GHC with one of these two features: 1) Implement native support for compiling modules in parallel (see #910http://hackage.haskell.org/trac/ghc/ticket/910). This will involve making the compilation pipeline thread-safe, implementing the logic for building modules in parallel (with an emphasis on keeping compiler output deterministic), and lots of testing and benchmarking. Being able to seamlessly build modules in parallel will shorten the time it takes to recompile a project and will therefore improve the life of every GHC user. 2) Improve existing constant folding, strength reduction and peephole optimizations on arithmetic and logical expressions, and optionally implement a core-to-core pass for optimizing nested comparisons (relevant tickets include #2132 http://hackage.haskell.org/trac/ghc/ticket/2132, #5615 http://hackage.haskell.org/trac/ghc/ticket/5615,#4101http://hackage.haskell.org/trac/ghc/ticket/4101). GHC currently performs some of these simplifications (via its BuiltinRule framework), but there is a lot of room for improvement. For instance, the core for this snippet is essentially identical to the Haskell source: foo :: Int - Int - Int - Int foo a b c = 10*((b+7+a+12+b+9)+4) + 5*(a+7+b+12+a+9) + 7 + b + c Yet the RHS is actually equivalent to 20*a + 26*b + c + 467 And: bar :: Int - Int - Int bar a b = a + b - a - b -- the RHS is should be optimized away to 0 Other optimizations include: multiplication and division by powers of two should be converted to shifts; multiple plusAddr calls with constant operands should be coalesced into a single plusAddr call; floating point functions should be constant folded, etc.. GHC should be able to perform all these algebraic simplifications. Of course, emphasis should be placed on the correctness of such transformations. A flag for performing unsafe optimizations like assuming floating point arithmetic is associative and distributive should be added. This proposal will benefit anybody writing or using numerically intensive code. I'm wondering what the community thinks of these projects. Which project is a better fit for GSoC, or are both a good fit? Is a mentor willing to supervise one of these projects? Thanks for your time. Patrick (A little about myself: I'm a Mathematics student in the US, and I've been programming in Haskell for about 3.5 years. Having a keen interest in Haskell and compilers, I began studying the GHC source about 1 year ago and I've since gotten a good understanding of its internals, contributing a few patches along the way.) ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Yi Google summer of code project
Hello, I'm interested in doing some improvments on the Yi editor for my Google summer of code by implementing a completion pop up list for any buffer and a smart completion for haskell. But I don't really know if this kind of project could be acceptable, for the following reason: 1) Should a GSoC project be mostly a working experience or also a learning one? I am not the most seasoned haskell programmer, but I really which to dive into a real-life project in this language. 2) Is it too long, too small? I don't have too many years of programmation in haskell behind me even if I have done some functionnal programming in other languages (Scala most of the time), hence I don't want to take something that will be overwhelming. 3) is Yi associated with haskell.org or it is not possible to have a project on Yi while being mentored by haskell? Thanks for your insight! Regards, Bastien Jacot-Guillarmod, 3rd CS bachelor student at EPFL (Lausanne) -- View this message in context: http://haskell.1045720.n5.nabble.com/Yi-Google-summer-of-code-project-tp5729090.html Sent from the Haskell - Haskell-Cafe mailing list archive at Nabble.com. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
[Haskell-cafe] Stream fusion and span/break/group/init/tails
Hello all, (cc'd stream fusion paper authors) I noticed that the current implementation of stream fusion does not support multiple-return stream combinators, e.g. break :: (a - Bool) - [a] - ([a], [a]). I thought a little bit about how might one go about implement this, but the problem seems nontrivial. (One possibility is to extend the definition of Step to support multiple return, but the details are a mess!) Nor, as far as I can tell, does the paper give any treatment of the subject. Has anyone thought about this subject in some detail? Thanks, Edward ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Stream fusion and span/break/group/init/tails
On 22/04/2013, at 11:07 , Edward Z. Yang ezy...@mit.edu wrote: Hello all, (cc'd stream fusion paper authors) I noticed that the current implementation of stream fusion does not support multiple-return stream combinators, e.g. break :: (a - Bool) - [a] - ([a], [a]). I thought a little bit about how might one go about implement this, but the problem seems nontrivial. (One possibility is to extend the definition of Step to support multiple return, but the details are a mess!) Nor, as far as I can tell, does the paper give any treatment of the subject. Has anyone thought about this subject in some detail? I've spent the last few months fighting this exact problem. The example you state is one instance of a more general limitation. Stream fusion (and most other short-cut fusion approaches) cannot fuse a producer into multiple consumers. The fusion systems don't support any unzip-like function, where elements from the input stream end up in multiple output streams. For example: unzip :: [(a, b)] - ([a], [b]) dup :: [a] - ([a], [a]) The general problem is that if elements of one output stream are demanded before the other, then the stream combinator must buffer elements until they are demanded by both outputs. John Hughes described this problem in his thesis, and gave an informal proof that it cannot be solved without some form of concurrency -- meaning the evaluation of the two consumers must be interleaved. I've got a solution for this problem and it will form the basis of Repa 4, which I'm hoping to finish a paper about for the upcoming Haskell Symposium. Ben. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Stream fusion and span/break/group/init/tails
I've got a solution for this problem and it will form the basis of Repa 4, which I'm hoping to finish a paper about for the upcoming Haskell Symposium. Sounds great! You should forward me a preprint when you have something in presentable shape. I suppose before then, I should look at repa-head/repa-stream to figure out what the details are? Edward ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Stream fusion and span/break/group/init/tails
On 22/04/2013, at 12:23 , Edward Z. Yang ezy...@mit.edu wrote: I've got a solution for this problem and it will form the basis of Repa 4, which I'm hoping to finish a paper about for the upcoming Haskell Symposium. Sounds great! You should forward me a preprint when you have something in presentable shape. I suppose before then, I should look at repa-head/repa-stream to figure out what the details are? The basic approach is already described in: Automatic Transformation of Series Expressions into Loops Richard Waters, TOPLAS 1991 The Anatomy of a Loop Olin Shivers, ICFP 2005 The contribution of the HS paper is planning to be: 1) How to extend the approach to the combinators we need for DPH 2) How to package it nicely into a Haskell library. I'm still working on the above... Ben. ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe
Re: [Haskell-cafe] Two GHC-related GSoC Proposals
GHC for the most part is already thread-safe. There are a few data structures that persist between module compilations, but they're mostly just caches that're already modified in an atomic fashion. Interleaving updates to these caches shouldn't cause any problems as long as the updates are done atomically. There is also a bit of top-level global state that needs attention, like the FastString table, the PersistentLinkerState and the unique-supply counter. The GHCi linker may need to be made thread-safe, and so will the interface loader. Deletion of temporary files will need to be handled more carefully. And.. that's about it as far as making the compiler thread safe goes, I think. The rest of the compilation pipeline just performs benign side effects, if any at all. Of course, unforeseen issues will pop up but I think I have enough baseline experience and familiarity with the GHC codebase to be able to complete the project within the given time frame. (Or maybe I'm just being naive -- there must be a good reason why it hasn't yet been done!) Thanks for your feedback. On Sun, Apr 21, 2013 at 4:35 PM, Alp Mestanogullari alpmes...@gmail.comwrote: Hi, I think the first proposal may be a bit too much for a GSoC, depending on how much you actually are familiar with the code base. If you can write down everything that needs to be done, in a detailed way (I mean a *lot* of details), for each of these steps, and if you sincerely consider all of these stuffs can be done in 3 months, yeah, that would be great! For example, making the compilation pipeline thread safe may end up being trickier than expected if it's not studied properly before making a proposal. The latter is a good idea, and a good proposal would ideally include some estimation on how it can impact some benchmarks/projects. It looks much less like a trap and if you propose enough improvements that it can fill a whole GSoC, considering how big the impact can be, yes, this would of course be a good idea. On Sun, Apr 21, 2013 at 6:20 PM, Patrick Palka patr...@parcs.ath.cxwrote: Hi, I'm interested in participating in the GSoC by improving GHC with one of these two features: 1) Implement native support for compiling modules in parallel (see #910http://hackage.haskell.org/trac/ghc/ticket/910). This will involve making the compilation pipeline thread-safe, implementing the logic for building modules in parallel (with an emphasis on keeping compiler output deterministic), and lots of testing and benchmarking. Being able to seamlessly build modules in parallel will shorten the time it takes to recompile a project and will therefore improve the life of every GHC user. 2) Improve existing constant folding, strength reduction and peephole optimizations on arithmetic and logical expressions, and optionally implement a core-to-core pass for optimizing nested comparisons (relevant tickets include #2132 http://hackage.haskell.org/trac/ghc/ticket/2132, #5615 http://hackage.haskell.org/trac/ghc/ticket/5615,#4101http://hackage.haskell.org/trac/ghc/ticket/4101). GHC currently performs some of these simplifications (via its BuiltinRule framework), but there is a lot of room for improvement. For instance, the core for this snippet is essentially identical to the Haskell source: foo :: Int - Int - Int - Int foo a b c = 10*((b+7+a+12+b+9)+4) + 5*(a+7+b+12+a+9) + 7 + b + c Yet the RHS is actually equivalent to 20*a + 26*b + c + 467 And: bar :: Int - Int - Int bar a b = a + b - a - b -- the RHS is should be optimized away to 0 Other optimizations include: multiplication and division by powers of two should be converted to shifts; multiple plusAddr calls with constant operands should be coalesced into a single plusAddr call; floating point functions should be constant folded, etc.. GHC should be able to perform all these algebraic simplifications. Of course, emphasis should be placed on the correctness of such transformations. A flag for performing unsafe optimizations like assuming floating point arithmetic is associative and distributive should be added. This proposal will benefit anybody writing or using numerically intensive code. I'm wondering what the community thinks of these projects. Which project is a better fit for GSoC, or are both a good fit? Is a mentor willing to supervise one of these projects? Thanks for your time. Patrick (A little about myself: I'm a Mathematics student in the US, and I've been programming in Haskell for about 3.5 years. Having a keen interest in Haskell and compilers, I began studying the GHC source about 1 year ago and I've since gotten a good understanding of its internals, contributing a few patches along the way.) ___ Haskell-Cafe mailing list Haskell-Cafe@haskell.org http://www.haskell.org/mailman/listinfo/haskell-cafe -- Alp Mestanogullari ___
Re: [Haskell-cafe] Two GHC-related GSoC Proposals
Good points. I did not take into account whether proposal #2 may be worth it in light of -fllvm. I suppose that even if the LLVM codegen is able to perform similar optimizations, it would still be beneficial to implement proposal #2 as a core-to-core pass because the transformations it performs would expose new information to subsequent core-to-core passes. Also, Haskell has different overflow rules than C does (whose rules I assume LLVM's optimizations are modeled from): In Haskell, integer overflow is undefined for all integral types, whereas in C it's only undefined for signed integral types. This limits the number of optimizations a C-based optimizer can perform on unsigned arithmetic. I'm not sure how I would break up the parallel compilation proposal into multiple self-contained units of work. I can only think of two units: making GHC thread safe, and writing the new parallel compilation driver. Other incidental units may come up during development (e.g. parallel compilation might exacerbate #4012http://hackage.haskell.org/trac/ghc/ticket/4012), but I still feel that three months of full time work is ample time to complete the project, especially with existing familiarity with the code base. Thanks for the feedback. On Sun, Apr 21, 2013 at 5:55 PM, Carter Schonwald carter.schonw...@gmail.com wrote: Hey Patrick, both are excellent ideas for work that would be really valuable for the community! (independent of whether or not they can be made into GSOC sided chunks ) --- I'm actually hoping to invest some time this summer investigating improving the numerics optimization story in ghc. This is in large part because I'm writing LOTs of numeric codes in haskell presently (hopefully on track to make some available to the community ). That said, its not entirely obvious (at least to me) what a tractable focused GSOC sized subset of the numerics optimization improvement would be, and that would have to also be a subset that has real performance impact and doesn't benefit from eg using -fllvm rather than -fasm . - For helping pave the way to better parallel builds, what are some self contained units of work on ghc (or related work on cabal) that might be tractable over a summer? If you can put together a clear roadmap of work chunks that are tractable over the course of the summer, I'd favor choosing that work, especially if you can give a clear outline of the plan per chunk and how to evaluate the success of each unit of work. basically: while both are high value projects, helping improve the parallel build tooling (whether in performance or correctness or both!) has a more obvious scope of community impact, and if you can layout a clear plan of work that GHC folks agree with and seems doable, i'd favor that project :) hope this feedback helps you sort out project ideas cheers -Carter On Sun, Apr 21, 2013 at 12:20 PM, Patrick Palka patr...@parcs.ath.cxwrote: Hi, I'm interested in participating in the GSoC by improving GHC with one of these two features: 1) Implement native support for compiling modules in parallel (see #910http://hackage.haskell.org/trac/ghc/ticket/910). This will involve making the compilation pipeline thread-safe, implementing the logic for building modules in parallel (with an emphasis on keeping compiler output deterministic), and lots of testing and benchmarking. Being able to seamlessly build modules in parallel will shorten the time it takes to recompile a project and will therefore improve the life of every GHC user. 2) Improve existing constant folding, strength reduction and peephole optimizations on arithmetic and logical expressions, and optionally implement a core-to-core pass for optimizing nested comparisons (relevant tickets include #2132 http://hackage.haskell.org/trac/ghc/ticket/2132, #5615 http://hackage.haskell.org/trac/ghc/ticket/5615,#4101http://hackage.haskell.org/trac/ghc/ticket/4101). GHC currently performs some of these simplifications (via its BuiltinRule framework), but there is a lot of room for improvement. For instance, the core for this snippet is essentially identical to the Haskell source: foo :: Int - Int - Int - Int foo a b c = 10*((b+7+a+12+b+9)+4) + 5*(a+7+b+12+a+9) + 7 + b + c Yet the RHS is actually equivalent to 20*a + 26*b + c + 467 And: bar :: Int - Int - Int bar a b = a + b - a - b -- the RHS is should be optimized away to 0 Other optimizations include: multiplication and division by powers of two should be converted to shifts; multiple plusAddr calls with constant operands should be coalesced into a single plusAddr call; floating point functions should be constant folded, etc.. GHC should be able to perform all these algebraic simplifications. Of course, emphasis should be placed on the correctness of such transformations. A flag for performing unsafe optimizations like assuming floating point arithmetic is associative and distributive should
