To tie the knot I summarised this as a blog post: http://neilmitchell.blogspot.co.uk/2016/07/why-did-stack-stop-using-shake.html
Thanks, Neil On Mon, Jul 11, 2016 at 8:24 PM, Neil Mitchell <[email protected]> wrote: > Very interesting comments from all. > > In my opinion, Shake excels at a few domains. > > * If you have lots of complex interdependencies, Shake lets you manage > them nicely. That's not really the case here, but is in large > heterogeneous build systems, e.g. the GHC build system. > > * If you are writing things quickly, Shake lets you manage > exceptions/retries/robustness quickly. For a project which has the > effort invested that Stack does, that's less important, but for things > like MinGHC (something Stack killed), it was critically important > because no one cared enough to do all this nasty engineering. > > * If you are experimenting, Shake provides a lot of pieces (resources, > parallelism, storage) that help explore the problem space without > having to do lots of work at each iteration. That might mean Shake is > more of a benefit at the start of a project than in a mature project. > > FWIW, the concurrent execution stuff in the middle of Stack actually > looks fairly hairy! I see a dependency based build system, with > features like staunch mode and parallelism. It's almost certainly an > O(n^2) algorithm in several places, although I suspect n is typically > small enough. I guess it assumes topological sorting and no dead links > - but I would assume all those are guaranteed in advance. esFinalLock > gives me a gentle shiver. esInAction looks like a space leak, unless > forced by actionDo. ActionContext worries me a bit, is it meant to > list all actions that aren't this one? That seems a very weird > argument. > > All those comments about Execute.hs are not intended to persuade you > to move to Shake, but I do see plenty of resemblances of Shake in > there. In many ways you have written a mini-Shake. > > Thanks, Neil > > On Mon, Jul 11, 2016 at 6:27 PM, Edward Z. Yang <[email protected]> wrote: >> No Dan, I think you're exactly right. >> >> I have a similar feeling when working with the cabal-install codebase. >> Without an API like Shake, you manually construct and run the graph of >> operations you want to do. Maybe it is simple, but there are so many >> other things which you could be caching / parallelizing more aggressively. >> >> Edward >> >> Excerpts from Dan Burton's message of 2016-07-11 10:20:13 -0700: >>> As I understand, shake is good at untangling "I need this to do that, I >>> need that to do the other" sorts of tasks. Stack has a lot of these things, >>> e.g. "I need to read the stack.yaml to...", "I need to download the >>> resolver build plan from stackage.org to...", and so forth. A deep dive >>> into using shake pervasively may prove beneficial, although I have >>> practically no experience with shake so I could be completely off base here. >>> >>> -- Dan Burton >>> >>> On Mon, Jul 11, 2016 at 9:55 AM, Michael Sloan <[email protected]> wrote: >>> >>> > Yeah, one thing I've wanted shake integration for is providing is >>> > collection and display of performance stats - a coarse impression of where >>> > time is spent. >>> > >>> > Thing is, we really aren't dealing with a make-like build system involving >>> > a variety of different interdependent tasks. This is shake's killer app / >>> > intended domain. Instead, we've just got some cabal builds to do. The >>> > implementation of concurrently executing such tasks is very simple: >>> > https://github.com/commercialhaskell/stack/blob/master/src/Control/Concurrent/Execute.hs >>> > >>> > So I'm also not sure if it's the right decision, but I'm not sure how much >>> > it matters. Stack is a lot more than just the "do build steps" bit, >>> > though >>> > that is certainly the core. >>> > >>> > It could be interesting to consider how difficult it would be to switch >>> > some choice bits over to shake, but I'd want to see a couple convincing >>> > advantages, and I'd hope for the code impact to be low. Reuse of the >>> > performance stats stuff is one potential advantage. >>> > >>> > -Michael >>> > >>> > On Sat, Jul 9, 2016 at 10:56 AM, Michael Snoyman <[email protected]> >>> > wrote: >>> > >>> >> It wasn't made for deep reasons, it was very likely done for bad reasons >>> >> driven by Chris and my lack of experience with Shake when working on the >>> >> initial Stack code. We had a lot of trouble getting reliably coordination >>> >> between Stack's, Cabal's, and (to a lesser extent) GHC's dependency >>> >> tracking, and adding in an extra layer we didn't have full understanding >>> >> of >>> >> made it worse. I have no idea if using Shake would have been better or >>> >> worse in the end, but doing it manually - a way Chris and I had a lot of >>> >> experience with - was the less risky path. >>> >> >>> >> I _will_ say that on other customer projects, we use Shake extensively, >>> >> so don't take the lack of usage in Stack as an aversion to Shake. It was >>> >> just a pragmatic decision when we were trying to churn out a first >>> >> version >>> >> quickly. >>> >> >>> >> On Sat, Jul 9, 2016 at 3:00 AM, Edward Yang <[email protected]> wrote: >>> >> >>> >>> This is not meant as a leading question, I am just curious and would >>> >>> like to know more. >>> >>> >>> >>> According to >>> >>> https://github.com/commercialhaskell/stack/wiki/Stack's-origins Stack >>> >>> used to use Shake. Looking at the dependencies of Stack, it no longer >>> >>> does >>> >>> so. What caused you to rebuild the functionality of Stack into Shake? >>> >>> >>> >>> -- >>> >>> You received this message because you are subscribed to the Google >>> >>> Groups "haskell-stack" group. >>> >>> To unsubscribe from this group and stop receiving emails from it, send >>> >>> an email to [email protected]. >>> >>> To post to this group, send email to [email protected]. >>> >>> To view this discussion on the web visit >>> >>> https://groups.google.com/d/msgid/haskell-stack/5c46f622-a261-487b-96fd-3c9a86edb4ce%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/haskell-stack/5c46f622-a261-487b-96fd-3c9a86edb4ce%40googlegroups.com?utm_medium=email&utm_source=footer> >>> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >>> >>> >> >>> >> -- >>> >> You received this message because you are subscribed to the Google Groups >>> >> "haskell-stack" group. >>> >> To unsubscribe from this group and stop receiving emails from it, send an >>> >> email to [email protected]. >>> >> To post to this group, send email to [email protected]. >>> >> To view this discussion on the web visit >>> >> https://groups.google.com/d/msgid/haskell-stack/CAKA2Jg%2BdzK7rBp9cJ6aLF1UbkBRkntFDWpnxa4KGbQ2VXCgVhQ%40mail.gmail.com >>> >> <https://groups.google.com/d/msgid/haskell-stack/CAKA2Jg%2BdzK7rBp9cJ6aLF1UbkBRkntFDWpnxa4KGbQ2VXCgVhQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> >> . >>> >> >>> >> For more options, visit https://groups.google.com/d/optout. >>> >> >>> > >>> > -- >>> > You received this message because you are subscribed to the Google Groups >>> > "haskell-stack" group. >>> > To unsubscribe from this group and stop receiving emails from it, send an >>> > email to [email protected]. >>> > To post to this group, send email to [email protected]. >>> > To view this discussion on the web visit >>> > https://groups.google.com/d/msgid/haskell-stack/CAEYHaY52y5QuFu1QL3fTcgKE57F4hTzT5Au9Vc3Nr6gAstoixA%40mail.gmail.com >>> > <https://groups.google.com/d/msgid/haskell-stack/CAEYHaY52y5QuFu1QL3fTcgKE57F4hTzT5Au9Vc3Nr6gAstoixA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> > . >>> > >>> > For more options, visit https://groups.google.com/d/optout. >>> > >> >> -- >> You received this message because you are subscribed to the Google Groups >> "haskell-stack" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/haskell-stack/1468257939-sup-6083%40sabre. >> For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "haskell-stack" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/haskell-stack/CAKcFpmKUjieorfDN_qx2ocNNW3xzF%3DfEOnkWvYT5z%2BCSrKsThg%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
