I know this opinion is probably going to be unpopular, but I just want to put out it there:
If you need to parallellize your build, then your problems getting parallel building working are certainly not your biggest problem. Even if you get a parallel build working, and it is certainly doable. Where is that going to go? Step 1 people will stop doing a pre-commit build - The fact that you need a parallel build means that it is almost certainly prohibitively time consuming for a dev to do one. Step 2, people start caring less about build efficiency. Why care, you can't run it locally anyway. The net effect of this is that your parallel build will eventually start taking as long or longer than your old serial one. Step 3 big refactorings become super scary, prohibitively time consuming or simply not doable. This is because you can't quickly re-run your specs anymore. A core refactor to be tested will have to go back and forth to the build server to be verified (probably many times). That means you are going to need another set of parallel builders focussing on another branch so that you don't go screwing up the deployable master. Step 4, the super smart guy/gal who built your parallel build system leaves the team and now you are in a world of pain :-) But seriously, a lot of effort will go into understanding, maintaining and expanding your build. Every project I have worked on that required a parallel build had 3 common threads: Lots of Factory based (database dependent) unit-level specs Large amounts of what should really be orthogonal, unit level specs being done in acceptance tests A large application with more than one candidate application extraction I have been privileged to work with some super smart people, and the projects suffering from the problems above were no exception, so it wasn't that they were poor developers or bad designers. Generally early extractions don't get done when there are tight deadlines. By the time you are feeling that your build is too slow and it is starting to get in the way of productivity, your application is already probably too big. At this point I recommend trying to identify groups of functionality that are candidates for an extraction. If they can operate completely independently and be deployed completely independently then that is great. Worst case scenario look to extract large pieces out as mountable engines. There are certainly non-trivial issues with maintaining multiple, interdependent apps, but all-in-all I think teams will maintain greater flexibility and development speed if effort is put into this kind of modularisation rather than ever more complicated build structures designed to stop the team needing to address the design issue. Adam Boas e: [email protected] m: +61 457 741 117 On 26/11/2012, at 9:37 AM, Thomas Egret <[email protected]> wrote: > I have been using parallel_tests as well and it's very awesome however > sometimes the tests suite just run smoothly and sometimes just fails so we > had to deactivate it for the moment (wonder if you use it as well on Travis). > > Wish some people could give some hints regarding how to ensure my tests to > pass using parallel_tests. > > > 2012/11/26 James Healy <[email protected]> > A previous project I worked on used Jenkins with a vagrant plugin to run > concurrent tests in isolated VMs. > > It took a bit of setting up, but once it got going it was amazing and dropped > our build time from >20 minutes to <5 minutes. > > The downside was that it became tricker to understand the "state of the > build" at a glance. Instead of checking 1 project, you had to check 6 (unit > tests, acceptance tests 1, acceptance tests 2, etc). > > We also tried parallel_tests for a while and it mostly worked, but we ran > into a few issues with parallel acceptance specs stepping on each others toes. > > James > > > > On 22 November 2012 13:12, Sebastian Porto <[email protected]> wrote: > Hi Guys > > There was a discussion the other day about CI server and it was mention > several times how good it is to run the tests in parallel. I am intrigued by > this and will like to implement this. But I don't know where to start. > > What is your approach for running CI test in parallel? We are using TeamCity > at the moment, were using Jenkins before. And Rspec. > > Thanks > Sebastian > > -- > You received this message because you are subscribed to the Google Groups > "Ruby or Rails Oceania" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/rails-oceania/-/HqhReVKyNx0J. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/rails-oceania?hl=en. > > > -- > You received this message because you are subscribed to the Google Groups > "Ruby or Rails Oceania" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/rails-oceania?hl=en. > > > -- > You received this message because you are subscribed to the Google Groups > "Ruby or Rails Oceania" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/rails-oceania?hl=en. -- You received this message because you are subscribed to the Google Groups "Ruby or Rails Oceania" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/rails-oceania?hl=en.
