Hi There was some work done for this by Craig it's not in the trunk because it is not complete yet as far as I know.
http://csut017.wordpress.com/?s=distributed&submit=Search with kind regards Ruben Willems On Mon, Oct 25, 2010 at 5:24 AM, Michael <[email protected]> wrote: > I'm trying to optimize my build times using as much parallelization as > I can - and I came across an interesting problem. > > I have 4 separate projects running in CCNet. Each project has two > tasks: compile and test. The compile task takes about 5 minutes and > the test takes about 10 - 15min total for each project. All 4 projects > are built together, so to try to improve total build time, I delegate > the projects to separate queues, so they can run in parallel. This > works fine, and so now instead of the projects running sequentially > and taking 60min to complete, I can run 2 projects at a time in > parallel (my server can't handle much more than that), reducing the > total build time to 30min. > > Now for the interesting part. > I'd like to get this down even shorter, and found some inefficiencies > in this process > > The compile tasks for all 4 projects run on one PC (Compile-Server) > and the test tasks run on another PC (Test-Server) - due to available > features on each PC. > > Because a single CCNet project contains both tasks, and 'blocks' the > spot in the queue until completion, the Compile-server is essentially > idle while the test task is running on the Test-server for that > project. And so, the next project in the queue can't start its compile > task until the previous project's test task is completed. I'd like to > figure out how to pipeline the compile and test tasks so no server is > unnecessarily idle > > One way I thought of was to break up each project into two separate > sub-projects: one for compiling and one for testing. This way I could > split the projects into separate queues, with the compile project > triggering the corresponding test project. The problem is, I want the > compile results and test results rolled up into one single build > status report - since they are essentially one project for a single > product. > > So, the question is: can results/artifacts from two CCNet projects be > merged into one project summary report ? > > thanks > Michael
