Hey there, I have a master job which runs all the different tests of a particular project. Let me call it 'masterjob' This masterjob has a multijob parallel phase which fans-out roughly 60-80 test jobs, these are run in parallel and spread out across different AWS regions.
The parallelism is there in order to achieve quick feedback and making sure the feedback loop times don't increase as the number of tests increases. It also gives me a quick point to check in how long it took to run all the tests as the total time is the time the masterjob took to complete. The problem I have is after the parallel job phase, where I need to aggregate all the downstream artifacts from all the jobs across every AWS region. This step is done sequencially and is currently taking 20% of the total duration of the masterjob. I am looking to run the aggregation of artifacts in parallel, and mantain the feedback for the total duration of all the jobs mapped to the main masterjob as I have today. I am thinking the best approach might be to build a piece of groovy code that retrieves the artifacts from the current jobs in parallel and discard the aggregate downstream artifacts plugins. Or as an alternative pushing those artifacts to a proper repository as part of the job execution, and aggregate them again in the main masterjob but in parallel through some shell/python/groovy step. Who else has this problem? Anyone -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/7c7cff81-ea28-42c6-85ca-802c8c923362%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
