Ghatage opened a new pull request #2278: Add forkCount option to parallelize build and tests URL: https://github.com/apache/bookkeeper/pull/2278 ### Motivation Current code forks a new JVM per module. (bookkeeper-server, bookkeeper-proto etc) This means one fork per module for build and testing, no parallelism within the module where majority of the time goes. We need parallelism within a module during the test execution so we can have the builds complete faster and have the artifacts shipped out quicker. ### Changes We use the maven surefire plugin but don't define the `forkCount` and hence set it to default of 1. This means it executes each module with one thread. This change sets `forkCount` to 5, enabling parallelism in testing and drastically reducing total turnaround time. (by about 2/3rds!) *Total build+test time without this change* ``` [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 01:06 h [INFO] Finished at: 2020-03-05T02:01:29-08:00 [INFO] ------------------------------------------------------------------------ ``` *Total build+test time with this change* ``` [INFO] ------------------------------------------------------------------------ [INFO] BUILD SUCCESS [INFO] ------------------------------------------------------------------------ [INFO] Total time: 18:23 min [INFO] Finished at: 2020-03-05T02:38:22-08:00 [INFO] ------------------------------------------------------------------------ ``` ### Things to watch Added parallelism may cause some flappers but with much trial and error I have come to the number `5`. The flappers are usually only from conflict in obtaining the same port number. If needed, we can increase the retryCount, but as of now I consistently don't see any flappers at a `forkCount` of 5
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services
