[
https://issues.apache.org/jira/browse/SAMZA-1885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16616478#comment-16616478
]
ASF GitHub Bot commented on SAMZA-1885:
---------------------------------------
GitHub user shanthoosh reopened a pull request:
https://github.com/apache/samza/pull/640
[WIP][Not ready for review]SAMZA-1885: Improve samza build time.
Samza codebase currently uses gradle 2.8 (which was released in 2014) and
builds the codebase serially executing each of the gradle tasks one at a time.
Not taking advantage of recent gradle features has caused slow builds (taking
around 15 minutes to build samza codebase locally). This hampers the overall
developer productivity within the team.
**Changes:**
1. Upgrade to gradle version: 4.7(changes in gradle.properties,
gradle-wrapper.jar).
2. Enable parallel gradle builds.
3. Set maximumParallelForks to 5(The maximum number of forked test
processes to execute in parallel. The default value is 1 (no parallel test
execution). It cannot exceed the value of max-workers for the current build.
Reference:
https://docs.gradle.org/current/dsl/org.gradle.api.tasks.testing.Test.html)
4. Fix the tests that cannot be run in parallel(TestsJobsResource,
TestTaskResource).
**Result:**
Reduces the build time by a factor of 2.
Before these changes, it took around 13 minutes 40 seconds to build the
open source samza codebase locally.
After these changes, it took around around 6 minutes 14 seconds to build
samza codebase locally.
**Pending:**
Ran the build locally multiple(50) times. After these changes, only the
TestRunner API tests fail consistently due to static partition state held in
it(Will fix it as a part of this patch).
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/shanthoosh/samza improve_build_time
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/samza/pull/640.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #640
----
commit b48a116bb7a91beeac014ae30f963555332223ef
Author: Shanthoosh Venkataraman <svenkata@...>
Date: 2018-09-15T18:43:08Z
Changes:
1. Switch to newer version of gradle.
2. Turn on gradle parallel builds.
----
> Improve samza build time.
> -------------------------
>
> Key: SAMZA-1885
> URL: https://issues.apache.org/jira/browse/SAMZA-1885
> Project: Samza
> Issue Type: New Feature
> Reporter: Shanthoosh Venkataraman
> Assignee: Shanthoosh Venkataraman
> Priority: Major
>
> Samza codebase currently uses gradle 2.8 (which was released in 2014) and
> builds the codebase serially executing each of the gradle tasks one at a
> time. Not taking advantage of recent gradle features has caused slow builds
> (taking around 15 minutes to build samza codebase locally). This hampers the
> overall developer productivity within the team.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
