[
https://issues.apache.org/jira/browse/HADOOP-18335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17565319#comment-17565319
]
Ayush Saxena edited comment on HADOOP-18335 at 7/12/22 7:12 AM:
----------------------------------------------------------------
On an exploratory stage to highlight the build time issue,
Just added my ideas in the description, which I feel can help, a bunch of them
may not work in real or might not be possible to do.
But the build times are really annoying now, We should definitely put some
effort to explore possibilities to reduce that.
Let me know if someone has any thoughts, suggestions or pointers
was (Author: ayushtkn):
On an exploratory stage to highlight the build time issue,
Just added my ideas in the description, which I feel can help, a bunch of them
may not work in real or might not be possible to do.
But the build times are really annoying now, We should definitely put some
effort to explore possibilities to reduce that.
> Improve Pre-Commit Time
> -----------------------
>
> Key: HADOOP-18335
> URL: https://issues.apache.org/jira/browse/HADOOP-18335
> Project: Hadoop Common
> Issue Type: Task
> Reporter: Ayush Saxena
> Priority: Critical
> Labels: buid
>
> As of now the complete build time has reached ~24 hours, which makes tracking
> Jira with root level changes very tough.
> Even at module level, it is on the higher side, kind of 6 hours or more,
> which is typically equivalent to 1 working day hours:
> Some areas to explore:
> * Enable Parallel-Test profile for the modules which don't have it.
> * Explore improvements in the area of our Test setup, increase memory,
> number of threads, or some parallel executions?
> * Remove the modules or atleast disable the test suites of modules which are
> no longer being used or maintained (Separate Mail Chain is there for this)
> * Spin Two Pre-Commit jobs rather than 1 and split some task between them,
> kind of one for Java-11 build & Javadoc, Checkstyle and related stuff.
> * Figure out & remove irrelevant or similar tests.
> * Improve existing tests, Like reusing MiniDfsClusters or so, rather than a
> bunch of test spinning one for themselves.
> * If possible, if there are multiple modules to be tested, run those modules
> test in parallel (Exploratory: may lead to OOM)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
