Ayush Saxena created HADOOP-18335:
-------------------------------------
Summary: Improve Pre-Commit Time
Key: HADOOP-18335
URL: https://issues.apache.org/jira/browse/HADOOP-18335
Project: Hadoop Common
Issue Type: Task
Reporter: Ayush Saxena
As of now the complete build time has reached ~24 hours, which makes tracking
Jira with root level changes very tough.
Even at module level, it is on the higher side, kind of 6 hours or more, which
is typically equivalent to 1 working day hours:
Some areas to explore:
* Enable Parallel-Test profile for the modules which don't have it.
* Explore improvements in the area of our Test setup, increase memory, number
of threads, or some parallel executions?
* Remove the modules or atleast disable the test suites of modules which are
no longer being used or maintained (Separate Mail Chain is there for this)
* Spin Two Pre-Commit jobs rather than 1 and split some task between them,
kind of one for Java-11 build & Javadoc, Checkstyle and related stuff.
* Figure out & remove irrelevant or similar tests.
* Improve existing tests, Like reusing MiniDfsClusters or so, rather than a
bunch of test spinning one for themselves.
* If possible, if there are multiple modules to be tested, run those modules
test in parallel (Exploratory: may lead to OOM)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-dev-h...@hadoop.apache.org
