Joe McDonnell created IMPALA-9107:
-------------------------------------
Summary: Reduce time spent downloading maven artifacts for
precommit tests
Key: IMPALA-9107
URL: https://issues.apache.org/jira/browse/IMPALA-9107
Project: IMPALA
Issue Type: Improvement
Components: Frontend, Infrastructure
Affects Versions: Impala 3.4.0
Reporter: Joe McDonnell
When building Impala from scratch with an empty .m2 directory, maven needs to
download a large number of jars and pom files. This is taking a long time and
adds about 15-20 minutes to the build. For example, here is some output from a
recent ubuntu-16.04-from-scratch, there is an 18 minute delay between the end
of building the backend tests and the end of building the frontend:
{code:java}
00:58:33 [100%] Built target hash-ring-util
...
01:16:37 [100%] Built target fe{code}
Almost all of that time is spent downloading maven artifacts. Quite a few come
from the maven central repository.
This is taking way too much time, and we need to reduce it. The total size of
artifacts being downloaded is not large. One approach would be to produce a
tarball with the jars/poms that don't come from the CDP or CDH GBN repos. We
can download that tarball and use it to either populate the .m2 directory or to
stash it in IMPALA_TOOLCHAIN and use it as a maven repository with a file://
URI.
This impacts all our jobs: ubuntu-16.04-from-scratch,
ubuntu-16.04-dockerised-tests, all-build-options-ub1604,
ubuntu-16.04-build-only, clang-tidy-ub1604
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
