Nate DAmico created BIGTOP-1906:
-----------------------------------
Summary: Optimize and enahnce build/CI process to reuse artifacts
Key: BIGTOP-1906
URL: https://issues.apache.org/jira/browse/BIGTOP-1906
Project: Bigtop
Issue Type: Sub-task
Components: build
Affects Versions: 1.0.0
Environment: all (host as well as any container based building)
Reporter: Nate DAmico
Fix For: 1.1.0
This was generated from latest "future bigtop" thread on mailing list,
capturing core details here from [~oflebbe]
cc: [~evans_ye]
==============================================
However we have a substantial problem hidden deep in th CI „2.0“ approach using
containers
You may know that we place artifacts (i.e. jars) we built with bigtop into the
local maven cache ~/.m2. (look for mvn install in do-component-build). The idea
is that later maven builds will pick these artifacts and use them rather
downloading them from maven central.
Placing artifacts into ~/.m2 will not have any effect if we use CI containers
the way we do now: The maven cache ~/.m2 is lost when the container ends.
[This triggered misfeature in JIRA
https://issues.apache.org/jira/browse/BIGTOP-1893 , BTW: gradle rpm/apt
behaved differently from a container build with artifacts from maven central.]
Option 1) Remove mvn install from all do-component-builds
Results:
+ We compile projects the way the upstream-developer does.
- local fixes and configurations will not propagated
Questions:
If we do not try to reuse our build-artifacts within compile we have to ask
ourself "why do we compile projects at all?“.
We can build a great test wether someone else has touched / manipulated the
maven central cache if we compare artifacts, but is this the really the point
of compiling ourselves?
Option 2) Use mvn install and reuse artifacts even in containers.
Consequences:
- Containers are not stateless any more
- We have to add depencies to CI jobs so they run in order
- single components may break the whole compile process.
- Compile does not scale any more
My Opinion:
The way we do now "mvn install“ , simply tainting the maven cache seems not a
really controlled way to propagate artifacts to me.
Option 3) Use 1) but reuse artifacts in packages by placing symlinks and
dependencies between them.
- Packages will break with subtile problems if we do symlink artifacts from
different releases.
----
Neither Option 1, Option 2 nor Option 3 seems a clever way to fix the problem.
Would like to hear comments regarding this issue:
In my humble opinion we should follow Option 2 with all the grave consequences.
But maybe reworking mvn install by placing the artifacts with a bigtop specific
name / groupid into the maven cache and upload them to maven central .
Olaf
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
