+1 - let's get rid of this crap.
On Thu, Nov 24, 2016 at 06:03PM, Olaf Flebbe wrote: > Hi, > > I think I got a clue why spark compile fails in our CI, but this may not > explain why compile fails for Ganesh. > > The compile job in CI is started with option > > docker run --rm -v `pwd`:/ws --workdir /ws -e COMPONENTS=$COMPONENTS > --net=container:nexus bigtop/slaves:trunk-$BUILD_ENVIRONMENTS \ > bash -c '. /etc/profile.d/bigtop.sh; ./gradlew allclean configure-nexus > $COMPONENTS-pkg' > > The "--net=container:nexus" is necessary to get access to the private subnet > of the nexus container to be used as a maven repository securely. > > However, if a process within the compile job happens to open a server port, > this port may be accessible from any other container running on the same > machine as well, since they are shareing their network! > > I just read about nailgun and I saw that the zinc server is started at spark > compile job unconditionally in the pom.xml. So the problem may be a "cross > docker compile" race condition: When the docker container servicing the zinc > server stops, all other compile host will fail at once, without any error > message. Or it will confuse libraries in the same namespace. There are sign > of this errors in the CI logs, too. > > Given that our build process is not an incremental compile, I vote for > disabling the zinc madness. > (i.e. remove useZincServer in pom.xml of spark) > > Other possible attempts is to give up caching of artifacts. > > > Thoughts? > Olaf >
