+1 I agree with disabling ZincServer On Thu, Nov 24, 2016 at 12:03 PM, Olaf Flebbe <[email protected]> wrote:
> Hi, > > I think I got a clue why spark compile fails in our CI, but this may not > explain why compile fails for Ganesh. > > The compile job in CI is started with option > > docker run --rm -v `pwd`:/ws --workdir /ws -e COMPONENTS=$COMPONENTS > --net=container:nexus bigtop/slaves:trunk-$BUILD_ENVIRONMENTS \ > bash -c '. /etc/profile.d/bigtop.sh; ./gradlew allclean configure-nexus > $COMPONENTS-pkg' > > The "--net=container:nexus" is necessary to get access to the private > subnet of the nexus container to be used as a maven repository securely. > > However, if a process within the compile job happens to open a server > port, this port may be accessible from any other container running on the > same machine as well, since they are shareing their network! > > I just read about nailgun and I saw that the zinc server is started at > spark compile job unconditionally in the pom.xml. So the problem may be a > "cross docker compile" race condition: When the docker container servicing > the zinc server stops, all other compile host will fail at once, without > any error message. Or it will confuse libraries in the same namespace. > There are sign of this errors in the CI logs, too. > > Given that our build process is not an incremental compile, I vote for > disabling the zinc madness. > (i.e. remove useZincServer in pom.xml of spark) > > Other possible attempts is to give up caching of artifacts. > > > Thoughts? > Olaf > >
