Hi, I think I got a clue why spark compile fails in our CI, but this may not explain why compile fails for Ganesh.
The compile job in CI is started with option docker run --rm -v `pwd`:/ws --workdir /ws -e COMPONENTS=$COMPONENTS --net=container:nexus bigtop/slaves:trunk-$BUILD_ENVIRONMENTS \ bash -c '. /etc/profile.d/bigtop.sh; ./gradlew allclean configure-nexus $COMPONENTS-pkg' The "--net=container:nexus" is necessary to get access to the private subnet of the nexus container to be used as a maven repository securely. However, if a process within the compile job happens to open a server port, this port may be accessible from any other container running on the same machine as well, since they are shareing their network! I just read about nailgun and I saw that the zinc server is started at spark compile job unconditionally in the pom.xml. So the problem may be a "cross docker compile" race condition: When the docker container servicing the zinc server stops, all other compile host will fail at once, without any error message. Or it will confuse libraries in the same namespace. There are sign of this errors in the CI logs, too. Given that our build process is not an incremental compile, I vote for disabling the zinc madness. (i.e. remove useZincServer in pom.xml of spark) Other possible attempts is to give up caching of artifacts. Thoughts? Olaf
