Thanks for the explanation! I was able to use nsenter on docker-worker-7 to check Nexus logs, and I see artifacts being retrieved (mostly Maven related). I also see some logs like the following (always for apache.snapshots), repeated every hour:
2022-02-20 05:57:06,677+0000 WARN [ar-7-thread-4] *TASK org.sonatype.nexus.proxy.maven.routing.internal.RemoteContentDiscovererImpl - Remote strategy prefix-file error on M2Repository(id=apache.snapshots): org.apache.http.conn.ConnectTimeoutException: Connect to repository.apache.org:443 [repository.apache.org/136.243.146.148] failed: connect timed out I also filed https://issues.apache.org/jira/browse/BIGTOP-3644 (and https://github.com/apache/bigtop/pull/865) as attempt to improve the Gradle download (that IIUC doesn't use any proxy but that could use some retry logic). WIll keep investigating, if anybody has suggestions please let me know :) Luca On Sat, Feb 19, 2022 at 7:21 PM Olaf Flebbe <[email protected]> wrote: > > Hi, > > glad you are asking how to make sure that a caching server is used: > > https://github.com/apache/bigtop/blob/8c323c4f12534508b6ffb45603db7cbf6e0a145f/build.gradle#L477 > > <https://github.com/apache/bigtop/blob/8c323c4f12534508b6ffb45603db7cbf6e0a145f/build.gradle#L477> > > The gradle task "configure-nexus“ > is configuring $HOME/.m2/settings.xml file for maven to download for instance > maven central via nexus instead of downloading from maven central directly. > > However this will only work for maven, not handling gradle or ivy builds > AFAIK. > > Best > Olaf > > > > Am 19.02.2022 um 14:23 schrieb Luca Toscano <[email protected]>: > > > > Hi everybody, > > > > I have one doubt related to Nexus, namely when it is used. I tried to > > check [1] as random example, and I see that we trigger at some point > > the configure-nexus gradle code, but when the debuild script kicks in, > > I see stuff like: > > > > + mvn clean install -DskipTests -Dhadoop.version=3.2.2 > > -Dmaven.buildNumber.revisionOnScmFailure=v2.4.1 -Phadoop-3 -Pyarn > > -Dmaven.repo.local=/var/lib/jenkins/.m2/repository > > > > Is the mvn command launched inside debuild using the nexus cache? > > > > Luca > > > > ---- > > [1] > > https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=alluxio,OS=debian-11/lastBuild/consoleFull > > > > On Thu, Feb 17, 2022 at 9:48 PM Olaf Flebbe <[email protected]> wrote: > >> > >> Hi everyone, > >> > >> Yesterday Luca Toscano and me had a call to look into improving the > >> situation of artifact downloads by caching . > >> > >> I was a bit surprised that the „nexus“ code is still in place and still > >> seem to work somehow. > >> > >> Since having a repository server (a specialized proxy for maven repos) is > >> technically a much cleaner solution than messing with the .m2/repository > >> maven cache -- since it can be shared across architectures and os and even > >> support more built tools -- I would like to step back from my proposal to > >> use docker volumes to share the raw m2 cache between instances. > >> > >> What need to be done is to either update to a current version of nexus or > >> switch to a different maven proxy which can be setup , updated and > >> configured more easily. > >> > >> I asked a search machine for alternatives and tripped over this project > >> https://github.com/jenkins-x/bucketrepo > >> which promises to be a low-footprint minimal replacement for nexus, which > >> could even use S3 as a backing store. > >> > >> There was a configuration example for nginx as well : > >> https://github.com/lkiesow/weblog.lkiesow.de/blob/master/20170413-nginx-as-fast-maven-repository-proxy.md > >> > >> Best > >> Olaf >
