Hi Luca, as I was mentioning in the JIRA Ticket. There seems to be an underdocumented gradle feature to use a maven mirror.
https://rahulsom.github.io/blog/2016/gradle-equivalent-of-maven-mirror.html <https://rahulsom.github.io/blog/2016/gradle-equivalent-of-maven-mirror.html> Olaf > Am 20.02.2022 um 11:00 schrieb Luca Toscano <[email protected]>: > > Thanks for the explanation! I was able to use nsenter on > docker-worker-7 to check Nexus logs, and I see artifacts being > retrieved (mostly Maven related). I also see some logs like the > following (always for apache.snapshots), repeated every hour: > > 2022-02-20 05:57:06,677+0000 WARN [ar-7-thread-4] *TASK > org.sonatype.nexus.proxy.maven.routing.internal.RemoteContentDiscovererImpl > - Remote strategy prefix-file error on > M2Repository(id=apache.snapshots): > org.apache.http.conn.ConnectTimeoutException: Connect to > repository.apache.org:443 [repository.apache.org/136.243.146.148] > failed: connect timed out > > I also filed https://issues.apache.org/jira/browse/BIGTOP-3644 (and > https://github.com/apache/bigtop/pull/865) as attempt to improve the > Gradle download (that IIUC doesn't use any proxy but that could use > some retry logic). > > WIll keep investigating, if anybody has suggestions please let me know :) > > Luca > > On Sat, Feb 19, 2022 at 7:21 PM Olaf Flebbe <[email protected]> wrote: >> >> Hi, >> >> glad you are asking how to make sure that a caching server is used: >> >> https://github.com/apache/bigtop/blob/8c323c4f12534508b6ffb45603db7cbf6e0a145f/build.gradle#L477 >> >> <https://github.com/apache/bigtop/blob/8c323c4f12534508b6ffb45603db7cbf6e0a145f/build.gradle#L477> >> >> The gradle task "configure-nexus“ >> is configuring $HOME/.m2/settings.xml file for maven to download for >> instance maven central via nexus instead of downloading from maven central >> directly. >> >> However this will only work for maven, not handling gradle or ivy builds >> AFAIK. >> >> Best >> Olaf >> >> >>> Am 19.02.2022 um 14:23 schrieb Luca Toscano <[email protected]>: >>> >>> Hi everybody, >>> >>> I have one doubt related to Nexus, namely when it is used. I tried to >>> check [1] as random example, and I see that we trigger at some point >>> the configure-nexus gradle code, but when the debuild script kicks in, >>> I see stuff like: >>> >>> + mvn clean install -DskipTests -Dhadoop.version=3.2.2 >>> -Dmaven.buildNumber.revisionOnScmFailure=v2.4.1 -Phadoop-3 -Pyarn >>> -Dmaven.repo.local=/var/lib/jenkins/.m2/repository >>> >>> Is the mvn command launched inside debuild using the nexus cache? >>> >>> Luca >>> >>> ---- >>> [1] >>> https://ci.bigtop.apache.org/view/Packages/job/Bigtop-trunk-packages/COMPONENTS=alluxio,OS=debian-11/lastBuild/consoleFull >>> >>> On Thu, Feb 17, 2022 at 9:48 PM Olaf Flebbe <[email protected]> wrote: >>>> >>>> Hi everyone, >>>> >>>> Yesterday Luca Toscano and me had a call to look into improving the >>>> situation of artifact downloads by caching . >>>> >>>> I was a bit surprised that the „nexus“ code is still in place and still >>>> seem to work somehow. >>>> >>>> Since having a repository server (a specialized proxy for maven repos) is >>>> technically a much cleaner solution than messing with the .m2/repository >>>> maven cache -- since it can be shared across architectures and os and even >>>> support more built tools -- I would like to step back from my proposal to >>>> use docker volumes to share the raw m2 cache between instances. >>>> >>>> What need to be done is to either update to a current version of nexus or >>>> switch to a different maven proxy which can be setup , updated and >>>> configured more easily. >>>> >>>> I asked a search machine for alternatives and tripped over this project >>>> https://github.com/jenkins-x/bucketrepo >>>> which promises to be a low-footprint minimal replacement for nexus, which >>>> could even use S3 as a backing store. >>>> >>>> There was a configuration example for nginx as well : >>>> https://github.com/lkiesow/weblog.lkiesow.de/blob/master/20170413-nginx-as-fast-maven-repository-proxy.md >>>> >>>> Best >>>> Olaf >>
