> -----Original Message----- > From: Jacques Etienne Beaudet <jebeau...@gmail.com> > Sent: Wednesday, March 23, 2022 3:14 PM > To: Maven Users List <users@maven.apache.org> > Subject: Re: Strategies for automatically fixing local maven repo > corruption? > > I'll copy paste something I wrote in another thread but yea basically > there's a neat way to help with this : > > Just chiming in on my experience using the redis locks with maven 3.8.x > and resolver 1.6. I've been running it on my Jenkins nodes for about a > year now with great success. Before it, we had corrupted local maven > repos (zip file empty kind of exceptions) a couple of times a week. > Then, we configured Jenkins to properly handle multiple concurrent > builds by not sharing the maven local repo and run each build in docker > which is wasteful since you have to redownload all your dependencies on > each build and build time is obviously very long. > > Then I bumped into the lock feature which is available on resolver 1.6 > (instructions here svn.apache.org/repos...x.html). It is a bit > cumbersome to setup but after that, we started sharing the same local > repos and our build time dramatically improved (something like from 4 > minutes to ~30 seconds) and we had a corrupted maven repo once or twice > throughout the year (and it could have been some job that a dev created > without the proper config, I didn't investigate much).
This note posted with that url truncated. Can you provide the full url? > Redis is really easy to setup and is available via docker or apt so it > should be a breeze to add it to your init script like we did. My nodes > had docker already installed so it was just adding this to the "Init > script" in the nodes configuration : > #!/bin/bash -ue > > docker run -d --network host --name mvn_redis redis:6.2-alpine > 23, 2022, 6:11 PM -0400, David Karr <davidmichaelk...@gmail.com>, wrote: > > Our enterprise has builds for a couple of hundred services running > > many times a day, on a set of Jenkins build nodes, most of which are > > Maven builds.The builds use a settings.xml that doesn't override the > > "localRepository", so it will use a local repository on the build node > > that is used by all builds. > > > > We often run into weird build errors like "Could not find artifact", > > for artifacts that I can clearly show are available in our intranet > repository. > > In every one of those cases, when we inspect the directory in the > > local maven repo on that build node, we see that it's "corrupted", in > > that perhaps the pom file is there, but the jar file is missing. > > > > We fix these errors by simply removing the directory where the key > > files are missing, then rerun the build, which correctly downloads the > > artifact, and the build succeeds. We often have to do this on multiple > > build nodes, although I don't have visibility to that (not on ops > > team), so I don't really know if the same artifact gets corrupted on > > multiple build nodes at the same time. I doubt that. > > > > I wish there was some way that we could either prevent some of these > > incidents, or somehow automatically fix them (by removing the > > corrupted directory). The latter seems like it's in the realm of > > possibilities, but it would have to be constantly scanning the entire > > repository, looking at leaf directories that haven't changed in > > several minutes, and somehow verifying the "sanity" of the contents > from a very limited perspective. > > > > Is there anything we can do to improve this situation?
