What’s the problem with adivisory locking, as long as Maven honors the advice it is the same as it’s a redis lock? (But much less footprint). In fact on the same machine it should even work without locking as Long as you use pidfiles?
Gruss Bernd -- http://bernd.eckenfels.net ________________________________ Von: Michael Osipov <[email protected]> Gesendet: Monday, October 4, 2021 9:15:23 PM An: [email protected] <[email protected]> Betreff: Re: Local repository accessed by multiple Maven instances - how is it solved ? Am 2021-10-04 um 14:31 schrieb Francois Marot: > Hello all, > > I would like clarifications on MNG-2802[*] that seems to be solved. If I > understand correctly it solves the problem where multiple simultaneous > Maven executions shared the same local repository. > I have been keeping for years a bit of code in my Jenkinsfiles ensuring the > local repos were accessed only by a single jenkins executor at the same > time, so it seems like I can get rid of this doesn't it ? > I'd like your input on this because reproducing the bug is difficult so I'd > like to be sure before simplifying my Jenkinsfile. > > Moreover, does anyone know how this problem is solved technically ? Using > files lock ? Can anybody explain ? > > Thanks you and thanks the Maven team for keeping up the good work at such > pace ! Salut François, I don't know what you exactly fiddled in your Jenkinsfile, but there is no way make it right just with Jenkinsfile foo when you want access *one* local repo with *multiple* processes you *must* use an external coordinator process. Forget also about file locks, they are all advisory, at least on POSIX-like filesystems. There is no way, or a very hard way to make them right. We tried and removed them. Now, let me give you bit of history: I have integrated/implemented a fully working multi-process Redisson-based synchronization lock factory for Maven Resolver 1.6.x which ships with Maven 3.8.x. (It has already been battle tested for several months by a user on a busy CI server). Tamás took it to the next level in Resolver 1.7.x and generalized it to a named lock approach where the Redisson Lock Factory plugs in nicely with other lock implementations. Almost nine months of work and it works very good now. Now here is the problem: Maven Resolver 1.7.x comes only with Maven 4.x. I have created a branch which reconciles best of breed: Maven 3.8.x and Maven Resolver 1.7.x. It just works (also verified by this user). I also ran more than a thousand builds with 8 parallel processes and 4 threads. It will easily handle thousands of locks in parralel with an one digit percent overhead. I don't know for what documentation you are waiting for, but everything is there, you can start *now* to leverage it: [2], [3]. With Resolver 2.0.0 and Maven 4.0.0 it will likely be even easier to integrate. Hazelcast will also work, but Tamás is the special here. Again: You *must* use an external process to synchronize access. No excuses. Michael [1] https://github.com/apache/maven/tree/maven-3.8.x-resolver-1.7.x [2] https://maven.apache.org/resolver/maven-resolver-named-locks/index.html [3] https://maven.apache.org/resolver/maven-resolver-named-locks-redisson/index.html --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
