Le 4 déc. 2017 21:45, "Eugene Kirpichov" <[email protected]> a écrit :
Romain - as far as I understand, Maven *does* have a retry strategy, but it is a poor retry strategy and there is no way to tweak it. In particular, if it uses https://maven.apache.org/guides/mini/guide-http-settings.html , that means it uses Apache Http Client 4.1.2 whose default retry settings are extremely conservative https://github.com/apache/httpcomponents-client/blob/4. 1.x/httpclient/src/main/java/org/apache/http/impl/client/ DefaultHttpRequestRetryHandler.java#L93 Right The errors we're hitting with Maven are some of those: "connection refused" and alike. Would a retry guarantee you it works. Should be quite easy to switch wagon with a configurable retry handler instance through mvnwrapper or custom mvn setup and test it but the logic looks sane to me. If it is on asf repo we must work with infra to fix it rather than working around the clients. If on a repo we dont control we can try to get rid of it maybe? On Mon, Dec 4, 2017 at 12:15 PM Romain Manni-Bucau <[email protected]> wrote: > 2017-12-04 18:58 GMT+01:00 Kenneth Knowles <[email protected]>: > > On Sat, Dec 2, 2017 at 3:06 AM, Romain Manni-Bucau < > [email protected]> > > wrote: > >> > >> Need to check but if plugin dependencies were not tuned it should be the > >> default with a retry of 3 IIRC. > >> > >> But arent you sure it was a repo/server issue any client cant solve? > > > > > > Not sure what you mean here. What we need is for mvn and gradle to > succeed > > even though there are frequent transient failures of the repo. Anything > that > > makes this happen is success. > > What I meant is you will likely never get it with whatever build > system until you go 100% local which wouldn't make the build > reproducible. > > Concretely you can do > > 1. > > for (int i = 0; i < 3; i++) { > if (download()) break; > } > > > 2. > > for (int i = 0; i < 3; i++) { > if (download()) break; > exponentialBackoff(); > } > > > Or other wait strategy with a timeout. > but you can't have: > > while (!download()) { > waitUntilItWorks(); > } > > > So concretely if your down time > reasonnable time you will see the > failure anyway. > > Maven default has a retry (you see it sometimes in the logs) so should > tolerante a restart or small downtime. Ivy (gradle) uses httpclient as > well so same kind of config is available but with the same limitation > by design. > > Having a local cache is the best way to avoid these issues but it > requires at least one green build for dependencies. > > What I often saw was to do a pre-build step just resolving > dependencies+plugins and use a .m2/repository caching. > > > > > Kenn > > > >> > >> > >> Le 1 déc. 2017 23:27, "Kenneth Knowles" <[email protected]> a écrit : > >>> > >>> How do you instruct maven or gradle to use that all the time? > >>> > >>> On Fri, Dec 1, 2017 at 1:58 PM, Romain Manni-Bucau > >>> <[email protected]> wrote: > >>>> > >>>> If you use wagon http - and not lightweigh - it should have retries by > >>>> default. > >>>> > >>>> Le 1 déc. 2017 22:31, "Valentyn Tymofieiev" <[email protected]> a > >>>> écrit : > >>>>> > >>>>> Has this ever been brought up in Maven dev community? Perhaps they > have > >>>>> some suggestions. It sounds like a reasonable feature request. > >>>>> > >>>>> On Fri, Dec 1, 2017 at 1:18 PM, Kenneth Knowles <[email protected]> > wrote: > >>>>>> > >>>>>> I've repeatedly searched around for a way to just add proper retry > to > >>>>>> maven or gradle, and haven't found anything :-( > >>>>>> > >>>>>> I had thought that we altered our builds in such a way that the .m2 > >>>>>> directory was permitted to survive across builds. True that it isn't > >>>>>> hermetic, precisely, but it is pretty safe to treat as a cache of > immutable > >>>>>> data, which is no more dangerous than having a caching incremental > build > >>>>>> system. > >>>>>> > >>>>>> Kenn > >>>>>> > >>>>>> On Wed, Nov 29, 2017 at 3:39 PM, Eugene Kirpichov > >>>>>> <[email protected]> wrote: > >>>>>>> > >>>>>>> Our builds often hit transient Maven network issues, e.g. this one > >>>>>>> > >>>>>>> https://builds.apache.org/job/beam_PostCommit_Java_ > MavenInstall/5331/consoleFull > >>>>>>> > >>>>>>> 2017-11-29T02:18:02.936 [ERROR] Failed to execute goal on project > >>>>>>> beam-sdks-java-io-hadoop-jdk1.8-tests: Could not resolve > dependencies for > >>>>>>> project > >>>>>>> org.apache.beam:beam-sdks-java-io-hadoop-jdk1.8-tests: > jar:2.3.0-SNAPSHOT: > >>>>>>> Could not transfer artifact org.apache.derby:derby:jar:10.10.2.0 > from/to > >>>>>>> central (https://repo.maven.apache.org/maven2): GET request of: > >>>>>>> org/apache/derby/derby/10.10.2.0/derby-10.10.2.0.jar from central > failed: > >>>>>>> Connection reset -> [Help 1] > >>>>>>> > >>>>>>> It'd be good to increase reliability of our builds. > >>>>>>> repo.maven.apache.org seems quite unreliable. > >>>>>>> > >>>>>>> I tried finding a way to configure Maven to retry such network > errors > >>>>>>> and it appears to be impossible [will be happy if someone proves > me wrong]. > >>>>>>> > >>>>>>> Would this issue be resolved if we used multiple mirrors? > >>>>>>> https://maven.apache.org/guides/mini/guide-mirror-settings.html > Any other > >>>>>>> suggestions? > >>>>>> > >>>>>> > >>>>> > >>> > > >
