I don't think it's timing-related either, as it fails very consistently, not intermittently.
As for choosing the closest mirror each time, does it actually do that? >From what I see, the Apache mirror is hardcoded to apache.osuosl.org in bigtop.bom. ~ Jonathan On Fri, Jan 15, 2016 at 11:39 PM Konstantin Boudnik <[email protected]> wrote: > Yeah, there's no way a local file removal could be causing this. Seems > like a > weird timing or something. I think, if in your first example you > run 'realclean download' in the same command a few times in a row you'll > see a > successful download. > > The situation might be also caused by the script that chooses the closest > mirror every time you hit the website. Either some mirrors are no good or > script might experience some issues like overload or else. > > Cos > > On Fri, Jan 15, 2016 at 11:02PM, Jonathan Kelly wrote: > > No, strangely enough it's the opposite. Running with "realclean" *causes* > > the failure (provided it's run in the same invocation of gradle as a task > > that downloads something from the Apache mirror and not in a separate > > invocation). Also, the failure is caused by an HTTP 403 error from the > > Apache mirror, definitely not caused by not being able to write to the > local > > file. > > > > # on EC2 or otherwise public internet > > gradle realclean someproject-download # fails > > gradle realclean; gradle someproject-download # works > > > > # on corporate network/VPN > > gradle someproject-download # fails > > (also fails with realclean whether run separately or in same invocation) > > > > > On Jan 15, 2016, at 8:49 PM, Konstantin Boudnik <[email protected]> > wrote: > > > > > >> On Fri, Jan 15, 2016 at 01:17AM, Jonathan Kelly wrote: > > >> No, it's not related to new app versions. This happens any time I try > (from > > >> my dev machine while on the corporate network) to build any app that > > >> downloads its source from the Apache mirror, which for some reason > seems to > > >> be hardcoded to http://apache.osuosl.org. > > >> > > >> What's even weirder is that I have also experienced this off the > corporate > > >> network (usually on our EC2 build nodes) if I run "gradle realclean > > >> someproject-rpm". That is, if I run the realclean target then some > target > > >> that ends up running a download target, where the download comes from > the > > >> Apache mirror, and I do so in the same invocation of gradle, it will > fail > > >> in this same way. If I run "gradle realclean" then "gradle > > >> someproject-rpm", it does not fail in the way. This makes even less > sense > > >> to me than the first problem. > > > > > > As I replied on the mahout JIRA it might be cause by some garbage left > from > > > previous unfinished download attempts. That junk can get in the way of > the > > > downloads and would be solved by 'realclean' which does exactly that: > cleans > > > up the dl/ Here's the code: > > > > > > task realclean (dependsOn: allclean, > > > description: "Removing $BUILD_DIR, $OUTPUT_DIR, $DIST_DIR, and > $DL_DIR", > > > group: PACKAGES_GROUP) << { > > > delete (DL_DIR) > > > } > > > > > > Cos > > > > > >>> On Thu, Jan 14, 2016 at 4:28 PM Konstantin Boudnik <[email protected]> > wrote: > > >>> > > >>> If a release has been published just recently, then it might not be > on all > > >>> mirrors yet. That's the only explanation I have. > > >>> > > >>> Cos > > >>> > > >>>> On Fri, Jan 15, 2016 at 12:11AM, Jonathan Kelly wrote: > > >>>> I have noticed errors like this as well. If I add "--stacktrace" to > the > > >>>> arguments, I see that the Apache mirror returned a 403 for the URL. > As > > >>>> Andrew says, doing a wget on the same exact URL works fine. Also, > I'm > > >>>> pretty sure I've noticed that the error only occurs while I'm in the > > >>> office > > >>>> on the corporate network or connected to VPN. This makes no sense > to me. > > >>>> > > >>>> ~ Jonathan > > >>>> > > >>>>> On Thu, Jan 14, 2016 at 4:04 PM Konstantin Boudnik <[email protected] > > > > >>>> wrote: > > >>>> > > >>>>> Could you please publish your patch to the JIRA? And I suggest to > move > > >>> this > > >>>>> discussion there as well. > > >>>>> > > >>>>> Cheers, > > >>>>> Cos > > >>>>> > > >>>>>> On Thu, Jan 14, 2016 at 03:37PM, Andrew Musselman wrote: > > >>>>>> Even though `wget > > >>> > http://apache.osuosl.org//mahout/0.11.0//apache-mahout-distribution-0.11.0-src.tar.gz` > <http://apache.osuosl.org//mahout/0.11.0//apache-mahout-distribution-0.11.0-src.tar.gz> > > >>> < > http://apache.osuosl.org//mahout/0.11.0//apache-mahout-distribution-0.11.0-src.tar.gz > > > > >>>>> < > > >>> > http://apache.osuosl.org//mahout/0.11.0//apache-mahout-distribution-0.11.0-src.tar.gz > > >>>> > > >>>>>> succeeds, I get this error: > > >>>>>> > > >>>>>> $ ./gradlew mahout-download > > >>>>>> :buildSrc:compileJava UP-TO-DATE > > >>>>>> :buildSrc:compileGroovy UP-TO-DATE > > >>>>>> :buildSrc:processResources UP-TO-DATE > > >>>>>> :buildSrc:classes UP-TO-DATE > > >>>>>> :buildSrc:jar UP-TO-DATE > > >>>>>> :buildSrc:assemble UP-TO-DATE > > >>>>>> :buildSrc:compileTestJava UP-TO-DATE > > >>>>>> :buildSrc:compileTestGroovy UP-TO-DATE > > >>>>>> :buildSrc:processTestResources UP-TO-DATE > > >>>>>> :buildSrc:testClasses UP-TO-DATE > > >>>>>> :buildSrc:test UP-TO-DATE > > >>>>>> :buildSrc:check UP-TO-DATE > > >>>>>> :buildSrc:build UP-TO-DATE > > >>>>>> :mahout_vardefines > > >>>>>> :mahout-download > > >>>>>> Download > > >>> > http://apache.osuosl.org//mahout/0.11.0//apache-mahout-distribution-0.11.0-src.tar.gz > > >>>>>> :mahout-download FAILED > > >>>>>> > > >>>>>> Same problem with `gradlew pig-download`; any ideas what's causing > > >>> it? > > >>> >
