I agree with Andrew that the external links checks are ultra-flaky and seldom strictly needed, so I filed a PR to make checking external links optional and disabled by default: https://github.com/apache/beam/pull/8318. Let me know what you all think.
Kyle Weaver 🕴️ Software Engineer 🕴️ github.com/ibzib 🕴️ [email protected] 🕴️ +16502035555 On Mon, Apr 1, 2019 at 11:05 AM Kenneth Knowles <[email protected]> wrote: > +1 thanks for noticing and raising yet another source of non-hermeticity > (plus the docker constraint) > > On Mon, Apr 1, 2019 at 9:09 AM Andrew Pilloud <[email protected]> wrote: > >> +1 on this, particularly removing the dead link checker from default >> tests. It is effectively testing that ~20 random websites are up. I wonder >> if there is a way to limit it to locally testing links within the beam site? >> >> On Mon, Apr 1, 2019 at 3:54 AM Michael Luckey <[email protected]> >> wrote: >> >>> Hi, >>> >>> after playing around with Gradle build for a while, I would like to >>> suggest to remove ':beam-website:testWebsite target from Gradle's check >>> task. >>> >>> Rationale: >>> - the task seems to be very flaky. In fact, I always need to add '-x >>> :beam-website:testWebsite' to my build [1] >>> - task uses docker, which imho adds a (unnecessary) severe constraint on >>> the build task. E.g. A part time user is unable to execute these tests in a >>> docker environment >>> - these tests are accessing production environment. So myself hitting >>> the build several times an hour could be considered a DOS attack. >>> >>> Of course, these tests add lots of value and should definitely be >>> executed, but wouldn't it be sufficient, to run this task only dedicated, >>> i.e. by an explicit call to ':beam-website:testWebsite' o >>> ':websitePreCommit'? Any thoughts? >>> >>> best, >>> >>> michel >>> >>> [1] https://issues.apache.org/jira/browse/BEAM-6760 >>> >>
