I really hate how INFRA just drops this stuff without any warning. I wish
they’d work with us for once. Anyway, this is going to totally destroy
precommit-admin. It needs a complete rewrite: it’s highly inefficient and
doesn’t work with pipeline jobs. But it’s going to take time and I know that
right now I personally don’t have any.
A few weeks ago (when I did have time) I was toying around with the
jenkins<->lira plugin for patch submission. It works by sending the full JIRA
key (e.g., YETUS-1) but it doesn’t appear to be possible to limit it to just
attachment changes (wether this is on the Jenkins side or the JIRA side, I’m
unsure). As a result of those experiments, I’ve been thinking that it would be
worthwhile to make a backwards incompatible precommit-admin that does a few
things differently:
* always sends the full key as JIRA_ISSUE_KEY to be compatible with the
web hook and with multibranch pipelines (since there will likely only be one
job for multiple JIRA projects, e.g., Hadoop)
* dumps out a file that lists what issue should be triggered and
includes some sample Jenkinsfile groovy code to read it and talk to pipelines[1]
* can use JIRA_ISSUE_KEY as input to determine if that job should be
processed; this eliminates the 10 minute wait(!)
* without JIRA_ISSUE_KEY, go ahead and query JIRA (via REST calls
rather than the search xml API); this deals with the “Jenkins was down” problem
Configuration would be very different than what we have today, obviously.
(Minimally, need a project -> job mapping so that a HDFS-xxx JIRA issue
triggers the Hadoop multibranch pipeline job.) It will probably also put us on
track to EOL the various ‘Precommit-XXX-admin’ jobs. Given the forced march to
Github, that may not necessarily be a bad thing.
1 - Jenkins pipelines don’t support token auth for some reason. This
completely breaks the ASF model and I can’t see a way to unbreak it easily.
Dynamic job triggering pretty much requires something plugged into the Jenkins
java code. :( :(
> On Jan 28, 2019, at 8:00 AM, Sean Busbey <[email protected]> wrote:
>
> heads up, since this could impact Release Doc Maker. I haven't done a
> pass of the code yet. Will try to find time this week.
>
> ---------- Forwarded message ---------
> From: Daniel Gruno <[email protected]>
> Date: Sun, Jan 27, 2019 at 11:10 AM
> Subject: [Notice] Rate limiting in effect on JIRA, BZ, Moin
> To: [email protected] <[email protected]>
>
>
> Hi folks,
>
> Over the past few days we have implemented rate limiting on selected
> services across the ASF; starting with JIRA, BugZilla and Moin Wiki.
>
> IF YOU ARE A NORMAL USER:
> #########################
> This very likely will never affect you, and you can go about your
> business just like normal :) If you DO experience errors or 429 (rate
> limited) response codes, please do let us know so we can address it.
>
>
> IF YOU ARE A ROBOT/CYBORG/COMPUTRON:
> ####################################
> There are now limits in place for how much CPU time you can use, varying
> from service to service. If you get limited, you will receive a HTTP 429
> response instead of the normal 200, and a short text blob will explain
> that you have crossed our resource limits and have been rate-limited. It
> will also explain why, and when you can expect to be unblocked again
> (generally within two minutes time). Scrapers, bots etc using our
> services should check for a 429 response code and act accordingly (or
> just slow down the discovery pace in general, as that benefits all of
> us).
>
> Rate limits are applied across IP blocks to discourage distributed
> abuse, thus if you have 1.2.3.4 abusing a service, 1.2.3.5 would
> potentially also be affected by the rate limits till they expire.
>
> Later this year, we will be rolling out rate limits on more services,
> and we encourage people automating tasks to honor the 429 responses
> across all ASF services.
>
> With regards, Daniel on behalf of ASF Infra.
>
>
>
> --
> busbey
