Re: github quota limit when scanning with the addition of tags

2018-10-04 Thread j . knurek
I've started looking into this issue again. It's been an ongoing problem, that only continues to get worse over time. We're using the basic-branch-build-strategies-plugin at the moment, which provides an option for not building tags older than a week. But this option provides no value in

Re: github quota limit when scanning with the addition of tags

2018-01-07 Thread R. Tyler Croy
Sender: jenkinsci-users@googlegroups.com On-Behalf-Of: ty...@monkeypox.org Subject: Re: github quota limit when scanning with the addition of tags Message-Id: <20180106042952.4wlg7enl4dbnn...@blackberry.coupleofllamas.com> Recipient: andrew.h...@nab.com.au The information contained in this

Re: github quota limit when scanning with the addition of tags

2018-01-05 Thread R. Tyler Croy
(replies inline) On Fri, 05 Jan 2018, Alicia Doblas wrote: > Hi, > > a couple of months ago we had the same problem. After improving the > discover filter of the branch/tag/PR calls, there weren't too much to > do...so we decided to "by-pass" the api of github by using different users. >

Re: github quota limit when scanning with the addition of tags

2018-01-05 Thread Alicia Doblas
Hi, a couple of months ago we had the same problem. After improving the discover filter of the branch/tag/PR calls, there weren't too much to do...so we decided to "by-pass" the api of github by using different users. Looks like the api limit applies to a single user, so the "solution" for us

Re: github quota limit when scanning with the addition of tags

2018-01-04 Thread Stephen Connolly
On 4 January 2018 at 16:43, wrote: > Ok, than I think I misunderstand what the scan is doing. > During the scan, Jenkins creates a list in memory of all branches, tags, > PRs. It does it from a single api call? Or from an api call for each type? > At least one API

Re: github quota limit when scanning with the addition of tags

2018-01-04 Thread j . knurek
Ok, than I think I misunderstand what the scan is doing. During the scan, Jenkins creates a list in memory of all branches, tags, PRs. It does it from a single api call? Or from an api call for each type? And then while iterating over that list, for each entity Jenkins makes an api call to get

Re: github quota limit when scanning with the addition of tags

2018-01-04 Thread Stephen Connolly
If you know those tags will never match, you could add a filter to exclude them from discovery. Part of the issue here is that Multibranch doesn't know if the SCMCriteria has changed from the last time it saw that revision (because Jenkins config is a filesystem, who knows what was restored,

Re: github quota limit when scanning with the addition of tags

2018-01-04 Thread j . knurek
I do want tags. I want tags very much. I'm very happy this feature is finally available. There just happens to be some tags in that repo that reference commits in which no Jenkinsfile exists, and I happened to copy those examples. Here is a better example: Checking tag v1.1.0

Re: github quota limit when scanning with the addition of tags

2018-01-04 Thread Stephen Connolly
On 4 January 2018 at 13:27, wrote: > @Stephen > You mention that caching the responses would "save about 50% of the > requests." That seems like a significant savings to me. > > I'm also wondering, I'm seeing a lot of things like this in the scan log: > > Checking

Re: github quota limit when scanning with the addition of tags

2018-01-04 Thread j . knurek
@Stephen You mention that caching the responses would "save about 50% of the requests." That seems like a significant savings to me. I'm also wondering, I'm seeing a lot of things like this in the scan log: Checking tag v0.28.1

Re: github quota limit when scanning with the addition of tags

2018-01-03 Thread j . knurek
> > There are only two good reasons to scan periodically: > 1. To recover from missed events (keep in mind that follow-up commits will > typically recover anyway, so the only case here is a commit before bedtime > not being built by morning because that event was not delivered by GitHub) >

Re: github quota limit when scanning with the addition of tags

2018-01-03 Thread Stephen Connolly
On 3 January 2018 at 14:41, Stephen Connolly < stephen.alan.conno...@gmail.com> wrote: > This is the limitation of 5000 requests per hour. > > Ideally we would look into caching the github responses so that duplicate > requests could be eliminated... but my preliminary analysis shows that > would

github quota limit when scanning with the addition of tags

2018-01-03 Thread j . knurek
Now that we've added *Discover tags*[1 ] and a *Build everything*[2 ] strategy, we're running into Github quota limits quite frequently. 18:58:09