Hi Fran,

That's.. certainly problematic.

A local mirror certainly works, though as you know it's a bit more to set
up, and ideally there wouldn't be a problem here :)

I'll look into python-github2. The way things are currently done, the
underlying code actually doesn't know about GitHub. The admin UI knows about
it and can provide some paths, but under the hood it's just accessing Git
servers and URLs. We may want to change that in general in the future, but
it's a bit of rethinking.

Rate limiting would be a good solution. I'm not sure off-hand how best to do
that just yet, but it's something we can think about.

I wonder if there's any additional information we get from the 403 Forbidden
from GitHub that could tell us we're being rate-limited. If so, perhaps we
could respond to that and wait a certain amount of time. Not holding my
breath on that though.

Something we have discussed doing in the past is writing a server (probably
for sale for a reasonable price) that allows scaling out of the repo
communication, patching, and diffing operations, meaning that stuff wouldn't
have to bog down the main web server, and could expand across multiple
servers for really heavy loads. I think incorporating rate limiting into
that would be beneficial, and it'd be easier to do from that point of view
because basically, things would go into a queue anyway, so it could manage
the time between processes.

Christian

-- 
Christian Hammond - chip...@chipx86.com
Review Board - http://www.reviewboard.org
VMware, Inc. - http://www.vmware.com


On Thu, Mar 17, 2011 at 5:08 AM, Fran Ruiz <quiesa...@gmail.com> wrote:

> Hi everyone,
>
> I am using Review Board with GitHub and I am getting some inaccurate errors
> while uploading big diff files:
>
> The file "<filename>" (revision <revision>) was not found in the repository
>
> And as the file does exist in the repository, I took a look at the Review 
> Board server log and the error was a *"**HTTP Error 403: Forbidden"*.
>
>
> After some research, I found at GitHub  <http://goog_644959457>General API 
> Information - Limitations <http://develop.github.com/p/general.html> that:
>
> Currently we are limiting API calls to 60 per minute. This may change in
> the future, or possibly per user at some point, but if you try to access the
> API more than 60 times in a minute, it will start giving you "access denied"
> errors.
>
> What means that diff files containing more than 60 files will probably fail 
> (and that's the case).
>
>
> Thus, I have a solution and a proposal. The *solution* for me has been to use 
> a local mirror of the GitHub repository, keeping it updated and changing the 
> Review Board repository configuration to use it instead of the remote one. 
> And the *proposal* is either to use the python-github2 
> <https://github.com/ask/python-github2> package or, as it's not finished, to 
> implement a *requests_per_second* limit (sleep...).
>
>
> What do you think about it?
>
>
> Fran Ruiz
>
> 2degrees Ltd. <http://www.2degreesnetwork.com/>
>
>  --
> Want to help the Review Board project? Donate today at
> http://www.reviewboard.org/donate/
> Happy user? Let us know at http://www.reviewboard.org/users/
> -~----------~----~----~----~------~----~------~--~---
> To unsubscribe from this group, send email to
> reviewboard+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/reviewboard?hl=en

-- 
Want to help the Review Board project? Donate today at 
http://www.reviewboard.org/donate/
Happy user? Let us know at http://www.reviewboard.org/users/
-~----------~----~----~----~------~----~------~--~---
To unsubscribe from this group, send email to 
reviewboard+unsubscr...@googlegroups.com
For more options, visit this group at 
http://groups.google.com/group/reviewboard?hl=en

Reply via email to