Re: git blame [was: Reducing CPU load on git server]

On wo, 2016-08-31 at 01:42 -0400, Jeff King wrote: > On Tue, Aug 30, 2016 at 12:46:20PM +0200, Jakub Narębski wrote: > > > I wonder if having support for 'git blame ' in Git core would > > be something interesting to Git users.  I once tried to implement it, > > but it went nowhere.  Would it be

Re: Reducing CPU load on git server

On Mon, Aug 29, 2016 at 03:41:59PM -0700, W. David Jarvis wrote: > We have an open support thread with the GHE support folks, but the > only feedback we've gotten so far on this subject is that they believe > our CPU load is being driven by the quantity of fetch operations (an > average of 16

Re: git blame [was: Reducing CPU load on git server]

On Tue, Aug 30, 2016 at 12:46:20PM +0200, Jakub Narębski wrote: > W dniu 29.08.2016 o 23:31, Jeff King pisze: > > > Blame-tree is a GitHub-specific command (it feeds the main repository > > view page), and is a known CPU hog. There's more clever caching for that > > coming down the pipe, but

git blame [was: Reducing CPU load on git server]

W dniu 29.08.2016 o 23:31, Jeff King pisze: > Blame-tree is a GitHub-specific command (it feeds the main repository > view page), and is a known CPU hog. There's more clever caching for that > coming down the pipe, but it's not shipped yet. I wonder if having support for 'git blame ' in Git core

Re: Reducing CPU load on git server

> Pegging CPU for a few seconds doesn't sound out-of-place for > pack-objects serving a fetch or clone on a large repository. And I can > certainly believe "minutes", especially if it was not serving a fetch, > but doing repository maintenance on a large repository. > > Talk to GitHub Enterprise

Re: Reducing CPU load on git server

On ma, 2016-08-29 at 13:57 -0700, W. David Jarvis wrote: > >  * If you do need branches consider archiving stale tags/branches > > after some time. I implemented this where I work, we just have a > > $REPO-archive.git with every tag/branch ever created for a given > > $REPO.git, and delete refs

Re: Reducing CPU load on git server

On Mon, Aug 29, 2016 at 12:16:20PM -0700, W. David Jarvis wrote: > > Do you know which processes are generating the load? git-upload-pack > > does the negotiation, and then pack-objects does the actual packing. > > When I look at expensive operations (ones that I can see consuming > 90%+ of a

Re: Reducing CPU load on git server

> * Consider having that queue of yours just send the pushed payload > instead of "pull this", see git-bundle. This can turn this sync entire > thing into a static file distribution problem. As far as I know, GHE doesn't support this out of the box. We've asked them for essentially this, though.

Re: Reducing CPU load on git server

On Sun, Aug 28, 2016 at 9:42 PM, W. David Jarvis wrote: > I've run into a problem that I'm looking for some help with. Let me > describe the situation, and then some thoughts. Just a few points that you may not have considered, and I didn't see mentioned in this

Re: Reducing CPU load on git server

> So your load is probably really spiky, as you get thundering herds of > fetchers after every push (the spikes may have a long flatline at the > top, as it takes time to process the whole herd). It is quite spiky, yes. At the moment, however, the replication fleet is relatively small (at the

Re: Reducing CPU load on git server

On Mon, Aug 29, 2016 at 12:46:27PM +0200, Jakub Narębski wrote: > > So your load is probably really spiky, as you get thundering herds of > > fetchers after every push (the spikes may have a long flatline at the > > top, as it takes time to process the whole herd). > > One solution I have heard

Re: Reducing CPU load on git server

W dniu 29.08.2016 o 07:47, Jeff King pisze: > On Sun, Aug 28, 2016 at 12:42:52PM -0700, W. David Jarvis wrote: > >> The actual replication process works as follows: >> >> 1. The primary git server receives a push and sends a webhook with the >> details of the push (repo, ref, sha, some metadata)

Re: Reducing CPU load on git server

On Sun, Aug 28, 2016 at 12:42:52PM -0700, W. David Jarvis wrote: > The actual replication process works as follows: > > 1. The primary git server receives a push and sends a webhook with the > details of the push (repo, ref, sha, some metadata) to a "publisher" > box > > 2. The publisher

Re: Reducing CPU load on git server

My assumption is that pack bitmaps are enabled since the primary server is a GitHub Enterprise instance, but I'll have to confirm. On Sun, Aug 28, 2016 at 2:20 PM, Jakub Narębski wrote: > W dniu 28.08.2016 o 21:42, W. David Jarvis pisze: > >> The ultimate goal for us is just

Re: Reducing CPU load on git server

W dniu 28.08.2016 o 21:42, W. David Jarvis pisze: > The ultimate goal for us is just figuring out how we can best reduce > the CPU load on the primary instance so that we don't find ourselves > in a situation where we're not able to run basic git operations > anymore. I assume that you have

Reducing CPU load on git server

Hi all - I've run into a problem that I'm looking for some help with. Let me describe the situation, and then some thoughts. The company I work for uses git. We use GitHub Enterprise as a frontend for our primary git server. We're using Chef solo to manage a fleet of upwards of 10,000 hosts,