Re: git-p4 out of memory for very large repository

2013-09-07 Thread Pete Wyckoff
cmt...@gmail.com wrote on Fri, 06 Sep 2013 15:03 -0400: > Finally, I claim success! Unfortunately I did not try either of the OOM > score or strace suggestions - sorry! After spending so much time on > this, I've gotten to the point that I'm more interested in getting it to > work than in figurin

Re: git-p4 out of memory for very large repository

2013-09-06 Thread Corey Thompson
On Mon, Sep 02, 2013 at 08:42:36PM +0100, Luke Diamand wrote: > I guess you could try changing the OOM score for git-fast-import. > > change /proc//oomadj. > > I think a value of -31 would make it very unlikely to be killed. > > On 29/08/13 23:46, Pete Wyckoff wrote: > >I usually just do "git p4

Re: git-p4 out of memory for very large repository

2013-09-02 Thread Luke Diamand
I guess you could try changing the OOM score for git-fast-import. change /proc//oomadj. I think a value of -31 would make it very unlikely to be killed. On 29/08/13 23:46, Pete Wyckoff wrote: cmt...@gmail.com wrote on Wed, 28 Aug 2013 11:41 -0400: On Mon, Aug 26, 2013 at 09:47:56AM -0400, Cor

Re: git-p4 out of memory for very large repository

2013-08-29 Thread Pete Wyckoff
cmt...@gmail.com wrote on Wed, 28 Aug 2013 11:41 -0400: > On Mon, Aug 26, 2013 at 09:47:56AM -0400, Corey Thompson wrote: > > You are correct that git-fast-import is killed by the OOM killer, but I > > was unclear about which process was malloc()ing so much memory that the > > OOM killer got invoke

Re: git-p4 out of memory for very large repository

2013-08-28 Thread Corey Thompson
On Mon, Aug 26, 2013 at 09:47:56AM -0400, Corey Thompson wrote: > You are correct that git-fast-import is killed by the OOM killer, but I > was unclear about which process was malloc()ing so much memory that the > OOM killer got invoked (as other completely unrelated processes usually > also get ki

Re: git-p4 out of memory for very large repository

2013-08-26 Thread Corey Thompson
On Sun, Aug 25, 2013 at 11:50:01AM -0400, Pete Wyckoff wrote: > Modern git, including your version, do "streaming" reads from p4, > so the git-p4 python process never even holds a whole file's > worth of data. You're seeing git-fast-import die, it seems. It > will hold onto the entire file conten

Re: git-p4 out of memory for very large repository

2013-08-25 Thread Pete Wyckoff
cmt...@gmail.com wrote on Fri, 23 Aug 2013 07:48 -0400: > On Fri, Aug 23, 2013 at 08:16:58AM +0100, Luke Diamand wrote: > > On 23/08/13 02:12, Corey Thompson wrote: > > >Hello, > > > > > >Has anyone actually gotten git-p4 to clone a large Perforce repository? > > > > Yes. I've cloned repos with a

Re: git-p4 out of memory for very large repository

2013-08-23 Thread Corey Thompson
On Fri, Aug 23, 2013 at 08:42:44PM +0100, Luke Diamand wrote: > > I think I've cloned files as large as that or larger. If you just want to > clone this and move on, perhaps you just need a bit more memory? What's the > size of your physical memory and swap partition? Per process memory limit? >

Re: git-p4 out of memory for very large repository

2013-08-23 Thread Luke Diamand
I think I've cloned files as large as that or larger. If you just want to clone this and move on, perhaps you just need a bit more memory? What's the size of your physical memory and swap partition? Per process memory limit? On 23 Aug 2013 12:59, "Corey Thompson" wrote: On 23/08/13 12:59, Corey

Re: git-p4 out of memory for very large repository

2013-08-23 Thread Corey Thompson
On Fri, Aug 23, 2013 at 07:48:56AM -0400, Corey Thompson wrote: > Sorry, I guess I could have included more details in my original post. > Since then, I have also made an attempt to clone another (slightly more > recent) branch, and at last had success. So I see this does indeed > work, it just se

Re: git-p4 out of memory for very large repository

2013-08-23 Thread Corey Thompson
On Fri, Aug 23, 2013 at 08:16:58AM +0100, Luke Diamand wrote: > On 23/08/13 02:12, Corey Thompson wrote: > >Hello, > > > >Has anyone actually gotten git-p4 to clone a large Perforce repository? > > Yes. I've cloned repos with a couple of Gig of files. > > >I have one codebase in particular that g

Re: git-p4 out of memory for very large repository

2013-08-23 Thread Luke Diamand
On 23/08/13 02:12, Corey Thompson wrote: Hello, Has anyone actually gotten git-p4 to clone a large Perforce repository? Yes. I've cloned repos with a couple of Gig of files. I have one codebase in particular that gets to about 67%, then consistently gets get-fast-import (and often times a fe

git-p4 out of memory for very large repository

2013-08-22 Thread Corey Thompson
Hello, Has anyone actually gotten git-p4 to clone a large Perforce repository? I have one codebase in particular that gets to about 67%, then consistently gets get-fast-import (and often times a few other processes) killed by the OOM killer. I've found some patches out there that claim to resolve