Re: filter-branch IO optimization

2012-10-12 Thread Enrico Weigelt
Hi, The usual advice is use an index-filter instead. It's *much* faster than a tree filter. However: I've tried the last example from git-filter-branch manpage, but failed. Seems like the GIT_INDEX_FILE env variable doesnt get honoured by git-update-index, no index.new file created, and so

Re: filter-branch IO optimization

2012-10-12 Thread Enrico Weigelt
snip Did some more experiments, and it seems that missing index file isn't automatically created. When I instead copy the original index file to the temporary location, it runs well. But I still have to wait for the final result to check whether it really overwrites the whole index or just adds

Re: filter-branch IO optimization

2012-10-12 Thread Enrico Weigelt
Hi folks, now finally managed the index-filter part. The main problem, IIRC, was that git-update-index didn't automatically create an empty index, so I needed to explicitly copy in (manually created it with an empty repo). My current filter code is: if [ ! $GIT_AUTHOR_EMAIL ] [ !

Re: filter-branch IO optimization

2012-10-12 Thread Jeff King
On Fri, Oct 12, 2012 at 04:49:54PM +0200, Enrico Weigelt wrote: The usual advice is use an index-filter instead. It's *much* faster than a tree filter. However: I've tried the last example from git-filter-branch manpage, but failed. Seems like the GIT_INDEX_FILE env variable doesnt

Re: filter-branch IO optimization

2012-10-11 Thread Johannes Sixt
Am 11.10.2012 17:39, schrieb Enrico Weigelt: The main goal of this filtering is splitting out many modules from a large upstream repo into their own downstream repos. ... The next step I have in mind is using --subdirectory-filter, but open questsions are: * does it suffer from the same

Re: filter-branch IO optimization

2012-10-11 Thread Thomas Rast
Enrico Weigelt enrico.weig...@vnc.biz writes: for certain projects, I need to regularily run filter-branch on quite large repos (10k commits), and that needs to be run multiple times, which takes several hours, so I'm looking for optimizations. [...] #2: run a tree-filter which: *