On Sun, May 27, 2012 at 11:42:55AM -0400, Joey Hess wrote:
maximilian attems wrote:
Trying out git annex on a rather large collection (~200k taking 120Gb)
of bzip2 data snapshot files. In contrast to other git commands git annex
operations seem not to properly scale to that number of files.
The scalabilty issue comes due to the symlink usage of git annex.
Modern fs are not yet made to be that fast for 100k of symlinks.
A rm -rf .git of any !annexed git repo is blasing fast,
but due to the symlink mess that git annex uses the same cmd takes
hours on the same repo if git annex is
maximilian attems wrote:
The scalabilty issue comes due to the symlink usage of git annex.
Modern fs are not yet made to be that fast for 100k of symlinks.
Hmm, I have never heard of any symlink specific speed limitations of
filesystems. What is it?
--
see shy jo
signature.asc
Description:
maximilian attems wrote:
Trying out git annex on a rather large collection (~200k taking 120Gb)
of bzip2 data snapshot files. In contrast to other git commands git annex
operations seem not to properly scale to that number of files.
What kind of operations?
The limiting factor is typically
Package: git-annex
Version: 3.20120511
Severity: normal
hello
Trying out git annex on a rather large collection (~200k taking 120Gb)
of bzip2 data snapshot files. In contrast to other git commands git annex
operations seem not to properly scale to that number of files.
I can't provide public
5 matches
Mail list logo
