As a software developer I've used git for years and have found it the
perfect solution for source control.
Lately I have found myself using git in a unique use-case - modifying
DNA/RNA sequences and storing them in git, which are essentially
software/source code for cells/life. For Bacteria and Viruses the
repo's are very small <10mb & compress nicely.
However on the extreme end of the spectrum a human genome can run in
at 50gb or say ~1gb per file/chromosome.
Now, this is not the binary problem and it is not the same as storing
media inside git - I have reviewed the solutions that exist for the
binary problem, such as git-annex, git-media & bup. But they don't
provide the featureset of git and the data i'm storing is more like
plaintext sourcecode with relatively small edits per commit.
I have googled and asked in #git which discussion mostly revolved
around these tools.
The only project that holds interest is a 2009 project, git-bigfiles -
however it is abit dated & the author is not interested in reviving
this project - referring me to git-annex. Unfortunately.
With that background;
I wanted to discuss the problems with git and how I can contribute to
the core project to best solve them.
>From my understanding the largest problem revolves around git's delta
discovery method, holding 2 files in memory at once - is there a
reason this could not be adapted to page/chunk the data in a sliding
window fashion ?
Are there any other issues I need to know about, is anyone else
working on making git more capable of handling large source files that
I can collaborate with?
Thanks for your time,
To unsubscribe from this list: send the line "unsubscribe git" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html