On 22.03.2012 16:23, Mark Phippard wrote: > On Thu, Mar 22, 2012 at 11:18 AM, Ashod Nakashian > <ashodnakash...@yahoo.com> wrote: >>> Design-wise I'm a bit surprised that the choice ended up being rolling >>> a custom file format. >> Personally I know not of any library that can deliver the requirements that >> we need (outlined in the doc). Again, if the requirements >> are in question, let's simplify them. If there is such a library, suggesting >> it will save us a lot of time and effort. Otherwise, using a >> Tar-like container will just not cut it. On the other hand, the proposed >> custom format is rather simple and its code shouldn't be >> complex. In fact, I suspect Tar is more complex (considering it must store >> more information than we do). > I am not sure what Daniel meant, but I had always just assumed we > would simply compress the files in the existing pristines. I think > your document does a nice job explaining why that is not good enough. > In that sense, I would also say that I was surprised by the choice of > a custom file format, but that does not mean I would question it. I > think your document does a nice job in revealing some of the subtle > complexities of this feature. That gives me more hope on progress > towards a solution.
I'd like to point out that there /is/ a library that handles storage, lookup, access and deletion of many small files in a single large one quite efficiently. Well tested, too, widely used, and configurable with regard to space reclamation. Moreover, we're already using that library. It's called SQLite. -- Brane
