On 2/3/2014 3:43 AM, Alexander Clouter wrote:
Maybe better to use sparse files[1] and then you get all the benefits of using mmap()? Of course if your target platform includes OS X you are out of luck for sparse files.
That would be useful to also have but I still would like the maps to be compressed. A combination of compression and sparse files would be a much better solution though so I'll probably try to get my maps to support it as well.
On 2/3/2014 11:51 AM, Lasse Collin wrote:
The XZ Index is near the end of the file. It stores the compressed and uncompressed sizes of the XZ Blocks. But this doesn't help you due to reasons explained above.
But wouldn't updating those indices be relatively fast? I realize the new XZ Block would be bigger which is why I'm looking to insert the data instead of overwrite it. I imagined something like: find block, insert new compressed data over block (while pushing down the blocks below), and then update indices to reflect changes. Or would this not be efficient?
There may be better ways to do it and I wouldn't be surprised if there were good libraries to do this kind of things (I don't know any but I haven't searched either). You don't need a full file system, something relatively simple should be enough.
I suppose my problem is that I don't really understand what it is I'm looking for. I knew I wanted the maps to be compressed and I knew I wanted them to support random access. What else should I be looking for?
-- *Brandon J. Fergerson*
