I have started to implement the code for archive manifests aka zip manifests aka containers in 0.7. The architecture is that the first time we come across a given container, we extract it to an encrypted cache directory. We then use LRU on the extracted files from all the different extracted archives, with a user-defined maximum cache size, and the files padded. Now, this may be a net gain for zip files; it is the only way to implement a cache securely afaics (apart from an in memory one), since ZipFile (which is the only java.util.zip API to allow reasonably efficient random access) requires a plaintext file on disk. It is *definitely* a gain for tar.bz2 containers though!
We can implement tar.bz2 and tar.gz containers using the Apache Commons Compress library: http://jakarta.apache.org/commons/sandbox/compress/ There is a minor licensing issue with that as it is ASL2; the result is that we have to add an exception to the GPL e.g.: As a special exception, the author(s) permit linking of the Freenet binaries with the Jakarta Compress library, which is licensed under the terms of the Apache Software License, version 2. Authors of derivative works may [not] choose to remove this exception (and corresponding source code) [at their discretion]. The actual code involved is very straightforward, and given the above architecture, the efficiency gains of tar.gz (or tar.bz2, subject to benchmarks convincing me that it's not a major performance loss for the file sizes we are talking about) over zip would be a good thing. It is likely that normally the node will construct containers (clients will send the individual files over FCP, tell the node to bundle them; the node makes the archive, makes the metadata to fetch the archive, which is probably a splitfile despite all the compression, and to tell the future client to use it as a zip manifest). So it makes sense to make this change now. Opinions? -- Matthew J Toseland - toad at amphibian.dyndns.org Freenet Project Official Codemonkey - http://freenetproject.org/ ICTHUS - Nothing is impossible. Our Boss says so. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: Digital signature URL: <https://emu.freenetproject.org/pipermail/devl/attachments/20051025/c82bb74d/attachment.pgp>
