On Monday 04 Aug 2003 15:03, [EMAIL PROTECTED] wrote: > On Mon, 04 Aug 2003 06:05:45 -0700 Gordan <[EMAIL PROTECTED]> wrote: > >Are there any objections to the above spec, other than objections > >to using any non-manual compression? > > I've been following this thread with increasing disgust so that the time > has come to express my thoughts on this issue. ;)
An unbiased view, then. :-) > I think there are to many forward, backward and inter-node compatibility > issues with general single file compression. And I doubt it will be > generally useful. I do recognize the fact that there would be inter-operability issues for some time. I do not, however, recognize that these issues would linger on for very long. Upgrading is easy, and the development cycle is currently short enough that all the old nodes would probably be replaced within a matter of weeks. > So, compression would be beneficial in only two cases: > A) compress compressible data, mostly text/plain and text/html > B) compress compressed data slightly to fit in a smaller log2size key I don't really consider those to be a separate case. If the compression doesn't yield a size reduction required to go to a smaller key size, then we throw away the file we were trying to compress and just insert it uncompressed. Compressing data that is already compressed shouldn't really be an option. > I only now of three applications, which would benefit from case A: > 1. Frost uses plain text messages and control-messages. So Frost should > improve its protocol and compress the messages on its own! Because of > the restricted domain the compression algorithm could be highly optimized. > 2. Freenet Message Board (FMB) uses plain text messages and control- > messages. So FMB should improve its protocol and ... see point 1. Ideally there should be a layer between the node and the client that provides a uniform access to insertion and retrieval of files. It would be this layer that should, IMHO, do the compression... That way a lot of compatibility issues are removed, and the node would be left to deal with files and routing, without having to worry about the content or file types. In the meantime, fproxy provides an interface for inserting and retrieving files using http, and most functions can be performed this way, using existing tools and libraries. This is why it was suggested that this should be implemented in fproxy. > 3. Freesites use plain text html files. I'm not much into this but in > the good ol' web my browser sends an "Accept-Encoding: gzip, deflate" > to the server and I know of at least one server sending me gzipped HTMLs. > Can this be extended to Freenet? E.g. by including in the content's > metadata a "Info.Encoding=gzip" line? I thought about this. The only difference there is whether you are going to leave the decompression to the end-client or the proxy. For uniformity, it should probably be the proxy. Additionally, seen as those of us supporting the idea of compression have kind of almost agreed on zip format, this would not work because zip compression is not compatible with gzip. > Case B (compress compressed data slightly) should happen very rarely. > It could be that an uncompressed 520KiB PDF occupying an 1 MiB key could > fit --when compressed-- into a 512 KiB key. But I think it's up to the > kindness of the content provider to compress the data where it's gonna > be useful. I don't believe in compressing compressed files, and I don't think this should be left entirely to the content provider. > To summarize: > - I see no need for an automatic compression done by Fproxy. And neither > by the node! ... good grief! I agree that the node is the wrong place to apply this. Node should handle the routing and the keys. Whether the file is compressed or not should be irrelevant at that logical level. Compression should be performed on a higher level, e.g. fproxy. > - Frost and FMB should compress their messages It depends on how big/compressible their messages are. I am not too sure that most messages would give worth-while compression, because they are so small (typically in the 4KB-8KB range, IIRC). > - Freesites could be gzipped, if this is a web standard and if it can > be done without reinventing the wheel. This would actually be a worse solution because fproxy would still have to understand how to decompress data - not all browsers support all decompression algorithms. It would create more exception handling, which is not desirable. Additionally, it would mean including the gzip library into the freenet distribution, when zip is already built in. Gordan _______________________________________________ devl mailing list [EMAIL PROTECTED] http://hawk.freenetproject.org:8080/cgi-bin/mailman/listinfo/devl
