* Ian Clarke <ian.clarke at gmail.com> [2009-12-13 17:08:37]: > As a test, I attempted to upload a 1GB avi file through FProxy. It > then proceeded to spend (at least) 20 minutes trying to compress it - > I say "at least" because I killed it before it was done. > > Of course, attempting to compress an already compressed file is > completely fruitless. >
Attempting to compress the file with the same compression algorithm is likely to be fruitless, yes... I had a patch somewhere which was trying to use file extensions to make educated guesses... but it never got merged because of conflicts (saces was working on metadatas) and lack of interrest on my side. Anyway, how do you determine if a file is already compressed or not without actually compressing it? Did you do the maths? In most cases, even though the data is already compressed it does make sense to recompress it with another algorithm (walltime-wise) before sending it over the (slow) wire. Iirc the node uses GZIP,BZIP2 and LZMA and inserts the smallest resulting file. At some point I even wanted to implement other algorithms like LZO and PAQ8P. After all, all we are talking about here is wasting some niced CPU cycles to earn both insert and download time! Florent
