It could also be that the C version is a lot more efficient than the Java version and it could take longer regardless. I could not find a benchmark on that, but C is usually better for bit twiddling.
wunder On 10/30/08 10:36 PM, "Otis Gospodnetic" <[EMAIL PROTECTED]> wrote: > man gzip: > > -# --fast --best > Regulate the speed of compression using the specified digit #, > where -1 or --fast indicates the fastest compres- > sion method (less compression) and -9 or --best indicates the > slowest compression method (best compression). The > default compression level is -6 (that is, biased towards high > compression at expense of speed). > > > So it could be better than the factor of 2, but also take longer. :) > > Otis > -- > Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch > > > > ----- Original Message ---- >> From: Walter Underwood <[EMAIL PROTECTED]> >> To: solr-user@lucene.apache.org >> Sent: Thursday, October 30, 2008 11:52:47 AM >> Subject: Re: replication handler - compression >> >> About a factor of 2 on a small, optimized index. Gzipping took 20 seconds, >> so it isn't free. >> >> $ cd index-copy >> $ du -sk >> 134336 . >> $ gzip * >> $ du -sk >> 62084 . >> >> wunder >> >> On 10/30/08 8:20 AM, "Otis Gospodnetic" wrote: >> >>> Yeah. I'm just not sure how much benefit in terms of data transfer this >>> will >>> save. Has anyone tested this to see if this is even worth it? >>> >>> >>> Otis >>> -- >>> Sematext -- http://sematext.com/ -- Lucene - Solr - Nutch >>> >>> >>> >>> ----- Original Message ---- >>>> From: Erik Hatcher >>>> To: solr-user@lucene.apache.org >>>> Sent: Thursday, October 30, 2008 9:54:28 AM >>>> Subject: Re: replication handler - compression >>>> >>>> +1 - the GzipServletFilter is the way to go. >>>> >>>> Regarding request handlers reading HTTP headers, yeah,... this will >>>> improve, >>>> for >>>> sure. >>>> >>>> Erik >>>> >>>> On Oct 30, 2008, at 12:18 AM, Chris Hostetter wrote: >>>> >>>>> >>>>> : You are partially right. Instead of the HTTP header , we use a request >>>>> : parameter. (RequestHandlers cannot read HTP headers). If the param is >>>>> >>>>> hmmm, i'm with walter: we shouldn't invent new mechanisms for >>>>> clients to request compression over HTTP from servers. >>>>> >>>>> replicatoin is both special enough and important enough that if we had to >>>>> add special support to make that information available to the handler on >>>>> the master we could. >>>>> >>>>> but frankly i don't think that's neccessary: the logic to turn on >>>>> compression if the client requests it using "Accept-Encoding: gzip" is >>>>> generic enough that there is no reason for it to be in a handler. we >>>>> could easily put it in the SolrDispatchFilter, or even in a new >>>>> ServletFilte (i'm guessing iv'e seen about 74 different implementations of >>>>> a GzipServletFilter in the wild that could be used as is. >>>>> >>>>> then we'd have double wins: compression for replication, and compression >>>>> of all responses generated by Solr if hte client requests it. >>>>> >>>>> -Hoss >>> >
