On Mon, Dec 15, 2014 at 11:49 AM, Eran Duchan <[email protected]> wrote: > On Monday, December 15, 2014 5:44:38 PM UTC+2, Robert Muir wrote: >> >> That is not the case, blocks of documents are compressed together: > > > Thanks, Robert. > > I unscientifically swam around the code pivoting around this and saw that: > > This isn't tweakable - I can't choose to compress in larger chunks > 2.0.0 will have an option to use deflate for better compression > > So if I can't tweak _source compression, can I shove a _source of my own as > posted originally in (1)?
Its not really tweakable at all before lucene 5, thats why we added a higher compression option. Note this option is not just deflate but also uses a higher blocksize and other internal parameters. Using a higher blocksize (64kb) for deflate is really a simple workaround to get the feature out sooner than later, with the idea that people that choose BEST_COMPRESSION are willing to sacrifice some retrieval speed. Increasing blocksize has a negative cost on retrieval performance and is not really the best way overall to get better compression when there is high redundancy across documents. In the future I hope we can add preset dictionary support for sharing across blocks. So the current blocksize should really be seen as an internal thing. -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/CAMUKNZX8-SGzuNvvZ%3D_-Sec_%2Bq3svtLBE-_d3L%2B70TR8Nm_%3Drw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
