Hi Adrian Thanks for quick response.
When I loaded nearly 45m documents of test data with 3 replicas (each document approx 2K+ bytes in size), I got following info on storage: *health status index pri rep docs.count docs.deleted store.size pri.store.size* *green open test_insert 5 3 44985382 0 414.9gb 106.4gb* This indicates there was hardly any compression on physical storage*.* Hence my question. How do I find /estimate how much storage would be used for X number of documents of average size of Y kilobytes each. From above result, it appears to be no compression at all on all stored data. Thanks Ajay On Tuesday, April 14, 2015 at 1:13:37 PM UTC-4, Adrien Grand wrote: > > Hi, > > Data are both duplicated to suit different access patterns and compressed. > There are so many compression algorithms in-place that it would be hard to > be exhaustive, but we have for instance Frame-Of-Reference compression for > postings lists, LZ4 for the document store, bit packing for numeric doc > values, ... > > There are no configurations options available to configure compression > besides disabling features that you don't need (such as norms on fields > that you don't score on). In the next major version of elasticsearch (2.0) > there will be a setting to enable heavier compression though (which in > practice will use DEFLATE instead of LZ4 for the document store): > https://github.com/elastic/elasticsearch/pull/8863 > > > On Tue, Apr 14, 2015 at 6:47 PM, <[email protected] <javascript:>> wrote: > >> I would like to know if Elasticsearch documents/indices are stored in >> compressed format on disk . If yes, what type of compression options are >> available and it's performance overheads. >> >> and if these compression options are configurable. >> >> Thanks >> Ajay >> >> -- >> You received this message because you are subscribed to the Google Groups >> "elasticsearch" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/elasticsearch/8c63c25f-4f49-47f4-8d0a-772d3301f45c%40googlegroups.com >> >> <https://groups.google.com/d/msgid/elasticsearch/8c63c25f-4f49-47f4-8d0a-772d3301f45c%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Adrien > -- You received this message because you are subscribed to the Google Groups "elasticsearch" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/elasticsearch/50726923-3199-457b-a53e-24978cb94510%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
