Hi Arief, please refer to https://www.elastic.co/de/blog/elasticsearch-storage-the-true-story-2.0 and https://www.elastic.co/guide/en/elasticsearch/reference/2.3/index-modules.html#_static_index_settings for details about the Lucene compression codecs and disk space requirements for Elasticsearch indices.
Cheers, Jochen On Wednesday, 20 July 2016 11:36:06 UTC+2, Arief Hydayat wrote: > > Hi Jochen, > > Thank you for your reply. After these 5 days the disk space utilization > increase quite high. > /dev/dm-0 212G 78G 126G 38% / > > Seems need to add more disk or just listed server that need to send all > those log to the OVA Graylog. What do you think? > > Anyway regarding to the "how well they can be compressed", by default > Graylog will compress the data under each index folder? > root@graylog:~# ls -lrt > /var/opt/graylog/data/elasticsearch/graylog/nodes/0/indices/graylog_7/0/index/ > total 1946504 > -rw------- 1 graylog graylog 0 Jul 8 05:59 write.lock > -rw------- 1 graylog graylog 974182 Jul 10 16:55 _2pkd.fdx > -rw------- 1 graylog graylog 513513754 Jul 10 16:55 _2pkd.fdt > -rw------- 1 graylog graylog 5164198 Jul 10 16:58 _2pkd_Lucene50_0.tip > -rw------- 1 graylog graylog 404791138 Jul 10 16:58 _2pkd_Lucene50_0.tim > -rw------- 1 graylog graylog 590583874 Jul 10 16:58 _2pkd_Lucene50_0.pos > -rw------- 1 graylog graylog 436255903 Jul 10 16:58 _2pkd_Lucene50_0.doc > -rw------- 1 graylog graylog 3126 Jul 10 16:59 _2pkd_Lucene54_0.dvm > -rw------- 1 graylog graylog 31883485 Jul 10 16:59 _2pkd_Lucene54_0.dvd > -rw------- 1 graylog graylog 98 Jul 10 16:59 _2pkd.nvm > -rw------- 1 graylog graylog 10000843 Jul 10 16:59 _2pkd.nvd > -rw------- 1 graylog graylog 4707 Jul 10 16:59 _2pkd.fnm > -rw------- 1 graylog graylog 568 Jul 10 16:59 _2pkd.si > -rw------- 1 graylog graylog 230 Jul 14 03:18 segments_35 > > Thank for the tools link. Been check between 30 - 50 messages/sec still > consider as High Availability setup :-| > > On Friday, July 15, 2016 at 4:49:58 PM UTC+8, Jochen Schalanda wrote: >> >> Hi Arief, >> >> That's impossible to say and depends on how many log messages those >> servers will send, how big they are, and how well they can be compressed. >> And of course it depends on how many indices with this number of documents >> you need to retain. >> >> You can get an educated guess about the hardware requirements at >> https://www.graylog.org/tools/sizing-estimator, so give it a try. >> >> Cheers, >> Jochen >> > -- You received this message because you are subscribed to the Google Groups "Graylog Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/graylog2/86c4c8a0-0898-46a7-a09b-42608b6f3812%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
