On Wednesday, October 19, 2016 at 5:45:33 AM UTC-6, Mathias Herberts wrote: > > Efficiency of compression depends on different factors, including interval > between timestamps, resolution of said timestamps, volatility and type of > values. > > The 2 or 3 bytes footprint is usually not achievable if you store your > timestamps with milli, micro or nanosecond precision as the delta between > consecutive timestamps will vary and will therefore not compress as well. > > For timestamps, the biggest factor is whether your values have regular or irregular time intervals. If they are recorded at regular intervals (i.e. every 10s, or 100ms), we can run length encode them regardless of precision. This can allow the footprint to drop well below 2-3 bytes.
If the intervals are irregular, it's really important that you store your timestamps at the right precision. For example, storing nanosecond precision timestamps that are 10s of seconds apart will not compress well. If you store them at seconds precision, they will compress much better. I usually suggest using a precision that matches the sampling interval. For example, if you sample at 100ms, use millisecond precision. If you collect every second, use seconds precision. If your values are seconds apart, use seconds precision. The best way to see how your data is compressed is to use influx_inspect on the larger tsm files. As compactions run, data becomes more and more compressed. The tool will show how each block is compressed and which encodings are being used as well as the compression ratio. > > On Wednesday, October 19, 2016 at 1:04:44 PM UTC+2, Guillaume Berthomé > wrote: >> >> Thank you for your answer. >> >> So let's admit that I have 2 metrics collected every 15 seconds during 24h >> >> - cpu with 20 values >> - mem with 10 values >> >> The total size of stored data will be 3 bytes * 8 metrics per minutes * >> 1440 minutes ? >> >> >> >> Le mardi 18 octobre 2016 18:57:53 UTC+2, Sean Beckett a écrit : >>> >>> After final compactions, InfluxDB uses between 2 and 3 bytes per field >>> value stored. If your metrics are all numbers, then the total steady-state >>> size will be a bit more than 3 bytes * (metrics per minute) * minutes of >>> data stored. >>> >>> See http://docs.influxdata.com/influxdb/v1.0/concepts/storage_engine/ >>> and >>> http://docs.influxdata.com/influxdb/v1.0/guides/hardware_sizing/#how-much-storage-do-i-need >>> for >>> details. >>> >>> On Tue, Oct 18, 2016 at 2:51 AM, <[email protected]> wrote: >>> >>>> Hi everybody, >>>> >>>> Is there any way to calculate the database's size from the number >>>> measurements done per minute ? I mean predict the databse behavior (wal >>>> files, tsm files ...) >>>> >>>> Thank you. >>>> >>>> -- >>>> Remember to include the version number! >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "InfluxData" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To post to this group, send email to [email protected]. >>>> Visit this group at https://groups.google.com/group/influxdb. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/influxdb/ee32b8d0-f9bf-40df-baf1-e77fa93cddf5%40googlegroups.com >>>> . >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> >>> >>> -- >>> Sean Beckett >>> Director of Support and Professional Services >>> InfluxDB >>> >> -- Remember to include the version number! --- You received this message because you are subscribed to the Google Groups "InfluxData" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/influxdb. To view this discussion on the web visit https://groups.google.com/d/msgid/influxdb/4dc90aab-e62a-44bc-9fb0-ea874fa727f6%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
