@Sean, Can I use the rule you gave me for integers ?
Le jeudi 20 octobre 2016 08:53:34 UTC+2, Guillaume Berthomé a écrit : > > Thank you everyone ! > > Le mercredi 19 octobre 2016 21:30:54 UTC+2, Sean Beckett a écrit : >> >> Mathias, I agree that irregular timestamps do lead to more space taken on >> disk. However, that number still falls under 3 bytes per numeric value, >> even with irregular nanosecond timestamps. Both what Jason and I are saying >> is true; they are not mutually exclusive. >> >> > The 2 or 3 bytes footprint is usually not achievable if you store your >> timestamps with milli, micro or nanosecond precision as the delta between >> consecutive timestamps will vary and will therefore not compress as well. >> >> That statement is not true, at least not in any of our testing >> experience. It would be fair to say that 2-3 bytes per value is not always >> achievable, as some rare edge cases can lead to larger footprint on disk, >> but to say that it is *usually* not achievable is simply incorrect. >> >> On Wed, Oct 19, 2016 at 1:10 PM, Mathias Herberts <[email protected]> >> wrote: >> >>> It seems Jason just posted the exact same findings as me, maybe check >>> with him directly, I persist in my saying, if your timestamps are not >>> regularly spaced then compression efficiency will decrease. >>> >>> On Wednesday, October 19, 2016 at 8:33:56 PM UTC+2, Sean Beckett wrote: >>>> >>>> Mathias, I question your findings on this. We've run extensive testing >>>> with nanosecond timestamps and routinely get < 3 bytes per recorded >>>> numeric >>>> value. >>>> >>>> Please note that this figure is only for fully compacted data, e.g. >>>> cold shards. Shards currently hot for writes will be much less compact. >>>> Over time the steady state of the system will approach ~2.5 bytes per >>>> numeric field. >>>> >>>> On Wed, Oct 19, 2016 at 5:45 AM, Mathias Herberts < >>>> [email protected]> wrote: >>>> >>>>> Efficiency of compression depends on different factors, including >>>>> interval between timestamps, resolution of said timestamps, volatility >>>>> and >>>>> type of values. >>>>> >>>>> The 2 or 3 bytes footprint is usually not achievable if you store your >>>>> timestamps with milli, micro or nanosecond precision as the delta between >>>>> consecutive timestamps will vary and will therefore not compress as well. >>>>> >>>>> >>>>> On Wednesday, October 19, 2016 at 1:04:44 PM UTC+2, Guillaume Berthomé >>>>> wrote: >>>>>> >>>>>> Thank you for your answer. >>>>>> >>>>>> So let's admit that I have 2 metrics collected every 15 seconds >>>>>> during 24h >>>>>> >>>>>> - cpu with 20 values >>>>>> - mem with 10 values >>>>>> >>>>>> The total size of stored data will be 3 bytes * 8 metrics per minutes >>>>>> * 1440 minutes ? >>>>>> >>>>>> >>>>>> >>>>>> Le mardi 18 octobre 2016 18:57:53 UTC+2, Sean Beckett a écrit : >>>>>>> >>>>>>> After final compactions, InfluxDB uses between 2 and 3 bytes per >>>>>>> field value stored. If your metrics are all numbers, then the total >>>>>>> steady-state size will be a bit more than 3 bytes * (metrics per >>>>>>> minute) * >>>>>>> minutes of data stored. >>>>>>> >>>>>>> See >>>>>>> http://docs.influxdata.com/influxdb/v1.0/concepts/storage_engine/ >>>>>>> and >>>>>>> http://docs.influxdata.com/influxdb/v1.0/guides/hardware_sizing/#how-much-storage-do-i-need >>>>>>> for >>>>>>> details. >>>>>>> >>>>>>> On Tue, Oct 18, 2016 at 2:51 AM, <[email protected]> wrote: >>>>>>> >>>>>>>> Hi everybody, >>>>>>>> >>>>>>>> Is there any way to calculate the database's size from the number >>>>>>>> measurements done per minute ? I mean predict the databse behavior >>>>>>>> (wal >>>>>>>> files, tsm files ...) >>>>>>>> >>>>>>>> Thank you. >>>>>>>> >>>>>>>> -- >>>>>>>> Remember to include the version number! >>>>>>>> --- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "InfluxData" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>>> To post to this group, send email to [email protected]. >>>>>>>> Visit this group at https://groups.google.com/group/influxdb. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/influxdb/ee32b8d0-f9bf-40df-baf1-e77fa93cddf5%40googlegroups.com >>>>>>>> . >>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Sean Beckett >>>>>>> Director of Support and Professional Services >>>>>>> InfluxDB >>>>>>> >>>>>> -- >>>>> Remember to include the version number! >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "InfluxData" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To post to this group, send email to [email protected]. >>>>> Visit this group at https://groups.google.com/group/influxdb. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/influxdb/b08ad8c0-341e-4390-a18c-8d6b67da26f0%40googlegroups.com >>>>> >>>>> <https://groups.google.com/d/msgid/influxdb/b08ad8c0-341e-4390-a18c-8d6b67da26f0%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> >>>> >>>> -- >>>> Sean Beckett >>>> Director of Support and Professional Services >>>> InfluxDB >>>> >>> -- >>> Remember to include the version number! >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "InfluxData" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> Visit this group at https://groups.google.com/group/influxdb. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/influxdb/4ba6c676-d0be-43a4-8f43-2c360ccafb3b%40googlegroups.com >>> >>> <https://groups.google.com/d/msgid/influxdb/4ba6c676-d0be-43a4-8f43-2c360ccafb3b%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> >> -- >> Sean Beckett >> Director of Support and Professional Services >> InfluxDB >> > -- Remember to include the version number! --- You received this message because you are subscribed to the Google Groups "InfluxData" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/influxdb. To view this discussion on the web visit https://groups.google.com/d/msgid/influxdb/5bb14b9d-afee-4412-873c-3b4253314f52%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
