Thank you everyone ! Le mercredi 19 octobre 2016 21:30:54 UTC+2, Sean Beckett a écrit : > > Mathias, I agree that irregular timestamps do lead to more space taken on > disk. However, that number still falls under 3 bytes per numeric value, > even with irregular nanosecond timestamps. Both what Jason and I are saying > is true; they are not mutually exclusive. > > > The 2 or 3 bytes footprint is usually not achievable if you store your > timestamps with milli, micro or nanosecond precision as the delta between > consecutive timestamps will vary and will therefore not compress as well. > > That statement is not true, at least not in any of our testing experience. > It would be fair to say that 2-3 bytes per value is not always achievable, > as some rare edge cases can lead to larger footprint on disk, but to say > that it is *usually* not achievable is simply incorrect. > > On Wed, Oct 19, 2016 at 1:10 PM, Mathias Herberts <[email protected] > <javascript:>> wrote: > >> It seems Jason just posted the exact same findings as me, maybe check >> with him directly, I persist in my saying, if your timestamps are not >> regularly spaced then compression efficiency will decrease. >> >> On Wednesday, October 19, 2016 at 8:33:56 PM UTC+2, Sean Beckett wrote: >>> >>> Mathias, I question your findings on this. We've run extensive testing >>> with nanosecond timestamps and routinely get < 3 bytes per recorded numeric >>> value. >>> >>> Please note that this figure is only for fully compacted data, e.g. cold >>> shards. Shards currently hot for writes will be much less compact. Over >>> time the steady state of the system will approach ~2.5 bytes per numeric >>> field. >>> >>> On Wed, Oct 19, 2016 at 5:45 AM, Mathias Herberts <[email protected] >>> > wrote: >>> >>>> Efficiency of compression depends on different factors, including >>>> interval between timestamps, resolution of said timestamps, volatility and >>>> type of values. >>>> >>>> The 2 or 3 bytes footprint is usually not achievable if you store your >>>> timestamps with milli, micro or nanosecond precision as the delta between >>>> consecutive timestamps will vary and will therefore not compress as well. >>>> >>>> >>>> On Wednesday, October 19, 2016 at 1:04:44 PM UTC+2, Guillaume Berthomé >>>> wrote: >>>>> >>>>> Thank you for your answer. >>>>> >>>>> So let's admit that I have 2 metrics collected every 15 seconds during >>>>> 24h >>>>> >>>>> - cpu with 20 values >>>>> - mem with 10 values >>>>> >>>>> The total size of stored data will be 3 bytes * 8 metrics per minutes >>>>> * 1440 minutes ? >>>>> >>>>> >>>>> >>>>> Le mardi 18 octobre 2016 18:57:53 UTC+2, Sean Beckett a écrit : >>>>>> >>>>>> After final compactions, InfluxDB uses between 2 and 3 bytes per >>>>>> field value stored. If your metrics are all numbers, then the total >>>>>> steady-state size will be a bit more than 3 bytes * (metrics per minute) >>>>>> * >>>>>> minutes of data stored. >>>>>> >>>>>> See http://docs.influxdata.com/influxdb/v1.0/concepts/storage_engine/ >>>>>> and >>>>>> http://docs.influxdata.com/influxdb/v1.0/guides/hardware_sizing/#how-much-storage-do-i-need >>>>>> for >>>>>> details. >>>>>> >>>>>> On Tue, Oct 18, 2016 at 2:51 AM, <[email protected]> wrote: >>>>>> >>>>>>> Hi everybody, >>>>>>> >>>>>>> Is there any way to calculate the database's size from the number >>>>>>> measurements done per minute ? I mean predict the databse behavior (wal >>>>>>> files, tsm files ...) >>>>>>> >>>>>>> Thank you. >>>>>>> >>>>>>> -- >>>>>>> Remember to include the version number! >>>>>>> --- >>>>>>> You received this message because you are subscribed to the Google >>>>>>> Groups "InfluxData" group. >>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>> send an email to [email protected]. >>>>>>> To post to this group, send email to [email protected]. >>>>>>> Visit this group at https://groups.google.com/group/influxdb. >>>>>>> To view this discussion on the web visit >>>>>>> https://groups.google.com/d/msgid/influxdb/ee32b8d0-f9bf-40df-baf1-e77fa93cddf5%40googlegroups.com >>>>>>> . >>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Sean Beckett >>>>>> Director of Support and Professional Services >>>>>> InfluxDB >>>>>> >>>>> -- >>>> Remember to include the version number! >>>> --- >>>> You received this message because you are subscribed to the Google >>>> Groups "InfluxData" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To post to this group, send email to [email protected]. >>>> Visit this group at https://groups.google.com/group/influxdb. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/influxdb/b08ad8c0-341e-4390-a18c-8d6b67da26f0%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/influxdb/b08ad8c0-341e-4390-a18c-8d6b67da26f0%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> >>> >>> -- >>> Sean Beckett >>> Director of Support and Professional Services >>> InfluxDB >>> >> -- >> Remember to include the version number! >> --- >> You received this message because you are subscribed to the Google Groups >> "InfluxData" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To post to this group, send email to [email protected] >> <javascript:>. >> Visit this group at https://groups.google.com/group/influxdb. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/influxdb/4ba6c676-d0be-43a4-8f43-2c360ccafb3b%40googlegroups.com >> >> <https://groups.google.com/d/msgid/influxdb/4ba6c676-d0be-43a4-8f43-2c360ccafb3b%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Sean Beckett > Director of Support and Professional Services > InfluxDB >
-- Remember to include the version number! --- You received this message because you are subscribed to the Google Groups "InfluxData" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/influxdb. To view this discussion on the web visit https://groups.google.com/d/msgid/influxdb/1b71ecba-41ea-4a47-a225-3c823d642048%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
