The cores are more than sufficient, but 64 million series would probably chew up even 128GB of RAM. If you have one or two very high cardinality tags (Docker container ID, PID, user email, lat/long, etc.) then consider making those tags fields instead. The tradeoff is that you won't be able to use them in a GROUP BY clause, and you won't be able to use regular expression operators to match them. If you want more assistance we offer a Developer Support plan <https://influxdata.com/pricing/#product_subscriptions> that is a simple monthly subscription. Alternately you can contact [email protected] for a schema consultation.
On Wed, Jul 27, 2016 at 2:37 AM, Pramodh Prakash <[email protected]> wrote: > Yup, Seems like it. InfluxDB is crashing after a few minutes due to high > memory usage on a system with 64GB ram and 24 cores. > > On Wed, Jul 27, 2016 at 6:41 AM, Sean Beckett <[email protected]> wrote: > >> 61 million series is a huge number for the database to handle. You might >> reconsider your schema to remove tags with high cardinality. >> >> On Tue, Jul 26, 2016 at 3:23 AM, <[email protected]> wrote: >> >>> Hey, >>> >>> So I upgraded influxDB to v1.0beta2 and the load times have come down to >>> less than 5 minutes. Thanks for the help. >>> >>> This is the output of the query you gave: >>> >>> name: database >>> -------------- >>> time sum >>> 1469524182533290092 61948572 >>> >>> >>> On Thursday, July 21, 2016 at 11:40:33 PM UTC+5:30, Sean Beckett wrote: >>>> >>>> The data folder size is a direct result of the size of the data you are >>>> writing. Numbers and booleans take ~2 bytes per field, strings vary based >>>> on compression. Or does your question about cause refer to something else? >>>> >>>> With that much data on disk you will have slower startup times, athough >>>> 10-20 minutes seems excessive. The fixes in the 1.0beta3 solve many of the >>>> issues with large shards, so the results of that upgrade will tell us >>>> whether there's something else going on. >>>> >>>> What is your series cardinality >>>> <https://docs.influxdata.com/influxdb/v0.13/concepts/glossary/#series-cardinality>? >>>> Run `select sum(numSeries) from _internal.."database" where time > now() - >>>> 10s` to find the value. >>>> >>>> >>>> >>>> >>>> On Tue, Jul 19, 2016 at 2:49 AM, Pramodh Prakash <[email protected] >>>> > wrote: >>>> >>>>> Hey, thanks for the reply. The size of our data folder is around 148G. >>>>> What is the cause of the for this? >>>>> >>>>> I will try with newer version you mentioned soon. >>>>> >>>>> Thanks, >>>>> Pramodh >>>>> >>>>> On Mon, Jul 18, 2016 at 9:28 PM, Sean Beckett <[email protected]> >>>>> wrote: >>>>> >>>>>> Can you try upgrading to 1.0 beta2? There's a specific fix that might >>>>>> help the load times: https://github.com/influxdata/influxdb/pull/6618 >>>>>> >>>>>> On Mon, Jul 18, 2016 at 8:28 AM, Sean Beckett <[email protected]> >>>>>> wrote: >>>>>> >>>>>>> There are no log levels for either InfluxDB or Telegraf. You can >>>>>>> enable and disable logging modules in the config file, but they are >>>>>>> either >>>>>>> fully on or fully off. >>>>>>> >>>>>>> How much data is in your system? (What's the size of >>>>>>> /var/lib/influxdb/data?) >>>>>>> >>>>>>> >>>>>>> >>>>>>> On Mon, Jul 18, 2016 at 6:24 AM, <[email protected]> wrote: >>>>>>> >>>>>>>> Hello, >>>>>>>> >>>>>>>> Since recently, our InfluxDB v0.13.0 is taking around 15 to 20 >>>>>>>> minutes to restart. After restart, even though it seems like influxdb >>>>>>>> is >>>>>>>> listening on port 8083 and 8086, telnet doesn't work. Can someone >>>>>>>> point me >>>>>>>> in a direction to debug this? Also, is there a way to increase logging >>>>>>>> verbosity for InfluxDB and Telegraf? >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Pramodh >>>>>>>> >>>>>>>> -- >>>>>>>> Remember to include the InfluxDB version number with all issue >>>>>>>> reports >>>>>>>> --- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "InfluxDB" group. >>>>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>>>> send an email to [email protected]. >>>>>>>> To post to this group, send email to [email protected]. >>>>>>>> Visit this group at https://groups.google.com/group/influxdb. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/msgid/influxdb/d4dbeb14-96c2-4ab5-8db2-d7d8ba9e90e6%40googlegroups.com >>>>>>>> <https://groups.google.com/d/msgid/influxdb/d4dbeb14-96c2-4ab5-8db2-d7d8ba9e90e6%40googlegroups.com?utm_medium=email&utm_source=footer> >>>>>>>> . >>>>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Sean Beckett >>>>>>> Director of Support and Professional Services >>>>>>> InfluxDB >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Sean Beckett >>>>>> Director of Support and Professional Services >>>>>> InfluxDB >>>>>> >>>>>> -- >>>>>> Remember to include the InfluxDB version number with all issue reports >>>>>> --- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "InfluxDB" group. >>>>>> To unsubscribe from this group and stop receiving emails from it, >>>>>> send an email to [email protected]. >>>>>> To post to this group, send email to [email protected]. >>>>>> Visit this group at https://groups.google.com/group/influxdb. >>>>>> To view this discussion on the web visit >>>>>> https://groups.google.com/d/msgid/influxdb/CALGqCvOKNdZGGYk9zLMs31%3DkFyQ0Wvm3iDGJhtbbw4rkTtm1ug%40mail.gmail.com >>>>>> <https://groups.google.com/d/msgid/influxdb/CALGqCvOKNdZGGYk9zLMs31%3DkFyQ0Wvm3iDGJhtbbw4rkTtm1ug%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>>> . >>>>>> >>>>>> For more options, visit https://groups.google.com/d/optout. >>>>>> >>>>> >>>>> -- >>>>> Remember to include the InfluxDB version number with all issue reports >>>>> --- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "InfluxDB" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to [email protected]. >>>>> To post to this group, send email to [email protected]. >>>>> Visit this group at https://groups.google.com/group/influxdb. >>>>> To view this discussion on the web visit >>>>> https://groups.google.com/d/msgid/influxdb/CAGquJLsJ6j4%3DdwUYrp%2Bj6T5fgDAPRaKTyz%2BQRN1L%2BTNJWKiKGw%40mail.gmail.com >>>>> <https://groups.google.com/d/msgid/influxdb/CAGquJLsJ6j4%3DdwUYrp%2Bj6T5fgDAPRaKTyz%2BQRN1L%2BTNJWKiKGw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>>>> . >>>>> >>>>> For more options, visit https://groups.google.com/d/optout. >>>>> >>>> >>>> >>>> >>>> -- >>>> Sean Beckett >>>> Director of Support and Professional Services >>>> InfluxDB >>>> >>> -- >>> Remember to include the InfluxDB version number with all issue reports >>> --- >>> You received this message because you are subscribed to the Google >>> Groups "InfluxDB" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To post to this group, send email to [email protected]. >>> Visit this group at https://groups.google.com/group/influxdb. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/influxdb/e0fc8157-b971-43da-ba46-466f17c6f170%40googlegroups.com >>> <https://groups.google.com/d/msgid/influxdb/e0fc8157-b971-43da-ba46-466f17c6f170%40googlegroups.com?utm_medium=email&utm_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> >> -- >> Sean Beckett >> Director of Support and Professional Services >> InfluxDB >> >> -- >> Remember to include the InfluxDB version number with all issue reports >> --- >> You received this message because you are subscribed to a topic in the >> Google Groups "InfluxDB" group. >> To unsubscribe from this topic, visit >> https://groups.google.com/d/topic/influxdb/_ORAU8zewGM/unsubscribe. >> To unsubscribe from this group and all its topics, send an email to >> [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at https://groups.google.com/group/influxdb. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/influxdb/CALGqCvNKwoDeV33SEcH89aDPWCAxS%2BxZxj%3Dme30TGCYbpySE4A%40mail.gmail.com >> <https://groups.google.com/d/msgid/influxdb/CALGqCvNKwoDeV33SEcH89aDPWCAxS%2BxZxj%3Dme30TGCYbpySE4A%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> >> For more options, visit https://groups.google.com/d/optout. >> > > -- > Remember to include the InfluxDB version number with all issue reports > --- > You received this message because you are subscribed to the Google Groups > "InfluxDB" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/influxdb. > To view this discussion on the web visit > https://groups.google.com/d/msgid/influxdb/CAGquJLups-Qvd%3DxSHoh3STwnzaPkHqv8NkRXDOJDRoGEkHpqdg%40mail.gmail.com > <https://groups.google.com/d/msgid/influxdb/CAGquJLups-Qvd%3DxSHoh3STwnzaPkHqv8NkRXDOJDRoGEkHpqdg%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- Sean Beckett Director of Support and Professional Services InfluxDB -- Remember to include the InfluxDB version number with all issue reports --- You received this message because you are subscribed to the Google Groups "InfluxDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/influxdb. To view this discussion on the web visit https://groups.google.com/d/msgid/influxdb/CALGqCvNfdR0_b9Wt%2ByRP5tAi1G0Fz8a%2BxEq-DiV5ZyNvn77AbA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
