> By missing the whole measurement I mean that for example we have one
measurement at10:00 pm in InfluxDB and in our file (we do write data in
file for debugging), then we have next measurement in 10:05 pm in both
InfluxDB and file, the next measurement in 10:10 we are missing in InfluxDB
but we do have that measurement in file and still we have 204 response
returned by InfluxDB.
Please be aware that "measurement" has a specific meaning in InfluxDB, and
does not refer to a single point or metric.
If you have a 204, the metric was recorded by the system.
How are you querying for the record?
Could there be a timestamp collision?
On Fri, Oct 14, 2016 at 11:57 AM, Sean Beckett <s...@influxdb.com> wrote:
> It looks like your CQ does lead to RAM spikes close to the capacity of the
> box. Your shard durations are what's tipping the issue, I believe. With 1
> day shards in a 90 day retention policy, there are a lot of housecleaning
> tasks to do each night at midnight UTC. When each shard expires, the series
> index has to be updated and a series of compactions kick off. Compactions
> are RAM and CPU intensive.
> First recommendation, use ALTER RETENTION POLICY to raise the shard
> duration for `three_months` to at least a week, but even a month would be
> good. It will reduce the frequency of the TSM compactions, and with fewer
> files the compactions will be less resource intensive.
> Also, queries should touch as few shards as possible. If you are often
> querying for more than 12 hours of data then raising the shard duration
> will reduce the RAM needs of those queries.
> On Fri, Oct 14, 2016 at 3:29 AM, <paveldi...@gmail.com> wrote:
>> Hi Sean,
>> here is the graph from out NMS about memory usage
>> and I would say we have spikes, but they are not every 24h but rather
>> every 30 minutes and I guess it's because of our CQ we use for
>> downsampling. I can post that CQ if that can help.
>> We are aware of cardinality when we designed our solution and currently
>> we have 81761 series which I guess it's quite ok for this amount of RAM.
>> > SHOW RETENTION POLICIES ON macdb
>> name duration shardGroupDuration replicaN
>> default 0 168h0m0s 1
>> seven_days 168h0m0s 24h0m0s 1
>> three_months 2160h0m0s 24h0m0s 1
>> Yes, we always have successful writes and we have 204 response returned
>> by InfluxDB. By missing the whole measurement I mean that for example we
>> have one measurement at 10:00 pm in InfluxDB and in our file (we do write
>> data in file for debugging), then we have next measurement in 10:05 pm in
>> both InfluxDB and file, the next measurement in 10:10 we are missing in
>> InfluxDB but we do have that measurement in file and still we have 204
>> response returned by InfluxDB.
>> Remember to include the version number!
>> You received this message because you are subscribed to the Google Groups
>> "InfluxData" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to influxdb+unsubscr...@googlegroups.com.
>> To post to this group, send email to firstname.lastname@example.org.
>> Visit this group at https://groups.google.com/group/influxdb.
>> To view this discussion on the web visit https://groups.google.com/d/ms
>> For more options, visit https://groups.google.com/d/optout.
> Sean Beckett
> Director of Support and Professional Services
Director of Support and Professional Services
Remember to include the version number!
You received this message because you are subscribed to the Google Groups
To unsubscribe from this group and stop receiving emails from it, send an email
To post to this group, send email to email@example.com.
Visit this group at https://groups.google.com/group/influxdb.
To view this discussion on the web visit
For more options, visit https://groups.google.com/d/optout.