Hi Mathias,

what's to be considered large amount of data in InfluxDB? I do have around
40k metrics every 5 minutes. Every metric has 2 tags and around 70 fields.
CQ runs every 30 minutes and downsamples data (I keep 7 days raw data and 3
months downsampled data). Is that too much for a 40GB RAM box?

On Mon, Oct 17, 2016 at 10:50 PM, Mathias Herberts <
mathias.herbe...@gmail.com> wrote:

> Your CQ completed in 3m27s, does it manipulate a very large amount of data?
>
>
> On Monday, October 17, 2016 at 10:43:22 PM UTC+2, pavel...@gmail.com
> wrote:
>>
>> Heh believe it or not once again I got an OOM error! And it's becomes
>> really 'funny' that it happens at the same time? Look at this:
>>
>> Oct 17 20:04:11 node1 kernel: kthreadd invoked oom-killer:
>> gfp_mask=0x3000d0, order=2, oom_score_adj=0
>>
>> When I look at InfluxDB log I see this:
>>
>> Oct 17 20:03:29 node1 influxd: [httpd] 192.168.11.24 - writter
>> [17/Oct/2016:20:03:29 +0200] "POST /write?db=macdb&precision=s HTTP/1.1"
>> 204 0
>> "-" "-" 0252f60c-9494-11e6-b227-000000000000 239168
>> Oct 17 20:03:33 node1 influxd: [httpd] 192.168.11.24 - writter
>> [17/Oct/2016:20:03:32 +0200] "POST /write?db=macdb&precision=s HTTP/1.1"
>> 204 0
>> "-" "-" 044496f0-9494-11e6-b228-000000000000 198616
>> Oct 17 20:03:36 node1 influxd: [httpd] 192.168.11.24 - writter
>> [17/Oct/2016:20:03:36 +0200] "POST /write?db=macdb&precision=s HTTP/1.1"
>> 204 0
>> "-" "-" 0655bab6-9494-11e6-b229-000000000000 141794
>> Oct 17 20:03:36 node1 influxd: [tsm1] 2016/10/17 20:03:36 Compacting
>> cache for /var/lib/influxdb/data/macdb/seven_days/579
>> Oct 17 20:03:37 node1 influxd: [continuous_querier] 2016/10/17 20:03:37
>> finished continuous query cq_30m (2016-10-17 19:30:00 +0200 CEST to
>> 2016-10-17 20
>> :00:00 +0200 CEST) in 3m37.079684067s
>> Oct 17 20:03:38 node1 influxd: [tsm1] 2016/10/17 20:03:38 Compacting
>> cache for /var/lib/influxdb/data/macdb/three_months/575
>> Oct 17 20:03:38 node1 influxd: [tsm1] 2016/10/17 20:03:38 Snapshot for
>> path /var/lib/influxdb/data/macdb/seven_days/579 deduplicated in
>> 368.123875m
>> s
>> Oct 17 20:03:40 node1 influxd: [tsm1] 2016/10/17 20:03:40 Snapshot for
>> path /var/lib/influxdb/data/macdb/three_months/575 deduplicated in
>> 485.55333
>> 4ms
>> Oct 17 20:04:02 node1 influxd: [tsm1wal] 2016/10/17 20:04:02 Removing
>> /var/lib/influxdb/wal/macdb/seven_days/579/_01024.wal
>> Oct 17 20:04:02 node1 influxd: [tsm1wal] 2016/10/17 20:04:02 Removing
>> /var/lib/influxdb/wal/macdb/seven_days/579/_01025.wal
>> Oct 17 20:04:02 node1 influxd: [tsm1wal] 2016/10/17 20:04:02 Removing
>> /var/lib/influxdb/wal/macdb/seven_days/579/_01026.wal
>> Oct 17 20:04:02 node1 influxd: [tsm1] 2016/10/17 20:04:02 Snapshot for
>> path /var/lib/influxdb/data/macdb/seven_days/579 written in 25.602303689s
>> Oct 17 20:04:02 node1 influxd: [tsm1] 2016/10/17 20:04:02 beginning level
>> 1 compaction of group 0, 2 TSM files
>> Oct 17 20:04:02 node1 influxd: [tsm1] 2016/10/17 20:04:02 compacting
>> level 1 group (0) /var/lib/influxdb/data/macdb/s
>> even_days/579/000000359-000000
>> 001.tsm (#0)
>> Oct 17 20:04:03 node1 influxd: [tsm1] 2016/10/17 20:04:02 compacting
>> level 1 group (0) /var/lib/influxdb/data/macdb/s
>> even_days/579/000000360-000000
>> 001.tsm (#1)
>> Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter
>> [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1"
>> 204 0
>> "-" "-" 1616a065-9494-11e6-b22f-000000000000 152030
>> Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter
>> [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1"
>> 204 0
>> "-" "-" 161979a7-9494-11e6-b238-000000000000 133364
>> Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter
>> [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1"
>> 204 0
>> "-" "-" 16169ee0-9494-11e6-b22b-000000000000 152118
>> Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter
>> [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1"
>> 204 0
>> "-" "-" 1616a01c-9494-11e6-b22e-000000000000 153616
>> Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter
>> [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1"
>> 204 0
>> "-" "-" 1616a070-9494-11e6-b230-000000000000 153639
>> Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter
>> [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1"
>> 204 0 "-" "-" 16169f73-9494-11e6-b22c-000000000000 153690
>> Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter
>> [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1"
>> 204 0 "-" "-" 16169f73-9494-11e6-b22c-000000000000 153690
>> Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter
>> [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1"
>> 204 0 "-" "-" 1616d89c-9494-11e6-b237-000000000000 152262
>> Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter
>> [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1"
>> 204 0 "-" "-" 16169fb1-9494-11e6-b22d-000000000000 153706
>> Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter
>> [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1"
>> 204 0 "-" "-" 1616c37e-9494-11e6-b234-000000000000 152707
>> Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter
>> [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1"
>> 204 0 "-" "-" 1616cbfb-9494-11e6-b235-000000000000 152636
>> Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter
>> [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1"
>> 204 0 "-" "-" 1616cc3e-9494-11e6-b236-000000000000 152641
>> Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter
>> [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1"
>> 204 0 "-" "-" 1616c2eb-9494-11e6-b233-000000000000 153000
>> Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter
>> [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1"
>> 204 0 "-" "-" 1616a080-9494-11e6-b231-000000000000 153896
>> Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter
>> [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1"
>> 204 0 "-" "-" 1616a0a7-9494-11e6-b232-000000000000 153927
>> Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter
>> [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1"
>> 204 0 "-" "-" 16169e92-9494-11e6-b22a-000000000000 153946
>> Oct 17 20:04:11 node1 kernel: kthreadd invoked oom-killer:
>> gfp_mask=0x3000d0, order=2, oom_score_adj=0
>> Oct 17 20:04:11 node1 kernel: kthreadd cpuset=/ mems_allowed=0
>>
>> Why it happens always at the same time? I don't have any cron jobs nor
>> any strange traffic patterns at this time.
>>
>>

-- 
Remember to include the version number!
--- 
You received this message because you are subscribed to the Google Groups 
"InfluxData" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to influxdb+unsubscr...@googlegroups.com.
To post to this group, send email to influxdb@googlegroups.com.
Visit this group at https://groups.google.com/group/influxdb.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/influxdb/CAKb_NupzgAgq6NK4ymmWYZSDnAVATAQnyXa89RaHDbQ6e%2B1e8g%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.

Reply via email to