Heh believe it or not once again I got an OOM error! And it's becomes really 'funny' that it happens at the same time? Look at this:
Oct 17 20:04:11 node1 kernel: kthreadd invoked oom-killer: gfp_mask=0x3000d0, order=2, oom_score_adj=0 When I look at InfluxDB log I see this: Oct 17 20:03:29 node1 influxd: [httpd] 192.168.11.24 - writter [17/Oct/2016:20:03:29 +0200] "POST /write?db=macdb&precision=s HTTP/1.1" 204 0 "-" "-" 0252f60c-9494-11e6-b227-000000000000 239168 Oct 17 20:03:33 node1 influxd: [httpd] 192.168.11.24 - writter [17/Oct/2016:20:03:32 +0200] "POST /write?db=macdb&precision=s HTTP/1.1" 204 0 "-" "-" 044496f0-9494-11e6-b228-000000000000 198616 Oct 17 20:03:36 node1 influxd: [httpd] 192.168.11.24 - writter [17/Oct/2016:20:03:36 +0200] "POST /write?db=macdb&precision=s HTTP/1.1" 204 0 "-" "-" 0655bab6-9494-11e6-b229-000000000000 141794 Oct 17 20:03:36 node1 influxd: [tsm1] 2016/10/17 20:03:36 Compacting cache for /var/lib/influxdb/data/macdb/seven_days/579 Oct 17 20:03:37 node1 influxd: [continuous_querier] 2016/10/17 20:03:37 finished continuous query cq_30m (2016-10-17 19:30:00 +0200 CEST to 2016-10-17 20 :00:00 +0200 CEST) in 3m37.079684067s Oct 17 20:03:38 node1 influxd: [tsm1] 2016/10/17 20:03:38 Compacting cache for /var/lib/influxdb/data/macdb/three_months/575 Oct 17 20:03:38 node1 influxd: [tsm1] 2016/10/17 20:03:38 Snapshot for path /var/lib/influxdb/data/macdb/seven_days/579 deduplicated in 368.123875m s Oct 17 20:03:40 node1 influxd: [tsm1] 2016/10/17 20:03:40 Snapshot for path /var/lib/influxdb/data/macdb/three_months/575 deduplicated in 485.55333 4ms Oct 17 20:04:02 node1 influxd: [tsm1wal] 2016/10/17 20:04:02 Removing /var/lib/influxdb/wal/macdb/seven_days/579/_01024.wal Oct 17 20:04:02 node1 influxd: [tsm1wal] 2016/10/17 20:04:02 Removing /var/lib/influxdb/wal/macdb/seven_days/579/_01025.wal Oct 17 20:04:02 node1 influxd: [tsm1wal] 2016/10/17 20:04:02 Removing /var/lib/influxdb/wal/macdb/seven_days/579/_01026.wal Oct 17 20:04:02 node1 influxd: [tsm1] 2016/10/17 20:04:02 Snapshot for path /var/lib/influxdb/data/macdb/seven_days/579 written in 25.602303689s Oct 17 20:04:02 node1 influxd: [tsm1] 2016/10/17 20:04:02 beginning level 1 compaction of group 0, 2 TSM files Oct 17 20:04:02 node1 influxd: [tsm1] 2016/10/17 20:04:02 compacting level 1 group (0) /var/lib/influxdb/data/macdb/seven_days/579/000000359-000000 001.tsm (#0) Oct 17 20:04:03 node1 influxd: [tsm1] 2016/10/17 20:04:02 compacting level 1 group (0) /var/lib/influxdb/data/macdb/seven_days/579/000000360-000000 001.tsm (#1) Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1" 204 0 "-" "-" 1616a065-9494-11e6-b22f-000000000000 152030 Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1" 204 0 "-" "-" 161979a7-9494-11e6-b238-000000000000 133364 Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1" 204 0 "-" "-" 16169ee0-9494-11e6-b22b-000000000000 152118 Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1" 204 0 "-" "-" 1616a01c-9494-11e6-b22e-000000000000 153616 Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1" 204 0 "-" "-" 1616a070-9494-11e6-b230-000000000000 153639 Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1" 204 0 "-" "-" 16169f73-9494-11e6-b22c-000000000000 153690 Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1" 204 0 "-" "-" 16169f73-9494-11e6-b22c-000000000000 153690 Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1" 204 0 "-" "-" 1616d89c-9494-11e6-b237-000000000000 152262 Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1" 204 0 "-" "-" 16169fb1-9494-11e6-b22d-000000000000 153706 Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1" 204 0 "-" "-" 1616c37e-9494-11e6-b234-000000000000 152707 Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1" 204 0 "-" "-" 1616cbfb-9494-11e6-b235-000000000000 152636 Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1" 204 0 "-" "-" 1616cc3e-9494-11e6-b236-000000000000 152641 Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1" 204 0 "-" "-" 1616c2eb-9494-11e6-b233-000000000000 153000 Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1" 204 0 "-" "-" 1616a080-9494-11e6-b231-000000000000 153896 Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1" 204 0 "-" "-" 1616a0a7-9494-11e6-b232-000000000000 153927 Oct 17 20:04:03 node1 influxd: [httpd] 192.168.11.24 - writter [17/Oct/2016:20:04:02 +0200] "POST /write?db=macdb&precision=s HTTP/1.1" 204 0 "-" "-" 16169e92-9494-11e6-b22a-000000000000 153946 Oct 17 20:04:11 node1 kernel: kthreadd invoked oom-killer: gfp_mask=0x3000d0, order=2, oom_score_adj=0 Oct 17 20:04:11 node1 kernel: kthreadd cpuset=/ mems_allowed=0 Why it happens always at the same time? I don't have any cron jobs nor any strange traffic patterns at this time. -- Remember to include the version number! --- You received this message because you are subscribed to the Google Groups "InfluxData" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/influxdb. To view this discussion on the web visit https://groups.google.com/d/msgid/influxdb/106877d9-6218-4a55-9f2e-5191efa4d135%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
