On Tuesday, November 8, 2016 at 9:18:08 PM UTC+2, [email protected] wrote: > Hi All > > I am inserting a low volume of data (less than 2000 points every 15 minutes). > I do this via a Java program (using a batch). What I have noticed is > inserting the same amount of data takes longer and longer and over time and > the CPU usage constantly increases (almost linearly) over 24 hours (It is not > swapping as far as I can see). > > If I restart InfluxDB the CPU usage goes back to normal for a while but it > slowly and steadily climbs with each insert. The only thing that reduces the > CPU load for writes (roughly the same amount of data every time) is if I > restart InfluxDB (I have tried stopping and starting the Java program and it > has no effect (I thought it might be holding onto ports etc). > > I am using a very small server (1G / 1 CPU) to test it (there is very little > data) but I was wondering if I am configuring InfluxDB properly. > > I am using version 1.0.2 of InfluxDB on Ubuntu Linux (on DigitalOcean) and > the "recommended" Java library. > > Here is my conf file > > reporting-disabled = false > bind-address = ":8088" > > [meta] > dir = "/var/lib/influxdb/meta" > retention-autocreate = true > logging-enabled = true > > [data] > dir = "/var/lib/influxdb/data" > engine = "tsm1" > wal-dir = "/var/lib/influxdb/wal" > wal-logging-enabled = true > query-log-enabled = false > cache-max-memory-size = 524288000 > cache-snapshot-memory-size = 26214400 > cache-snapshot-write-cold-duration = "1h0m0s" > compact-full-write-cold-duration = "24h0m0s" > max-points-per-block = 0 > max-series-per-database = 1000000 > trace-logging-enabled = false > > [coordinator] > write-timeout = "10s" > max-concurrent-queries = 0 > query-timeout = "0" > log-queries-after = "0" > max-select-point = 0 > max-select-series = 0 > max-select-buckets = 0 > > [retention] > enabled = true > check-interval = "30m0s" > > [shard-precreation] > enabled = true > check-interval = "10m0s" > advance-period = "30m0s" > > [admin] > enabled = true > bind-address = ":8083" > https-enabled = false > https-certificate = "/etc/ssl/influxdb.pem" > > [monitor] > store-enabled = true > store-database = "_internal" > store-interval = "10s" > > [subscriber] > enabled = true > http-timeout = "30s" > write-concurrency = 40 > write-buffer-size = 1000 > > [http] > enabled = true > bind-address = ":8086" > auth-enabled = false > log-enabled = false > write-tracing = false > https-enabled = false > https-certificate = "/etc/ssl/influxdb.pem" > https-private-key = "" > max-row-limit = 10000 > max-connection-limit = 0 > shared-secret = "" > realm = "InfluxDB" > > [continuous_queries] > log-enabled = true > enabled = true > run-interval = "1s" > > Any pointers will be appreciated. > > Regards > Steve van der Merwe
My apologies here is a log snippet: Nov 8 19:06:35 db influxd[17415]: [tsm1] 2016/11/08 19:06:35 compacted level 2 group 0 of 2 files into 1 files in 452.912177ms Nov 8 19:06:36 db influxd[17415]: [tsm1] 2016/11/08 19:06:36 beginning level 3 compaction of group 0, 4 TSM files Nov 8 19:06:36 db influxd[17415]: [tsm1] 2016/11/08 19:06:36 compacting level 3 group (0) /var/lib/influxdb/data/db/autogen/50/000000551-000000003.tsm (#0) Nov 8 19:06:36 db influxd[17415]: [tsm1] 2016/11/08 19:06:36 compacting level 3 group (0) /var/lib/influxdb/data/db/autogen/50/000000555-000000003.tsm (#1) Nov 8 19:06:36 db influxd[17415]: [tsm1] 2016/11/08 19:06:36 compacting level 3 group (0) /var/lib/influxdb/data/db/autogen/50/000000559-000000003.tsm (#2) Nov 8 19:06:36 db influxd[17415]: [tsm1] 2016/11/08 19:06:36 compacting level 3 group (0) /var/lib/influxdb/data/db/autogen/50/000000563-000000003.tsm (#3) Nov 8 19:06:37 db influxd[17415]: [tsm1] 2016/11/08 19:06:37 compacted level 3 group (0) into /var/lib/influxdb/data/db/autogen/50/000000563-000000004.tsm.tmp (#0) Nov 8 19:06:37 db influxd[17415]: [tsm1] 2016/11/08 19:06:37 compacted level 3 group 0 of 4 files into 1 files in 798.550193ms Nov 8 19:07:05 db influxd[17415]: [tsm1] 2016/11/08 19:07:05 Snapshot for path /var/lib/influxdb/data/db/autogen/50 written in 2.163763999s Nov 8 19:07:37 db influxd[17415]: [tsm1] 2016/11/08 19:07:37 Snapshot for path /var/lib/influxdb/data/db/autogen/50 written in 2.6193556s Nov 8 19:07:37 db influxd[17415]: [tsm1] 2016/11/08 19:07:37 beginning level 1 compaction of group 0, 2 TSM files Nov 8 19:07:37 db influxd[17415]: [tsm1] 2016/11/08 19:07:37 compacting level 1 group (0) /var/lib/influxdb/data/db/autogen/50/000000564-000000001.tsm (#0) Nov 8 19:07:37 db influxd[17415]: [tsm1] 2016/11/08 19:07:37 compacting level 1 group (0) /var/lib/influxdb/data/db/autogen/50/000000565-000000001.tsm (#1) Nov 8 19:07:38 db influxd[17415]: [tsm1] 2016/11/08 19:07:38 compacted level 1 group (0) into /var/lib/influxdb/data/db/autogen/50/000000565-000000002.tsm.tmp (#0) Nov 8 19:07:38 db influxd[17415]: [tsm1] 2016/11/08 19:07:38 compacted level 1 group 0 of 2 files into 1 files in 490.727946ms Nov 8 19:12:16 db influxd[17415]: [tsm1] 2016/11/08 19:12:16 Snapshot for path /var/lib/influxdb/data/db/autogen/50 written in 3.521008625s Nov 8 19:12:51 db influxd[17415]: [tsm1] 2016/11/08 19:12:51 Snapshot for path /var/lib/influxdb/data/db/autogen/50 written in 2.915170587s Nov 8 19:12:51 db influxd[17415]: [tsm1] 2016/11/08 19:12:51 beginning level 1 compaction of group 0, 2 TSM files Nov 8 19:12:51 db influxd[17415]: [tsm1] 2016/11/08 19:12:51 compacting level 1 group (0) /var/lib/influxdb/data/db/autogen/50/000000566-000000001.tsm (#0) Nov 8 19:12:51 db influxd[17415]: [tsm1] 2016/11/08 19:12:51 compacting level 1 group (0) /var/lib/influxdb/data/db/autogen/50/000000567-000000001.tsm (#1) Nov 8 19:12:52 db influxd[17415]: [tsm1] 2016/11/08 19:12:52 compacted level 1 group (0) into /var/lib/influxdb/data/db/autogen/50/000000567-000000002.tsm.tmp (#0) Nov 8 19:12:52 db influxd[17415]: [tsm1] 2016/11/08 19:12:52 compacted level 1 group 0 of 2 files into 1 files in 421.739431ms Nov 8 19:12:52 db influxd[17415]: [tsm1] 2016/11/08 19:12:52 beginning level 2 compaction of group 0, 2 TSM files Nov 8 19:12:52 db influxd[17415]: [tsm1] 2016/11/08 19:12:52 compacting level 2 group (0) /var/lib/influxdb/data/db/autogen/50/000000565-000000002.tsm (#0) Nov 8 19:12:52 db influxd[17415]: [tsm1] 2016/11/08 19:12:52 compacting level 2 group (0) /var/lib/influxdb/data/db/autogen/50/000000567-000000002.tsm (#1) Nov 8 19:12:53 db influxd[17415]: [tsm1] 2016/11/08 19:12:53 compacted level 2 group (0) into /var/lib/influxdb/data/db/autogen/50/000000567-000000003.tsm.tmp (#0) Nov 8 19:12:53 db influxd[17415]: [tsm1] 2016/11/08 19:12:53 compacted level 2 group 0 of 2 files into 1 files in 573.971276ms Nov 8 19:13:24 db influxd[17415]: [tsm1] 2016/11/08 19:13:24 Snapshot for path /var/lib/influxdb/data/db/autogen/50 written in 1.842949453s -- Remember to include the version number! --- You received this message because you are subscribed to the Google Groups "InfluxData" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/influxdb. To view this discussion on the web visit https://groups.google.com/d/msgid/influxdb/57d726a8-fcc7-488e-8fce-365ab39e467c%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
