Hi Sean, The current verified number of series is : 41712
27 CQ, I am using enableBatch: influxClient.enableBatch(10000, 3000, TimeUnit.MILLISECONDS); at least 8192 calls of: batch_points.point(metric_point); before calling write: influxClient.write(batch_points); Please let me know if you have more questions (more stats are available in the end of this email), I appreciate your help, Thank you! --Rafit some more stats: select * from runtime where time > now() - 10s name: runtime ------------- time Alloc Frees HeapAlloc HeapIdle HeapInUse HeapObjects HeapReleased HeapSys Lookups Mallocs NumGC NumGoroutine PauseTotalNs Sys TotalAlloc clusterID hostname nodeID 1473312983000000000 153833696 345337583 153833696 29990912 204431360 2751982 0 234422272 491 348089565 246 134 122381212 248941072 15169333136 2765793636717751283 ??????.?????????????.??? 0 select * from "database" where time > now() - 10s name: database -------------- time clusterID database hostname nodeID numMeasurements numSeries 1473313073000000000 2765793636717751283 _internal ??????.????????????.??? 0 11 131 1473313073000000000 2765793636717751283 analytics ??????.????????????.??? 0 25 41581 select * from "shard" where time > now() - 10s name: shard ----------- time clusterID database engine fieldsCreate hostname id nodeID path retentionPolicy seriesCreate writePointsOk writeReq 1473313223000000000 2765793636717751283 _internal tsm1 0 ??????.????????????.??? 31 0 /var/lib/influxdb/data/_internal/monitor/31 monitor 9391 122 select * from "write" where time > now() - 10s name: write ----------- time clusterID hostname nodeID pointReq pointReqLocal req subWriteDrop subWriteOk writeOk 1473313293000000000 2765793636717751283 ??????.????????????.??? 0 9930 9930 129 129 129 show stats name: cq -------- queryOk 36342 name: database tags: database=_internal numMeasurements numSeries --------------- --------- 11 131 name: database tags: database=analytics numMeasurements numSeries --------------- --------- 25 41581 name: httpd tags: bind=:8086 pingReq queryReq queryReqDurationNs queryRespBytes req reqActive reqDurationNs ------- -------- ------------------ -------------- --- --------- ------------- 1 51 764949154 2509363 52 1 773122523 name: queryExecutor ------------------- queriesActive queryDurationNs 1 1677180844 name: shard tags: database=_internal, engine=tsm1, id=31, path=/var/lib/influxdb/data/_internal/monitor/31, retentionPolicy=monitor fieldsCreate writePointsOk writeReq ------------ ------------- -------- 0 10623 138 name: subscriber ---------------- pointsWritten 10623 name: tsm1_cache tags: database=_internal, path=/var/lib/influxdb/data/_internal/monitor/1, retentionPolicy=monitor WALCompactionTimeMs cacheAgeMs cachedBytes diskBytes memBytes snapshotCount ------------------- ---------- ----------- --------- -------- ------------- 0 1427909 0 0 0 0 name: tsm1_cache tags: database=_internal, path=/var/lib/influxdb/data/_internal/monitor/10, retentionPolicy=monitor WALCompactionTimeMs cacheAgeMs cachedBytes diskBytes memBytes snapshotCount ------------------- ---------- ----------- --------- -------- ------------- 0 1427908 0 0 0 0 name: tsm1_cache tags: database=_internal, path=/var/lib/influxdb/data/_internal/monitor/31, retentionPolicy=monitor WALCompactionTimeMs cacheAgeMs cachedBytes diskBytes memBytes snapshotCount ------------------- ---------- ----------- --------- -------- ------------- 0 1428182 0 0 9594304 0 name: tsm1_cache tags: database=analytics, path=/var/lib/influxdb/data/analytics/default/2, retentionPolicy=default WALCompactionTimeMs cacheAgeMs cachedBytes diskBytes memBytes snapshotCount ------------------- ---------- ----------- --------- -------- ------------- 0 1427908 0 0 0 0 name: tsm1_cache tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_day/17, retentionPolicy=one_day WALCompactionTimeMs cacheAgeMs cachedBytes diskBytes memBytes snapshotCount ------------------- ---------- ----------- --------- -------- ------------- 0 1427909 0 0 0 0 name: tsm1_cache tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_day/18, retentionPolicy=one_day WALCompactionTimeMs cacheAgeMs cachedBytes diskBytes memBytes snapshotCount ------------------- ---------- ----------- --------- -------- ------------- 0 1404711 0 0 0 0 name: tsm1_cache tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_day/19, retentionPolicy=one_day WALCompactionTimeMs cacheAgeMs cachedBytes diskBytes memBytes snapshotCount ------------------- ---------- ----------- --------- -------- ------------- 0 1389395 0 0 0 0 name: tsm1_cache tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_day/20, retentionPolicy=one_day WALCompactionTimeMs cacheAgeMs cachedBytes diskBytes memBytes snapshotCount ------------------- ---------- ----------- --------- -------- ------------- 0 1405509 0 0 0 0 name: tsm1_cache tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_day/21, retentionPolicy=one_day WALCompactionTimeMs cacheAgeMs cachedBytes diskBytes memBytes snapshotCount ------------------- ---------- ----------- --------- -------- ------------- 0 1397407 0 0 0 0 name: tsm1_cache tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_day/22, retentionPolicy=one_day WALCompactionTimeMs cacheAgeMs cachedBytes diskBytes memBytes snapshotCount ------------------- ---------- ----------- --------- -------- ------------- 0 1396496 0 0 0 0 name: tsm1_cache tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_day/23, retentionPolicy=one_day WALCompactionTimeMs cacheAgeMs cachedBytes diskBytes memBytes snapshotCount ------------------- ---------- ----------- --------- -------- ------------- 0 1396236 0 0 0 0 name: tsm1_cache tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_day/24, retentionPolicy=one_day WALCompactionTimeMs cacheAgeMs cachedBytes diskBytes memBytes snapshotCount ------------------- ---------- ----------- --------- -------- ------------- 0 1403847 0 0 0 0 name: tsm1_cache tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_day/25, retentionPolicy=one_day WALCompactionTimeMs cacheAgeMs cachedBytes diskBytes memBytes snapshotCount ------------------- ---------- ----------- --------- -------- ------------- 0 1414572 0 0 0 0 name: tsm1_cache tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_day/26, retentionPolicy=one_day WALCompactionTimeMs cacheAgeMs cachedBytes diskBytes memBytes snapshotCount ------------------- ---------- ----------- --------- -------- ------------- 0 1414691 0 0 0 0 name: tsm1_cache tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_day/27, retentionPolicy=one_day WALCompactionTimeMs cacheAgeMs cachedBytes diskBytes memBytes snapshotCount ------------------- ---------- ----------- --------- -------- ------------- 0 1405726 0 0 0 0 name: tsm1_cache tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_day/28, retentionPolicy=one_day WALCompactionTimeMs cacheAgeMs cachedBytes diskBytes memBytes snapshotCount ------------------- ---------- ----------- --------- -------- ------------- 0 1395702 0 0 0 0 name: tsm1_cache tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_day/29, retentionPolicy=one_day WALCompactionTimeMs cacheAgeMs cachedBytes diskBytes memBytes snapshotCount ------------------- ---------- ----------- --------- -------- ------------- 0 1416754 0 0 0 0 name: tsm1_cache tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_day/30, retentionPolicy=one_day WALCompactionTimeMs cacheAgeMs cachedBytes diskBytes memBytes snapshotCount ------------------- ---------- ----------- --------- -------- ------------- 0 1419778 0 0 0 0 name: tsm1_cache tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_month/13, retentionPolicy=one_month WALCompactionTimeMs cacheAgeMs cachedBytes diskBytes memBytes snapshotCount ------------------- ---------- ----------- --------- -------- ------------- 0 1427868 0 0 0 0 name: tsm1_cache tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_month/5, retentionPolicy=one_month WALCompactionTimeMs cacheAgeMs cachedBytes diskBytes memBytes snapshotCount ------------------- ---------- ----------- --------- -------- ------------- 0 1423398 0 0 0 0 name: tsm1_cache tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_week/12, retentionPolicy=one_week WALCompactionTimeMs cacheAgeMs cachedBytes diskBytes memBytes snapshotCount ------------------- ---------- ----------- --------- -------- ------------- 0 1422570 0 0 0 0 name: tsm1_cache tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_week/4, retentionPolicy=one_week WALCompactionTimeMs cacheAgeMs cachedBytes diskBytes memBytes snapshotCount ------------------- ---------- ----------- --------- -------- ------------- 0 1420522 0 0 0 0 name: tsm1_cache tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_year/6, retentionPolicy=one_year WALCompactionTimeMs cacheAgeMs cachedBytes diskBytes memBytes snapshotCount ------------------- ---------- ----------- --------- -------- ------------- 0 1426600 0 0 0 0 name: tsm1_filestore tags: database=_internal, path=/var/lib/influxdb/data/_internal/monitor/1, retentionPolicy=monitor diskBytes --------- 330422 name: tsm1_filestore tags: database=_internal, path=/var/lib/influxdb/data/_internal/monitor/10, retentionPolicy=monitor diskBytes --------- 2321520 name: tsm1_filestore tags: database=analytics, path=/var/lib/influxdb/data/analytics/default/2, retentionPolicy=default diskBytes --------- 22524306 name: tsm1_filestore tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_day/17, retentionPolicy=one_day diskBytes --------- 243080989 name: tsm1_filestore tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_day/18, retentionPolicy=one_day diskBytes --------- 239262941 name: tsm1_filestore tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_day/19, retentionPolicy=one_day diskBytes --------- 248413477 name: tsm1_filestore tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_day/20, retentionPolicy=one_day diskBytes --------- 244294838 name: tsm1_filestore tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_day/21, retentionPolicy=one_day diskBytes --------- 239914567 name: tsm1_filestore tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_day/22, retentionPolicy=one_day diskBytes --------- 252412434 name: tsm1_filestore tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_day/23, retentionPolicy=one_day diskBytes --------- 245078392 name: tsm1_filestore tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_day/24, retentionPolicy=one_day diskBytes --------- 239291222 name: tsm1_filestore tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_day/25, retentionPolicy=one_day diskBytes --------- 250779604 name: tsm1_filestore tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_day/26, retentionPolicy=one_day diskBytes --------- 243886546 name: tsm1_filestore tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_day/27, retentionPolicy=one_day diskBytes --------- 238989758 name: tsm1_filestore tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_day/28, retentionPolicy=one_day diskBytes --------- 250148686 name: tsm1_filestore tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_day/29, retentionPolicy=one_day diskBytes --------- 174987063 name: tsm1_filestore tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_day/30, retentionPolicy=one_day diskBytes --------- 68603390 name: tsm1_filestore tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_month/13, retentionPolicy=one_month diskBytes --------- 31862665 name: tsm1_filestore tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_month/5, retentionPolicy=one_month diskBytes --------- 29406087 name: tsm1_filestore tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_week/12, retentionPolicy=one_week diskBytes --------- 90157429 name: tsm1_filestore tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_week/4, retentionPolicy=one_week diskBytes --------- 144493656 name: tsm1_filestore tags: database=analytics, path=/var/lib/influxdb/data/analytics/one_year/6, retentionPolicy=one_year diskBytes --------- 49154983 name: tsm1_wal tags: database=_internal, path=/var/lib/influxdb/wal/_internal/monitor/1, retentionPolicy=monitor currentSegmentDiskBytes oldSegmentsDiskBytes ----------------------- -------------------- 0 0 name: tsm1_wal tags: database=_internal, path=/var/lib/influxdb/wal/_internal/monitor/10, retentionPolicy=monitor currentSegmentDiskBytes oldSegmentsDiskBytes ----------------------- -------------------- 0 0 name: tsm1_wal tags: database=_internal, path=/var/lib/influxdb/wal/_internal/monitor/31, retentionPolicy=monitor currentSegmentDiskBytes oldSegmentsDiskBytes ----------------------- -------------------- 725550 11936787 name: tsm1_wal tags: database=analytics, path=/var/lib/influxdb/wal/analytics/default/2, retentionPolicy=default currentSegmentDiskBytes oldSegmentsDiskBytes ----------------------- -------------------- 0 0 name: tsm1_wal tags: database=analytics, path=/var/lib/influxdb/wal/analytics/one_day/17, retentionPolicy=one_day currentSegmentDiskBytes oldSegmentsDiskBytes ----------------------- -------------------- 0 0 name: tsm1_wal tags: database=analytics, path=/var/lib/influxdb/wal/analytics/one_day/18, retentionPolicy=one_day currentSegmentDiskBytes oldSegmentsDiskBytes ----------------------- -------------------- 0 0 name: tsm1_wal tags: database=analytics, path=/var/lib/influxdb/wal/analytics/one_day/19, retentionPolicy=one_day currentSegmentDiskBytes oldSegmentsDiskBytes ----------------------- -------------------- 0 0 name: tsm1_wal tags: database=analytics, path=/var/lib/influxdb/wal/analytics/one_day/20, retentionPolicy=one_day currentSegmentDiskBytes oldSegmentsDiskBytes ----------------------- -------------------- 0 0 name: tsm1_wal tags: database=analytics, path=/var/lib/influxdb/wal/analytics/one_day/21, retentionPolicy=one_day currentSegmentDiskBytes oldSegmentsDiskBytes ----------------------- -------------------- 0 0 name: tsm1_wal tags: database=analytics, path=/var/lib/influxdb/wal/analytics/one_day/22, retentionPolicy=one_day currentSegmentDiskBytes oldSegmentsDiskBytes ----------------------- -------------------- 0 0 name: tsm1_wal tags: database=analytics, path=/var/lib/influxdb/wal/analytics/one_day/23, retentionPolicy=one_day currentSegmentDiskBytes oldSegmentsDiskBytes ----------------------- -------------------- 0 0 name: tsm1_wal tags: database=analytics, path=/var/lib/influxdb/wal/analytics/one_day/24, retentionPolicy=one_day currentSegmentDiskBytes oldSegmentsDiskBytes ----------------------- -------------------- 0 0 name: tsm1_wal tags: database=analytics, path=/var/lib/influxdb/wal/analytics/one_day/25, retentionPolicy=one_day currentSegmentDiskBytes oldSegmentsDiskBytes ----------------------- -------------------- 0 0 name: tsm1_wal tags: database=analytics, path=/var/lib/influxdb/wal/analytics/one_day/26, retentionPolicy=one_day currentSegmentDiskBytes oldSegmentsDiskBytes ----------------------- -------------------- 0 0 name: tsm1_wal tags: database=analytics, path=/var/lib/influxdb/wal/analytics/one_day/27, retentionPolicy=one_day currentSegmentDiskBytes oldSegmentsDiskBytes ----------------------- -------------------- 0 0 name: tsm1_wal tags: database=analytics, path=/var/lib/influxdb/wal/analytics/one_day/28, retentionPolicy=one_day currentSegmentDiskBytes oldSegmentsDiskBytes ----------------------- -------------------- 0 0 name: tsm1_wal tags: database=analytics, path=/var/lib/influxdb/wal/analytics/one_day/29, retentionPolicy=one_day currentSegmentDiskBytes oldSegmentsDiskBytes ----------------------- -------------------- 0 0 name: tsm1_wal tags: database=analytics, path=/var/lib/influxdb/wal/analytics/one_day/30, retentionPolicy=one_day currentSegmentDiskBytes oldSegmentsDiskBytes ----------------------- -------------------- 0 0 name: tsm1_wal tags: database=analytics, path=/var/lib/influxdb/wal/analytics/one_month/13, retentionPolicy=one_month currentSegmentDiskBytes oldSegmentsDiskBytes ----------------------- -------------------- 0 0 name: tsm1_wal tags: database=analytics, path=/var/lib/influxdb/wal/analytics/one_month/5, retentionPolicy=one_month currentSegmentDiskBytes oldSegmentsDiskBytes ----------------------- -------------------- 0 0 name: tsm1_wal tags: database=analytics, path=/var/lib/influxdb/wal/analytics/one_week/12, retentionPolicy=one_week currentSegmentDiskBytes oldSegmentsDiskBytes ----------------------- -------------------- 0 0 name: tsm1_wal tags: database=analytics, path=/var/lib/influxdb/wal/analytics/one_week/4, retentionPolicy=one_week currentSegmentDiskBytes oldSegmentsDiskBytes ----------------------- -------------------- 0 0 name: tsm1_wal tags: database=analytics, path=/var/lib/influxdb/wal/analytics/one_year/6, retentionPolicy=one_year currentSegmentDiskBytes oldSegmentsDiskBytes ----------------------- -------------------- 0 0 name: write ----------- pointReq pointReqLocal req subWriteOk writeOk 10623 10623 138 138 138 name: runtime ------------- Alloc Frees HeapAlloc HeapIdle HeapInUse HeapObjects HeapReleased HeapSys Lookups Mallocs NumGC NumGoroutine PauseTotalNs Sys TotalAlloc 175354504 384279146 175354504 18333696 221331456 3414371 0 239665152 518 387693517 263 139 132846894 254351888 16680835416 On Wed, Sep 7, 2016 at 8:36 PM, Sean Beckett <[email protected]> wrote: > 45k series should not consume 64GB of RAM. Are there any CQs running? How > many writes per second? Are they batched? What's the user-initiated query > load? > > Can you verify the series count using > > SELECT sum(numSeries) AS “total_series" FROM “_internal".."database" WHERE > time > now() - 10s > > On Wed, Sep 7, 2016 at 6:07 PM, <[email protected]> wrote: > >> Hi Sean, >> I am running into a similar problem, >> RAM size is 64G >> number of series is 45,158 >> influxdb uses all the memory and crashes, >> Once it crashes, after restarting is will fill up the memory within 10 >> minutes or so and will crash again. >> I am using the java api to push data, >> >> Thanks, >> --Rafit >> >> -- >> Remember to include the InfluxDB version number with all issue reports >> --- >> You received this message because you are subscribed to the Google Groups >> "InfluxDB" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at https://groups.google.com/group/influxdb. >> To view this discussion on the web visit https://groups.google.com/d/ms >> gid/influxdb/6d3e06d7-c36c-4da1-908c-9b0407646e31%40googlegroups.com. >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Sean Beckett > Director of Support and Professional Services > InfluxDB > > -- > Remember to include the InfluxDB version number with all issue reports > --- > You received this message because you are subscribed to a topic in the > Google Groups "InfluxDB" group. > To unsubscribe from this topic, visit https://groups.google.com/d/ > topic/influxdb/fiQzEy1aWP8/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/influxdb. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/influxdb/CALGqCvNN10UeGLDq00juNdjrRD-qpRD219bFtKzJPYHXDxg%3D9Q% > 40mail.gmail.com > <https://groups.google.com/d/msgid/influxdb/CALGqCvNN10UeGLDq00juNdjrRD-qpRD219bFtKzJPYHXDxg%3D9Q%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- Rafit Izhak Ratzin, PhD Computer Science Department UCLA -- Remember to include the InfluxDB version number with all issue reports --- You received this message because you are subscribed to the Google Groups "InfluxDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/influxdb. To view this discussion on the web visit https://groups.google.com/d/msgid/influxdb/CAFAManek-igJM61g_2Qz%3Dhf1q1Ccro91-ZFEecPX_j5zUkY_4Q%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
