Thanks Sean for the clarification.I shall try with millisecond/nanosecond time stamp.
On Thursday, September 8, 2016 at 9:11:09 AM UTC+5:30, Sean Beckett wrote: > > When no explicit timestamp is supplied, InfluxDB applies the same > timestamp to all points in the batch. Since all points belong to the same > series, InfluxDB treats them as duplicate points, and stores the union of > the points, with the last field winning. > > > https://docs.influxdata.com/influxdb/v0.13/troubleshooting/frequently_encountered_issues/#writing-duplicate-points > > To avoid this, you need to explicitly supply timestamps (preferred) or > write the points in separate batches so they will inherit unique nanosecond > timestamps. Note that not batching writes will lead to significantly > reduced write throughput, due to the high HTTP overhead. > > On Wed, Sep 7, 2016 at 7:46 AM, indranilr <[email protected] > <javascript:>> wrote: > >> I am consuming messages from per-populated Kafka topic and trying to POST >> to influxdb (tried via telegraf and node.js client) but seeing that >> multiple POSTs issued in short span of time is not getting reflected in >> influxdb database.I wasn't sure if this is an actual bug or an issue with >> some configuration in my code.Kindly advise. >> >> Following are the points I have populated in Kafka queue : >> >> {"value":40,"host":"host1"} >> {"value":50,"host":"host1"} >> {"value":60,"host":"host1"} >> {"value":70,"host":"host1"} >> {"value":80,"host":"host1"} >> Once node.js client consumes (using kafka-node) the same it writes it to >> influx (using node-influx) using *writePoint *method (post some parsing) >> >> influxClient.writePoint(target_metric, {value : vl}, {host : ht} , {db: >> db_name},callback); >> >> Please note* no timestamp* is being sent to influx. >> >> However, when I check in target influx measurements I only fewer points >> than originally sent (you can see only 4 points below). >> >> name: test_ms_1 >> --------------- >> time host value >> 1473251241835000000 host1 40 >> 1473251241838000000 host1 80 >> 1473251241860000000 host1 60 >> 1473251241861000000 host1 50 >> >> >> Could anyone explain the above behavior is due to missing timestamp in >> input data or something else is wrong. >> >> I checked influx log and I can see equivalent number of POST queries and >> relevant message bodies (I have turned on body tracing) >> >> [httpd] 2016/09/07 17:57:21 write body received by handler: >>> test_ms_1,host=host1 value=40 >>> [tsm1wal] 2016/09/07 17:57:21 tsm1 WAL starting with 10485760 segment >>> size >>> [tsm1wal] 2016/09/07 17:57:21 tsm1 WAL writing to >>> /var/lib/influxdb/wal/eridb6/default/12 >>> [shard] 2016/09/07 17:57:21 /var/lib/influxdb/data/eridb6/default/12 >>> database index loaded in 986ns >>> [httpd] 2016/09/07 17:57:21 write body received by handler: >>> test_ms_1,host=host1 value=80 >>> [httpd] 2016/09/07 17:57:21 write body received by handler: >>> test_ms_1,host=host1 value=70 >>> [httpd] 2016/09/07 17:57:21 write body received by handler: >>> test_ms_1,host=host1 value=60 >>> [httpd] 2016/09/07 17:57:21 write body received by handler: >>> test_ms_1,host=host1 value=50 >>> [httpd] 2016/09/07 17:57:21 10.0.2.2 - root [07/Sep/2016:17:57:21 +0530] >>> POST /write?database=&db=eridb6&p=%5BREDACTED%5D&precision=ms&u=root >>> HTTP/1.1 204 0 - - 6cd985c1-74f6-11e6-8013-000000000000 26.071054ms >>> [httpd] 2016/09/07 17:57:21 10.0.2.2 - root [07/Sep/2016:17:57:21 +0530] >>> POST /write?database=&db=eridb6&p=%5BREDACTED%5D&precision=ms&u=root >>> HTTP/1.1 204 0 - - 6cda049b-74f6-11e6-8014-000000000000 29.699655ms >>> [httpd] 2016/09/07 17:57:21 10.0.2.2 - root [07/Sep/2016:17:57:21 +0530] >>> POST /write?database=&db=eridb6&p=%5BREDACTED%5D&precision=ms&u=root >>> HTTP/1.1 204 0 - - 6cdd57e2-74f6-11e6-8015-000000000000 10.920647ms >>> [httpd] 2016/09/07 17:57:21 10.0.2.2 - root [07/Sep/2016:17:57:21 +0530] >>> POST /write?database=&db=eridb6&p=%5BREDACTED%5D&precision=ms&u=root >>> HTTP/1.1 204 0 - - 6cdd765b-74f6-11e6-8016-000000000000 11.097619ms >> >> [httpd] 2016/09/07 17:57:21 10.0.2.2 - root [07/Sep/2016:17:57:21 +0530] >>> POST /write?database=&db=eridb6&p=%5BREDACTED%5D&precision=ms&u=root >>> HTTP/1.1 204 0 - - 6cdd7c1a-74f6-11e6-8017-000000000000 11.845628ms >>> >>> >> I am using influx 0.13 on CentOS Linux release 7.2.1511 (Core) (64 bit) >> on Virtual box: >> >> Static hostname: localhost.centos >> Icon name: computer-vm >> Chassis: vm >> Machine ID: XXXXXXXXXX >> Boot ID: XXXXXXXXXXXXXXXX >> Virtualization: kvm >> Operating System: CentOS Linux 7 (Core) >> CPE OS Name: cpe:/o:centos:centos:7 >> Kernel: Linux 3.10.0-327.el7.x86_64 >> Architecture: x86-64 >> >> System resources : >> >> *Memory :* >> >> MemTotal: 1016860 kB >> MemFree: 699836 kB >> MemAvailable: 716180 kB >> Cached: 126384 kB >> SwapCached: 0 kB >> SwapTotal: 839676 kB >> SwapFree: 839676 kB >> >> >> *CPU:* >> >> cache size : 3072 KB >> physical id : 0 >> siblings : 1 >> core id : 0 >> cpu cores : 1 >> apicid : 0 >> initial apicid : 0 >> fpu : yes >> fpu_exception : yes >> cpuid level : 13 >> wp : yes >> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge >> mca cmov pat pse36 clflush mmx fxsr sse sse2 syscall nx rdtscp lm >> constant_tsc rep_good nopl xtopology nonstop_tsc pni pclmulqdq monitor >> ssse3 cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx rdrand hypervisor >> lahf_lm abm >> bogomips : 5188.00 >> clflush size : 64 >> cache_alignment : 64 >> address sizes : 39 bits physical, 48 bits virtual >> >> >> -- >> Remember to include the InfluxDB version number with all issue reports >> --- >> You received this message because you are subscribed to the Google Groups >> "InfluxDB" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> To post to this group, send email to [email protected] >> <javascript:>. >> Visit this group at https://groups.google.com/group/influxdb. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/influxdb/e42d40c9-f743-44ab-b2d9-7a0c18a6bfeb%40googlegroups.com >> >> <https://groups.google.com/d/msgid/influxdb/e42d40c9-f743-44ab-b2d9-7a0c18a6bfeb%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Sean Beckett > Director of Support and Professional Services > InfluxDB > -- Remember to include the InfluxDB version number with all issue reports --- You received this message because you are subscribed to the Google Groups "InfluxDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/influxdb. To view this discussion on the web visit https://groups.google.com/d/msgid/influxdb/9610ea74-29be-4d17-b042-81acf34dd5b8%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
