On Tue, Jul 26, 2016 at 8:21 AM, <[email protected]> wrote: > On Monday, July 25, 2016 at 5:57:53 PM UTC-4, Sean Beckett wrote: > > It is expected behavior that pointsRx will be larger than pointsTx, but > not for more than 1s or 5000 points. pointsRx is a count of the points > received by the UDP listener. Those points are fed to a batcher running in > the background which batches up points every 1s or 5000 points, whichever > comes first. Those batches are then written to the database, which > increments pointsTx. > > > > > > Are you certain none of your points are duplicates? (Same measurement, > tag set, and timestamp) > > > > > > On Fri, Jul 22, 2016 at 3:01 PM, Hiro <[email protected]> wrote: > > I have been sending UDP data over to one of my databases, and some > points are not coming through. While investigating where this is happening > (I am aware of the nature of UDP but the volume of data is quite small), I > was looking at the UDP measurement in the _internal database. I can see > that at some times, pointsTx (points transmitted) is lower than pointsRx > (points received), and was curious as to what is meant by points > transmitted.. are these points that made it into the database? I am just > curious about the relationship between the two and if there is any way to > observe that behavior further? Any help would be greatly appreciated! > > > > > > > > Thanks, > > > > Hiro > > > > > > > > -- > > > > Remember to include the InfluxDB version number with all issue reports > > > > --- > > > > You received this message because you are subscribed to the Google > Groups "InfluxDB" group. > > > > To unsubscribe from this group and stop receiving emails from it, send > an email to [email protected]. > > > > To post to this group, send email to [email protected]. > > > > Visit this group at https://groups.google.com/group/influxdb. > > > > To view this discussion on the web visit > https://groups.google.com/d/msgid/influxdb/8438915b-1c5a-4e10-99a9-75cd716d5807%40googlegroups.com > . > > > > For more options, visit https://groups.google.com/d/optout. > > > > > > > > > > > > -- > > > > > > Sean Beckett > > Director of Support and Professional Services > > InfluxDB > > Thanks for the explanation! Our system seems to far worse than expected, > as we are often (basically always) farther ahead with pointsRx than with > pointsTx. Here is a graph that shows that relationship a few times, as you > can see pointsTx lags behind by a lot... > > https://i.sli.mg/kL67PV.png > > And this is how our data looks coming in (it seems to be fine for a while > initially, but then points stop arriving consistently and our data is chunky > > https://i.sli.mg/NCfKRE.png > > As for duplicates, I am sure that none of the points are the same, they > all differ. I generated some text files with all our data points and ran > "sort <file> | uniq -d" on our ubuntu server, and all of the files had > unique lines (Tested this with 25532 points from 26 seconds of data and > 68680 points for 98 seconds of data) >
The field keys and values don't contribute to uniqueness. Points are identified by their measurement, full tag set, and timestamp. foo,tag=a field=X 1000 foo,tag=a field=Y 1000 foo,tag=a field=Y,field2=Z 1000 All three of the above are the same point. With that in mind, are you still sure you aren't writing duplicate points? (I no longer think this is the issue, but I'd like to be sure.) > Is it possible that I just configured UDP wrong and it can't handle this > large of an amount of data? > > In the influxdb.conf file it is set up as follows: > > [[udp]] > enabled = true > bind-address = ":8089" > database = "udp" > batch-size = 5000 > batch-timeout = "2s" > read-buffer = 8388608 > I don't have a lot of personal experience tuning the UDP input. I'll ask. > > -- > Remember to include the InfluxDB version number with all issue reports > --- > You received this message because you are subscribed to the Google Groups > "InfluxDB" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/influxdb. > To view this discussion on the web visit > https://groups.google.com/d/msgid/influxdb/200761f9-609a-46d6-a238-417f5aec71d8%40googlegroups.com > . > For more options, visit https://groups.google.com/d/optout. > -- Sean Beckett Director of Support and Professional Services InfluxDB -- Remember to include the InfluxDB version number with all issue reports --- You received this message because you are subscribed to the Google Groups "InfluxDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/influxdb. To view this discussion on the web visit https://groups.google.com/d/msgid/influxdb/CALGqCvOW1hw1QqrA5NY_Xke2pSv-gGyjGeVc6%3DwWyZDy%3DJ8U9A%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
