On Monday, July 25, 2016 at 5:57:53 PM UTC-4, Sean Beckett wrote: > It is expected behavior that pointsRx will be larger than pointsTx, but not > for more than 1s or 5000 points. pointsRx is a count of the points received > by the UDP listener. Those points are fed to a batcher running in the > background which batches up points every 1s or 5000 points, whichever comes > first. Those batches are then written to the database, which increments > pointsTx. > > > Are you certain none of your points are duplicates? (Same measurement, tag > set, and timestamp) > > > On Fri, Jul 22, 2016 at 3:01 PM, Hiro <[email protected]> wrote: > I have been sending UDP data over to one of my databases, and some points are > not coming through. While investigating where this is happening (I am aware > of the nature of UDP but the volume of data is quite small), I was looking at > the UDP measurement in the _internal database. I can see that at some times, > pointsTx (points transmitted) is lower than pointsRx (points received), and > was curious as to what is meant by points transmitted.. are these points that > made it into the database? I am just curious about the relationship between > the two and if there is any way to observe that behavior further? Any help > would be greatly appreciated! > > > > Thanks, > > Hiro > > > > -- > > Remember to include the InfluxDB version number with all issue reports > > --- > > You received this message because you are subscribed to the Google Groups > "InfluxDB" group. > > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > > To post to this group, send email to [email protected]. > > Visit this group at https://groups.google.com/group/influxdb. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/influxdb/8438915b-1c5a-4e10-99a9-75cd716d5807%40googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. > > > > > > -- > > > Sean Beckett > Director of Support and Professional Services > InfluxDB
Thanks for the explanation! Our system seems to far worse than expected, as we are often (basically always) farther ahead with pointsRx than with pointsTx. Here is a graph that shows that relationship a few times, as you can see pointsTx lags behind by a lot... https://i.sli.mg/kL67PV.png And this is how our data looks coming in (it seems to be fine for a while initially, but then points stop arriving consistently and our data is chunky https://i.sli.mg/NCfKRE.png As for duplicates, I am sure that none of the points are the same, they all differ. I generated some text files with all our data points and ran "sort <file> | uniq -d" on our ubuntu server, and all of the files had unique lines (Tested this with 25532 points from 26 seconds of data and 68680 points for 98 seconds of data) Is it possible that I just configured UDP wrong and it can't handle this large of an amount of data? In the influxdb.conf file it is set up as follows: [[udp]] enabled = true bind-address = ":8089" database = "udp" batch-size = 5000 batch-timeout = "2s" read-buffer = 8388608 -- Remember to include the InfluxDB version number with all issue reports --- You received this message because you are subscribed to the Google Groups "InfluxDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/influxdb. To view this discussion on the web visit https://groups.google.com/d/msgid/influxdb/200761f9-609a-46d6-a238-417f5aec71d8%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
