> - When sending data to TimeSeries DB like InfluxDB, with current
> precision of millis, we can end up overwriting some value if user
> wants to send data to InfluxDB as in the same MS you can trigger many same
> Transactions.
>
> If we go up to Nanos we would be able to handle such cases without issues.
^ Referring to the precision of "time of event", so not the precision of the
likes of "response time".
JMeter should not have the responsibility of changing the precision of its
time_of_event to solve a problem of interaction with InfluxDB. It would
actually not entirely solve the problem and is also not the correct way to use
InfluxDB - read below why and how.
InfluxDB (or Graphite or Prometheus too?) will only overwrite the data point if
all "tags" + time are the same. Example:
Tag - Hostname
Tag - ThreadName
Tag - ThreadNumber
Tag - sample Label
Value ("field") - response time
Time - time_of_event_ms
^ in order for the data point to be overwritten, the Tags *plus* the Time of
the data point all need to be the same. So the very same thread would have to
produce the same data point within the very same millisecond. We can make this
occur although not under normal circumstances (zero pauses in-between requests
combined with sub-ms response times). If we were to increase the precision,
collisions would happen more rarely but it would still not be guaranteed,
collisions could still happen.
Identifying data points with a reasonably unique set of Tags decreases the
likelihood of collisions but it will never be guaranteed, and that is
regardless of the precision of the time_of_event. <- are you by chance trying
to upload the data points with a small set of identification Tags? That could
indeed run into frequent collisions...
The proper way to handle this and the *only* way to guarantee that no data
points get lost is to adjust how the data is exported to InfluxDB - the data
should be pre-aggregated before shipping to InfluxDB, data that isn't a
time-series should be transformed from non-time-series data into
time-series-data - that's how Influx is meant to be used, it's a time-series DB.
Ex, transforming the non-time-series JMeter results set into a time-series data
of «every minute» rate:
Tag - Hostname
Tag - ThreadName
Tag - ThreadNumber
Tag - sample Label
Value ("field") - Min response time
Value ("field") - Mean response time
Value ("field") - Median response time
Value ("field") - 90th Percentile response time
Value ("field") - 95th Percentile response time
Value ("field") - Max response time
Value ("field") - Number of successful requests
Value ("field") - Number of failed requests
Time - time_of_event_minute
I wrote my own plugin to export the data to InfluxDB but I'm under the
impression that JMeter's built-in BackendListener plugin is already doing these
aggregations, transforming the JMeter data into a time-series. Are you by
change using something like Logstash or Telegraf to monitor the produced CSV
files and collect that data? If you are, make sure you are parsing the right
columns into appropriate Tags identifying the data points and also you should
have some manner of pause in-between requests - or alternatively consider using
the built-in BackendListener plugin that exports the results directly into
InfluxDB (or Graphite).
Regards,
Paulo Augusto Maia Borges
The information in this e-mail and any attachments is confidential and may be
legally privileged. It is intended solely for the addressee or addressees. Any
use or disclosure of the contents of this e-mail/attachments by a not intended
recipient is unauthorized and may be unlawful. If you have received this e-mail
in error please notify the sender. Please note that any views or opinions
presented in this e-mail are solely those of the author and do not necessarily
represent those of TEMENOS. We recommend that you check this e-mail and any
attachments against viruses. TEMENOS accepts no liability for any damage caused
by any malicious code or virus transmitted by this e-mail.
