Thanks so much for the detailed response! On Fri, Jan 6, 2017 at 6:54 PM, Mark Rushakoff <[email protected]> wrote:
> > should I send that information to statsd/telegraf and have it > process/summarize those metrics (such as generating count, lower, mean, > upper, etc) - or should I just write each measurement directly to influxdb > and use native influxdb functions to compute those computed values if I > want them. > > I'd recommend starting by recording the raw values into InfluxDB. From the > raw data it's easy to apply whichever aggregates you need to help you > discover whatever it is you're interested in learning. Having the raw data > available makes it easy for you to vary GROUP BY intervals [1], for > instance. > > Depending on what queries you're running, and how expensive they are and > how frequently you run them, you may want to eventually use continuous > queries [2] to do the heavy lifting on those aggregates only once, which > would roughly be akin to the statsd preprocessing. I wouldn't worry about > CQs until you've settled more into your use patterns. > > > what are the consideration for implementing the url as a tag vs a field > > You're almost certainly going to want to keep URLs as tags, for the sake > of quick searches and for GROUP BY. See the schema design docs [3] for more > details. > > > our application could have thousands of unique urls, so the cardinality > is perhaps high. > > Cardinality in the thousands or tens of thousands will be no problem on > modern hardware. It depends how your URLs are structured, too. You probably > don't want to store "/posts?id=123" as the URL when just "/posts" would > suffice. If you do care to capture id=123 in that case, one option would be > to have one measurement per URL/controller, where every measurement has the > "ms" field, and individual measurements have other URL-specific fields. > > > What troubles will I run into? > > The one-tag one-field approach is a great starting point for application > metrics. > > You'll want to find the right balance between writing at the lowest > acceptable time precision [4], and avoiding or accepting duplicate points > for a series [5]. > > You're not going to want to hold onto your application metrics data > forever, so read up on retention policies [6] to understand more about how > old data is purged. > > [1] https://docs.influxdata.com/influxdb/v1.1/query_ > language/data_exploration/#group-by-time-intervals > [2] https://docs.influxdata.com/influxdb/v1.1/query_ > language/continuous_queries/ > [3] https://docs.influxdata.com/influxdb/v1.1/concepts/ > schema_and_data_layout/ > [4] https://docs.influxdata.com/influxdb/v1.1/troubleshooting/ > frequently-asked-questions/#does-the-precision-of-the-timestamp-matter > [5] https://docs.influxdata.com/influxdb/v1.1/troubleshooting/frequently- > asked-questions/#how-does-influxdb-handle-duplicate-points > [6] https://docs.influxdata.com/influxdb/v1.1/query_ > language/database_management/#retention-policy-management > > On Fri, Jan 6, 2017 at 3:13 PM, <[email protected]> wrote: > >> I'm migrating from a graphite install and I'm trying to figure out the >> best way to design a measurement for capturing performance metrics for an >> application that will allow me to break down performance for specific URLs >> (or controllers, etc...) >> >> So assume I have a url, and a time in ms it took to handle that url, I >> plan on saving this to a measurement called "url_performance". >> >> My first question is - should I send that information to statsd/telegraf >> and have it process/summarize those metrics (such as generating count, >> lower, mean, upper, etc) - or should I just write each measurement directly >> to influxdb and use native influxdb functions to compute those computed >> values if I want them. What are the considerations for one over the >> other? I'm leaning towards just writing this data directly to influxdb >> >> The second question is - what are the consideration for implementing the >> url as a tag vs a field. I want to search by the url (even searching by >> regex to lump urls together), so that would suggest a tag, but our >> application could have thousands of unique urls, so the cardinality is >> perhaps high. >> >> So right now, I'm thinking of this simple measurement >> >> url_performance >> tag: url >> field: ms >> >> What troubles will I run into? >> >> Thanks! >> >> -- >> Remember to include the version number! >> --- >> You received this message because you are subscribed to the Google Groups >> "InfluxData" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at https://groups.google.com/group/influxdb. >> To view this discussion on the web visit https://groups.google.com/d/ms >> gid/influxdb/b8671a64-c862-4f0a-8531-483ac70e57e4%40googlegroups.com. >> For more options, visit https://groups.google.com/d/optout. >> > > -- > Remember to include the version number! > --- > You received this message because you are subscribed to a topic in the > Google Groups "InfluxData" group. > To unsubscribe from this topic, visit https://groups.google.com/d/ > topic/influxdb/And25FCx0M8/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/influxdb. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/influxdb/CALxJwdN3XnEmis9sJSjgSPv8w_ri9R5QtoPqa6CJevaYWRh1ig% > 40mail.gmail.com > <https://groups.google.com/d/msgid/influxdb/CALxJwdN3XnEmis9sJSjgSPv8w_ri9R5QtoPqa6CJevaYWRh1ig%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- Remember to include the version number! --- You received this message because you are subscribed to the Google Groups "InfluxData" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/influxdb. To view this discussion on the web visit https://groups.google.com/d/msgid/influxdb/CAJRP2Y_2gHW%2BqnmxciyHzDf%2BTdGyGYL336SJc5xt06jwTkuuQw%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
