We’d like to do analysis across both, so preferably, yes, we’d like to do group-by’s on both. Or at least I think that’s what we want – minimally, we’d want to chart number of users for a given AP in something like 10–30 minute intervals. That would require both being tags, right? I mean if users were stored in fields, it’d mean no two users could share the same timestamp. Perhaps that’s not a big issue – I’ll go inspect our logs for timestamp overlap – but it’s something I’d conceptually rather not worry about.
On 18 November 2016 at 17:48:33, Sean Beckett ([email protected]<mailto:[email protected]>) wrote: It's not possible to predict accurately how much RAM you will need, but with 4 million series (roughly a day of unique user and AP ids) you would need something on the order of 32-64GB for the system to be healthy. http://docs.influxdata.com/influxdb/v1.1/guides/hardware_sizing/ With 30k users and 7.5k APs, the theoretical max series is about 225 million, which is current infeasible, although much work is being done to allow for cardinalities up to 1 billion. We expect the 1.2 release to have the experimental version of the new index. You can also shard the series across data nodes in our commercial clustering offering. With a 4 node cluster, doubly replicated, each node only handles half the series cardinality. With a 10 node cluster, doubly replicated, each node only handles 1/5th of the series cardinality. Do you need to GROUP BY both users and APs? Perhaps there's a way to structure your queries so that one or the other can be a field and not a tag. On Fri, Nov 18, 2016 at 8:02 AM, <[email protected]<mailto:[email protected]>> wrote: Op dinsdag 15 december 2015 02:16:02 UTC+1 schreef Sean Beckett: > If you have highly dynamic data, that should be stored as a field, not a tag. > That makes it impossible to use with GROUP BY clauses, however, which I > suspect is a non-starter for you. Is there a way to guesstimate the memory requirements from the cardinality? We're looking to analyse how people flow through our building by analysing our radius logs. From these, we get a measurement approx every 30 seconds per user that couples a user (or device) to an access point. Picking a random day from the logs puts us at 7.5k access points to which some 500 users connected. That'd get us a cardinality of about 3.75M... that's not going to end well I take it then. If we broaden the sample, we're likely to approach 30k users over the lifetime of the analysis. -- Remember to include the version number! --- You received this message because you are subscribed to the Google Groups "InfluxData" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]<mailto:influxdb%[email protected]>. To post to this group, send email to [email protected]<mailto:[email protected]>. Visit this group at https://groups.google.com/group/influxdb. To view this discussion on the web visit https://groups.google.com/d/msgid/influxdb/3d709e55-7a67-44c8-9b35-79693e26e65c%40googlegroups.com. For more options, visit https://groups.google.com/d/optout. -- Sean Beckett Director of Support and Professional Services InfluxDB -- Remember to include the version number! --- You received this message because you are subscribed to a topic in the Google Groups "InfluxData" group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/influxdb/iyCFGavNaeU/unsubscribe. To unsubscribe from this group and all its topics, send an email to [email protected]<mailto:[email protected]>. To post to this group, send email to [email protected]<mailto:[email protected]>. Visit this group at https://groups.google.com/group/influxdb. To view this discussion on the web visit https://groups.google.com/d/msgid/influxdb/CALGqCvNdJTuPeuYGOQwOKpAhrFNcMcbhEMbvEGozx4EyKsswhQ%40mail.gmail.com<https://groups.google.com/d/msgid/influxdb/CALGqCvNdJTuPeuYGOQwOKpAhrFNcMcbhEMbvEGozx4EyKsswhQ%40mail.gmail.com?utm_medium=email&utm_source=footer>. For more options, visit https://groups.google.com/d/optout. -- Groet, Emiliano Heyns -- Remember to include the version number! --- You received this message because you are subscribed to the Google Groups "InfluxData" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/influxdb. To view this discussion on the web visit https://groups.google.com/d/msgid/influxdb/etPan.582f7931.72099a8b.4ef%40han.nl. For more options, visit https://groups.google.com/d/optout.
