Hi Nathaniel, Thanks for replying, I forgot to mention that I am using v0.12 since our prod influxdb is still on 0.12.
Join works on the collectd and httpd measurement that I tried on in _internal database. Here's data returned by my queries, copied from the kapacitor.log - they both have exact timestamps but no joins. *from Kapacitor.log * 1. Data from Query 1 <https://gist.github.com/vmahedia/2643abc35284a59befdecee11a584ce2> 2. Data from Query 2 <https://gist.github.com/vmahedia/ec34da47857182e28aab96e020dba5cc> Most likely there's something with my data but I can't seem to figure it out :-( Further what I have already tried based on your suggestions: I simplified the query and tried following queries. Actually I tried way more queries than these ones but this should give you the flavor. 1. Group both batch by 'instance' allow 5s tolerance - does not join, sample data included <https://gist.github.com/vmahedia/52be4f5a7fa2372a85177c1a3ba93ace> 2. No grouping, trying to join on timestamp (default) - does not join <https://gist.github.com/vmahedia/0ae09de5d07e7af5681d9f8425254e7d> 3. Join using a `field`, "app_name", does not join <https://gist.github.com/vmahedia/0560c3ab3d00f50b3c0523d0f6217720> 4. Removed aggregator and trying to join on the field, "app_name" - does not work <https://gist.github.com/vmahedia/df1c3a3d8093bf0c2353b7eef7d2164c> Considering all my other queries might have some problem with data being returned at least No.2 should work as the timestamps are matching since both measurement are sent from same collectd plugin where we put the same timestamp for both. On Wed, Oct 5, 2016 at 8:38 AM, <[email protected]> wrote: > First the union is working as expected, the script has three parents to > the union node: expected_instances, expected_instances, and running_instances. > Expected_instances is given twice, as a result if you do 72985 + 38927 38927 > = 150839 > 150782, which just means that some messages are still in > flight. > > For the join the reason it is not processing any results is that not > points are matching up. In order for two points to be joined they must have > the same timestamp (or be within a specified tolerance) and they must be in > the same group*. In the example above the expected_instances is grouped > by 'instance' while running_instances is grouped by 'instance' and > 'type_instance'. > As a result non of the points in the expected_instances parent are in the > same group as the running_instances parent. > > > * It is possible to join values that are not in the same group, in this > case the join "on" property must be specified. The 'on' property must be a > list of tags to join on and must be a subset of the parents tags. > > For example: > > running_instances > |join(expected_instances) > .as('running', 'expected') > .on('instance') > |httpOut('join_result') > > The above will join a point from expected_instances with each of the > points from running_instances that have the same instance tag. The result > will be joined points grouped by instance and type_instance. > > > Since its not clear what you are trying to accomplish I am not sure > whether you should change the groups of the parents of use the join "on" > property. > > > On Tuesday, October 4, 2016 at 5:38:39 PM UTC-6, Vinit wrote: >> >> Join is not working, it processes zero records. Union works but the >> processes more than the two parent batches `processed` combined. >> >> Since nobody is complaining, it is likely that I am doing something wrong >> but I am out of ideas. Also both these table have exact same timestamps to >> millis. >> >> >> // Get number of instances that should be running >> var expected_instances = batch >> |query('''select first(value) as desired, instance as app_name from >> "collectd_db"."default".marathon_apps_value where >> type_instance='expected' ''') >> .period(2s) >> .every(5s) >> .groupBy('instance') >> >> var running_instances = batch >> |query('''select first(value), instance as app_name, type_instance >> from "collectd_db"."default".marathon_tasks_value where value > 60000 >> ''') >> .period(1s) >> .every(2s) >> .groupBy('instance', 'type_instance') >> >> expected_instances >> |union(expected_instances, running_instances) >> .rename('union_result') >> |httpOut('union_result') >> >> // join this two tables, the app name is same for the same types of >> columns >> running_instances >> |join(expected_instances) >> .as('running', 'expected') >> |httpOut('join_result') >> >> >> >> kapacitor show flapping_tasks >> >> DOT: >> digraph flapping_tasks { >> graph [throughput="0.00 batches/s"]; >> >> batch2 [avg_exec_time_ns="856.130728ms" connect_errors="0" >> query_errors="0" ]; >> batch2 -> join8 [processed="72985"]; >> batch2 -> union5 [processed="*72985*"]; >> >> batch1 [avg_exec_time_ns="273.525217ms" connect_errors="0" >> query_errors="0" ]; >> batch1 -> join8 [processed="38927"]; >> batch1 -> union5 [processed="38927"]; >> batch1 -> union5 [processed="*38927*"]; >> >> join8 [avg_exec_time_ns="30.272µs" ]; >> join8 -> http_out9 [processed="0"]; >> >> http_out9 [avg_exec_time_ns="0" ]; >> >> union5 [avg_exec_time_ns="2.023µs" ]; >> union5 -> http_out6 [processed="150782"]; >> >> http_out6 [avg_exec_time_ns="9.795µs" ]; >> } >> >> -- > Remember to include the InfluxDB version number with all issue reports > --- > You received this message because you are subscribed to a topic in the > Google Groups "InfluxDB" group. > To unsubscribe from this topic, visit https://groups.google.com/d/ > topic/influxdb/wQ3HxSYQdXs/unsubscribe. > To unsubscribe from this group and all its topics, send an email to > [email protected]. > To post to this group, send email to [email protected]. > Visit this group at https://groups.google.com/group/influxdb. > To view this discussion on the web visit https://groups.google.com/d/ > msgid/influxdb/4876112e-ec65-48cb-8062-ed4390fbc28d%40googlegroups.com > <https://groups.google.com/d/msgid/influxdb/4876112e-ec65-48cb-8062-ed4390fbc28d%40googlegroups.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- ~Vinit -- Remember to include the InfluxDB version number with all issue reports --- You received this message because you are subscribed to the Google Groups "InfluxDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/influxdb. To view this discussion on the web visit https://groups.google.com/d/msgid/influxdb/CAKSnE9n7zoon_h4MJWRZa0CSDfS2R8njMxPTaYRa3K07%2BObRVA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
