That is most unfortunate... I am wondering if it is possible to somehow combine stream and batch mode in Kapacitor to collect list of affected intervals (timestamps rounded to interval) via stream and then, as needed, run a batch command to resample for those intervals in addition to latest ones?? (would be really nice if that was part of CQ too, would me much simpler...)
-M On Sunday, November 20, 2016 at 1:56:53 PM UTC-8, Sean Beckett wrote: > Yes, that is correct. All CQs operate on recent data by timestamp, not by > write time. You can use the RESAMPLE FOR clause in a CQ to go back, but it > might be prohibitive to make every CQ go back over the past two or three days > every 10 minutes. > > > I would suggest using the CLI in non-interactive mode to set up a cron job to > run nightly and resample older data using a SELECT...INTO query. > > > On Fri, Nov 18, 2016 at 7:42 PM, <[email protected]> wrote: > I have a (common) scenario where data for measurements comes from multiple > remote sources, some of which may lose connectivity, and data may arrive late > (as much as a few days late in some scenarios.) Since the data is cached, it > is not a big deal, as once the data is sent, the series are populated with > proper data with proper timestamps (which are sent along with the data) > > > > We are now looking into aggregating this data using CQs and retention > policies (or alternatively using Kapacitor to do the same) and I have a > concern about late data. If I understand this correctly, if I aggregate data > in say 10 minute increments, it really only checks the last 10 minutes for > that data. If data is not there when it runs (i.e. over 10 mins late), it > will arrive into the raw measurement in influx, but would never be aggregated > unless I manually run a catch-up "SELECT ...INTO" statement. > > > > First of all, Am I understanding this correctly? Is this really a problem? Or > will it just magically work? > > > > And if so, what is the proper way to aggregate data that may be being > recorded late? > > > > Thanks, > > > > -M > > > > -- > > Remember to include the version number! > > --- > > You received this message because you are subscribed to the Google Groups > "InfluxData" group. > > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > > To post to this group, send email to [email protected]. > > Visit this group at https://groups.google.com/group/influxdb. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/influxdb/9b6d848d-1f43-437b-a774-5d509da458cd%40googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. > > > > > > -- > > > Sean Beckett > Director of Support and Professional Services > InfluxDB -- Remember to include the version number! --- You received this message because you are subscribed to the Google Groups "InfluxData" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/influxdb. To view this discussion on the web visit https://groups.google.com/d/msgid/influxdb/ee445b59-9453-4fa2-8873-8c5ce7e0412d%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
