Victor, Yes, both QDT and GTF would generate something like "SELECT * from myTable where event_time > X", and QDT will execute it and update X. So if event_time is always increasing, it will continue to pick up the same row(s).
That's a curious use case, maybe NiFi could handle other parts of it so you wouldn't need to update a single row in an external database table? Regards, Matt On Mon, Oct 29, 2018 at 12:36 PM l vic <[email protected]> wrote: > > What if have only one row and update the values in it? Will QDT fetch updates? > Thank you, > Victor > > > On Mon, Oct 29, 2018 at 11:54 AM Matt Burgess <[email protected]> wrote: >> >> You can use QueryDatabaseTable (QDT) for this, you'd set your >> "event_time" column as the "Maximum Value Column(s)" property in the >> processor. The first time QDT executes, it will fetch all the rows >> (since it has not seen event_time before), then it will keep track of >> the largest value of event_time. As new rows are added (with larger >> event_time values), QDT will only fetch the rows whose event_time is >> greater than the largest one it's seen. Then it updates its "largest >> seen value" and so on. >> >> GenerateTableFetch (GTF) is another option, it works in a similar >> fashion, except that it does not fetch the rows itself, instead it >> generates flow files containing SQL statements that you can send >> downstream to perhaps ExecuteSQL in order to actually fetch the rows. >> GTF is often used in place of QDT if you'll be fetching a large number >> of rows in each statement, as you can distribute the SQL flow files >> among the nodes in a cluster, to do the fetch in parallel. >> >> Regards, >> Matt >> >> On Mon, Oct 29, 2018 at 11:13 AM l vic <[email protected]> wrote: >> > >> > Hi, >> > i have "event_time" field in SQLite database that means epoch time for >> > triggering of external event. What processor(s) can i use to implement >> > schedule monitoring/ execution based on change in "event_time" value? >> > Thanks,
