Hi Hendrik, The behaviour you describe sounds like the first GenerateTableFetch processor doesn’t find any records with a higher max_value than the initial value. In that case, it will not emit any flowfile (unless you have Output Empty FlowFile on Zero Results set to true). Since it hasn’t found any records, it also hasn’t increased the max_value in the state.
In the second case, it does find records new than your initial value and generates a select statement up to the highest value it sees (which appears to be close to realtime from your description). The state should then be the new observed max value. Both are expected behavior for the respective cases. I don’t believe the processor *ever* sets the state to now(), only to the highest observed value. The most likely explanations are that: * The MySQL server table received very recent records that you didn’t expect * The SQL server’s time value is not getting converted the way you expect and that’s why it’s not finding any new records. Maybe a microseconds vs milliseconds thing? You could verify this by manually executing a select with the specified starting values and checking the results. Regards, Isha Van: Hendrik Ruijter <[email protected]> Verzonden: vrijdag 14 januari 2022 16:43 Aan: [email protected] Onderwerp: Inconsistent behaviour in GenerateTableFetch 1.15.0 Hello, in GenerateTableFetch 1.15.0, I use the dynamic property initial.maxvalue.<max_value_column> when I move timestamp state to a new cluster from old GenerateTableFetch 1.13.2 processors. First use case: SQL Server initial.maxvalue.tm 1326581088 State is set correctly to 1326581088 and the flowfile is terminated by GenerateTableFetch 1.15.0. Second use case: MySQL Server initial.maxvalue.time 2022-01-14 06:13:57.0 State is set correctly to Now() UTC (not to the dynamic property 2022-01-14 06:13:57.0) and the flowfile contains a correct select statement with a where clause to execute. Is the expected behavior to terminate the flowfile when the state is set as in the first use case? Or, is the expected behavior to set the state and execute the generated select statement as in second use case? Sorry, I am unable to find the answer in the documentation and my Java source code knowledge is very limited.
