Github user mattyb149 commented on the issue:
https://github.com/apache/nifi/pull/1407
I ran into a snag when trying to add incoming flow file support to
QueryDatabaseTable. For GenerateTableFetch, a SQL query is made to the
database to get the max values (and column types) for the table. If the table
names and/or max-value columns are static (like before), the column types can
be fetched when the processor is scheduled. If not, they can be determined from
the SQL query results the first time it is issued (no max-values yet, e.g.).
However QueryDatabaseTable supports "Initial Max Value" dynamic properties, and
does not perform a query ahead of time to get the max values. Instead it
performs the actual fetch query, and determines the max value as it iterates
over the result set. With an initial max value, the initial query needs to
have a WHERE clause, but we don't know the type of the column(s) to determine
how to represent the initial max value as a literal.
So as to avoid this issue, I enabled Expression Language support for the
QueryDatabaseTable properties but kept the annotation as INPUT_FORBIDDEN. We
can revisit this issue as a later improvement to QDT and/or when Initial Max
Value is implemented for GenerateTableFetch (as it may have this same
limitation at that point).
How does that sound? The updated PR has the aforementioned behavior, both
processors support EL (variable-based table names and max-value columns, e.g.)
and GenerateTableFetch supports incoming flow files (so attributes can be used
in the EL expressions).
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
