Matt Burgess created NIFI-2881:
----------------------------------
Summary: Allow Database Fetch processors to accept incoming flow
files and use Expression Language
Key: NIFI-2881
URL: https://issues.apache.org/jira/browse/NIFI-2881
Project: Apache NiFi
Issue Type: Improvement
Components: Extensions
Reporter: Matt Burgess
The QueryDatabaseTable and GenerateTableFetch processors do not allow
Expression Language to be used in the properties, mainly because they also do
not allow incoming connections. This means if the user desires to fetch from
multiple tables, they currently need one instance of the processor for each
table, and those table names must be hard-coded.
To support the same capabilities for multiple tables and more flexible
configuration via Expression Language, these processors should have properties
that accept Expression Language, and should accept (optional) incoming
connections.
Conversation about the behavior of the processors is welcomed and encouraged.
For example, if an incoming flow file is available, do we also still run the
incremental fetch logic for tables that aren't specified by this flow file, or
do we just do incremental fetching when the processor is scheduled but there is
no incoming flow file. The latter implies a denial-of-service could take place,
by flooding the processor with flow files and not letting it do its original
job of querying the table, keeping track of maximum values, etc.
This is likely a breaking change to the processors because of how state
management is implemented. Currently since the table name is hard coded, only
the column name comprises the key in the state. This would have to be extended
to have a compound key that represents table name, max-value column name, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
