[
https://issues.apache.org/jira/browse/NIFI-2881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15826199#comment-15826199
]
ASF GitHub Bot commented on NIFI-2881:
--------------------------------------
Github user mattyb149 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1407#discussion_r96434505
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GenerateTableFetch.java
---
@@ -115,20 +128,36 @@ public GenerateTableFetch() {
@OnScheduled
public void setup(final ProcessContext context) {
+ // The processor is invalid if there is an incoming connection and
max-value columns are defined
+ if (context.getProperty(MAX_VALUE_COLUMN_NAMES).isSet() &&
context.hasIncomingConnection()) {
+ throw new ProcessException("If an incoming connection is
supplied, no max-value column names may be specified");
--- End diff --
For backwards-compatibility, what do you think about only falling back to
the column name as the key if the table name property does not contain
Expression Language? I'm thinking the only use case we should have to support
from before is if the table name is static (as it was required to be before
now). This would not allow for incoming flow files to specify via attribute the
same table name as before, but I don't see that being a high-priority use case
necessarily. With this approach, currently configured processors would continue
to work as-is, but once you change to incoming flow files (or use EL), the new
scheme would apply. Even currently configured processors would automatically
switch to the new scheme as soon as a new maximum value is observed.
> Allow Database Fetch processor(s) to accept incoming flow files and use
> Expression Language
> -------------------------------------------------------------------------------------------
>
> Key: NIFI-2881
> URL: https://issues.apache.org/jira/browse/NIFI-2881
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: Matt Burgess
> Assignee: Matt Burgess
>
> The QueryDatabaseTable and GenerateTableFetch processors do not allow
> Expression Language to be used in the properties, mainly because they also do
> not allow incoming connections. This means if the user desires to fetch from
> multiple tables, they currently need one instance of the processor for each
> table, and those table names must be hard-coded.
> To support the same capabilities for multiple tables and more flexible
> configuration via Expression Language, these processors should have
> properties that accept Expression Language, and GenerateTableFetch should
> accept (optional) incoming connections.
> Conversation about the behavior of the processors is welcomed and encouraged.
> For example, if an incoming flow file is available, do we also still run the
> incremental fetch logic for tables that aren't specified by this flow file,
> or do we just do incremental fetching when the processor is scheduled but
> there is no incoming flow file. The latter implies a denial-of-service could
> take place, by flooding the processor with flow files and not letting it do
> its original job of querying the table, keeping track of maximum values, etc.
> This is likely a breaking change to the processors because of how state
> management is implemented. Currently since the table name is hard coded, only
> the column name comprises the key in the state. This would have to be
> extended to have a compound key that represents table name, max-value column
> name, etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
