Github user ijokarumawak commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1407#discussion_r98604070
--- Diff:
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/AbstractDatabaseFetchProcessor.java
---
@@ -117,9 +124,11 @@
+ "can be used to retrieve only those rows that have
been added/updated since the last retrieval. Note that some "
+ "JDBC types such as bit/boolean are not conducive to
maintaining maximum value, so columns of these "
+ "types should not be listed in this property, and
will result in error(s) during processing. If no columns "
- + "are provided, all rows from the table will be
considered, which could have a performance impact.")
+ + "are provided, all rows from the table will be
considered, which could have a performance impact.\nNOTE: If Expression
Language is "
+ + "present for this property and it refers to flow
file attribute(s), then the Table Name property must also contain Expression
Language.")
--- End diff --
What does the 'NOTE' imply? There's no validation code for this condition,
so I understand it as a recommendation. Is it correct?
I think it's possible that an user intentionally use static Table name and
dynamic Max value columns together, if they would like to fetch `brand new`
items by `created_timestamp`, and also `updated` items by `updated_timestamp`
from the same table. This works fine.
What doesn't work (or dangerous) is probably a situation that different Max
value columns are passed with the same table name and those columns overlap
each other, as such usage will corrupt state management.
- E.g.1 : FF1(table=T1, maxColumn=b, a) but FF2(table=T1, maxColumn=a
only). This can be problematic.
- E.g.2 : FF1(table=T1, maxColumn=b, a), FF2(table=T1, maxColumn=c). This
actually work fine.
If the note was added to explain this risk, then the 'NOTE' can be written
like: "NOTE: It's important to use consistent Max value columns for a given
table for increment fetch to work properly"?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
