[
https://issues.apache.org/jira/browse/NIFI-8101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17260889#comment-17260889
]
Matt Burgess commented on NIFI-8101:
------------------------------------
This looks related to NIFI-7862, except here the use case is that when data
comes in that doesn't match the table, the new property can be set to refresh
the table in the expectation that the table has been updated and should accept
the new data. This feels like a sort of more manual "data drift" capability;
NIFI-7862 would add (and possibly delete) columns in the table based on the
incoming data, so the table doesn't have to be externally updated to accept the
new data.
Having said that, if I understand the intent of this Jira correctly, we
wouldn't want fields/columns to be ignored or throw errors when the table has
been externally altered. Could we instead just re-fetch the table schema if
something doesn't match, and then check again (regardless of whether we ignore,
warn, or fail)? Seems like the worst that would happen is the same that would
happen if we had stopped and restarted the processor after the table had been
altered. If, after updating the schema, the fields don't match the current
columns, the processor would do what it is configured to do. That way we don't
need a new property, we just use a possible error condition to bring things
up-to-date and check again. Or do you see use cases when we'd only want to
fetch the table schema if fields didn't match columns, or vice versa? IMO if
something doesn't match, we'd just make sure we have the latest information and
proceed on that basis. What are your thoughts?
> Improvement PutDatabaseRecord for refresh table schema cache
> -------------------------------------------------------------
>
> Key: NIFI-8101
> URL: https://issues.apache.org/jira/browse/NIFI-8101
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: ZhangCheng
> Assignee: ZhangCheng
> Priority: Major
> Time Spent: 10m
> Remaining Estimate: 0h
>
> * Sometimes, the target table has changed and the `PutDatabaseRecord` cached
> outdated table schema informatiion. Maybe we need a new property to tell
> the `PutDatabaseRecord` to refresh the table schema cache under certain
> conditions.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
