[
https://issues.apache.org/jira/browse/NIFI-8101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17261747#comment-17261747
]
ZhangCheng commented on NIFI-8101:
----------------------------------
Hi [~mattyb149],happy new year!
Firstly ,it would be better to generate a new Processor for creating(alter
drop) table. In this way, the function of the Processor is single and
independent, and we can design the process more flexibly.(And if we want to
develop new Processors for synchronize the table structure, i think some
thoughts in Kettle is very useful)
Secondly, I think the 'PutDatabaseRecord' function should focus on the `data`,
and strive to ensure that the correct data is written to the target table, with
no missing data and no dirty data. And we should follow the table structure of
the target table, instead of modifying the target table according to the data
structure. Modify the target table structure based on the structure of the
data, I know this is useful sometimes, but the data content in the FlowFile
should be treated as indeterminate (including the structure of the data).
Even if we provide the ability to synchronize the table structure in our flow,
I think that modifying the target table should happen before writing data to
the target table, not when writing data to the target table using
'putDatabaseRecord' or after some error occurs.I always think that modifying
the structure of the target table is a very serious thing, and we should do it
explicitly and visibly when we need to.
Additionally, 'Unmatched Field Behavior' and 'Unmatched Column Behavior' are
very useful (I really really like this design),There are always situations
where the incoming data is more or less columns than the target table, and
these columns are likely to be the ones the designer wants to ignore. NIFI-8101
is just the enhancement and supplement of Unmatched Field Behavior' and
'Unmatched Column Behavior'. If the user already knows how to use
PutDatabaseRecord and understands the 'Unmatched Field Behavior' and 'Unmatched
Column Behavior', then I believe they will be able to easily accept the
'Refresh Cached Schema'(Refresh Unmatched Fields/Refresh Unmatched Columns...).
That is my thoughts, what do you think about.
> Improvement PutDatabaseRecord for refresh table schema cache
> -------------------------------------------------------------
>
> Key: NIFI-8101
> URL: https://issues.apache.org/jira/browse/NIFI-8101
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: ZhangCheng
> Assignee: ZhangCheng
> Priority: Major
> Time Spent: 20m
> Remaining Estimate: 0h
>
> * Sometimes, the target table has changed and the `PutDatabaseRecord` cached
> outdated table schema informatiion. Maybe we need a new property to tell
> the `PutDatabaseRecord` to refresh the table schema cache under certain
> conditions.
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
