[
https://issues.apache.org/jira/browse/NIFI-853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702239#comment-14702239
]
Aldrin Piri commented on NIFI-853:
----------------------------------
Verified functionality again with the updated patches. Everything looks good
so far from that standpoint and am getting into the details in terms of
conventions and user experience.
normalizeColumnName is a bit confusing to process and needs some additional
documentation or an associated strategy for how this is carried out. I
understand that it is trying to perform the auto matching of SQL columns to the
JSON fields presented, but I have seen a significant amount of JSON that is _
spaced (particularly scripting languages). The intent is good, we just need it
to be more explicit about what is happening. While the normalization would
likely be okay in either case, there is nothing technically wrong with a
database having both _FIRST_NAME and FIRST_NAME (besides poor schema design)
and this could lead to some very hard to discover bugs when rolled out.
Continuing on with my review. Will relay additional comments.
> Create Processors to put JSON data to a Relational Database
> -----------------------------------------------------------
>
> Key: NIFI-853
> URL: https://issues.apache.org/jira/browse/NIFI-853
> Project: Apache NiFi
> Issue Type: Task
> Components: Extensions
> Reporter: Mark Payne
> Assignee: Mark Payne
> Fix For: 0.3.0
>
> Attachments:
> 0001-NIFI-853-Added-processors-ConvertFlatJSONToSQL-PutSQ.patch,
> 0002-NIFI-853-Made-updates-to-processors.patch
>
>
> Most of the discussion/design for these processors happened in the comments
> of NIFI-293, which was the initial ticket for implementing JDBC functionality
> in NiFi, but was closed in a previous version, so this ticket was created to
> do the work.
> The idea is to have a processor that will take in FlowFiles whose contents
> are arbitrary SQL INSERT/UPDATE commands. The commands can be parameterized
> with the parameters' values and types in FlowFile attributes.
> We then should have a processor that converts a JSON document into a SQL
> command to either update or insert data into a database table. We will also
> want some other processors in the future probably to handle other data types,
> such as converting XML, CSV, Avro, etc. into SQL commands.
> This breakout gives us a nice coherence to the "do only one thing and do it
> well" principle by separating the logic of handling all of the incoming
> formats from the logic of updating the database.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
