[
https://issues.apache.org/jira/browse/NIFI-853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14702268#comment-14702268
]
Aldrin Piri commented on NIFI-853:
----------------------------------
I think things are close, but playing around with some things a bit, there is
one thing I think needs to be addressed and another that is questionable in
terms of necessity. Both of these are centered around multiple inserts.
Consider the following json adapted from the persons.json provided in tests:
{code}
[
{
"id": 1,
"name": "Mark",
"code": 48
},
{
"id": 2,
"code": 48,
"name": "George"
},
{
"code": 21,
"id": 3,
"name": "Harry"
},
{
"id": 4,
"name": "Julie",
"code": 48
},
{
"puppy": 12,
"code": 16,
"name": "Frank Henry",
"id": 82
}
]
{code}
# JSON objects have no strict ordering of their elements to be considered
equivalent. Accordingly, while all items have the requisite fields needed,
items are currently just iterated across in order. I think the fix is to
create a map of anticipated fields.
# Coincides with the concept of the prior item and would make use of the map,
but when presented with an array of objects, how do we remedy items that are in
excess of the representative item? It seems like this should likely just be an
error condition and the associated flow file routed off. We can't be sure of
the intent of the user. From the segment above, if the object with id 82 was
first, all other elements would appear to be short a field whereas, in its
current configuration, it appears to have an excess field.
> Create Processors to put JSON data to a Relational Database
> -----------------------------------------------------------
>
> Key: NIFI-853
> URL: https://issues.apache.org/jira/browse/NIFI-853
> Project: Apache NiFi
> Issue Type: Task
> Components: Extensions
> Reporter: Mark Payne
> Assignee: Mark Payne
> Fix For: 0.3.0
>
> Attachments:
> 0001-NIFI-853-Added-processors-ConvertFlatJSONToSQL-PutSQ.patch,
> 0002-NIFI-853-Made-updates-to-processors.patch
>
>
> Most of the discussion/design for these processors happened in the comments
> of NIFI-293, which was the initial ticket for implementing JDBC functionality
> in NiFi, but was closed in a previous version, so this ticket was created to
> do the work.
> The idea is to have a processor that will take in FlowFiles whose contents
> are arbitrary SQL INSERT/UPDATE commands. The commands can be parameterized
> with the parameters' values and types in FlowFile attributes.
> We then should have a processor that converts a JSON document into a SQL
> command to either update or insert data into a database table. We will also
> want some other processors in the future probably to handle other data types,
> such as converting XML, CSV, Avro, etc. into SQL commands.
> This breakout gives us a nice coherence to the "do only one thing and do it
> well" principle by separating the logic of handling all of the incoming
> formats from the logic of updating the database.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
