[
https://issues.apache.org/jira/browse/NIFI-853?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14700534#comment-14700534
]
Mark Payne commented on NIFI-853:
---------------------------------
I uploaded a new patch to this ticket. It adds the Batch Size property as
discussed above.
[~aldrin]: the approach that I took to handle a failure in the batch scenario:
If a BatchUpdateException is thrown, you can obtain from it the number of rows
updated for each statement before the error.
Because of that, we can determine exactly which FlowFile caused failure.
Those that were updated before the problematic FlowFile can move on to success.
Problematic FlowFile is routed to failure. All others in that Batch are routed
to 'retry'.
If an update fails and it was part of a transaction that spans multiple
FlowFiles, though, we rollback the entire database transaction and route all
FlowFiles in that transaction to failure.
I've created unit tests for the edge cases that I found, but please do let me
know if you find anything that seems off, any way to create a better design, or
if you can think of an edge case that was missed.
Thanks
-Mark
> Create Processors to put JSON data to a Relational Database
> -----------------------------------------------------------
>
> Key: NIFI-853
> URL: https://issues.apache.org/jira/browse/NIFI-853
> Project: Apache NiFi
> Issue Type: Task
> Components: Extensions
> Reporter: Mark Payne
> Assignee: Mark Payne
> Fix For: 0.3.0
>
> Attachments:
> 0001-NIFI-853-Added-processors-ConvertFlatJSONToSQL-PutSQ.patch,
> 0002-NIFI-853-Made-updates-to-processors.patch
>
>
> Most of the discussion/design for these processors happened in the comments
> of NIFI-293, which was the initial ticket for implementing JDBC functionality
> in NiFi, but was closed in a previous version, so this ticket was created to
> do the work.
> The idea is to have a processor that will take in FlowFiles whose contents
> are arbitrary SQL INSERT/UPDATE commands. The commands can be parameterized
> with the parameters' values and types in FlowFile attributes.
> We then should have a processor that converts a JSON document into a SQL
> command to either update or insert data into a database table. We will also
> want some other processors in the future probably to handle other data types,
> such as converting XML, CSV, Avro, etc. into SQL commands.
> This breakout gives us a nice coherence to the "do only one thing and do it
> well" principle by separating the logic of handling all of the incoming
> formats from the logic of updating the database.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
