[
https://issues.apache.org/jira/browse/NIFI-2958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15616297#comment-15616297
]
Justin M Cavinee commented on NIFI-2958:
----------------------------------------
This might be more appropriate as a second ticket, but I think the two problems
are related.
The problem that really made the lack of an error message frustrating was that
the records were being placed back on the input queue, and thus replayed (where
they proceeded to error again, Ad infinitum). After sometime the queue would
fill up, and then all of my processing would stop.
After doing some experimentation, I found that if I set the batch size down to
one, the record(s) that were failing would be properly routed to my failure
queue, at which point I could fix / replay them as needed. Batch size of one
appears to be considerably slower than doing batch processing though, so it's
not something I'll want to do long-term. That said, it at least has my
processor working so it's no longer a blocking issue, but this is going to be a
major concern once I get close to going to production.
Ideal Solution:
The single record that caused the failure would be forward to the failure
route, while the successful ones would be retried later
Acceptable Solution / quick fix:
The whole block is sent to the failure, at which point I could have a
secondary processor that tries them individually.
One other concern I have that I haven't tested yet is if it attempts to insert
a batch of 100, and 1 fails, whether or not the others made it to elastic
search, or if the whole set is done as a transaction and rolled back on the
first failure.
> Missing Error Message in PutElasticsearchHttp
> ---------------------------------------------
>
> Key: NIFI-2958
> URL: https://issues.apache.org/jira/browse/NIFI-2958
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Reporter: Justin M Cavinee
> Priority: Minor
>
> Error messages are not being shown properly when the push to Elastic Search
> fails.
> When I get a failure, it just reads:
> Failed to Insert StandardFlowFileRecord[..] into Elastic Search due to ,
> transferring to failure.
> With nothing following the due to. If I use curl to directly push to content
> to ElasticSearch then I do get a proper error message, which should be put
> into the Elastic Search log.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
