[ https://issues.apache.org/jira/browse/NIFI-6724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pierre Villard updated NIFI-6724: --------------------------------- Fix Version/s: 1.11.0 Resolution: Fixed Status: Resolved (was: Patch Available) > PutSQS does not check for individual status in batch message > ------------------------------------------------------------ > > Key: NIFI-6724 > URL: https://issues.apache.org/jira/browse/NIFI-6724 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions > Affects Versions: 1.9.2 > Environment: Docker running in AWS > Reporter: Martin > Assignee: Pierre Villard > Priority: Critical > Labels: Bug, PutSQS, SQS > Fix For: 1.11.0 > > Original Estimate: 2h > Time Spent: 0.5h > Remaining Estimate: 1.5h > > PutSQS does not check for individual status in batch message. This could lead > to PutSQS reporting success and dropping files even when individual files > actually failed in SQS. > As specified in the AWSSimpleQueueService API Reference for > [SendMessageBatch|https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/API_SendMessageBatch.html]: > {quote}The result of sending each message is reported individually in the > response. Because the batch request can result in a combination of successful > and unsuccessful actions, *you should check for batch errors even when the > call returns an HTTP status code of 200.* > {quote} > > We learned this as we did not Base64-encode compressed files (gzip) before > sending them to PutSQS. All the compressed files failed to queue in SQS, but > SQS acknowledged the batch message and reported status 200 (success). Thus > PutSQS also reported success and dropped the files, when they actually > failed. > I suggest a simple fix by implementing > [SendMessageBatchResult|https://docs.aws.amazon.com/AWSJavaSDK/latest/javadoc/com/amazonaws/services/sqs/model/SendMessageBatchResult.html] > to check status on individual files by implementing something similar to this > {code:java} > SendMessageBatchResult sendMessageBatchResult = > sqsClient.sendMessageBatch(request); > if (!sendMessageBatchResult.getFailed().isEmpty()) { > // Error > } > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)