[
https://issues.apache.org/jira/browse/CAMEL-8431?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15260726#comment-15260726
]
Claus Ibsen commented on CAMEL-8431:
------------------------------------
the nextMarker is only if the result has more data (eg if it was truncated) and
you need to call it again to get more data, eg in a while loop.
If there is no delete after read, then you need some kind of idempotent
consumer to detect duplicates, you can do that by adding the idempotent
consumer EIP
http://camel.apache.org/idempotent-consumer.html
> Consume all files in aws S3 bucket where deleteAfterRead = false
> ----------------------------------------------------------------
>
> Key: CAMEL-8431
> URL: https://issues.apache.org/jira/browse/CAMEL-8431
> Project: Camel
> Issue Type: Bug
> Components: camel-aws
> Affects Versions: 2.14.0
> Reporter: Yap Poh Soon
> Assignee: Willem Jiang
> Fix For: 2.15.4, 2.16.1, 2.17.0
>
>
> The current AWS S3Consumer class has a problem if user supply the parameter
> deleteAfterRead=false. The S3Consumer will always consume the same files
> over and over again with the size of getMaxMessagesPerPoll().
> After some code chasing, i think, the root cause is because the
> ListObjectsRequest does not has the previous marker value. Hence, i wonder if
> we could do something like below:
> {quote}
> {color:blue}private string marker; // new line to define the marker string
> {color}
> ..
> ..
> ..
> ListObjectsRequest listObjectsRequest = new ListObjectsRequest();
> listObjectsRequest.setBucketName(bucketName);
> listObjectsRequest.setPrefix(getConfiguration().getPrefix());
> listObjectsRequest.setMaxKeys(maxMessagesPerPoll);
> {color:blue}listObjectsRequest.setMarker(marker); // new line to set the
> marker {color}
> ObjectListing listObjects =
> getAmazonS3Client().listObjects(listObjectsRequest);
> {color:blue}marker = listObjects.getMarker(); // where marker is track
>
> {color}{quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
