[
https://issues.apache.org/jira/browse/CAMEL-8539?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14956342#comment-14956342
]
James Lindstorff commented on CAMEL-8539:
-----------------------------------------
As I understand the AWS S3 client the marker is only for paging through a
listObjects result and can not be use for detecting new items in a bucket. I
think something like the idempotent option on the file-component is needed to
use the S3consumer with deleteAfterRead set to false. There is a isTruncated()
method on the objectlisting that indicates if there are more pages in the
listing.
> S3 marker does not work
> -----------------------
>
> Key: CAMEL-8539
> URL: https://issues.apache.org/jira/browse/CAMEL-8539
> Project: Camel
> Issue Type: Bug
> Components: camel-aws
> Affects Versions: 2.15.0
> Reporter: Alexey Hanin
>
> S3Consumer doesn't set marker correctly, when {{deleteAfterRead}} is set to
> {{false}}. Therefore, on every subsequent poll, it starts from the beginning
> of the list.
> The reason for that is the following code in S3Consumer:
> {code:lang=java|title=S3Consumer.java}
> if (!getConfiguration().isDeleteAfterRead()) {
> // where marker is track
> marker = listObjects.getMarker();
> }
> {code}
> Accordingly to S3 client documentation, {{getMarker()}} return a marker
> previously set in request. Instead, {{getNextMarker()}} shall be used.
> Although, simply changing it to use {{getNextMarker()}} introduces an endless
> loop iterating over the same set of keys under the prefix, because when
> reached the end of the key set, {{getNextMarker()}} returns {{null}}, and on
> the next poll, the head of the list will be returned, and so on.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
