[
https://issues.apache.org/jira/browse/STREAMS-166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14140806#comment-14140806
]
ASF GitHub Bot commented on STREAMS-166:
----------------------------------------
Github user smashew commented on the pull request:
https://github.com/apache/incubator-streams/pull/80#issuecomment-56198437
@renato2099 With the way the reader works, there is a price for an object
listing... (1 cent per 1,000), seeing as the stream is exhausting the entire
directory instead of reading just a portion, the constant of 50 || 100 is
inconsequential. bumping to 500 will limit the # of round trip calls however.
Agree, that it should be a setting, but presumably, one does run the risk
of having soo many settings that it is too difficult to use. :o)
> S3PersistReader does not correctly handle directory listing
> -----------------------------------------------------------
>
> Key: STREAMS-166
> URL: https://issues.apache.org/jira/browse/STREAMS-166
> Project: Streams
> Issue Type: Bug
> Reporter: Robert Douglas
> Original Estimate: 3h
> Remaining Estimate: 3h
>
> In its current form, the S3PeristReader is not able to pick up on files
> inside of a given directory. In addition to checking for common prefixes, we
> need to ensure there are no object summaries in the listing response before
> we assume we are dealing with a single file.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
