Thank you for this information. There's no internal notion of
directories in S3, only objects, so I suspect I'm good if I only set the

On 2017-08-08 09:55, Joe Skora wrote:

> Laurens, 
> The S3 User Guide Working with Folders [1] page explains how S3 provides a 
> conceptual directory hierarchy using key name prefixes but that buckets 
> really just hold a flat collection of objects. 
> ListS3 will query S3 for the list of objects and then uses the object 
> timestamp as James pointed out to determine what's new to be processed.  
> (Though, it uses last modified timestamp not the last read timestamp.)  You 
> can populate the "Prefix" property of the processor so that to S3 can filter 
> the object list (as if for a directory tree) before sending the list back to 
> NiFi to make things more efficient when dealing with subsets of the bucket 
> contents. 
> Regards, 
> Joe S 
> On Tue, Aug 8, 2017 at 11:22 AM, Laurens Vets <> wrote:
>> Hi list,
>> Does the ListS3 processor keep state of multiple directories in a bucket?
>> For instance, suppose I have a directory "logs" with subdirectories "host1", 
>> "host2" & "host3". Each directory contains logfiles which are added dailty.
>> Will ListS3 keep state correctly here for all 3 subdirectories?
>> Thanks in advance.
>> Laurens



Reply via email to