Thank you for this information. There's no internal notion of directories in S3, only objects, so I suspect I'm good if I only set the bucketname?
On 2017-08-08 09:55, Joe Skora wrote: > Laurens, > > The S3 User Guide Working with Folders [1] page explains how S3 provides a > conceptual directory hierarchy using key name prefixes but that buckets > really just hold a flat collection of objects. > > ListS3 will query S3 for the list of objects and then uses the object > timestamp as James pointed out to determine what's new to be processed. > (Though, it uses last modified timestamp not the last read timestamp.) You > can populate the "Prefix" property of the processor so that to S3 can filter > the object list (as if for a directory tree) before sending the list back to > NiFi to make things more efficient when dealing with subsets of the bucket > contents. > > Regards, > Joe S > > On Tue, Aug 8, 2017 at 11:22 AM, Laurens Vets <laur...@daemon.be> wrote: > >> Hi list, >> >> Does the ListS3 processor keep state of multiple directories in a bucket? >> >> For instance, suppose I have a directory "logs" with subdirectories "host1", >> "host2" & "host3". Each directory contains logfiles which are added dailty. >> >> Will ListS3 keep state correctly here for all 3 subdirectories? >> >> Thanks in advance. >> >> Laurens Links: ------ [1] http://docs.aws.amazon.com/AmazonS3/latest/UG/FolderOperations.html
