Yes, that's my understanding too. On Tue, Aug 8, 2017 at 1:14 PM, Laurens Vets <[email protected]> wrote:
> Thank you for this information. There's no internal notion of directories > in S3, only objects, so I suspect I'm good if I only set the bucketname? > > On 2017-08-08 09:55, Joe Skora wrote: > > Laurens, > > The S3 User Guide Working with Folders > <http://docs.aws.amazon.com/AmazonS3/latest/UG/FolderOperations.html> page > explains how S3 provides a conceptual directory hierarchy using key name > prefixes but that buckets really just hold a flat collection of objects. > > ListS3 will query S3 for the list of objects and then uses the object > timestamp as James pointed out to determine what's new to be processed. > (Though, it uses last modified timestamp not the last read timestamp.) > You can populate the "Prefix" property of the processor so that to S3 can > filter the object list (as if for a directory tree) before sending the list > back to NiFi to make things more efficient when dealing with subsets of the > bucket contents. > > Regards, > Joe S > > > On Tue, Aug 8, 2017 at 11:22 AM, Laurens Vets <[email protected]> wrote: > >> Hi list, >> >> Does the ListS3 processor keep state of multiple directories in a bucket? >> >> For instance, suppose I have a directory "logs" with subdirectories >> "host1", "host2" & "host3". Each directory contains logfiles which are >> added dailty. >> >> Will ListS3 keep state correctly here for all 3 subdirectories? >> >> Thanks in advance. >> >> Laurens >> > >
