Kyle Let us know if that doesn't get you what you need. We have a decent set of templates but I didn't see one that demonstrates interaction with amazon services.
Thanks Joe On Jan 30, 2016 12:56 PM, "Joey Frazee" <[email protected]> wrote: > Kyle, > > I think you can do what you want right now without ListS3 by using S3 > event notifications. You can configure an event notification to publish to > SQS and then use GetSQS to retrieve the events and FetchS3Object to get the > JSON file and the rest of the flow could be written as you have in mind. > > Depending on your scale, this might be preferable because it's > slow/expensive to do listings on S3 prefixes that have a lot of file > matches. > > > -joey > > On Jan 30, 2016, at 11:40 AM, Joe Skora <[email protected]> wrote: > > Kyle, > > Processors exist to Put, Fetch, and Delete S3Objects, but ListS3 is in the > backlog on ticket NIFI-840 > <https://issues.apache.org/jira/browse/NIFI-840> at the moment. It > should fit the List/Fetch metaphor like the List/Fetch processors pairs for > xFile, xHDFS, xSFTP, etc. > > Regards, > Joe Skora > > On Sat, Jan 30, 2016 at 10:14 AM, Kyle Burke <[email protected]> > wrote: > >> All, >> I'm trying to get Nifi set up to a move data around S3. My first >> attempt is to just monitor a S3 folder where json files are placed and then >> copy the file, convert it to Avro, and the drop it in a different S3 >> folder. The documentation is pretty slim for working with S3. I can't seem >> to get it working and was wondering if anyone had any S3 examples for >> monitoring an S3 folder (i.e.. something like a ListS3 processer similar to >> what is available on a local file system?) >> >> Respectfully, >> >> *Kyle Burke *| Data Science Engineer >> *IgnitionOne - *Marketing Technology. Simplified. >> Office: 1545 Peachtree St NE, Suite 500 | Atlanta, GA | 30309 >> Direct: 404.961.3918 >> >> >
