There is already a ticket (NIFI-840 <https://issues.apache.org/jira/browse/NIFI-840>) in the hopper to create a ListS3Objects processor that can track bucket contents and trigger FetchS3Object.
In the meantime, if you want it triggered automatically (as opposed to generate flow file) you may be able to accomplish something similar with a combination of HTTP and XML processors. On Thu, Oct 8, 2015 at 2:32 PM, Chakrader Dewaragatla < [email protected]> wrote: > Thanks Aldrin. GenerateFlowFile helped to trigger. > > Thanks, > -Chakri > > From: Aldrin Piri <[email protected]> > Reply-To: "[email protected]" <[email protected]> > Date: Thursday, October 8, 2015 at 11:22 AM > > To: "[email protected]" <[email protected]> > Subject: Re: FetchS3Object processor > > Chakri, > > The FetchS3Object processor does not act as a source directly and is > instead driven by incoming FlowFiles. The use case was that one could > utilize Expression Language to interact with buckets and paths in a dynamic > fashion. Alternatively, if you are looking to just grab a specific file, > you can use GenerateFlowFile to "trigger" the FetchS3Object with its static > configuration. > > On Thu, Oct 8, 2015 at 1:45 PM, Chakrader Dewaragatla < > [email protected]> wrote: > >> Thanks Joe. >> I have S3fetch and putfile processors setup, files are not fetching >> despite files exist on s3. Am I missing anything ? Does S3fetch processor >> need any additional processor to feed data ? >> >> Thanks, >> -Chakri >> >> >> From: Joe Skora <[email protected]> >> Reply-To: "[email protected]" <[email protected]> >> Date: Thursday, October 8, 2015 at 3:38 AM >> To: "[email protected]" <[email protected]> >> Subject: Re: FetchS3Object processor >> >> Chakri, >> >> The Amazon docs explain here >> <http://docs.aws.amazon.com/AmazonS3/latest/UG/FolderOperations.html> >> that S3 doesn't support pathes within a bucket, but you can embed path-like >> naming into the "Object Key" parameter to FetchS3Object. >> >> For your example, you would use bucket="s3://<bucket>" and >> key="<path>/<objectfile>" to incorporate the path as you store the object. >> >> I hope that helps. >> >> Regards, >> Joe >> >> On Thu, Oct 8, 2015 at 2:39 AM, Chakrader Dewaragatla < >> [email protected]> wrote: >> >>> Nifi users - FetchS3Object processor do not have option to specify >>> Object file path, how do you set that ? >>> >>> Eg : s3://<bucket>/<path>/<objectfile> >>> >>> Thanks, >>> -Chakri >>> ------------------------------ >>> The information contained in this transmission may contain privileged >>> and confidential information. It is intended only for the use of the >>> person(s) named above. If you are not the intended recipient, you are >>> hereby notified that any review, dissemination, distribution or duplication >>> of this communication is strictly prohibited. If you are not the intended >>> recipient, please contact the sender by reply email and destroy all copies >>> of the original message. >>> ------------------------------ >>> >> >> ------------------------------ >> The information contained in this transmission may contain privileged and >> confidential information. It is intended only for the use of the person(s) >> named above. If you are not the intended recipient, you are hereby notified >> that any review, dissemination, distribution or duplication of this >> communication is strictly prohibited. If you are not the intended >> recipient, please contact the sender by reply email and destroy all copies >> of the original message. >> ------------------------------ >> > > ------------------------------ > The information contained in this transmission may contain privileged and > confidential information. It is intended only for the use of the person(s) > named above. If you are not the intended recipient, you are hereby notified > that any review, dissemination, distribution or duplication of this > communication is strictly prohibited. If you are not the intended > recipient, please contact the sender by reply email and destroy all copies > of the original message. > ------------------------------ >
