Thank you Adam and James. This has been very helpful, and gives me a number of options to explore. I am all set, thanks again for your help! -Jim
On Fri, Mar 17, 2017 at 5:33 PM, Adam Lamar <[email protected]> wrote: > Jim, > > Absolutely that's one way. Depending on how many directories you have, you > can also do it directly with RouteOnAttribute and the expression language: > > Property name: s3exists > Property value: ${outputTarget:equals('foo'):or(outputTarget:equals(' > bar'))} > > Then route the s3exists relationship to PutS3Object. > > The python script strategy you mentioned may be good for a small to medium > number of directories. > > The ListS3 strategy mentioned by James might be a better fit if the list > is too large to easily maintain by hand. > > Hope that helps, > Adam > > > On Fri, Mar 17, 2017 at 3:07 PM, James McMahon <[email protected]> > wrote: > >> So keep my list in a python script dictionary called by an ExecuteScript >> processor, and toss my outputTarget value against that. Set a new attribute >> s3exists to true or false in my script based on that result, and then use >> RouteAttribute to direct the output. Is that what you have in mind? -Jim >> >> On Fri, Mar 17, 2017 at 4:59 PM, Adam Lamar <[email protected]> wrote: >> >>> Jim, >>> >>> Also keep in mind that as an object store, S3 uses "directories" only as >>> a grouping concept, and not as a hierarchal storage mechanism. That's why >>> the initial PutS3Object doesn't fail with a new "directory". See >>> http://docs.aws.amazon.com/AmazonS3/latest/UG/FolderOperations.html >>> >>> I think James' advice is spot on - to accomplish what you need, you'll >>> likely want to keep a list of known outputTargets in NiFi. >>> >>> Cheers, >>> Adam >>> >> >> >
