Jim,
Absolutely that's one way. Depending on how many directories you have, you
can also do it directly with RouteOnAttribute and the expression language:
Property name: s3exists
Property value: ${outputTarget:equals('foo'):or(outputTarget:equals('bar'))}
Then route the s3exists relationship to PutS3Object.
The python script strategy you mentioned may be good for a small to medium
number of directories.
The ListS3 strategy mentioned by James might be a better fit if the list is
too large to easily maintain by hand.
Hope that helps,
Adam
On Fri, Mar 17, 2017 at 3:07 PM, James McMahon <[email protected]> wrote:
> So keep my list in a python script dictionary called by an ExecuteScript
> processor, and toss my outputTarget value against that. Set a new attribute
> s3exists to true or false in my script based on that result, and then use
> RouteAttribute to direct the output. Is that what you have in mind? -Jim
>
> On Fri, Mar 17, 2017 at 4:59 PM, Adam Lamar <[email protected]> wrote:
>
>> Jim,
>>
>> Also keep in mind that as an object store, S3 uses "directories" only as
>> a grouping concept, and not as a hierarchal storage mechanism. That's why
>> the initial PutS3Object doesn't fail with a new "directory". See
>> http://docs.aws.amazon.com/AmazonS3/latest/UG/FolderOperations.html
>>
>> I think James' advice is spot on - to accomplish what you need, you'll
>> likely want to keep a list of known outputTargets in NiFi.
>>
>> Cheers,
>> Adam
>>
>
>
