[
https://issues.apache.org/jira/browse/NIFI-6762?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16948046#comment-16948046
]
Aram S Openden edited comment on NIFI-6762 at 10/11/19 9:55 PM:
----------------------------------------------------------------
Please note: In the course of my work for my project team at IBM Watson Health,
_I've done almost all of the work for this Processor already_ in a segregated
proprietary project for our team.
My plan is to port that code over to the NiFi main source project in the
{{nifi-nar-bundles}} sub-project, making a small set of changes/refactorings
(mostly around common code factored out) on a feature branch and then submit my
PR for Review to the NiFi team.
note: Feature branch in my fork of the main NiFi project:
[https://github.com/aramcodz/nifi/tree/nifi-6762]
--Aram
was (Author: aramcodez):
Please note: In the course of my work for my project team at IBM Watson Health,
_I've done almost all of the work for this Processor already_ in a segregated
proprietary project for our team.
My plan is to port that code over to the NiFi main source project in the
{{nifi-nar-bundles}} sub-project, making a small set of changes/refactorings
(mostly around common code factored out) on a feature branch and then submit my
PR for Review to the NiFi team.
note: Feature branch in my fork of the main NiFi project:
[https://github.com/aramcodz/nifi/tree/nifi-6762]
--Aram
> New Processor - Watch Multiple S3 Buckets for content changes
> -------------------------------------------------------------
>
> Key: NIFI-6762
> URL: https://issues.apache.org/jira/browse/NIFI-6762
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
> Affects Versions: 1.9.2
> Reporter: Aram S Openden
> Priority: Major
> Labels: Processor, S3, watch
> Original Estimate: 384h
> Remaining Estimate: 384h
>
> Proposing New S3 processor that is a "variation" on the existing ListS3
> Processor. For lack of better term, I will call this "Watch Multiple S3
> Buckets Processor" (naming here is open to feedback). This new Processor
> takes an input attribute "Bucket" from the FlowFile, which could use NiFi
> ExpressionLanguage to compute that value. Hence, the Processor could "watch"
> different buckets in an S3 Object Store.
> This processor must handle the State Management logic for multiple S3
> buckets. The simplest solution for this problem add to modify the State
> Management logic to prepend the bucket name to this key/objectName that is
> being used to track state.
> Finally, the Dev should document the maximum number of buckets it will
> monitor at once, maximum amount of state/storage it can have to track the
> various listings in various buckets.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
