Hi. The team I work in is doing a good deal of work with NiFi S3 Processors amongst others and writing some of our own custom processors. Our team has a similar use-case requirement for a variation on the ListS3 Processor as Martijn Dekkers in this post here < http://apache-nifi-users-list.2361937.n4.nabble.com/Listing-S3-tp5777p5850.html > .
For context, the reader may wish to refer to this entire thread from the beginning < http://apache-nifi-users-list.2361937.n4.nabble.com/Listing-S3-td5777.html#a5850 > . In our case, we would like the processor to allow for incoming FlowFiles and be able to change the S3 bucket it "listens to" by making the /s3.bucket/ attribute modifiable using the NiFi expression language *while continuing to maintain the internal state* of the Processor. We would simultaneously restrict the /prefix/ property from being updated, making it a fixed value for the entire lifetime of the Processor's running. In other words, we want a *WatchMultipleS3Buckets* Processor that maintains state for multiple buckets. In practice this means, for our new processor, we would modify the internal state management logic. Currently, the value for each entry in the StateMap is simply the "filename" of the S3 object. Our suggested change in the StateMap's HashMap would have this value now be of the/ bucketName + some delimiter + filename/ of the S3 Object. Our team is working on our variation of the processor, this *WatchMultipleS3Buckets*. We would like to offer to contribute back to the community this effort as follows. Since there will be a great deal of common code between the current ListS3 Processor and our newly proposed WatchMultipleS3Buckets Processor, we refactor to create a new Abstract class: AbstractS3WatchProcessor with the existing ListS3 and the new WatchMultipleS3Buckets processors as subclasses of this new AbstractS3WatchProcessor. Is this addition/modification something the community would be interested in? If yes, can someone please provide me a link to the instructions on the logistics & contribution guidelines we should follow in order to contributing this change? Thanks. --Aram ----- https://github.com/aramcodz
