[
https://issues.apache.org/jira/browse/NIFI-786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15303464#comment-15303464
]
ASF GitHub Bot commented on NIFI-786:
-------------------------------------
Github user jvwing commented on the pull request:
https://github.com/apache/nifi/pull/239#issuecomment-222057686
Your suggestion about the plain old SDK Kinesis API is interesting, but I
guess I don't recommend it. Using the plain SDK would let NiFi handle the
multithreading, retries, batching, etc. It would be a win for the NiFi model.
But then we would also have to pick up at least some of the KCL's other
features, like balancing traffic across shards, or leave that as an excercise
for the user. It sounds complicated and hard, and I'm lazy. Also, the KCL is
recommended by Amazon, as you pointed out earlier, and not using it might be a
point of confusion as to why we didn't do it the "right" way, especially if a
new version of the KCL is released with features we hadn't thought of. So even
though the KCL is an awkward fit in NiFi, as long as it is an opt-in feature,
it seems like a good addition to NiFi's AWS interoperability story.
Part of why I would prefer to not change the base AWS processor classes is
to keep it opt-in. I'm not sure how the other AWS Get* processors would
benefit from the Kinesis-like threading model, they do not have applications or
threads to drive activity outside of NiFi's scheduling. But I can see lobbying
for a change to `AbstractProcessor` to remove `onTrigger`'s `final` modifier
for cases like these without divorcing the class hierarchies.
A recently merged [change in the 0.x
branch](https://github.com/apache/nifi/commit/de7ecd719a2e9907042628ea3a8283cfe2d4fbac)
for NIFI-786 has some shared credential property descriptors and validation
logic that you may be able to use, that will hopefully make it easier to
implement separate base classes. Let me know if I can help with that.
> Add other supporting options for configuring credentials for AWS processors
> ---------------------------------------------------------------------------
>
> Key: NIFI-786
> URL: https://issues.apache.org/jira/browse/NIFI-786
> Project: Apache NiFi
> Issue Type: Improvement
> Affects Versions: 0.3.0
> Reporter: Michael Kobit
> Assignee: James Wing
> Priority: Minor
> Fix For: 0.7.0
>
>
> I was looking at https://issues.apache.org/jira/browse/NIFI-770 and looked at
> how the AWS processors credentials are currently configured. As a NFM you
> have a few options with the properties right now:
> 1) set basic, static credentials
> 2) set a credentials properties filepath
> 3) set neither, use anonymous credentials
> I think it would be better if each AWS could rely on a ControllerService that
> returns `AWSCredentialsProvider` (instead of `AWSCredentials`) that gives
> all of the possible implementations that could be used, rather than relying
> on a static credentials. *Provider implementations can be refreshed and can
> also other more complicated implementations, but already have built in
> support for the Static and Properties file that are provided by NiFi today.
> My thinking is that the controller service would be something like
> public interface AwsCredentialsProviderService extends ControllerService {
> AWSCredentialsProvider getCredentialsProvider();
> }
> and you could have `StaticAwsCredentialsProviderService`,
> `PropertiesFileAwsCredentialsProviderService`, and
> `AnonymousAwsCredentialsProviderService` to provide the functionality that is
> supported right now. Additional credential providers could be added later, as
> there a bunch more AWS provided versions that I think could fit in well.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
