Github user mce commented on a diff in the pull request:
https://github.com/apache/spark/pull/5439#discussion_r28419555
--- Diff:
extras/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisReceiver.scala
---
@@ -82,15 +82,19 @@ private[kinesis] class KinesisReceiver(
var workerId: String = null
/*
- * This impl uses the DefaultAWSCredentialsProviderChain and searches
for credentials
- * in the following order of precedence:
+ * This impl uses the DefaultAWSCredentialsProviderChain unless it's
provided by constructor
+ * and searches for credentials in the following order of precedence:
* Environment Variables - AWS_ACCESS_KEY_ID and AWS_SECRET_KEY
* Java System Properties - aws.accessKeyId and aws.secretKey
* Credential profiles file at the default location (~/.aws/credentials)
shared by all
* AWS SDKs and the AWS CLI
* Instance profile credentials delivered through the Amazon EC2
metadata service
*/
- var credentialsProvider: AWSCredentialsProvider = null
+ var credentialsProvider: AWSCredentialsProvider = (credentials map { cr
=> new AWSCredentialsProvider {
--- End diff --
I just tested the new code by running the example applications and it seems
the serialization problem still happening. I'll move the initialization into
onStart method and run the tests again.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
