etspaceman commented on a change in pull request #24801:
[SPARK-27950][DSTREAMS][Kinesis] dynamoDBEndpointUrl and cloudWatchMetricsLevel
for Kinesis
URL: https://github.com/apache/spark/pull/24801#discussion_r290724897
##########
File path:
external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala
##########
@@ -268,6 +273,30 @@ object KinesisInputDStream {
this
}
+ /**
+ * Sets the AWS DynamoDB endpoint URL. Defaults to
+ * "https://dynamodb.us-east-1.amazonaws.com"; if no custom value is
specified
Review comment:
The KCL will attempt to connect to the typical dynamo instance in the
configured region. The KinesisInputDStream's default for the region is
`us-east-1`, so naturally the default dynamodb endpoint would be
`dynamodb.us-east-1.amazonaws.com`.
It is not explicitly typed anywhere though. I could update this to be more
indicative of the value in the `KinesisReceiver` (an `Option`), and specify
that the default is `None` if that is preferable.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
