Cole Murray created SPARK-24218:
-----------------------------------
Summary: Allow Configuration of DynamoDbEndpointUrl in
KinesisReceiver
Key: SPARK-24218
URL: https://issues.apache.org/jira/browse/SPARK-24218
Project: Spark
Issue Type: Improvement
Components: DStreams
Affects Versions: 2.3.0
Reporter: Cole Murray
KinesisReceiver does not allow users to set DynamoDbEndpointUrl. When testing
Kinesis locally, users are unable to utilize local checkpoints through a local
Dynamodb.
An update to the constructor would allow other Apis, specifically,
KinesisInputDStream.builder to add support for local DynamoDb endpointUrls.
Resources:
[https://github.com/apache/spark/blob/master/external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisReceiver.scala|KinesisReceiver]
[https://github.com/apache/spark/blob/master/external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisReceiver.scala#L162|Line
To Add EndpointUrl to KinesisClientLibConfiguration]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
