[
https://issues.apache.org/jira/browse/SPARK-6514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14495605#comment-14495605
]
Chris Fregly commented on SPARK-6514:
-------------------------------------
hey pawel!
i think keeping it regionName is fine. i just reviewed the KCL code and docs
again - and realized that they also use this regionName for CloudWatch.
and by exposing the implementation, i meant that calling something "Dynamo
Region" would be awkward if AWS ever changes the implementation to be something
other than Dynamo. that's the level of implementation that i was referring to.
also, if we ever move off of KCL - or no longer need to set this region for
whatever reason - it may bite us later. just something to think about.
lastly, i think it's OK not to extract the region from the stream URL -
especially if we make regionName a required field on the API. if it's
optional, however, i would use the one in the stream location. this may be
overly complicated. you're call.
> For Kinesis Streaming, use the same region for DynamoDB (KCL checkpoints) as
> the Kinesis stream itself
> --------------------------------------------------------------------------------------------------------
>
> Key: SPARK-6514
> URL: https://issues.apache.org/jira/browse/SPARK-6514
> Project: Spark
> Issue Type: Improvement
> Components: Streaming
> Affects Versions: 1.3.0
> Reporter: Chris Fregly
>
> context: i started the original Kinesis impl with KCL 1.0 (not supported),
> then finished on KCL 1.1 (supported) without realizing that it's supported.
> also, we should upgrade to the latest Kinesis Client Library (KCL) which is
> currently v1.2 right now, i believe.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
