[
https://issues.apache.org/jira/browse/FLINK-8267?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16326639#comment-16326639
]
ASF GitHub Bot commented on FLINK-8267:
---------------------------------------
GitHub user bowenli86 opened a pull request:
https://github.com/apache/flink/pull/5301
[FLINK-8267] [Kinesis Connector] update Kinesis Producer example for
setting Region key
## What is the purpose of the change
Update doc to guide users to use KPL's native keys to set regions.
This originates from a bug that we forgot to set region keys explicitly for
kinesis connector, which has been fixed. According to the previous discussion
between @tzulitai and I, we want to remove AWSConfigConstants in 2.0 because it
basically copies/translates config keys of KPL/KCL, which doesn't add much
value.
Guide users to use KPL's native keys to set regions can be the first step
that facilitates the migration.
## Brief change log
- updated doc
## Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
## Does this pull request potentially affect one of the following parts:
## Documentation
- Does this pull request introduce a new feature? (no)
cc @tzulitai
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/bowenli86/flink FLINK-8267
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/flink/pull/5301.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #5301
----
commit b4034f67e727fef68740221e3b31cd131c905df1
Author: Bowen Li <bowenli86@...>
Date: 2018-01-02T19:21:28Z
update local branch
commit 11e9c255cd51304c5281d55226fbb6fe8360d8a2
Author: Bowen Li <bowenli86@...>
Date: 2018-01-04T01:35:11Z
remove sh
commit e322a5416b0f4f89c366b98bb3571fbf6b7d460a
Author: Bowen Li <bowenli86@...>
Date: 2017-12-17T06:18:55Z
update doc
commit 1b447633df4a8bfe7c4c19e7ae91aab6157756d7
Author: Bowen Li <bowenli86@...>
Date: 2018-01-15T23:48:38Z
format code snippet
commit 4ede4b5a89d4bfcda8dcc845ab1da42177d22358
Author: Bowen Li <bowenli86@...>
Date: 2018-01-15T23:49:37Z
remove ';' from scala code
----
> update Kinesis Producer example for setting Region key
> ------------------------------------------------------
>
> Key: FLINK-8267
> URL: https://issues.apache.org/jira/browse/FLINK-8267
> Project: Flink
> Issue Type: Bug
> Components: Kinesis Connector
> Reporter: Dyana Rose
> Assignee: Bowen Li
> Priority: Minor
>
> https://ci.apache.org/projects/flink/flink-docs-release-1.4/dev/connectors/kinesis.html#kinesis-producer
> In the example code for the kinesis producer the region key is set like:
> {code:java}
> producerConfig.put(AWSConfigConstants.AWS_REGION, "us-east-1");
> {code}
> However, the AWS Kinesis Producer Library requires that the region key be
> Region
> (https://github.com/awslabs/amazon-kinesis-producer/blob/94789ff4bb2f5dfa05aafe2d8437d3889293f264/java/amazon-kinesis-producer-sample/default_config.properties#L269)
> so the setting at this point should be:
> {code:java}
> producerConfig.put(AWSConfigConstants.AWS_REGION, "us-east-1");
> producerConfig.put("Region", "us-east-1");
> {code}
> When you run the Kinesis Producer you can see the effect of not setting the
> Region key by a log line
> {noformat}
> org.apache.flink.streaming.connectors.kinesis.FlinkKinesisProducer - Started
> Kinesis producer instance for region ''
> {noformat}
> The KPL also then assumes it's running on EC2 and attempts to determine it's
> own region, which fails.
> {noformat}
> (EC2MetadataClient)Http request to Ec2MetadataService failed.
> [error] [main.cc:266] Could not configure the region. It was not given in the
> config and we were unable to retrieve it from EC2 metadata
> {noformat}
> At the least I'd say the documentation should mention the difference between
> these two keys and when you are required to also set the Region key.
> On the other hand, is this even the intended behaviour of this connector? Was
> it intended that the AWSConfigConstants.AWS_REGION key also set the region of
> the of the kinesis stream? The documentation for the example states
> {noformat}
> The example demonstrates producing a single Kinesis stream in the AWS region
> “us-east-1”.
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
