[
https://issues.apache.org/jira/browse/FLINK-4197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15373043#comment-15373043
]
ASF GitHub Bot commented on FLINK-4197:
---------------------------------------
Github user tzulitai commented on a diff in the pull request:
https://github.com/apache/flink/pull/2227#discussion_r70459536
--- Diff: docs/apis/streaming/connectors/kinesis.md ---
@@ -299,3 +299,41 @@ Otherwise, the returned stream name is used.
Other optional configuration keys can be found in `KinesisConfigConstants`.
+### Using Non-AWS Kinesis Endpoints for Testing
+
+It is sometimes desirable to have Flink operate as a consumer or producer
against a non-AWS Kinesis endpoint such as kinesalite; this is especially
useful when performing functional testing of a Flink application. The AWS
endpoint that would normally be inferred by the AWS region set in the Flink
configuration must be overriden via a configuration property.
+
+To override the AWS endpoint, set the
`KinesisConfigConstants.CONFIG_AWS_ENDPOINT` property in the Flink
configuration, in addition to the `KinesisConfigConstants.CONFIG_AWS_REGION`
required by Flink. Although the region is required, it will not be used to
determine the AWS endpoint URL.
+
+The following example shows how one might supply the
`KinesisConfigConstants.CONFIG_AWS_ENDPOINT` configuration property:
+
+<div class="codetabs" markdown="1">
+<div data-lang="java" markdown="1">
+{% highlight java %}
+Properties kinesisProducerConfig = new Properties();
+kinesisProducerConfig.put(KinesisConfigConstants.CONFIG_AWS_REGION,
"us-east-1");
+kinesisProducerConfig.put(
+
KinesisConfigConstants.CONFIG_AWS_CREDENTIALS_PROVIDER_BASIC_ACCESSKEYID,
+ "aws_access_key_id_here");
+kinesisProducerConfig.put(
+ KinesisConfigConstants.CONFIG_AWS_CREDENTIALS_PROVIDER_BASIC_SECRETKEY,
+ "aws_secret_key_here");
+kinesisProducerConfig.put(
+ KinesisConfigConstants.CONFIG_AWS_ENDPOINT,
+ "http://localhost:4567";);
+{% endhighlight %}
+</div>
+<div data-lang="scala" markdown="1">
+{% highlight scala %}
+val kinesisProducerConfig = new Properties();
+kinesisProducerConfig.put(KinesisConfigConstants.CONFIG_AWS_REGION,
"us-east-1");
+kinesisProducerConfig.put(
+
KinesisConfigConstants.CONFIG_AWS_CREDENTIALS_PROVIDER_BASIC_ACCESSKEYID,
+ "aws_access_key_id_here");
+kinesisProducerConfig.put(
+ KinesisConfigConstants.CONFIG_AWS_CREDENTIALS_PROVIDER_BASIC_SECRETKEY,
+ "aws_secret_key_here");
+kinesisProducerConfig.put(KinesisConfigConstants.CONFIG_AWS_ENDPOINT,
"http://localhost:4567";);
--- End diff --
nit: This config is written as a single line in the Scala example, but
spans 3 lines in the Java example. Would be good to keep them align :)
> Allow Kinesis Endpoint to be Overridden via Config
> --------------------------------------------------
>
> Key: FLINK-4197
> URL: https://issues.apache.org/jira/browse/FLINK-4197
> Project: Flink
> Issue Type: Improvement
> Components: Kinesis Connector
> Affects Versions: 1.0.3
> Reporter: Scott Kidder
> Priority: Minor
> Labels: easyfix
> Fix For: 1.0.4
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> I perform local testing of my application stack with Flink configured as a
> consumer on a Kinesis stream provided by Kinesalite, an implementation of
> Kinesis built on LevelDB. This requires me to override the AWS endpoint to
> refer to my local Kinesalite server rather than reference the real AWS
> endpoint. I'd like to add a configuration property to the Kinesis streaming
> connector that allows the AWS endpoint to be specified explicitly.
> This should be a fairly small change and provide a lot of flexibility to
> people looking to integrate Flink with Kinesis in a non-production setup.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
