[
https://issues.apache.org/jira/browse/FLINK-4197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15373256#comment-15373256
]
ASF GitHub Bot commented on FLINK-4197:
---------------------------------------
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/2227#discussion_r70478621
--- Diff: docs/apis/streaming/connectors/kinesis.md ---
@@ -299,3 +299,43 @@ Otherwise, the returned stream name is used.
Other optional configuration keys can be found in `KinesisConfigConstants`.
+### Using Non-AWS Kinesis Endpoints for Testing
+
+It is sometimes desirable to have Flink operate as a consumer or producer
against a non-AWS Kinesis endpoint such as kinesalite; this is especially
useful when performing functional testing of a Flink application. The AWS
endpoint that would normally be inferred by the AWS region set in the Flink
configuration must be overriden via a configuration property.
+
+To override the AWS endpoint, set the
`KinesisConfigConstants.CONFIG_AWS_ENDPOINT` property in the Flink
configuration, in addition to the `KinesisConfigConstants.CONFIG_AWS_REGION`
required by Flink. Although the region is required, it will not be used to
determine the AWS endpoint URL.
+
+The following example shows how one might supply the
`KinesisConfigConstants.CONFIG_AWS_ENDPOINT` configuration property:
+
+<div class="codetabs" markdown="1">
+<div data-lang="java" markdown="1">
+{% highlight java %}
+Properties kinesisProducerConfig = new Properties();
+kinesisProducerConfig.put(KinesisConfigConstants.CONFIG_AWS_REGION,
"us-east-1");
+kinesisProducerConfig.put(
+
KinesisConfigConstants.CONFIG_AWS_CREDENTIALS_PROVIDER_BASIC_ACCESSKEYID,
+ "aws_access_key_id_here");
+kinesisProducerConfig.put(
+ KinesisConfigConstants.CONFIG_AWS_CREDENTIALS_PROVIDER_BASIC_SECRETKEY,
+ "aws_secret_key_here");
+kinesisProducerConfig.put(
+ KinesisConfigConstants.CONFIG_AWS_ENDPOINT,
+ "http://localhost:4567";);
+{% endhighlight %}
+</div>
+<div data-lang="scala" markdown="1">
+{% highlight scala %}
+val kinesisProducerConfig = new Properties();
+kinesisProducerConfig.put(KinesisConfigConstants.CONFIG_AWS_REGION,
"us-east-1");
+kinesisProducerConfig.put(
+
KinesisConfigConstants.CONFIG_AWS_CREDENTIALS_PROVIDER_BASIC_ACCESSKEYID,
+ "aws_access_key_id_here");
+kinesisProducerConfig.put(
+ KinesisConfigConstants.CONFIG_AWS_CREDENTIALS_PROVIDER_BASIC_SECRETKEY,
+ "aws_secret_key_here");
+kinesisProducerConfig.put(
+ KinesisConfigConstants.CONFIG_AWS_ENDPOINT,
+ "http://localhost:4567";);
--- End diff --
Mh, I'm undecided on this one. Its a lot of "code" for showing how to use
one config property, but since its on the bottom of the page, regular user's
are probably not distracted by this.
I'd say lets keep it for now. If the kinesis documentation keeps growing,
we can reconsider.
> Allow Kinesis Endpoint to be Overridden via Config
> --------------------------------------------------
>
> Key: FLINK-4197
> URL: https://issues.apache.org/jira/browse/FLINK-4197
> Project: Flink
> Issue Type: Improvement
> Components: Kinesis Connector
> Affects Versions: 1.0.3
> Reporter: Scott Kidder
> Priority: Minor
> Labels: easyfix
> Fix For: 1.0.4
>
> Original Estimate: 1h
> Remaining Estimate: 1h
>
> I perform local testing of my application stack with Flink configured as a
> consumer on a Kinesis stream provided by Kinesalite, an implementation of
> Kinesis built on LevelDB. This requires me to override the AWS endpoint to
> refer to my local Kinesalite server rather than reference the real AWS
> endpoint. I'd like to add a configuration property to the Kinesis streaming
> connector that allows the AWS endpoint to be specified explicitly.
> This should be a fairly small change and provide a lot of flexibility to
> people looking to integrate Flink with Kinesis in a non-production setup.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
