Jack Tuck edited comment on FLINK-13111 at 8/14/19 9:56 AM:

Also, just FYI. Looks like current e2e tests in Flink are working around the 
issue with KPL. 

was (Author: jacktuck):
Also, just FYI. Looks like current e2e tests in Flink are working around the 
issue with KPL and the consumer. 

> FlinkKinesisConsumer fails with endpoint and region used together
> -----------------------------------------------------------------
>                 Key: FLINK-13111
>                 URL: https://issues.apache.org/jira/browse/FLINK-13111
>             Project: Flink
>          Issue Type: Bug
>          Components: Connectors / Kinesis
>            Reporter: Jack Tuck
>            Assignee: Yu Li
>            Priority: Major
> So far I have followed the instructions documented for Flink's kinesis 
> connector to use a local Kinesis. (Using Flink 1.8 and Kinesis connector 1.8)
> [https://ci.apache.org/projects/flink/flink-docs-stable/dev/connectors/kinesis.html#using-non-aws-kinesis-endpoints-for-testing]
> {code:java}
> Properties producerConfig = new Properties();
> producerConfig.put(AWSConfigConstants.AWS_REGION, "us-east-1");
> producerConfig.put(AWSConfigConstants.AWS_ACCESS_KEY_ID, "aws_access_key_id");
> producerConfig.put(AWSConfigConstants.AWS_SECRET_ACCESS_KEY, 
> "aws_secret_access_key");
> producerConfig.put(AWSConfigConstants.AWS_ENDPOINT, 
> "http://localhost:4567";);{code}
> With a Flink producer, these instructions work with a local kinesis (I use 
> Kinesalite).
> However, with a Flink consumer, I get an exception that `aws.region` and 
> `aws.endpoint` are not *both* allowed.
> {noformat}
> org.apache.flink.client.program.ProgramInvocationException: The main method 
> caused an error: For FlinkKinesisConsumer either AWS region ('aws.region') or 
> AWS endpoint ('aws.endpoint') must be set in the config.{noformat}
> Is this a bug in the connector? I found the PR which fixed this but maybe for 
> only the producer [https://github.com/apache/flink/pull/6045] .
> I found a [workaround on Flink's mailing 
> list]([http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Can-t-get-the-FlinkKinesisProducer-to-work-against-Kinesalite-for-tests-td23438.html]),
>  but their issue is with the producer rather than the consumer but perhaps 
> they got that the wrong way around, it is after all weird how there are two 
> codepaths for consumer/producer.

This message was sent by Atlassian JIRA

Reply via email to