Re: Incompatible Apache Http lib in Flink kinesis connector

Thanks a lot of looking into this Bowen! On 21 June 2017 at 5:02:55 AM, Bowen Li (bowen...@offerupnow.com) wrote: Guys, This is the PR https://github.com/apache/flink/pull/4150 On Tue, Jun 20, 2017 at 1:37 PM, Bowen Li wrote: > Hi Ted and Gordon, > I found

Re: Incompatible Apache Http lib in Flink kinesis connector

Guys, This is the PR https://github.com/apache/flink/pull/4150 On Tue, Jun 20, 2017 at 1:37 PM, Bowen Li wrote: > Hi Ted and Gordon, > I found the root cause and a solution. Basically > https://ci.apache.org/projects/flink/flink-docs- >

Re: Incompatible Apache Http lib in Flink kinesis connector

Hi Ted and Gordon, I found the root cause and a solution. Basically https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/aws.html#flink-for-hadoop-27 is out of date. Adding httpcore-4.3.6.jar and httpclient-4.3.3.jar rather than httpcore-4.2.5.jar and httpclient-4.2.5.jar to /lib

Re: Incompatible Apache Http lib in Flink kinesis connector

>From >aws-sdk-java/aws-java-sdk-core/src/main/java/com/amazonaws/http/conn/SdkConnectionKeepAliveStrategy.java : import org.apache.http.impl.client.DefaultConnectionKeepAliveStrategy; I checked out 4.2.x branch of httpcomponents-client There is no INSTANCE in

Re: Incompatible Apache Http lib in Flink kinesis connector

Bowen: The picture didn't come thru. Can you pastebin the contents of /lib dir ? Cheers On Mon, Jun 19, 2017 at 11:22 PM, Bowen Li wrote: > Hi Gordon, > I double checked that I'm not using any of httpclient/httpcore > or aws-java-sdk-xxx jars in my application. >

Re: Incompatible Apache Http lib in Flink kinesis connector

Hi Bowen, Thanks for the info. I checked the 1.3.0 release jars, and they do not have unshaded httpcomponent dependencies, so that shouldn’t be the problem. Looking back into the stack trace you posted, the conflict seems to be a different problem. The conflict seems to be with clashes with

Re: Incompatible Apache Http lib in Flink kinesis connector

Hi, We’ve seen this issue before [1]. The usual reason is that the httpcomponent dependencies weren’t properly shaded in the flink-dist jar. Having them properly shaded should solve the issue. cc Bowen: Are you building Flink yourself? If yes, what Maven version are you using? If you’re using

Re: Incompatible Apache Http lib in Flink kinesis connector

I logged FLINK-6951, referencing this thread. We can continue discussion there. Thanks On Mon, Jun 19, 2017 at 9:06 PM, Bowen Li wrote: > Thanks, Ted! woo, this is unexpected. https://ci.apache. > org/projects/flink/flink-docs-release-1.3/setup/aws.html is out of

Re: Incompatible Apache Http lib in Flink kinesis connector

Thanks, Ted! woo, this is unexpected. https://ci.apache. org/projects/flink/flink-docs-release-1.3/setup/aws.html is out of date. I bet anyone using Kinesis with Flink will run into this issue. I can try to build Flink myself and resolve this problem. But talking about a feasible permanent

Re: Incompatible Apache Http lib in Flink kinesis connector

Here is the dependency in the flink-connector-kinesis module: [INFO] +- com.amazonaws:aws-java-sdk-kinesis:jar:1.10.71:compile [INFO] | \- com.amazonaws:aws-java-sdk-core:jar:1.10.71:compile [INFO] | +- org.apache.httpcomponents:httpclient:jar:4.3.6:compile [INFO] | +-

Incompatible Apache Http lib in Flink kinesis connector

Hi guys, I'm trying to enable Flink's checkpoint on our Flink app. I got the following Apache http jar compatibility error, and cannot figure out how to resolve it. Here's the stacktrace: ``` 2017-06-19 15:07:39,828 INFO org.apache.flink.runtime.taskmanager.Task