[
https://issues.apache.org/jira/browse/FLINK-14896?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Danny Cranmer updated FLINK-14896:
----------------------------------
Affects Version/s: 1.16.0
> Kinesis connector doesn't shade jackson dependency
> --------------------------------------------------
>
> Key: FLINK-14896
> URL: https://issues.apache.org/jira/browse/FLINK-14896
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kinesis
> Affects Versions: 1.9.0, 1.16.0, 1.15.2
> Environment: AWS EMR 5.28.0
> Reporter: Michel Davit
> Assignee: Danny Cranmer
> Priority: Not a Priority
> Labels: pull-request-available
> Fix For: 1.17.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> flink-kinesis-connector depends on aws java sdk which is shaded to
> {{org.apache.flink.kinesis.shaded.com.amazonaws.}}
>
> {{However, the aws sdk has a transitive dependency to jackson wich is not
> shaded in the artifact.}}
>
> {{This creates problem when running flink on YARN: }}{{The aws sdk requires
> jackson-core v2.6 but hadoop pulls in 2.3. See
> [here|https://github.com/apache/flink/blob/e7c11ed672013512e5b159e7e892b27b1ef60a1b/flink-yarn/pom.xml#L133].}}
>
> {{If YARN uses the loads wrong jackson version from classpath. Jod fails
> with}}
> {code:java}
> 2019-11-20 17:23:11,563 ERROR
> org.apache.flink.runtime.webmonitor.handlers.JarRunHandler - Unhandled
> exception.org.apache.flink.client.program.ProgramInvocationException: The
> program caused an error: at
> org.apache.flink.client.program.OptimizerPlanEnvironment.getOptimizedPlan(OptimizerPlanEnvironment.java:93)
> at
> org.apache.flink.client.program.PackagedProgramUtils.createJobGraph(PackagedProgramUtils.java:80)
> at
> org.apache.flink.runtime.webmonitor.handlers.utils.JarHandlerUtils$JarHandlerContext.toJobGraph(JarHandlerUtils.java:126)
> at
> org.apache.flink.runtime.webmonitor.handlers.JarRunHandler.lambda$getJobGraphAsync$6(JarRunHandler.java:142)
> at
> java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1590)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)Caused by:
> java.lang.NoSuchMethodError:
> com.fasterxml.jackson.databind.ObjectMapper.enable([Lcom/fasterxml/jackson/core/JsonParser$Feature;)Lcom/fasterxml/jackson/databind/ObjectMapper;
> at
> com.amazonaws.partitions.PartitionsLoader.<clinit>(PartitionsLoader.java:54)
> at
> com.amazonaws.regions.RegionMetadataFactory.create(RegionMetadataFactory.java:30)
> at com.amazonaws.regions.RegionUtils.initialize(RegionUtils.java:65)
> at com.amazonaws.regions.RegionUtils.getRegionMetadata(RegionUtils.java:53)
> at com.amazonaws.regions.RegionUtils.getRegion(RegionUtils.java:107) at
> com.amazonaws.client.builder.AwsClientBuilder.getRegionObject(AwsClientBuilder.java:256)
> at
> com.amazonaws.client.builder.AwsClientBuilder.setRegion(AwsClientBuilder.java:460)
> at
> com.amazonaws.client.builder.AwsClientBuilder.configureMutableProperties(AwsClientBuilder.java:424)
> at
> com.amazonaws.client.builder.AwsAsyncClientBuilder.build(AwsAsyncClientBuilder.java:80)
> ...
> {code}
> The flink-kinesis-connector should do as other connectors: shade jackson or
> use the flink-shaded-jackson core dependency
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
