[
https://issues.apache.org/jira/browse/FLINK-20617?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17250762#comment-17250762
]
Georger commented on FLINK-20617:
---------------------------------
it seems the
org.apache.kafka.common.serialization.Deserializer has loaded twice, here is
the detail log:
[Loaded org.apache.kafka.common.serialization.Deserializer from
file:/mnt/data1/yarn/nm/usercache/hdfs/appcache/application_1606119213964_1657/blobStore-a0f843a0-737f-491c-9d8e-a7ec76568e07/job_24c0a330e49ff08ff3355fc9efeaa7bf/blob_p-bf8f8652268b5550585fcc184b2925c5e613a255-932296ba280a1ae7a544c24b249e68cd]
....
[Loaded org.apache.kafka.common.serialization.Deserializer from
file:/mnt/data0/yarn/nm/usercache/hdfs/appcache/application_1606119213964_1657/filecache/14/flink-examples-1.0-SNAPSHOT201217_104230_459_r57.jar]
> Kafka Consumer Deserializer Exception on application mode
> ---------------------------------------------------------
>
> Key: FLINK-20617
> URL: https://issues.apache.org/jira/browse/FLINK-20617
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka
> Affects Versions: 1.11.2
> Environment: application mode
> flink 1.11.2 with hadoop 2.6.0-cdh5.15.0
> Reporter: Georger
> Priority: Critical
>
> Kafka source may has some issues on application mode
>
> when i run it with application mode on flink 1.11.2 it can't startup
> the detail Excetion is:
> org.apache.kafka.common.KafkaException: Failed to construct kafka consumer
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:789)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:643)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:623)
> at
> org.apache.flink.streaming.connectors.kafka.internal.KafkaPartitionDiscoverer.initializeConnections(KafkaPartitionDiscoverer.java:58)
> at
> org.apache.flink.streaming.connectors.kafka.internals.AbstractPartitionDiscoverer.open(AbstractPartitionDiscoverer.java:94)
> at
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.open(FlinkKafkaConsumerBase.java:550)
> at
> org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
> at
> org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain.initializeStateAndOpenOperators(OperatorChain.java:291)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$0(StreamTask.java:479)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:92)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:475)
> at
> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:528)
> at org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:721)
> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:546)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.kafka.common.KafkaException:
> org.apache.kafka.common.serialization.ByteArrayDeserializer is not an
> instance of org.apache.kafka.common.serialization.Deserializer
> at
> org.apache.kafka.common.config.AbstractConfig.getConfiguredInstance(AbstractConfig.java:263)
> at
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:688)
> ... 15 more
> The pom is:
> <dependency>
> <groupId>org.apache.flink</groupId>
> <artifactId>flink-connector-kafka_2.11</artifactId>
> <version>${flink.version}</version>
> <exclusions>
> <exclusion>
> <groupId>org.slf4j</groupId>
> <artifactId>slf4j-api</artifactId>
> </exclusion>
> <exclusion>
> <groupId>org.apache.kafka</groupId>
> <artifactId>kafka-clients</artifactId>
> </exclusion>
> </exclusions>
> </dependency>
> <dependency>
> <groupId>org.apache.kafka</groupId>
> <artifactId>kafka-clients</artifactId>
> <version>1.0.1</version>
> </dependency>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
