?????? Kafka Consumer??????????????

Even Mon, 01 Jun 2020 00:19:12 -0700

??????????pom??????????????????provided??????????????????????????????????jar??????????????????????????????????????????????????????????????





------------------&nbsp;????????&nbsp;------------------
??????:&nbsp;"tison"<[email protected]&gt;;
????????:&nbsp;2020??5??29??(??????) ????7:47
??????:&nbsp;"user-zh"<[email protected]&gt;;

????:&nbsp;Re: Kafka Consumer??????????????



????????????????????????????????????????

https://ci.apache.org/projects/flink/flink-docs-stable/monitoring/debugging_classloading.html

Best,
tison.


tison <[email protected]&gt; ??2020??5??29?????? ????7:46??????

&gt; ?????????????????????????????????????????? child-first ???????????????? 
perjob 1.10 ??????????????????????????
&gt;
&gt; ????????????
&gt; 
https://ci.apache.org/projects/flink/flink-docs-stable/ops/deployment/yarn_setup.html#user-jars--classpath
&gt;
&gt; Best,
&gt; tison.
&gt;
&gt;
&gt; Even <[email protected]&gt; ??2020??5??29?????? ????6:48??????
&gt;
&gt;&gt; ????????????????????????????????????
&gt;&gt;
&gt;&gt;
&gt;&gt;
&gt;&gt;
&gt;&gt; ------------------&amp;nbsp;????????&amp;nbsp;------------------
&gt;&gt; ??????:&amp;nbsp;"zz zhang"<[email protected]&amp;gt;;
&gt;&gt; ????????:&amp;nbsp;2020??5??29??(??????) ????5:16
&gt;&gt; ??????:&amp;nbsp;"user-zh"<[email protected]&amp;gt;;"????"<
&gt;&gt; [email protected]&amp;gt;;
&gt;&gt;
&gt;&gt; ????:&amp;nbsp;Re: Kafka Consumer??????????????
&gt;&gt;
&gt;&gt;
&gt;&gt;
&gt;&gt; ??????maven-shade??????????????
&gt;&gt;
&gt;&gt; 
????class????org.apache.kafka.common.serialization.ByteArrayDeserializer????????org.
&gt;&gt; apache.flink.kafka.shaded.org
&gt;&gt; 
.apache.kafka.common.serialization.ByteArrayDeserializer??????????????????Deserializer????shade????????????????org.apache.flink.kafka.shaded.org.apache.kafka.common.serialization.Deserializer??????????????
&gt;&gt;
&gt;&gt; ???? <[email protected]&amp;gt; ??2020??5??29?????? 
????4:33??????
&gt;&gt; &amp;gt;
&gt;&gt; &amp;gt; ??????????????????jar??????
&gt;&gt; &amp;gt;
&gt;&gt; &amp;gt;
&gt;&gt; &amp;gt; 
------------------------------------------------------------------
&gt;&gt; &amp;gt; ????????Even <[email protected]&amp;gt;
&gt;&gt; &amp;gt; ??????????2020??5??29??(??????) 16:17
&gt;&gt; &amp;gt; ????????user-zh <[email protected]&amp;gt;
&gt;&gt; &amp;gt; ?? ????Kafka Consumer??????????????
&gt;&gt; &amp;gt;
&gt;&gt; &amp;gt; Hi??
&gt;&gt; &amp;gt; ????????Kafka Consumer????????????
&gt;&gt; &amp;gt; ????kafka&amp;amp;nbsp;consumer&amp;amp;nbsp;job ??????Flink 
session
&gt;&gt; cluster????????????????????????????Flink per-job cluster 
????kafka??????????????????????????
&gt;&gt; &amp;gt; 
????flink??????1.10,kafka??????kafka_2.12-2.1.0????????consumer??????val data =
&gt;&gt; env.addSource(new FlinkKafkaConsumer[String](topic, new
&gt;&gt; SimpleStringSchema(), properties))
&gt;&gt; &amp;gt; 2020-05-27&amp;amp;nbsp;17:05:22
&gt;&gt; &amp;gt; org.apache.kafka.common.KafkaException: Failed to construct 
kafka
&gt;&gt; consumer
&gt;&gt; &amp;gt;&amp;nbsp; at
&gt;&gt; 
org.apache.kafka.clients.consumer.KafkaConsumer.<init&amp;amp;gt;(KafkaConsumer.java:811)
&gt;&gt; &amp;gt;&amp;nbsp; at
&gt;&gt; 
org.apache.kafka.clients.consumer.KafkaConsumer.<init&amp;amp;gt;(KafkaConsumer.java:659)
&gt;&gt; &amp;gt;&amp;nbsp; at
&gt;&gt; 
org.apache.kafka.clients.consumer.KafkaConsumer.<init&amp;amp;gt;(KafkaConsumer.java:639)
&gt;&gt; &amp;gt;&amp;nbsp; at
&gt;&gt; 
org.apache.flink.streaming.connectors.kafka.internal.KafkaPartitionDiscoverer.initializeConnections(KafkaPartitionDiscoverer.java:58)
&gt;&gt; &amp;gt;&amp;nbsp; at
&gt;&gt; 
org.apache.flink.streaming.connectors.kafka.internals.AbstractPartitionDiscoverer.open(AbstractPartitionDiscoverer.java:94)
&gt;&gt; &amp;gt;&amp;nbsp; at
&gt;&gt; 
org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumerBase.open(FlinkKafkaConsumerBase.java:505)
&gt;&gt; &amp;gt;&amp;nbsp; at
&gt;&gt; 
org.apache.flink.api.common.functions.util.FunctionUtils.openFunction(FunctionUtils.java:36)
&gt;&gt; &amp;gt;&amp;nbsp; at
&gt;&gt; 
org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.open(AbstractUdfStreamOperator.java:102)
&gt;&gt; &amp;gt;&amp;nbsp; at
&gt;&gt; 
org.apache.flink.streaming.runtime.tasks.StreamTask.initializeStateAndOpen(StreamTask.java:1007)
&gt;&gt; &amp;gt;&amp;nbsp; at
&gt;&gt; 
org.apache.flink.streaming.runtime.tasks.StreamTask.lambda$beforeInvoke$0(StreamTask.java:454)
&gt;&gt; &amp;gt;&amp;nbsp; at
&gt;&gt; 
org.apache.flink.streaming.runtime.tasks.StreamTaskActionExecutor$SynchronizedStreamTaskActionExecutor.runThrowing(StreamTaskActionExecutor.java:94)
&gt;&gt; &amp;gt;&amp;nbsp; at
&gt;&gt; 
org.apache.flink.streaming.runtime.tasks.StreamTask.beforeInvoke(StreamTask.java:449)
&gt;&gt; &amp;gt;&amp;nbsp; at
&gt;&gt; 
org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:461)
&gt;&gt; &amp;gt;&amp;nbsp; at
&gt;&gt; org.apache.flink.runtime.taskmanager.Task.doRun(Task.java:707)
&gt;&gt; &amp;gt;&amp;nbsp; at 
org.apache.flink.runtime.taskmanager.Task.run(Task.java:532)
&gt;&gt; &amp;gt;&amp;nbsp; at java.lang.Thread.run(Thread.java:748)
&gt;&gt; &amp;gt; Caused by: org.apache.kafka.common.KafkaException:
&gt;&gt; 
org.apache.flink.kafka.shaded.org.apache.kafka.common.serialization.ByteArrayDeserializer
&gt;&gt; is not an instance of 
org.apache.kafka.common.serialization.Deserializer
&gt;&gt; &amp;gt;&amp;nbsp; at
&gt;&gt; 
org.apache.kafka.common.config.AbstractConfig.getConfiguredInstance(AbstractConfig.java:304)
&gt;&gt; &amp;gt;&amp;nbsp; at
&gt;&gt; 
org.apache.kafka.clients.consumer.KafkaConsumer.<init&amp;amp;gt;(KafkaConsumer.java:705)
&gt;&gt; &amp;gt;&amp;nbsp; ... 15 more
&gt;&gt;
&gt;&gt;
&gt;&gt;
&gt;&gt; --
&gt;&gt; Best,
&gt;&gt; zz zhang
&gt;
&gt;

?????? Kafka Consumer??????????????

回复