BohanZhang0222 opened a new issue #14239:
URL: https://github.com/apache/pulsar/issues/14239


   #### Expected behavior
   
   按照官方示例:https://pulsar.apache.org/docs/en/io-kafka-source/
   通过 sources localrun 的方式,试图完成kafka数据导入到pulsar。
   
   #### Actual behavior
   
   实际上,我的操作并没有成功。
   出现报错:
   java.lang.RuntimeException: User class must be in class path
        at 
org.apache.pulsar.common.util.Reflections.createInstance(Reflections.java:113) 
~[org.apache.pulsar-pulsar-common-2.9.1.jar:2.9.1]
        at 
org.apache.pulsar.functions.instance.JavaInstanceRunnable.setupInput(JavaInstanceRunnable.java:735)
 ~[org.apache.pulsar-pulsar-functions-instance-2.9.1.jar:2.9.1]
        at 
org.apache.pulsar.functions.instance.JavaInstanceRunnable.setup(JavaInstanceRunnable.java:232)
 ~[org.apache.pulsar-pulsar-functions-instance-2.9.1.jar:2.9.1]
        at 
org.apache.pulsar.functions.instance.JavaInstanceRunnable.run(JavaInstanceRunnable.java:260)
 ~[org.apache.pulsar-pulsar-functions-instance-2.9.1.jar:2.9.1]
        at java.lang.Thread.run(Thread.java:748) [?:1.8.0_162]
   Caused by: java.lang.NoClassDefFoundError: 
org/apache/kafka/clients/consumer/Consumer
        at java.lang.Class.forName0(Native Method) ~[?:1.8.0_162]
        at java.lang.Class.forName(Class.java:348) ~[?:1.8.0_162]
        at 
org.apache.pulsar.common.util.Reflections.createInstance(Reflections.java:111) 
~[org.apache.pulsar-pulsar-common-2.9.1.jar:2.9.1]
        ... 4 more
   Caused by: java.lang.ClassNotFoundException: 
org.apache.kafka.clients.consumer.Consumer
        at java.net.URLClassLoader.findClass(URLClassLoader.java:381) 
~[?:1.8.0_162]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[?:1.8.0_162]
        at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ~[?:1.8.0_162]
        at java.lang.Class.forName0(Native Method) ~[?:1.8.0_162]
        at java.lang.Class.forName(Class.java:348) ~[?:1.8.0_162]
        at 
org.apache.pulsar.common.util.Reflections.createInstance(Reflections.java:111) 
~[org.apache.pulsar-pulsar-common-2.9.1.jar:2.9.1]
        ... 4 more
   
   
![image](https://user-images.githubusercontent.com/18319597/153570785-4490701a-4041-46f8-9fd7-b8cb73d1db3e.png)
   
   #### Steps to reproduce
   
   
   
   **我创建了目录connectors,并且放入了文件pulsar-io-kafka-2.9.1.nar。**
   
![image](https://user-images.githubusercontent.com/18319597/153571338-6d398679-c2ed-49ad-804c-4eaaafb2766b.png)
   
   
   **我的kafka-source配置文件:**
   {
       "bootstrapServers": 
"bigdata-kafka-01-ontest.chj.cloud:6667,bigdata-kafka-02-ontest.chj.cloud:6667,bigdata-kafka-03-ontest.chj.cloud:6667",
       "groupId": "test-pulsar-io",
       "topic": "test-zbh",
       "sessionTimeoutMs": "10000",
       "autoCommitEnabled": false
   }
   
![image](https://user-images.githubusercontent.com/18319597/153571124-8e5cf58c-6557-4bbd-ad45-7409a16a31dc.png)
   
   **我执行的命令:**
   bin/pulsar-admin sources localrun --archive 
/chj/pulsarCluster/connectors/pulsar-io-kafka-2.9.1.nar --classname 
org.apache.pulsar.io.kafka.KafkaBytesSource --tenant public --namespace default 
--name kafka --destination-topic-name  test-zbh --source-config-file 
./sourceConfig/kafka-source.json --parallelism 1
   
   执行就会报出User class must be in class path,
   由于Caused by: java.lang.ClassNotFoundException: 
org.apache.kafka.clients.consumer.Consumer
   
   
**所以我认为是pulsar的lib中,可能不包含我使用的kafka版本的client,于是我在pulsar/lib中上传了一个kafka-client的jar包**
   
![image](https://user-images.githubusercontent.com/18319597/153571785-3cdf17ed-5672-4204-976f-310fd5c34504.png)
   
   **重新执行sources localrun命令后,我得到了不同的报错结果。**
   
![image](https://user-images.githubusercontent.com/18319597/153571912-c8005b72-30c1-44c5-b409-b126c9dc5226.png)
   
   
   #### System configuration
   
   **我的来源kafka:**
   版本:1.1.1
   3个节点的集群部署
   
   **我的Pulsar集群:**
   版本:2.9.1
   3个节点的集群部署(zk、broker、bookie)。
   
   另外,kafka和pulsar使用了不同的zk。
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


Reply via email to