It seems that you have a typo when specifying the pipeline classpath. "file:///flink-jar/flink-connector-rabbitmq_2.12-1.14.4.jar" -> "file:///flink-jars/flink-connector-rabbitmq_2.12-1.14.4.jar"
If this is not the root cause, maybe you could have a try with downloading the connector jars to /opt/flink/usrlib. The usrlib will be loaded to the user classloader automatically without any configuration. BTW, I am not aware of any other bugs which will cause pipeline classpath not take effect except FLINK-21289[1]. [1]. https://issues.apache.org/jira/browse/FLINK-21289 Best, Yang Francis Conroy <francis.con...@switchdin.com> 于2022年4月7日周四 15:14写道： > Hi all, > thanks in advance for any tips. > > I've been trying to specify some additional classpaths in my kubernetes > yaml file when using the official flink operator and nothing seems to work. > > I know the technique for getting my job jar works fine since it's finding > the class ok, but I cannot get the RabbitMQ connector jar to load. > > apiVersion: flink.apache.org/v1alpha1 > kind: FlinkDeployment > metadata: > namespace: default > name: http-over-mqtt > spec: > image: flink:1.14.4-scala_2.12-java11 > flinkVersion: v1_14 > flinkConfiguration: > taskmanager.numberOfTaskSlots: "2" > pipeline.classpaths: > "file:///flink-jar/flink-connector-rabbitmq_2.12-1.14.4.jar" > serviceAccount: flink > jobManager: > replicas: 1 > resource: > memory: "1024m" > cpu: 1 > taskManager: > resource: > memory: "1024m" > cpu: 1 > podTemplate: > spec: > serviceAccount: flink > containers: > - name: flink-main-container > volumeMounts: > - mountPath: /flink-job > name: flink-jobs > - mountPath: /flink-jars > name: flink-jars > initContainers: > - name: grab-mqtt-over-http-jar > image: busybox > command: [ '/bin/sh', '-c', > 'cd /tmp/job; wget > https://jenkins/job/platform_flink/job/master/39/artifact/src-java/switchdin-topologies/target/switchdin-topologies-1.0-SNAPSHOT.jar > --no-check-certificate;', > 'cd /tmp/jar; wget > https://repo1.maven.org/maven2/org/apache/flink/flink-connector-rabbitmq_2.12/1.14.4/flink-connector-rabbitmq_2.12-1.14.4.jar' > ] > volumeMounts: > - name: flink-jobs > mountPath: /tmp/job > - name: flink-jars > mountPath: /tmp/jar > volumes: > - name: flink-jobs > emptyDir: { } > - name: flink-jars > emptyDir: { } > job: > jarURI: local:///flink-job/switchdin-topologies-1.0-SNAPSHOT.jar > entryClass: org.switchdin.HTTPOverMQTT > parallelism: 1 > upgradeMode: stateless > state: running > > Any ideas? I've looked at the ConfigMaps that result and they also look > fine. > apiVersion: v1 > data: > flink-conf.yaml: "blob.server.port: 6124\nkubernetes.jobmanager.replicas: > 1\njobmanager.rpc.address: > http-over-mqtt.default\nkubernetes.taskmanager.cpu: 1.0\n > kubernetes.service-account: > flink\nkubernetes.cluster-id: http-over-mqtt\n > $internal.application.program-args: > \nkubernetes.container.image: flink:1.14.4-scala_2.12-java11\n > parallelism.default: > 1\nkubernetes.namespace: default\ntaskmanager.numberOfTaskSlots: 2\n > kubernetes.rest-service.exposed.type: > ClusterIP\n$internal.application.main: org.switchdin.HTTPOverMQTT\n > taskmanager.memory.process.size: > 1024m\nkubernetes.internal.jobmanager.entrypoint.class: > org.apache.flink.kubernetes.entrypoint.KubernetesApplicationClusterEntrypoint > \nkubernetes.pod-template-file: > /tmp/podTemplate_11292791104169595925.yaml\n > kubernetes.pod-template-file.taskmanager: > /tmp/podTemplate_17362225267763549900.yaml\nexecution.target: > kubernetes-application\njobmanager.memory.process.size: > 1024m\njobmanager.rpc.port: 6123\ntaskmanager.rpc.port: 6122\n > internal.cluster.execution-mode: > NORMAL\nqueryable-state.proxy.ports: 6125\npipeline.jars: > local:///flink-job/switchdin-topologies-1.0-SNAPSHOT.jar\n > kubernetes.jobmanager.cpu: > 1.0\npipeline.classpath: > file:///flink-jars/flink-connector-rabbitmq_2.12-1.14.4.jar\n > kubernetes.pod-template-file.jobmanager: > /tmp/podTemplate_17029501154997462433.yaml\n" > > > > This email and any attachments are proprietary and confidential and are > intended solely for the use of the individual to whom it is addressed. Any > views or opinions expressed are solely those of the author and do not > necessarily reflect or represent those of SwitchDin Pty Ltd. If you have > received this email in error, please let us know immediately by reply email > and delete it from your system. You may not use, disseminate, distribute or > copy this message nor disclose its contents to anyone. > SwitchDin Pty Ltd (ABN 29 154893857) PO Box 1165, Newcastle NSW 2300 > Australia >
