yanmxa opened a new issue, #1594: URL: https://github.com/apache/camel-kafka-connector/issues/1594
The pods created by the `KafkaConnect` can't start, when I followed the [document](https://camel.apache.org/camel-kafka-connector/next/user-guide/getting-started/try-it-out-on-kubernetes.html) to create a `KafkaConnect` to backup/sink the Kafka data to aws s3. I'm wondering if there other external steps that need to be added to the guideline. **Troubleshooting** - create the `kafkaconnect` ```bash cat <<EOF | oc apply -f - apiVersion: kafka.strimzi.io/v1beta2 kind: KafkaConnect metadata: name: kafka-connect-cluster annotations: # enable connect operator strimzi.io/use-connector-resources: "true" spec: image: $REGISTRY_URL/$REGISTRY_USR/camel-kafka-connector-plugin:latest version: 2.3.0 replicas: 1 bootstrapServers: kafka-kafka-bootstrap.multicluster-global-hub.svc:9092 externalConfiguration: env: - name: AWS_ACCESS_KEY_ID valueFrom: secretKeyRef: name: aws-creds key: awsAccessKey - name: AWS_SECRET_ACCESS_KEY valueFrom: secretKeyRef: name: aws-creds key: awsSecretAccessKey resources: requests: cpu: 250m memory: 512Mi limits: cpu: 500m memory: 1Gi jvmOptions: gcLoggingEnabled: false config: group.id: kafka-connect-cluster key.converter: org.apache.kafka.connect.storage.StringConverter value.converter: org.apache.kafka.connect.storage.StringConverter offset.storage.topic: kafka-connect-cluster-offsets config.storage.topic: kafka-connect-cluster-configs status.storage.topic: kafka-connect-cluster-status config.storage.replication.factor: 1 offset.storage.replication.factor: 1 status.storage.replication.factor: 1 EOF kafkaconnect.kafka.strimzi.io/kafka-connect-cluster created ``` - get pod ```bash $ oc get pods NAME READY STATUS RESTARTS AGE kafka-connect-cluster-connect-58cd465cc7-qsgng 0/1 Running 2 (29s ago) 55s ``` - describe pod ```bash $ oc describe pod kafka-connect-cluster-connect-58cd465cc7-qsgng Name: kafka-connect-cluster-connect-58cd465cc7-qsgng Namespace: multicluster-global-hub Priority: 0 Node: ip-10-0-132-6.ec2.internal/10.0.132.6 Start Time: Mon, 29 Jan 2024 12:59:03 +0000 Labels: app.kubernetes.io/instance=kafka-connect-cluster app.kubernetes.io/managed-by=strimzi-cluster-operator app.kubernetes.io/name=kafka-connect app.kubernetes.io/part-of=strimzi-kafka-connect-cluster pod-template-hash=58cd465cc7 strimzi.io/cluster=kafka-connect-cluster strimzi.io/component-type=kafka-connect strimzi.io/kind=KafkaConnect strimzi.io/name=kafka-connect-cluster-connect Annotations: k8s.v1.cni.cncf.io/network-status: [{ "name": "openshift-sdn", "interface": "eth0", "ips": [ "10.128.1.50" ], "default": true, "dns": {} }] k8s.v1.cni.cncf.io/networks-status: [{ "name": "openshift-sdn", "interface": "eth0", "ips": [ "10.128.1.50" ], "default": true, "dns": {} }] openshift.io/scc: restricted-v2 seccomp.security.alpha.kubernetes.io/pod: runtime/default strimzi.io/auth-hash: 0 strimzi.io/logging-appenders-hash: 06ee78c4 Status: Running IP: 10.128.1.50 IPs: IP: 10.128.1.50 Controlled By: ReplicaSet/kafka-connect-cluster-connect-58cd465cc7 Containers: kafka-connect-cluster-connect: Container ID: cri-o://e5cc2aadcfe9877521bda830254be72fcfaca434d467714458b66b4560ca13c8 Image: quay.io/myan/camel-kafka-connector-plugin:latest Image ID: quay.io/myan/camel-kafka-connector-plugin@sha256:bfb98f6f879b216f9d65634a1d2da25b405fd8f5ad17e7c53bb8e3b6a1236237 Port: 8083/TCP Host Port: 0/TCP Args: /opt/kafka/kafka_connect_run.sh State: Waiting Reason: CrashLoopBackOff Last State: Terminated Reason: Error Exit Code: 1 Started: Mon, 29 Jan 2024 12:59:48 +0000 Finished: Mon, 29 Jan 2024 12:59:59 +0000 Ready: False Restart Count: 2 Liveness: http-get http://:rest-api/ delay=60s timeout=5s period=10s #success=1 #failure=3 Readiness: http-get http://:rest-api/ delay=60s timeout=5s period=10s #success=1 #failure=3 Environment: KAFKA_CONNECT_CONFIGURATION: offset.storage.topic=connect-cluster-offsets value.converter=org.apache.kafka.connect.json.JsonConverter config.storage.topic=connect-cluster-configs key.converter=org.apache.kafka.connect.json.JsonConverter group.id=connect-cluster status.storage.topic=connect-cluster-status KAFKA_CONNECT_METRICS_ENABLED: false KAFKA_CONNECT_BOOTSTRAP_SERVERS: kafka-kafka-bootstrap.multicluster-global-hub.svc:9092 STRIMZI_KAFKA_GC_LOG_ENABLED: false KAFKA_HEAP_OPTS: -Xms128M AWS_ACCESS_KEY_ID: <set to the key 'awsAccessKey' in secret 'aws-creds'> Optional: false AWS_SECRET_ACCESS_KEY: <set to the key 'awsSecretAccessKey' in secret 'aws-creds'> Optional: false Mounts: /opt/kafka/custom-config/ from kafka-metrics-and-logging (rw) /tmp from strimzi-tmp (rw) /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-srz7b (ro) Conditions: Type Status Initialized True Ready False ContainersReady False PodScheduled True Volumes: strimzi-tmp: Type: EmptyDir (a temporary directory that shares a pod's lifetime) Medium: Memory SizeLimit: 5Mi kafka-metrics-and-logging: Type: ConfigMap (a volume populated by a ConfigMap) Name: kafka-connect-cluster-connect-config Optional: false # Please edit the object below. Lines beginning with a '#' will be ignored, kube-api-access-srz7b: Type: Projected (a volume that contains injected data from multiple sources) TokenExpirationSeconds: 3607 ConfigMapName: kube-root-ca.crt ConfigMapOptional: <nil> DownwardAPI: true ConfigMapName: openshift-service-ca.crt ConfigMapOptional: <nil> QoS Class: BestEffort Node-Selectors: <none> Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s node.kubernetes.io/unreachable:NoExecute op=Exists for 300s Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled 72s default-scheduler Successfully assigned multicluster-global-hub/kafka-connect-cluster-connect-58cd465cc7-qsgng to ip-10-0-132-6.ec2.internal Normal AddedInterface 70s multus Add eth0 [10.128.1.50/23] from openshift-sdn Normal Pulled 70s kubelet Successfully pulled image "quay.io/myan/camel-kafka-connector-plugin:latest" in 171.0038ms (171.01734ms including waiting) Normal Pulled 57s kubelet Successfully pulled image "quay.io/myan/camel-kafka-connector-plugin:latest" in 160.465058ms (160.478918ms including waiting) Normal Pulling 26s (x3 over 70s) kubelet Pulling image "quay.io/myan/camel-kafka-connector-plugin:latest" Normal Created 26s (x3 over 69s) kubelet Created container kafka-connect-cluster-connect Normal Started 26s (x3 over 69s) kubelet Started container kafka-connect-cluster-connect Normal Pulled 26s kubelet Successfully pulled image "quay.io/myan/camel-kafka-connector-plugin:latest" in 171.659524ms (171.671444ms including waiting) Warning BackOff 11s (x4 over 45s) kubelet Back-off restarting failed container ``` - pod log ```bash $ oc logs -f kafka-connect-cluster-connect-58cd465cc7-qsgng Preparing truststore Preparing truststore is complete Starting Kafka Connect with configuration: # Bootstrap servers bootstrap.servers=kafka-kafka-bootstrap.multicluster-global-hub.svc:9092 # REST Listeners ... 2024-01-29 13:47:33,292 INFO Kafka version: 2.6.0 (org.apache.kafka.common.utils.AppInfoParser) [DistributedHerder-connect-1-1] 2024-01-29 13:47:33,292 INFO Kafka commitId: 62abe01bee039651 (org.apache.kafka.common.utils.AppInfoParser) [DistributedHerder-connect-1-1] 2024-01-29 13:47:33,293 INFO Kafka startTimeMs: 1706536053292 (org.apache.kafka.common.utils.AppInfoParser) [DistributedHerder-connect-1-1] 2024-01-29 13:47:33,297 INFO [Consumer clientId=consumer-kafka-connect-cluster-1, groupId=kafka-connect-cluster] Cluster ID: IbXwdCfQRzqSPgS22-GhAg (org.apache.kafka.clients.Metadata) [DistributedHerder-connect-1-1] 2024-01-29 13:47:33,299 ERROR [Worker clientId=connect-1, groupId=kafka-connect-cluster] Uncaught exception in herder work thread, exiting: (org.apache.kafka.connect.runtime.distributed.DistributedHerder) [DistributedHerder-connect-1-1] org.apache.kafka.common.errors.TopicAuthorizationException: Not authorized to access topics: [kafka-connect-cluster-offsets] 2024-01-29 13:47:33,393 INFO Stopped http_8083@15d42ccb{HTTP/1.1,[http/1.1]}{0.0.0.0:8083} (org.eclipse.jetty.server.AbstractConnector) [Thread-3] 2024-01-29 13:47:33,403 INFO DefaultSessionIdManager workerName=node0 (org.eclipse.jetty.server.session) [main] 2024-01-29 13:47:33,403 INFO No SessionScavenger set, using defaults (org.eclipse.jetty.server.session) [main] 2024-01-29 13:47:33,404 INFO node0 Scavenging every 600000ms (org.eclipse.jetty.server.session) [main] 2024-01-29 13:47:35,090 INFO Started o.e.j.s.ServletContextHandler@22bf9122{/,null,AVAILABLE} (org.eclipse.jetty.server.handler.ContextHandler) [main] 2024-01-29 13:47:35,090 INFO REST resources initialized; server is started and ready to handle requests (org.apache.kafka.connect.runtime.rest.RestServer) [main] 2024-01-29 13:47:35,090 INFO Kafka Connect started (org.apache.kafka.connect.runtime.Connect) [main] 2024-01-29 13:47:35,090 INFO Kafka Connect stopping (org.apache.kafka.connect.runtime.Connect) [connect-shutdown-hook] 2024-01-29 13:47:35,090 INFO Stopping REST server (org.apache.kafka.connect.runtime.rest.RestServer) [connect-shutdown-hook] 2024-01-29 13:47:35,091 INFO REST server stopped (org.apache.kafka.connect.runtime.rest.RestServer) [connect-shutdown-hook] 2024-01-29 13:47:35,091 INFO [Worker clientId=connect-1, groupId=kafka-connect-cluster] Herder stopping (org.apache.kafka.connect.runtime.distributed.DistributedHerder) [connect-shutdown-hook] 2024-01-29 13:47:40,091 INFO [Worker clientId=connect-1, groupId=kafka-connect-cluster] Herder stopped (org.apache.kafka.connect.runtime.distributed.DistributedHerder) [connect-shutdown-hook] 2024-01-29 13:47:40,091 INFO Kafka Connect stopped (org.apache.kafka.connect.runtime.Connect) [connect-shutdown-hook] ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
