arjun180 commented on issue #1263: URL: https://github.com/apache/camel-kafka-connector/issues/1263#issuecomment-957885299
Thank you @oscerd and @scholzj for your comments. Based on those - I managed to create a AWS Kafka s3 source connector with the STS jar. I used this link to setup : https://strimzi.io/blog/2020/05/07/camel-kafka-connectors/. The jar files I currently have : ``` LICENSE.txt camel-main-3.10.0.jar netty-buffer-4.1.54.Final.jar NOTICE.txt camel-management-api-3.10.0.jar netty-codec-4.1.54.Final.jar README.adoc camel-seda-3.10.0.jar netty-codec-http-4.1.54.Final.jar annotations-13.0.jar camel-support-3.10.0.jar netty-codec-http2-4.1.54.Final.jar annotations-2.16.62.jar camel-util-3.10.0.jar netty-common-4.1.54.Final.jar apache-client-2.16.62.jar commons-codec-1.15.jar netty-handler-4.1.54.Final.jar apicurio-registry-common-1.3.2.Final.jar commons-compress-1.20.jar netty-nio-client-2.16.62.jar apicurio-registry-rest-client-1.3.2.Final.jar commons-logging-1.2.jar netty-reactive-streams-2.0.5.jar apicurio-registry-utils-converter-1.3.2.Final.jar connect-json-2.6.0.jar netty-reactive-streams-http-2.0.5.jar apicurio-registry-utils-serde-1.3.2.Final.jar converter-jackson-2.9.0.jar netty-resolver-4.1.54.Final.jar arns-2.16.62.jar eventstream-1.0.1.jar netty-transport-4.1.54.Final.jar auth-2.16.62.jar http-client-spi-2.16.62.jar netty-transport-native-epoll-4.1.54.Final-linux-x86_64.jar avro-1.10.2.jar httpclient-4.5.13.jar netty-transport-native-unix-common-4.1.54.Final.jar aws-core-2.16.62.jar httpcore-4.4.14.jar okhttp-4.8.1.jar aws-query-protocol-2.16.62.jar jackson-annotations-2.12.3.jar okio-2.7.0.jar aws-xml-protocol-2.16.62.jar jackson-core-2.12.3.jar profiles-2.16.62.jar camel-api-3.10.0.jar jackson-databind-2.12.3.jar protobuf-java-3.13.0.jar camel-aws2-s3-3.10.0.jar jackson-dataformat-avro-2.11.3.jar protocol-core-2.16.62.jar camel-aws2-s3-kafka-connector-0.10.1.jar jackson-datatype-jdk8-2.11.3.jar reactive-streams-1.0.3.jar camel-base-3.10.0.jar javax.annotation-api-1.3.2.jar regions-2.16.62.jar camel-base-engine-3.10.0.jar jboss-jaxrs-api_2.1_spec-2.0.1.Final.jar retrofit-2.9.0.jar camel-core-engine-3.10.0.jar jctools-core-3.3.0.jar s3-2.16.62.jar camel-core-languages-3.10.0.jar kafka-clients-2.8.0.jar sdk-core-2.16.62.jar camel-core-model-3.10.0.jar kotlin-reflect-1.3.20.jar slf4j-api-1.7.30.jar camel-core-processor-3.10.0.jar kotlin-stdlib-1.3.20.jar snappy-java-1.1.8.1.jar camel-core-reifier-3.10.0.jar kotlin-stdlib-common-1.3.20.jar sts-2.17.51.jar camel-direct-3.10.0.jar lz4-java-1.7.1.jar utils-2.16.62.jar camel-jackson-3.10.0.jar medeia-validator-core-1.1.1.jar zstd-jni-1.4.9-1.jar camel-kafka-3.10.0.jar medeia-validator-jackson-1.1.1.jar camel-kafka-connector-0.10.1.jar metrics-spi-2.16.62.jar ``` This includes the sts-2.17.51 jar as well. My Kafka connect configuration looks like this : ``` apiVersion: kafka.strimzi.io/v1beta2 kind: KafkaConnect metadata: namespace : my-kafka name: my-dev-kafka-connect-cluster annotations: strimzi.io/use-connector-resources: "true" spec: image: my-kafka-connect:latest template: serviceAccount: metadata: annotations: eks.amazonaws.com/role-arn:arn:aws:iam::xxxxxxxxxxxx:role/my-kafka-sa replicas: 1 bootstrapServers: my-kafka-dev.com:9093 tls: trustedCertificates: - secretName: my-kafka-secret certificate: my_server_com.crt authentication: type: oauth tokenEndpointUri: <token> clientId: <client_id> clientSecret: key: secret secretName: my-oauth-secret scope: kafka config: group.id: my-connect-cluster-cluster offset.storage.topic: my-connect-cluster-offsets config.storage.topic: my-connect-cluster-configs status.storage.topic: my-connect-cluster-status key.converter: org.apache.kafka.connect.json.JsonConverter value.converter: org.apache.kafka.connect.json.JsonConverter key.converter.schemas.enable: true value.converter.schemas.enable: true config.storage.replication.factor: 1 offset.storage.replication.factor: 1 status.storage.replication.factor: 1 ``` Once I started up the Kafka connector - I got the same error : ``` Caused by: software.amazon.awssdk.core.exception.SdkClientException: Unable to load credentials from any of the providers in the chain AwsCredentialsProviderChain(credentialsProviders=[SystemPropertyCredentialsProvider(), EnvironmentVariableCredentialsProvider(), WebIdentityTokenCredentialsProvider(), ProfileCredentialsProvider(), ContainerCredentialsProvider(), InstanceProfileCredentialsProvider()]) : [SystemPropertyCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., EnvironmentVariableCredentialsProvider(): Unable to load credentials from system settings. Access key must be specified either via environment variable (AWS_ACCESS_KEY_ID) or system property (aws.accessKeyId)., WebIdentityTokenCredentialsProvider(): Not authorized to perform sts:AssumeRoleWithWebIdentity (Service: Sts, Status Code: 403, Request ID: 739494d8-e385-4d3f-88b5-583aedf f9252, Extended Request ID: null), ProfileCredentialsProvider(): Profile file contained no credentials for profile 'default': ProfileFile(profiles=[]), ContainerCredentialsProvider(): Cannot fetch credentials from container - neither AWS_CONTAINER_CREDENTIALS_FULL_URI or AWS_CONTAINER_CREDENTIALS_RELATIVE_URI environment variables are set., InstanceProfileCredentialsProvider(): Unable to load credentials from service endpoint.] ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
