rajarshisarkar commented on issue #4804: URL: https://github.com/apache/iceberg/issues/4804#issuecomment-1131501318
Seems like there is an issue to load the credentials. Is the AWS SDK v2 library in the classpath? Possibly the `--jars` path is breaking at one place before the AWS SDK v2 library due to `'s3-data-bucket'`: ``` "sparkSubmitParameters": "--jars .....,s3://'s3-data-bucket'/spark-sql-kinesis_2.12-1.2.1_spark-3.0-SNAPSHOT.jar,https://repo1.maven.org/maven2/software/amazon/awssdk/bundle/2.15.40/bundle-2.15.40.jar,https://repo1.maven.org/maven2/software/amazon/awssdk/url-connection-client/2.15.40/url-connection-client-2.15.40.jar"; ``` If it is in the classpath, can you please explicitly try providing the access key specified either via environment variable (`AWS_ACCESS_KEY_ID`) or system property (`aws.accessKeyId`) for testing purpose. Also, I couldn't find the Dynamo DB lock manager Spark configuration: `--conf spark.sql.catalog.my_catalog.lock-impl=org.apache.iceberg.aws.glue.DynamoLockManager`. Refer: [DynamoDB for Commit Locking](https://iceberg.apache.org/docs/latest/aws/#dynamodb-for-commit-locking) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
