dramaticlly commented on issue #7570: URL: https://github.com/apache/iceberg/issues/7570#issuecomment-1544919981
I think this looks like runtime failure where region cannot be loaded properly. Although I saw your set `%env AWS_REGION=us-east-1` but somehow it's not picked up by iceberg when trying to initialize S3 fileIO. I would try the following - read env in spark after `AWS_REGION` is set to make sure it's actually set - try add region as jvm argument as a workaround ``` spark.driver.extraJavaOptions=-Daws.region=us-east-1 spark.executor.extraJavaOptions=-Daws.region=us-east-1 ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
