flyrain commented on issue #214: URL: https://github.com/apache/polaris/issues/214#issuecomment-2318190441
Thanks @anuragmantri for chiming in. It'd be ideal to use Iceberg `FileIO` in `removeOrphanFile`, so that we don't have to config Spark file system differently, which is a duplication to avoid. I will take a look at the Iceberg PR. We will still need a workaround at this moment though, as the Iceberg change and release will take a while. You can customize your iceberg lib of course, but not every user is able to do that. @dennishuo mentioned a workaround [here](https://apache-iceberg.slack.com/archives/C07HADW86HZ/p1724909442884259?thread_ts=1724680467.488829&cid=C07HADW86HZ). It doesn't work for me locally, but worth to try. cc @TheerachotTle ``` spark.hadoop.fs.s3.impl=org.apache.hadoop.fs.s3a.S3AFileSystem spark.hadoop.fs.AbstractFileSystem.s3.impl=org.apache.hadoop.fs.s3a.S3A ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
