sdaberdaku commented on PR #17862: URL: https://github.com/apache/iceberg/pull/17862#issuecomment-5457363944
@yangshangqing95 thanks for the review. I agree the derivation was guessing at what `table.io()` does rather than mirroring it, so I'll narrow this to the second option you suggested: - Drop `applyCatalogAssumeRole`. `hadoopConfCatalogOverrides` keeps its current contract. - The action builds its Hadoop configuration from that helper instead of the session, and the procedure passes the catalog it was invoked on. - Add a procedure-level test in `TestRemoveOrphanFilesProcedure` that fails if the wiring is reverted. - Document that the listing honours `spark.sql.catalog.<name>.hadoop.fs.s3a.*`, so a catalog that assumes a role can get the same role on the S3A side, per catalog. That limits the change to "the orphan file walk sees the catalog's own Hadoop overrides", which is what the issue is really about, and keeps the backports small. If there's interest in deriving the role later, I think it should read `table.io().properties()`, gate on the client factory, and map the full property set; I can open a follow-up issue for that. One open question on the `catalogName(...)` thread before I push. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
