surawut-jirasaktavee opened a new issue, #7667: URL: https://github.com/apache/datafusion-comet/issues/7667
### What is the problem the feature request solves? A common way to give Spark on EKS access to a Glue catalog is the Iceberg docs' assume-role setup: the pod's own identity (Pod Identity / IRSA) holds only `sts:AssumeRole`, and catalog and table-data permissions (Glue, Lake Formation, bucket policies) are granted to a separate role: ``` spark.sql.catalog.glue=org.apache.iceberg.spark.SparkCatalog spark.sql.catalog.glue.catalog-impl=org.apache.iceberg.aws.glue.GlueCatalog spark.sql.catalog.glue.io-impl=org.apache.iceberg.aws.s3.S3FileIO spark.sql.catalog.glue.client.factory=org.apache.iceberg.aws.AssumeRoleAwsClientFactory spark.sql.catalog.glue.client.assume-role.arn=arn:aws:iam::<account>:role/<data-role> spark.sql.catalog.glue.client.assume-role.region=<region> ``` With `spark.comet.write.iceberg.enabled=true` on 1.2.0, every write to these tables falls back: ``` IcebergWrite [COMET: unsupported S3 FileIO settings: client.assume-role.region, client.factory] ``` This fallback is intentional (#6139, PR #6441). It replaced silently ignoring the settings and writing with a different identity. But the native writer already supports the assume-role keys, because `client.assume-role.arn`, `client.assume-role.external-id` and `client.assume-role.session-name` are on `SupportedS3FileIOProperties`. The fallback comes only from how iceberg-java asks for that behavior: - `client.factory=org.apache.iceberg.aws.AssumeRoleAwsClientFactory` is how iceberg-java turns on the assume-role path. Without it, iceberg-java ignores `client.assume-role.arn`, so users can't drop the key to pass the gate. The catalog (Glue) and the commit would then run as the pod identity. - `client.assume-role.region` is the region `AssumeRoleAwsClientFactory` uses for its STS and S3 clients. The pinned iceberg-rust (`de0a6ac`) has no such key. It reads the region from `client.region` / `s3.region`. So these setups can't use native writes at all, unless they grant data permissions to the pod identity directly. That is what the assume-role setup is designed to avoid. ### Describe the potential solution Keep the gate an allowlist, and accept this one well-defined factory configuration: 1. Treat `client.factory` as supported only when its value is exactly `org.apache.iceberg.aws.AssumeRoleAwsClientFactory` and `client.assume-role.arn` is set. Any other factory class still falls back, because a custom factory can build an arbitrary client. 2. Accept `client.assume-role.region` and pass it on as the native writer's region when neither `client.region` nor `s3.region` is set. If either is set and the values differ, fall back, since iceberg-java's S3 client would use the assume-role region. 3. Keep falling back on the assume-role settings the native writer can't honor yet, such as `client.assume-role.tags.*`. Possibly also `client.assume-role.timeout-sec`, unless it's judged not to change the writing identity. Session duration only affects refresh, which the native side handles itself. This could be a sub-item of #6957 (admitting S3 settings one at a time). ### Additional context - Related: #6139 / PR #6441 (fail closed on unsupported S3 settings), #6957 (umbrella for admitting S3 settings), PR #7604 (explicit S3A credential provider lists for native writes, which covers the `HadoopFileIO` / S3A assumed-role case but not `S3FileIO` + `client.factory`). - Seen on Comet 1.2.0 + Spark on EKS with a Glue catalog and `S3FileIO`. ### Willingness to contribute I can contribute a fix for this bug independently -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
