tiborkiss opened a new issue #2636: URL: https://github.com/apache/iceberg/issues/2636
We have assume role based AWS setup and I would like to try Iceberg with Flink 1.11 . Unfortunately I cannot figure out the setup in case of Flink. In the [documentation](https://iceberg.apache.org/aws) there is a section **Cross-Account and Cross-Region Access**. That in case of Spark gives an example how to pass ``` --conf spark.sql.catalog.my_catalog.client.factory=org.apache.iceberg.aws.AssumeRoleAwsClientFactory \ --conf spark.sql.catalog.my_catalog.client.assume-role.arn=arn:aws:iam::123456789:role/myRoleToAssume \ --conf spark.sql.catalog.my_catalog.client.assume-role.region=ap-northeast-1 ``` I cannot figure out how to apply that for Flink. Under flink there is a `conf/sql-client-defaults.yaml` which has a section ``` catalogs: - name: tiko-dev type: iceberg warehouse: s3://<mybucket>/<warehouse> catalog-impl: org.apache.iceberg.aws.glue.GlueCatalog io-impl: org.apache.iceberg.aws.s3.S3FileIO lock-impl: org.apache.iceberg.aws.glue.DynamoLockManager lock.table: myGlueLockTable ``` Unfortunately it cannot parse if I add ``` client.factory=org.apache.iceberg.aws.AssumeRoleAwsClientFactory client.assume-role.arn=arn:aws:iam::1212121121212:role/rolename client.assume-role.region=eu-central-1 ``` fails with ``` No default environment specified. Searching for '/home/tibor/flink/flink-1.11.3/conf/sql-client-defaults.yaml'...found. Reading default environment from: file:/home/tibor/flink/flink-1.11.3/conf/sql-client-defaults.yaml Exception in thread "main" org.apache.flink.table.client.SqlClientException: Could not parse environment file. Cause: while scanning a simple key in 'reader', line 85, column 5: client.factory=org.apache.iceber ... ^ could not find expected ':' in 'reader', line 86, column 5: client.assume-role.arn=arn:aws:i ... ^ at [Source: (URL); line: 84, column: 19] (through reference chain: org.apache.flink.table.client.config.Environment["catalogs"]->java.util.ArrayList[0]) at org.apache.flink.table.client.config.Environment.parse(Environment.java:215) at org.apache.flink.table.client.gateway.local.LocalExecutor.<init>(LocalExecutor.java:163) at org.apache.flink.table.client.SqlClient.start(SqlClient.java:94) at org.apache.flink.table.client.SqlClient.main(SqlClient.java:201) ``` Maybe someone knows how to configure AWS + Flink + Iceberg with AssumeRoleAwsClientFactory. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
