jennifer-xiong25 opened a new issue, #18934: URL: https://github.com/apache/iceberg/issues/18934
### Apache Iceberg version 1.12.0 (latest release) ### Query engine Flink ### Please describe the bug 🐞 **What happens.** We use a REST catalog that vends storage credentials (BigLake on GCS), and the job's own service account has no direct bucket access. In that setup, a Flink batch query on a metadata table fails, for example `SELECT * FROM t$files`. The JobManager plans the scan fine. Every TaskManager then fails reading the manifests with `403 storage.objects.get`, and the error names the job's own service account. A normal data scan of the same table works. **Why.** The FLIP-27 source sends each split to the TaskManagers as JSON. For metadata-table tasks, that JSON includes the FileIO, written by `FileIOParser`. `FileIOParser` writes only `io.properties()` ([L47](https://github.com/apache/iceberg/blob/55e54026ddc44c26265609427a83618d584a1eef/core/src/main/java/org/apache/iceberg/io/FileIOParser.java#L47)) and rebuilds the FileIO without credentials ([L79](https://github.com/apache/iceberg/blob/55e54026ddc44c26265609427a83618d584a1eef/core/src/main/java/org/apache/iceberg/io/FileIOParser.java#L79)). `GCSFileIO` keeps vended credentials in a separate field, not in `properties()` ([L85](https://github.com/apache/iceberg/blob/55e54026ddc44c26265609427a83618d584a1eef/gcp/src/main/java/org/apache/iceberg/gcp/gcs/GCSFileIO.java#L85)). So on the TaskManager the rebuilt FileIO has no credentials and falls back to the environment's identity. Data scans aren't affected, because their reader's FileIO travels by Java serialization, which keeps the credentials. `$entries` and `$all_manifests` use the same parser, so they are probably affected too. `S3FileIO` stores credentials the same way ([L114](https://github.com/apache/iceberg/blob/55e54026ddc44c26265609427a83618d584a1eef/aws/src/main/java/org/apache/iceberg/aws/s3/S3FileIO.java#L114)), but we've only tested GCS. Related: #10735 added this JSON path, and #10926 fixed a similar gap in the same parser. **Repro (unit test, no cloud needed).** The test builds a `$files` task whose `GCSFileIO` holds one credential, then round-trips it through `ScanTaskParser`. The credentials come back empty on 1.11.0 and 1.12.0. The same task round-tripped through Java serialization keeps them. <details><summary>Test</summary> ```java package org.apache.iceberg; import static org.junit.jupiter.api.Assertions.assertEquals; import java.io.File; import java.lang.reflect.Field; import java.util.List; import java.util.Map; import org.apache.iceberg.expressions.Expressions; import org.apache.iceberg.gcp.gcs.GCSFileIO; import org.apache.iceberg.io.StorageCredential; import org.apache.iceberg.types.Types; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.io.TempDir; public class TestFilesTaskCredentials { @Test public void splitJsonKeepsVendedCredentials(@TempDir File dir) throws Exception { PartitionSpec spec = PartitionSpec.unpartitioned(); ManifestWriter<DataFile> writer = ManifestFiles.write(2, spec, Files.localOutput(new File(dir, "m.avro")), 1L); writer.add(DataFiles.builder(spec).withPath("gs://bucket/t/data/a.parquet") .withFormat(FileFormat.PARQUET).withFileSizeInBytes(10).withRecordCount(1).build()); writer.close(); StorageCredential cred = StorageCredential.create("gs://bucket/", Map.of("gcs.oauth2.token", "vended-token", "gcs.oauth2.token-expires-at", "4102444800000")); GCSFileIO io = new GCSFileIO(); io.initialize(Map.of("gcs.project-id", "p")); io.setCredentials(List.of(cred)); Schema data = new Schema(Types.NestedField.required(1, "x", Types.LongType.get())); Schema projection = new Schema(DataFile.getType(Types.StructType.of()).fields()); FileScanTask task = new BaseFilesTable.ManifestReadTask( data, io, Map.of(0, spec), writer.toManifestFile(), projection, Expressions.alwaysTrue()); FileScanTask after = ScanTaskParser.fromJson(ScanTaskParser.toJson(task), true); Field f = BaseFilesTable.ManifestReadTask.class.getDeclaredField("io"); f.setAccessible(true); assertEquals(List.of(cred), ((GCSFileIO) f.get(after)).credentials()); // fails: [] } } ``` </details> **Possible fix.** When the FileIO implements `SupportsStorageCredentials`, have `FileIOParser` write `credentials()` to the JSON and pass them back to the 4-argument `CatalogUtil.loadFileIO` when reading. **Workaround.** `SET 'table.exec.iceberg.use-flip27-source' = 'false';` The legacy source Java-serializes its splits, so the credentials survive. That fixed it for us on the same cluster and table. The legacy source is deprecated, though. ### Willingness to contribute - [ ] I can contribute a fix for this bug independently - [x] I would be willing to contribute a fix for this bug with guidance from the Iceberg community - [ ] I cannot contribute a fix for this bug at this time -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
