Yanam opened a new pull request #3641: URL: https://github.com/apache/iceberg/pull/3641
The purpose of this PR is to split the io() methods that operate on data files and meta files. The reason is that when I implemented the custom catalog, I found that the API can only customize the read/write logic of meta.json files, but had no control over the manifest files that also belong to metadata. Since the data file and manifest file are basically operated by the FileIO object obtained by [Table.io() ](https://github.com/apache/iceberg/blob/master/api/src/main/java/org/apache/iceberg/Table.java)or [TableOperations.io() ](https://github.com/apache/iceberg/blob/master/core/src/main/java/org/apache/iceberg/TableOperations.java)method, the read/write logic of the two files cannot be split. Separating the FileIO of the meta file may be more conducive to custom catalog. Of course there is still a lot of code involved to fully implement the splitting of data files and manifest files, so if this solution is feasible, there is still a lot of work to follow. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
