jackye1995 commented on issue #3546: URL: https://github.com/apache/iceberg/issues/3546#issuecomment-968465627
I see, it looks like you want to use Hadoop catalog layout instead of just read and write data files using the s3:// scheme. These are 2 different concepts in Iceberg. The FIleIO concept is specifically used for only reading and writing files. The HadoopCatalog is a catalog implementation that relies on Hadoop file system, so if you want to have that also support s3:// scheme, just register the scheme through your Hadoop config, something like this: https://github.com/hanborq/hadoop/blob/master/src/core/core-default.xml#L174-L178 And you can choose whatever file system implementation you like for s3://, the recommended one is https://github.com/apache/hadoop/blob/trunk/hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
