kbendick commented on code in PR #5972:
URL: https://github.com/apache/iceberg/pull/5972#discussion_r996383639
##########
core/src/main/java/org/apache/iceberg/hadoop/HadoopFileIO.java:
##########
@@ -78,7 +78,7 @@ public OutputFile newOutputFile(String path) {
@Override
public void deleteFile(String path) {
- Path toDelete = new Path(path);
+ Path toDelete = new Path(path.replaceAll("s3", "s3a"));
Review Comment:
I think in some places (including maybe S3FileIO), equivalency is
configurable already (or at least accounted for). Might be good to check how
that works.
Some people might want more s3 prefixes to be considered, such as `s3n`.
And even cross-cloud or HDFS to S3, So I'm more of a fan of configurability for
the general hadop case.
But I've been OOO for a bit, so might not be fully up to date. I would be
interested to see what @flyrain thinks about this change to HadoopFileIO, as I
believe he introduced the configuration for allowing different prefixes to be
considered equivalent.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
