RussellSpitzer commented on a change in pull request #2380:
URL: https://github.com/apache/iceberg/pull/2380#discussion_r603234562
##########
File path:
spark3/src/main/java/org/apache/iceberg/actions/Spark3SnapshotAction.java
##########
@@ -135,10 +140,23 @@ protected TableCatalog checkSourceCatalog(CatalogPlugin
catalog) {
@Override
public SnapshotAction withLocation(String location) {
- Preconditions.checkArgument(!sourceTableLocation().equals(location),
+ String qualifiedLocation = makeQualifiedLocation(location);
+
Preconditions.checkArgument(!sourceTableLocation().equals(qualifiedLocation),
"Cannot create snapshot where destination location is the same as the
source location." +
" This would cause a mixing of original table created and snapshot
created files.");
this.destTableLocation = location;
return this;
}
+
+ private String makeQualifiedLocation(String location) {
+ try {
+ Path path = new Path(location);
+ Configuration conf = spark().sessionState().newHadoopConf();
+ FileSystem fs = FileSystem.get(path.toUri(), conf);
+ return CatalogUtils.URIToString(fs.makeQualified(path).toUri());
+ } catch (IOException ioe) {
+ LOG.error("Failed to initialize hadoop filesystem.", ioe);
+ return location;
Review comment:
We should probably just fail, after all if the location string cannot be
resolved isn't it going to fail anyway?
This makes me think we should be attempting to make a qualified path before
comparing locations, so we can fail with cannot resolve uri before we have a
chance to compare.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
