github-actions[bot] commented on code in PR #65034:
URL: https://github.com/apache/doris/pull/65034#discussion_r3543319813
##########
fe/fe-core/src/main/java/org/apache/doris/fs/SpiSwitchingFileSystem.java:
##########
@@ -114,69 +122,101 @@ public FileSystem forLocation(Location location) throws
IOException {
@Override
public boolean exists(Location location) throws IOException {
- return forLocation(location).exists(location);
+ ResolvedLocation resolvedLocation = resolveLocation(location);
+ return resolvedLocation.fileSystem.exists(resolvedLocation.location);
}
@Override
public void mkdirs(Location location) throws IOException {
- forLocation(location).mkdirs(location);
+ ResolvedLocation resolvedLocation = resolveLocation(location);
+ resolvedLocation.fileSystem.mkdirs(resolvedLocation.location);
}
@Override
public void delete(Location location, boolean recursive) throws
IOException {
- forLocation(location).delete(location, recursive);
+ ResolvedLocation resolvedLocation = resolveLocation(location);
+ resolvedLocation.fileSystem.delete(resolvedLocation.location,
recursive);
}
@Override
public void rename(Location src, Location dst) throws IOException {
- forLocation(src).rename(src, dst);
+ ResolvedLocation resolvedSrc = resolveLocation(src);
Review Comment:
One HMS commit path can still reach this method with `src` already
normalized to `s3://...` while `dst` is the raw OSS bucket-endpoint location.
`HiveTableSink` sends FILE_S3 writes to `locationPath.getNormalizedLocation()`,
but `prepareInsertExistingTable` compares that write path with the raw HMS
table location; for `oss://bucket.oss-cn-beijing.aliyuncs.com/t` versus
`s3://bucket/t`, `PathUtils.equalsIgnoreSchemeIfOneIsS3` compares authorities
`bucket.oss-cn-beijing.aliyuncs.com` and `bucket`, so it schedules
`fs.rename(s3://bucket/t/file,
oss://bucket.oss-cn-beijing.aliyuncs.com/t/file)`.
Here `resolveLocation(src)` then selects the S3 storage properties when S3
is configured, while `normalizeLocation(dst)` rewrites the destination to the
same `s3://bucket/t/file`. That can fail the append because the delegate sees
the destination as the source object itself, and staged rename paths would
still run with S3 credentials instead of the OSS properties. Please
normalize/compare the two operands before scheduling the rename, or carry the
storage properties selected from the original OSS location through the commit
rename path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
