CTTY commented on code in PR #6882:
URL: https://github.com/apache/hudi/pull/6882#discussion_r989551863
##########
hudi-common/src/main/java/org/apache/hudi/metadata/HoodieMetadataPayload.java:
##########
@@ -473,21 +479,26 @@ public Option<HoodieMetadataColumnStats>
getColumnStatMetadata() {
/**
* Returns the files added as part of this record.
*/
- public FileStatus[] getFileStatuses(Configuration hadoopConf, Path
partitionPath) throws IOException {
- FileSystem fs = partitionPath.getFileSystem(hadoopConf);
- return getFileStatuses(fs, partitionPath);
- }
+ public FileStatus[] getFileStatuses(FileSystem fs, String partitionPath,
+ HoodieStorageStrategy storageStrategy) {
+ URI uri = fs.getUri();
+ String scheme = uri.getScheme();
+ String authority = uri.getAuthority();
- /**
- * Returns the files added as part of this record.
- */
- public FileStatus[] getFileStatuses(FileSystem fs, Path partitionPath) {
- long blockSize = fs.getDefaultBlockSize(partitionPath);
return filterFileInfoEntries(false)
.map(e -> {
+ String fileName = e.getKey();
+ String fileId = FSUtils.getFileId(fileName);
+
+ // Convert logical path to physical path
+ Path physicalPartitionPath = new Path(scheme, authority,
+ storageStrategy.storageLocation(partitionPath, fileId));
+
Review Comment:
reading side path conversion. Only code that goes through metadata payload
would be able to reach physical path with storage strategy
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
