yihua commented on code in PR #13007:
URL: https://github.com/apache/hudi/pull/13007#discussion_r2017709403
##########
hudi-spark-datasource/hudi-spark/src/test/java/org/apache/hudi/client/functional/TestHoodieBackedMetadata.java:
##########
@@ -256,6 +259,7 @@ public void testMetadataTableBootstrap(HoodieTableType
tableType, boolean addRol
validateMetadata(testTable, true);
}
+ @Disabled("HUDI-9221")
Review Comment:
What's the reason of disabling these tests? The tests seem to be unrelated
to the changes in this PR?
##########
hudi-io/src/main/java/org/apache/hudi/storage/StorageSchemes.java:
##########
@@ -86,6 +88,11 @@ public enum StorageSchemes {
// Hopsworks File System
HOPSFS("hopsfs", false, true);
+ // list files may bring pressure to storage with centralized meta service
like HDFS.
+ // when we want to get only part of files under a directory rather than all
files, use getStatus may be more friendly than listStatus.
+ // here is a trade-off between rpc times and throughput of storage meta
service
+ private static final Set<String> LIST_STATUS_FRIENDLY_SCHEMES = new
HashSet<>(Arrays.asList(FILE.scheme, S3.scheme, S3A.scheme, GCS.scheme));
Review Comment:
Add it as part of the scheme definition
```
StorageSchemes(String scheme, Boolean isWriteTransactional, Boolean
supportAtomicCreation, Boolean isListStatusFriendly)
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
