bharatviswa504 commented on a change in pull request #1569:
URL: https://github.com/apache/ozone/pull/1569#discussion_r524753212
##########
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java
##########
@@ -2209,14 +2205,18 @@ private void listStatusFindKeyInTableCache(
bucketName);
}
+ List<OmKeyInfo> keyInfoList = new ArrayList<>(fileStatusList.size());
for (OzoneFileStatus fileStatus : fileStatusList) {
- if (args.getRefreshPipeline()) {
- refreshPipeline(fileStatus.getKeyInfo());
- }
- if (args.getSortDatanodes()) {
+ keyInfoList.add(fileStatus.getKeyInfo());
+ }
+ refreshPipeline(keyInfoList);
Review comment:
Just a question listStatus is limited by batch size.
But we are making a single Rpc Call to SCM for all the containers we got
from this list, is this fine here? (My question is if it has a large number of
containerIDs due to larger key sizes, will it have an issue in Rpc Response
size.
##########
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java
##########
@@ -2209,14 +2205,18 @@ private void listStatusFindKeyInTableCache(
bucketName);
}
+ List<OmKeyInfo> keyInfoList = new ArrayList<>(fileStatusList.size());
for (OzoneFileStatus fileStatus : fileStatusList) {
- if (args.getRefreshPipeline()) {
- refreshPipeline(fileStatus.getKeyInfo());
- }
- if (args.getSortDatanodes()) {
+ keyInfoList.add(fileStatus.getKeyInfo());
+ }
+ refreshPipeline(keyInfoList);
+
+ if (args.getSortDatanodes()) {
+ for (OzoneFileStatus fileStatus : fileStatusList) {
sortDatanodeInPipeline(fileStatus.getKeyInfo(), clientAddress);
Review comment:
This call still makes number of entries in listStatus call to put nodes
in sorted order in pipeline when ozone.network.topology.aware.read is set.
(Default false, so this will not be a problem, but we have still the problem of
making number of entries in listStatus calls to SCM.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
