avijayanhwx commented on a change in pull request #194: HDDS-2241. Optimize the
refresh pipeline logic used by KeyManagerImpl…
URL: https://github.com/apache/hadoop-ozone/pull/194#discussion_r347117599
##########
File path:
hadoop-ozone/ozone-manager/src/main/java/org/apache/hadoop/ozone/om/KeyManagerImpl.java
##########
@@ -688,6 +663,48 @@ public OmKeyInfo lookupKey(OmKeyArgs args, String
clientAddress)
}
}
+ /**
+ * Refresh pipeline info in OM by asking SCM.
+ * @param value OmKeyInfo
+ */
+ @VisibleForTesting
+ protected void refreshPipeline(OmKeyInfo value) {
+ // Refresh container pipeline info from SCM
+ // based on OmKeyArgs.refreshPipeline flag
+ // 1. Client send initial read request OmKeyArgs.refreshPipeline = false
+ // and uses the pipeline cached in OM to access datanode
+ // 2. If succeeded, done.
+ // 3. If failed due to pipeline does not exist or invalid pipeline state
+ // exception, client should retry lookupKey with
+ // OmKeyArgs.refreshPipeline = true
+ Map<Long, ContainerWithPipeline> containerWithPipelineMap = new
HashMap<>();
+ for (OmKeyLocationInfoGroup key : value.getKeyLocationVersions()) {
+ key.getLocationList().forEach(k -> {
+ // TODO: fix Some tests that may not initialize container client
+ // The production should always have containerClient initialized.
+ if (scmClient.getContainerClient() != null) {
+ ContainerWithPipeline cp = containerWithPipelineMap
+ .computeIfAbsent(k.getContainerID(), cId -> {
+ try {
+ return scmClient.getContainerClient()
+ .getContainerWithPipeline(cId);
+ } catch (IOException e) {
+ LOG.error("Unable to update pipeline for container:{}",
+ k.getContainerID());
+ return null;
+ }
+ });
+ if (cp == null) {
+ return;
Review comment:
The old logic would have thrown an NPE in Line 665 if the
ContainerWithPipeline got from SCM is null. I have changed that to a return
from method here. I have not added a fail fast logic.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
