[
https://issues.apache.org/jira/browse/HDDS-2241?focusedWorklogId=346954&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-346954
]
ASF GitHub Bot logged work on HDDS-2241:
----------------------------------------
Author: ASF GitHub Bot
Created on: 20/Nov/19 19:21
Start Date: 20/Nov/19 19:21
Worklog Time Spent: 10m
Work Description: bharatviswa504 commented on pull request #194:
HDDS-2241. Optimize the refresh pipeline logic used by KeyManagerImpl…
URL: https://github.com/apache/hadoop-ozone/pull/194
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 346954)
Time Spent: 20m (was: 10m)
> Optimize the refresh pipeline logic used by KeyManagerImpl to obtain the
> pipelines for a key
> --------------------------------------------------------------------------------------------
>
> Key: HDDS-2241
> URL: https://issues.apache.org/jira/browse/HDDS-2241
> Project: Hadoop Distributed Data Store
> Issue Type: Bug
> Components: Ozone Manager
> Reporter: Aravindan Vijayan
> Assignee: Aravindan Vijayan
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.5.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> Currently, while looking up a key, the Ozone Manager gets the pipeline
> information from SCM through an RPC for every block in the key. For large
> files > 1GB, we may end up making a lot of RPC calls for this. This can be
> optimized in a couple of ways
> * We can implement a batch getContainerWithPipeline API in SCM using which we
> can get the pipeline info locations for all the blocks for a file. To keep
> the number of containers passed in to SCM in a single call, we can have a
> fixed container batch size on the OM side. _Here, Number of calls = 1 (or k
> depending on batch size)_
> * Instead, a simpler change would be to have a map (method local) of
> ContainerID -> Pipeline that we get from SCM so that we don't need to make
> repeated calls to SCM for the same containerID for a key. _Here, Number of
> calls = Number of unique containerIDs_
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
