[
https://issues.apache.org/jira/browse/HDDS-9343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated HDDS-9343:
---------------------------------
Labels: blackbird pull-request-available (was: blackbird)
> Shift sortDatanodes logic to OM
> -------------------------------
>
> Key: HDDS-9343
> URL: https://issues.apache.org/jira/browse/HDDS-9343
> Project: Apache Ozone
> Issue Type: Improvement
> Reporter: Tanvi Penumudy
> Assignee: Tanvi Penumudy
> Priority: Major
> Labels: blackbird, pull-request-available
>
> Motivation behind this change:
> * When OM has to call SCM then the performance of the objects on the
> read-path can affect SCM’s scaling requirements, so SCM needs to perform at
> the same level as that of OM for the read-path - harming the performance of
> OM by going to SCM for every read.
> * Instead, a more efficient approach would be to perform sorting within OM
> itself, eliminating the need to rely on SCM for every read.
>
> Steps how this can be achieved:
> I. Add API in SCM:
> * The current sorting logic of datanodes in the container pipelines is
> present in SCM (SCMBlockProtocolServer).
> * SCM holds a configuration regarding the layout of the datanodes (similar
> to that of Hadoop) as part of an .xml file (NodeSchemaManager#init) -
> Configuration key: ozone.scm.network.topology.schema.file, value:
> network-topology-default.xml
> * The first step would be to add an API for SCM to serve this .xml file as
> part of an RPC call to OM.
> * We can do this such that OM has enough information to take the client
> location and the layout information for sorting the datanodes.
> II. OM Cache and Refresh:
> * OM will need to cache the layout information and periodically refresh it.
> * The periodic refreshes would involve refetching the updated layout
> information from SCM.
> III. Refactor NetworkTopology + Sort:
> * The NetworkTopology calculation can be refactored and moved to a common
> location so that both OM and SCM can utilize it.
> * On top of the NetworkTopology object obtained,
> NetworkTopology#sortByDistanceCost could be called.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
