[
https://issues.apache.org/jira/browse/HDDS-15729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ivan Andika updated HDDS-15729:
-------------------------------
Description:
The SCM placement policy code structure (both Pipeline Placement Policy and the
Container Placement Policy) is poorly abstracted which cause very hard to
reason about. There are a lot of leaky abstractions or logic specific to a
replication config (e.g. Ratis logic in interface that can also be applied to
EC). I found myself getting lost every time I look at the placement policy
logic since it's hard to build a mental model of the whole process. This task
is to identify these abstractions and fix them for better readability and
maintainability, as well as document the exact contracts and responsibilities
of these policies. This will minimize any surprises and potential bugs due to
misunderstandings and reduce cognitive overloads of all the possible APIs of
parent classes.
For example (not exhaustive)
* PlacementPolicy is supposed to be generic, but we have container placement
specific logic like validateContainerPlacement and
replicasToCopyToFixMisReplication and replicasToRemoveToFixOverreplication
* PipelinePlacementPolicy seems to only used for RATIS pipeline, but we have
EC pipeline, so it gives an impression that PipelinePlacementPolicy can also be
used for EC, which AFAIK is not the case.
*
** PipelinePlacementPolicy is only used for RATIS, then we should make it
explicit by changing the name to RatisPipelinePlacementPolicy
** PipelinePlacementPolicy also have internal "Ratis" related pipelines like
* We are still using PlacementPolicy for everything, even if
PipelinePlacementPolicy is more appropriate for pipeline related matters, which
forces PlacementPolicy to implement interfaces that it should not be
implementing
** For example, in RatisPipelineProvider we can change PlacementPolicy to
PipelinePlacementPolicy since we are dealing with pipeline
** We also don't have a separate ContainerPlacementPolicy interface that have
the logic to handle container placement, we use a generic
SCMCommonPlacementPolicy, which is extended by PipelinePlacementPolicy
* Domain concepts like "anchor node" is not properly defined
* Most of the placement policy is still based on the assumption of rack/leaf
hierarchies
** We need to be able to be more flexible in supporting other schema
(datacenter, region, hall, spine, etc)
** For example, Ceph CRUSH allows administrator to arbitrarily setup the
schema and the CRUSH algorithm will be able to adapt to it (of course there is
risk on this and complexity, so we don't need to go to this far, but it's good
to consider)
** For example, if we only support rack related logic, then we should make it
explicit that placement policy should be used only if the schema is rack/leaf,
and nothing else, since different schema might cause issues (e.g. data loss due
to deletion or replication storms)
was:
The SCM placement policy code structure (both Pipeline Placement Policy and the
Container Placement Policy) is poorly abstracted which cause very hard to
reason about. There are a lot of leaky abstractions or logic specific to a
replication config (e.g. Ratis logic in interface that can also be applied to
EC). I found myself getting lost every time I look at the placement policy
logic since it's hard to build a mental model of the whole process. This task
is to identify these abstractions and fix them for better readability and
maintainability, as well as document the exact contracts and responsibilities
of these policies. This will minimize any surprises and potential bugs due to
misunderstandings and reduce cognitive overloads of all the possible APIs of
parent classes.
For example (not exhaustive)
* PlacementPolicy is supposed to be generic, but we have container placement
specific logic like validateContainerPlacement and
replicasToCopyToFixMisReplication and replicasToRemoveToFixOverreplication
* PipelinePlacementPolicy seems to only used for RATIS pipeline, but we have
EC pipeline, so it gives an impression that PipelinePlacementPolicy can also be
used for EC, which AFAIK is not the case.
*
** PipelinePlacementPolicy is only used for RATIS, then we should make it
explicit by changing the name to RatisPipelinePlacementPolicy
** PipelinePlacementPolicy also have internal "Ratis" related pipelines like
* We are still using PlacementPolicy for everything, even if
PipelinePlacementPolicy is more appropriate for pipeline related matters, which
forces PlacementPolicy to implement interfaces that it should not be
implementing
** For example, in RatisPipelineProvider we can change PlacementPolicy to
PipelinePlacementPolicy since we are dealing with pipeline
** We also don't have a separate ContainerPlacementPolicy interface that have
the logic to handle container placement, we use a generic
SCMCommonPlacementPolicy, which is extended by PipelinePlacementPolicy
* Domain concepts like "anchor node" is not properly defined
* Most of the placement policy is still based on the assumption of rack/leaf
hierarchies
** We need to be able to be more flexible in supporting other schema
(datacenter, region, hall, spine, etc)
** For example, Ceph CRUSH allows administrator to arbitrarily setup the
schema and the CRUSH algorithm will be able to adapt to it (of course there is
risk on this and complexity, so we don't need to go to this far, but it's good
to consider)
> PlacementPolicy API improvements
> --------------------------------
>
> Key: HDDS-15729
> URL: https://issues.apache.org/jira/browse/HDDS-15729
> Project: Apache Ozone
> Issue Type: Improvement
> Reporter: Ivan Andika
> Assignee: Ivan Andika
> Priority: Major
>
> The SCM placement policy code structure (both Pipeline Placement Policy and
> the Container Placement Policy) is poorly abstracted which cause very hard to
> reason about. There are a lot of leaky abstractions or logic specific to a
> replication config (e.g. Ratis logic in interface that can also be applied to
> EC). I found myself getting lost every time I look at the placement policy
> logic since it's hard to build a mental model of the whole process. This task
> is to identify these abstractions and fix them for better readability and
> maintainability, as well as document the exact contracts and responsibilities
> of these policies. This will minimize any surprises and potential bugs due to
> misunderstandings and reduce cognitive overloads of all the possible APIs of
> parent classes.
> For example (not exhaustive)
> * PlacementPolicy is supposed to be generic, but we have container placement
> specific logic like validateContainerPlacement and
> replicasToCopyToFixMisReplication and replicasToRemoveToFixOverreplication
> * PipelinePlacementPolicy seems to only used for RATIS pipeline, but we have
> EC pipeline, so it gives an impression that PipelinePlacementPolicy can also
> be used for EC, which AFAIK is not the case.
> *
> ** PipelinePlacementPolicy is only used for RATIS, then we should make it
> explicit by changing the name to RatisPipelinePlacementPolicy
> ** PipelinePlacementPolicy also have internal "Ratis" related pipelines like
> * We are still using PlacementPolicy for everything, even if
> PipelinePlacementPolicy is more appropriate for pipeline related matters,
> which forces PlacementPolicy to implement interfaces that it should not be
> implementing
> ** For example, in RatisPipelineProvider we can change PlacementPolicy to
> PipelinePlacementPolicy since we are dealing with pipeline
> ** We also don't have a separate ContainerPlacementPolicy interface that
> have the logic to handle container placement, we use a generic
> SCMCommonPlacementPolicy, which is extended by PipelinePlacementPolicy
> * Domain concepts like "anchor node" is not properly defined
> * Most of the placement policy is still based on the assumption of rack/leaf
> hierarchies
> ** We need to be able to be more flexible in supporting other schema
> (datacenter, region, hall, spine, etc)
> ** For example, Ceph CRUSH allows administrator to arbitrarily setup the
> schema and the CRUSH algorithm will be able to adapt to it (of course there
> is risk on this and complexity, so we don't need to go to this far, but it's
> good to consider)
> ** For example, if we only support rack related logic, then we should make
> it explicit that placement policy should be used only if the schema is
> rack/leaf, and nothing else, since different schema might cause issues (e.g.
> data loss due to deletion or replication storms)
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
