Jiajun Wang created HELIX-659:
---------------------------------
Summary: Support Additional Associate States
Key: HELIX-659
URL: https://issues.apache.org/jira/browse/HELIX-659
Project: Apache Helix
Issue Type: New Feature
Components: helix-core
Affects Versions: 0.6.x
Reporter: Jiajun Wang
Currently, Helix only supports management a single state for all
resources/partitions. However, in the real world, cluster management
requirements may be more complicated than that.
In Pinot, for example, each partition need to be assigned a version for
ensuring data consistency.
When a new version comes, the system needs to replace the old partition with
the new one. And the replacement is done one partition by one partition. So any
reads during this period will get inconsistent data.
Pinot system cannot directly put the version information into the
section(partition) state field because it is already occupied by the main state
(offline-online for instance) used by Helix controller.
So Pinot team relies on some workarounds to implement their application logic:
creating a new resource with the latest version and replace them after the
resource is fully loaded. And for Helix controller, version is unknown.
Another option is Pinot team maintaining their own config item or property
store item for recording versions.
Both ways require Pinot team implementing version control themselves.
Another requirement is from Ambry team. Where partition can be "ONLINE:READ" or
"ONLINE:WRITE".
In both cases, single state mechanism is not sufficient for applications'
requirement.
It would be very helpful to provide a framework level feature that supports
more than one states for each partition.
Benefits:
# The application doesn't need to write additional code for managing additional
states.
# Avoid potential conflict when multiple states transition happens concurrently.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
