[
https://issues.apache.org/jira/browse/HELIX-659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jiajun Wang updated HELIX-659:
------------------------------
Attachment: Associate States Processing Flow.pdf
https://www.lucidchart.com/documents/view/5217edea-896d-4ddc-8a93-c31eeb273f38
> Support Additional Associate States
> -----------------------------------
>
> Key: HELIX-659
> URL: https://issues.apache.org/jira/browse/HELIX-659
> Project: Apache Helix
> Issue Type: New Feature
> Components: helix-core
> Affects Versions: 0.6.x
> Reporter: Jiajun Wang
> Attachments: Associate States Processing Flow.pdf
>
>
> Currently, Helix only supports management a single state for all
> resources/partitions. However, in the real world, cluster management
> requirements may be more complicated than that.
> In Pinot, for example, each partition need to be assigned a version for
> ensuring data consistency.
> When a new version comes, the system needs to replace the old partition with
> the new one. And the replacement is done one partition by one partition. So
> any reads during this period will get inconsistent data.
> Pinot system cannot directly put the version information into the
> section(partition) state field because it is already occupied by the main
> state (offline-online for instance) used by Helix controller.
> So Pinot team relies on some workarounds to implement their application
> logic: creating a new resource with the latest version and replace them after
> the resource is fully loaded. And for Helix controller, version is unknown.
> Another option is Pinot team maintaining their own config item or property
> store item for recording versions.
> Both ways require Pinot team implementing version control themselves.
> Another requirement is from Ambry team. Where partition can be "ONLINE:READ"
> or "ONLINE:WRITE".
> In both cases, single state mechanism is not sufficient for applications'
> requirement.
> It would be very helpful to provide a framework level feature that supports
> more than one states for each partition.
> Benefits:
> # The application doesn't need to write additional code for managing
> additional states.
> # Avoid potential conflict when multiple states transition happens
> concurrently.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
