GitHub user zhan849 opened a pull request: https://github.com/apache/helix/pull/294
Implement view cluster aggregator Based on #266 , design of helix view aggregator, here is the implementation. Helix view aggregator will be a different module under helix repo, and the impl will use components from helix-core. Here is a debrief of this PR: - View cluster related information will be added to `ClusterConfig`, and related java apis will be added to helix-core. - `SourceClusterDataProvider` is a component that watches changes from source cluster, updates its cached data, and notify data change event via a given channel - `ViewClusterRefresher` is a component that does the actual refresh operation of the view cluster. It computes diff between source clusters and view cluster, make changes to view cluster accordingly - `SourceClusterConfigChangeAction` is a wrapper containing information about what to update for a source cluster. It takes in old and new `ClusterConfig` and compute actions to adopt view cluster to the changes - `HelixViewAggregator` hooks up small components and contains the main reconciliation loop for refreshing view cluster - Metrics recording mechanism is added - starting helix view aggregator in a stand-alone mode and distributed mode (by adding state model into helix participant) is supported - related unit tests and integration tests are added You can merge this pull request into a Git repository by running: $ git pull https://github.com/zhan849/helix harry/view-aggregator-impl Alternatively you can review and apply these changes as the patch at: https://github.com/apache/helix/pull/294.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #294 ---- commit d6b98f236d85522866e4e7584561a02ce525d142 Author: Harry Zhang <hrzhang@...> Date: 2018-11-01T22:26:45Z [HELIX-781]: add java apis for view cluster config support commit 759cf8a9a48805851c50681afc98b2cafe309ff7 Author: Harry Zhang <hrzhang@...> Date: 2018-11-02T21:28:23Z [HELIX-781] set up module structure for helix view aggregator commit 2ae85f6127b43aa7928776ede7e9bbbfe94e9385 Author: Harry Zhang <hrzhang@...> Date: 2018-11-02T21:38:14Z [HELIX-781] implement SourceClusterDataProvider and add tests commit 1df6c7e15bc31fb424883e3febcd9f8cd019de82 Author: Harry Zhang <hrzhang@...> Date: 2018-11-02T21:42:56Z [HELIX-781] implement ViewClusterRefresher and add tests commit 7802cf2443d45092c69cb529b4bba17a3c45dc31 Author: Harry Zhang <hrzhang@...> Date: 2018-11-02T21:50:35Z [HELIX-781] implement SourceClusterConfigChangeAction and added tests commit 985a4ac2bcc43738d9c7ab463ad606f8691f7298 Author: Harry Zhang <hrzhang@...> Date: 2018-11-02T21:59:21Z [HELIX-781] implement helix view aggregator main logic and added tests commit 3e2cfb695720fc09f181b55a3a31adc2b44f94a4 Author: Harry Zhang <hrzhang@...> Date: 2018-11-02T22:08:43Z [HELIX-781] added metrics to helix view aggregator and added tests commit 82d33e0aad1ad2279f864fea6d5851129ba56610 Author: Harry Zhang <hrzhang@...> Date: 2018-11-02T22:11:00Z [HELIX-781] added main function to start helix view aggregator via bash commit 11a7c126ca8bb9298754cb9127af588148af0902 Author: Harry Zhang <hrzhang@...> Date: 2018-11-02T22:12:57Z [HELIX-781] support deploy helix view aggregator in a distributed fashion using helix participant ---- ---