(helix) branch master updated: [apache/helix] -- [Part-2] Enable JDK 1.8 (backward) compatibility for meta-client (#2802)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new c11cddf91 [apache/helix] -- [Part-2] Enable JDK 1.8 (backward) compatibility for meta-client (#2802) c11cddf91 is described below commit c11cddf916fa7809ea99a10d97aa18dc8bc90a87 Author: Himanshu Kandwal AuthorDate: Thu May 9 13:44:49 2024 -0700 [apache/helix] -- [Part-2] Enable JDK 1.8 (backward) compatibility for meta-client (#2802) We would like to provide a backward compatible support to our consumers where they also have an option to use JDK-8 compiled meta-client jar, if they have such a requirement. By default we will generate JDK-11 jars and JDK-8 jars using a classifier. --- meta-client/pom.xml | 30 ++ 1 file changed, 30 insertions(+) diff --git a/meta-client/pom.xml b/meta-client/pom.xml index e08872c8c..a4762eb37 100644 --- a/meta-client/pom.xml +++ b/meta-client/pom.xml @@ -89,6 +89,36 @@ under the License. + +org.apache.maven.plugins +maven-compiler-plugin +3.12.1 + + +JDK 8 +compile + + compile + + + ${project.build.outputDirectory}_jdk8 + 8 + true + + + +JDK 11 +compile + + compile + + + 11 + true + + + + org.apache.maven.plugins maven-jar-plugin
(helix) branch master updated: [apache/helix] -- Provide JDK 1.8 (backward) compatibility for meta-client (#2799)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new 93c59a763 [apache/helix] -- Provide JDK 1.8 (backward) compatibility for meta-client (#2799) 93c59a763 is described below commit 93c59a763c1f4ec72aeae8c654292b15d0f50ed5 Author: Himanshu Kandwal AuthorDate: Wed May 8 18:25:45 2024 -0700 [apache/helix] -- Provide JDK 1.8 (backward) compatibility for meta-client (#2799) We would like to provide a backward compatible support to our consumers where they also have an option to use JDK-8 compiled helix-core and meta-client jar, if they have such a requirement. By default we will generate JDK-11 jars and JDK-8 jars using a classifier. --- helix-core/pom.xml | 1 - meta-client/pom.xml | 18 ++ 2 files changed, 18 insertions(+), 1 deletion(-) diff --git a/helix-core/pom.xml b/helix-core/pom.xml index ba3d7b7e5..c8c5a504d 100644 --- a/helix-core/pom.xml +++ b/helix-core/pom.xml @@ -219,7 +219,6 @@ package jar - test-jar ${project.build.outputDirectory}_jdk8 diff --git a/meta-client/pom.xml b/meta-client/pom.xml index 29092ef1e..e08872c8c 100644 --- a/meta-client/pom.xml +++ b/meta-client/pom.xml @@ -89,6 +89,24 @@ under the License. + +org.apache.maven.plugins +maven-jar-plugin +3.3.0 + + +default-package-jdk11 +package + + jar + + + ${project.build.outputDirectory}_jdk8 + jdk8 + + + + org.apache.maven.plugins maven-assembly-plugin
(helix) branch master updated: Disable the unstable test for task framework
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new ca6ad6bc3 Disable the unstable test for task framework ca6ad6bc3 is described below commit ca6ad6bc35ad9a83607392e6b141722d854be7e5 Author: Junkai Xue AuthorDate: Wed May 8 15:42:32 2024 -0700 Disable the unstable test for task framework --- .../helix/integration/task/TestTaskSchedulingTwoCurrentStates.java | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/helix-core/src/test/java/org/apache/helix/integration/task/TestTaskSchedulingTwoCurrentStates.java b/helix-core/src/test/java/org/apache/helix/integration/task/TestTaskSchedulingTwoCurrentStates.java index ada5157a3..4181c4b82 100644 --- a/helix-core/src/test/java/org/apache/helix/integration/task/TestTaskSchedulingTwoCurrentStates.java +++ b/helix-core/src/test/java/org/apache/helix/integration/task/TestTaskSchedulingTwoCurrentStates.java @@ -96,7 +96,7 @@ public class TestTaskSchedulingTwoCurrentStates extends TaskTestBase { super.afterClass(); } - @Test + @Test (enabled = false) public void testTargetedTaskTwoCurrentStates() throws Exception { _gSetupTool.addResourceToCluster(CLUSTER_NAME, DATABASE, _numPartitions, MASTER_SLAVE_STATE_MODEL, IdealState.RebalanceMode.SEMI_AUTO.name());
(helix) branch master updated: [apache/helix] -- Add SetPartitionToError for participants to self annotate a node to ERROR state (#2792)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new 1d47d6b51 [apache/helix] -- Add SetPartitionToError for participants to self annotate a node to ERROR state (#2792) 1d47d6b51 is described below commit 1d47d6b51ee27764008685cf6ec76d4108915807 Author: Charanya Sudharsanan AuthorDate: Tue May 7 21:44:40 2024 -0700 [apache/helix] -- Add SetPartitionToError for participants to self annotate a node to ERROR state (#2792) Co-authored-by: Charanya Sudharsanan What: An API endpoint that validates the incoming request and sends a state transition message to sets one or more partitions from any current state to ERROR state. Why: Currently, the participants are unable to set a partition to an ERROR state explicitly when they seem to be stuck in a specific current state. The only way a replica can be set to ERROR is from within a state model. Having an endpoint to allow this behavior would allow the clients to call the resetPartition endpoint to set it back to INIT state and recover the replica. resetPartition works only on partitions in error state. --- .../src/main/java/org/apache/helix/HelixAdmin.java | 12 + .../org/apache/helix/manager/zk/ZKHelixAdmin.java | 256 - .../handling/HelixStateTransitionHandler.java | 3 +- .../apache/helix/messaging/handling/HelixTask.java | 1 + .../helix/participant/statemachine/StateModel.java | 11 + .../java/org/apache/helix/tools/ClusterSetup.java | 21 ++ .../integration/TestSetPartitionsToErrorState.java | 99 .../apache/helix/manager/zk/TestZkHelixAdmin.java | 119 +- .../java/org/apache/helix/mock/MockHelixAdmin.java | 6 + .../rest/server/resources/AbstractResource.java| 3 +- .../resources/helix/PerInstanceAccessor.java | 10 + .../helix/rest/server/TestPerInstanceAccessor.java | 43 +++- 12 files changed, 458 insertions(+), 126 deletions(-) diff --git a/helix-core/src/main/java/org/apache/helix/HelixAdmin.java b/helix-core/src/main/java/org/apache/helix/HelixAdmin.java index d2e0c2681..84a7154b1 100644 --- a/helix-core/src/main/java/org/apache/helix/HelixAdmin.java +++ b/helix-core/src/main/java/org/apache/helix/HelixAdmin.java @@ -421,6 +421,18 @@ public interface HelixAdmin { */ ClusterManagementMode getClusterManagementMode(String clusterName); + /** + * Set a list of partitions for an instance to ERROR state from any state. + * The partitions could be in any state and setPartitionsToError will bring them to ERROR + * state. ANY to ERROR state transition is required for this. + * @param clusterName + * @param instanceName + * @param resourceName + * @param partitionNames + */ + void setPartitionsToError(String clusterName, String instanceName, String resourceName, + List partitionNames); + /** * Reset a list of partitions in error state for an instance * The partitions are assume to be in error state and reset will bring them from error diff --git a/helix-core/src/main/java/org/apache/helix/manager/zk/ZKHelixAdmin.java b/helix-core/src/main/java/org/apache/helix/manager/zk/ZKHelixAdmin.java index c7fe0861b..8c873b4cd 100644 --- a/helix-core/src/main/java/org/apache/helix/manager/zk/ZKHelixAdmin.java +++ b/helix-core/src/main/java/org/apache/helix/manager/zk/ZKHelixAdmin.java @@ -1035,6 +1035,136 @@ public class ZKHelixAdmin implements HelixAdmin { : new ClusterManagementMode(status.getManagementMode(), status.getManagementModeStatus()); } + @Override + public void setPartitionsToError(String clusterName, String instanceName, String resourceName, + List partitionNames) { +logger.info("Set partitions {} for resource {} on instance {} in cluster {} to ERROR state.", +partitionNames == null ? "NULL" : HelixUtil.serializeByComma(partitionNames), resourceName, +instanceName, clusterName); +sendStateTransitionMessage(clusterName, instanceName, resourceName, partitionNames, +StateTransitionType.SET_TO_ERROR); + } + + private void sendStateTransitionMessage(String clusterName, String instanceName, + String resourceName, List partitionNames, StateTransitionType stateTransitionType) { +HelixDataAccessor accessor = +new ZKHelixDataAccessor(clusterName, new ZkBaseDataAccessor(_zkClient)); +PropertyKey.Builder keyBuilder = accessor.keyBuilder(); + +// check the instance is alive +LiveInstance liveInstance = accessor.getProperty(keyBuilder.liveInstance(instanceName)); +if (liveInstance == null) { + // check if the instance exists in the cluster + String instanceConfigPath = PropertyPathBuilder.instanceConfig(clusterName, instanceName); + throw new HelixException(String.format( +
(helix) branch master updated: Bump tar from 6.1.11 to 6.2.1 in /helix-front (#2789)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new f12e63e3a Bump tar from 6.1.11 to 6.2.1 in /helix-front (#2789) f12e63e3a is described below commit f12e63e3a7169c98f36d7a2d36c38fd803dd702a Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> AuthorDate: Thu Apr 11 10:12:52 2024 -0700 Bump tar from 6.1.11 to 6.2.1 in /helix-front (#2789) Bumps [tar](https://github.com/isaacs/node-tar) from 6.1.11 to 6.2.1. - [Release notes](https://github.com/isaacs/node-tar/releases) - [Changelog](https://github.com/isaacs/node-tar/blob/main/CHANGELOG.md) - [Commits](https://github.com/isaacs/node-tar/compare/v6.1.11...v6.2.1) --- updated-dependencies: - dependency-name: tar dependency-type: indirect ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- helix-front/yarn.lock | 13 + 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/helix-front/yarn.lock b/helix-front/yarn.lock index 57efb4142..4f3148651 100644 --- a/helix-front/yarn.lock +++ b/helix-front/yarn.lock @@ -11567,6 +11567,11 @@ minipass@^3.0.0, minipass@^3.1.0, minipass@^3.1.1, minipass@^3.1.3, minipass@^3. dependencies: yallist "^4.0.0" +minipass@^5.0.0: + version "5.0.0" + resolved "https://registry.yarnpkg.com/minipass/-/minipass-5.0.0.tgz#3e9788ffb90b694a5d0ec94479a45b5d8738133d; + integrity sha512-3FnjYuehv9k6ovOEbyOswadCDPX1piCfhV8ncmYtHOjuPwylVWsghTLo7rabjC3Rx5xD4HDx8Wm1xnMF7S5qFQ== + minizlib@^2.0.0, minizlib@^2.1.1, minizlib@^2.1.2: version "2.1.2" resolved "https://registry.yarnpkg.com/minizlib/-/minizlib-2.1.2.tgz#e90d3466ba209b932451508a11ce3d3632145931; @@ -14895,13 +14900,13 @@ tapable@^2.0.0, tapable@^2.1.1, tapable@^2.2.0: integrity sha512-GNzQvQTOIP6RyTfE2Qxb8ZVlNmw0n88vp1szwWRimP02mnTsx3Wtn5qRdqY9w2XduFNUgvOwhNnQsjwCp+kqaQ== tar@^6.0.2, tar@^6.1.0, tar@^6.1.11, tar@^6.1.2: - version "6.1.11" - resolved "https://registry.yarnpkg.com/tar/-/tar-6.1.11.tgz#6760a38f003afa1b2ffd0ffe9e9abbd0eab3d621; - integrity sha512-an/KZQzQUkZCkuoAA64hM92X0Urb6VpRhAFllDzz44U2mcD5scmT3zBc4VgVpkugF580+DQn8eAFSyoQt0tznA== + version "6.2.1" + resolved "https://registry.yarnpkg.com/tar/-/tar-6.2.1.tgz#717549c541bc3c2af15751bea94b1dd068d4b03a; + integrity sha512-DZ4yORTwrbTj/7MZYq2w+/ZFdI6OZ/f9SFHR+71gIVUZhOQPHzVCLpvRnPgyaMpfWxxk/4ONva3GQSyNIKRv6A== dependencies: chownr "^2.0.0" fs-minipass "^2.0.0" -minipass "^3.0.0" +minipass "^5.0.0" minizlib "^2.1.1" mkdirp "^1.0.3" yallist "^4.0.0"
(helix) branch master updated: [apache/helix] -- Issue during onboarding resources without instances (#2782)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new c480eac01 [apache/helix] -- Issue during onboarding resources without instances (#2782) c480eac01 is described below commit c480eac018932214886aaf55f3c28107d46e84b0 Author: Himanshu Kandwal AuthorDate: Mon Apr 1 11:06:40 2024 -0700 [apache/helix] -- Issue during onboarding resources without instances (#2782) When adding a new WAGED resource with a tag and without any instances against that tag, we are observing NPE coming from the system. To solve this issue we are adding a check in the ResourceComputationStage to have such resources excluded from the pipeline computation and only be considered when there are actual resource partitions (>0) to be assigned to the instances. --- .../stages/ResourceComputationStage.java | 2 +- ...xpansionWithAddingResourcesBeforeInstances.java | 189 + 2 files changed, 190 insertions(+), 1 deletion(-) diff --git a/helix-core/src/main/java/org/apache/helix/controller/stages/ResourceComputationStage.java b/helix-core/src/main/java/org/apache/helix/controller/stages/ResourceComputationStage.java index 00b2fd71b..e1d9a9215 100644 --- a/helix-core/src/main/java/org/apache/helix/controller/stages/ResourceComputationStage.java +++ b/helix-core/src/main/java/org/apache/helix/controller/stages/ResourceComputationStage.java @@ -91,7 +91,7 @@ public class ResourceComputationStage extends AbstractBaseStage { Map idealStates, boolean isTaskCache) { if (idealStates != null && idealStates.size() > 0) { for (IdealState idealState : idealStates.values()) { -if (idealState == null) { +if (idealState == null || idealState.getNumPartitions() == 0) { continue; } diff --git a/helix-core/src/test/java/org/apache/helix/integration/rebalancer/WagedRebalancer/TestWagedClusterExpansionWithAddingResourcesBeforeInstances.java b/helix-core/src/test/java/org/apache/helix/integration/rebalancer/WagedRebalancer/TestWagedClusterExpansionWithAddingResourcesBeforeInstances.java new file mode 100644 index 0..de7aff693 --- /dev/null +++ b/helix-core/src/test/java/org/apache/helix/integration/rebalancer/WagedRebalancer/TestWagedClusterExpansionWithAddingResourcesBeforeInstances.java @@ -0,0 +1,189 @@ +package org.apache.helix.integration.rebalancer.WagedRebalancer; + +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import java.util.ArrayList; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.concurrent.atomic.AtomicLong; + +import javax.management.MalformedObjectNameException; +import javax.management.ObjectName; + +import org.apache.helix.ConfigAccessor; +import org.apache.helix.TestHelper; +import org.apache.helix.common.ZkTestBase; +import org.apache.helix.integration.manager.ClusterControllerManager; +import org.apache.helix.integration.manager.MockParticipantManager; +import org.apache.helix.model.ClusterConfig; +import org.apache.helix.model.ExternalView; +import org.apache.helix.model.IdealState; +import org.apache.helix.model.InstanceConfig; +import org.apache.helix.monitoring.mbeans.MonitorDomainNames; +import org.apache.helix.tools.ClusterVerifiers.BestPossibleExternalViewVerifier; +import org.apache.helix.tools.ClusterVerifiers.HelixClusterVerifier; +import org.apache.helix.tools.ClusterVerifiers.StrictMatchExternalViewVerifier; +import org.apache.helix.tools.ClusterVerifiers.ZkHelixClusterVerifier; +import org.testng.Assert; +import org.testng.annotations.AfterClass; +import org.testng.annotations.BeforeClass; +import org.testng.annotations.Test; + +import static org.apache.helix.model.BuiltInStateModelDefinitions.LeaderStandby; +import static org.apache.helix.monitoring.mbeans.ClusterStatusMonitor.CLUSTER_DN_KEY; + +public class TestWagedClusterExpansionWithAddingResourcesBeforeInstances extends ZkTestBase { + private stat
(helix) branch master updated: Bump express from 4.18.2 to 4.19.2 in /helix-front (#2786)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new e606cbe29 Bump express from 4.18.2 to 4.19.2 in /helix-front (#2786) e606cbe29 is described below commit e606cbe295b086a4751be0f536a2c16b6a096c10 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> AuthorDate: Tue Mar 26 17:18:36 2024 -0700 Bump express from 4.18.2 to 4.19.2 in /helix-front (#2786) Bumps [express](https://github.com/expressjs/express) from 4.18.2 to 4.19.2. - [Release notes](https://github.com/expressjs/express/releases) - [Changelog](https://github.com/expressjs/express/blob/master/History.md) - [Commits](https://github.com/expressjs/express/compare/4.18.2...4.19.2) --- updated-dependencies: - dependency-name: express dependency-type: direct:production ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- helix-front/package.json | 2 +- helix-front/yarn.lock| 52 +++- 2 files changed, 22 insertions(+), 32 deletions(-) diff --git a/helix-front/package.json b/helix-front/package.json index b7256f6be..19f7efdb5 100644 --- a/helix-front/package.json +++ b/helix-front/package.json @@ -92,7 +92,7 @@ "core-js": "^2.4.1", "d3-shape": "^1.2.0", "dotenv": "^4.0.0", -"express": "^4.15.3", +"express": "^4.19.2", "express-session": "^1.15.6", "keycharm": "^0.3.0", "ldapjs": "^1.0.2", diff --git a/helix-front/yarn.lock b/helix-front/yarn.lock index 0df1350c3..57efb4142 100644 --- a/helix-front/yarn.lock +++ b/helix-front/yarn.lock @@ -5070,13 +5070,13 @@ bluebird@^3.7.2: resolved "https://registry.yarnpkg.com/bluebird/-/bluebird-3.7.2.tgz#9f229c15be272454ffa973ace0dbee79a1b0c36f; integrity sha512-XpNj6GDQzdfW+r2Wnn7xiSAd7TM3jzkxGXBGTtWKuSXv1xUV+azxAm8jdWZN06QTQk+2N2XB9jRDkvbmQmcRtg== -body-parser@1.20.1, body-parser@^1.17.2: - version "1.20.1" - resolved "https://registry.yarnpkg.com/body-parser/-/body-parser-1.20.1.tgz#b1812a8912c195cd371a3ee5e66faa2338a5c668; - integrity sha512-jWi7abTbYwajOytWCQc37VulmWiRae5RyTpaCyDcS5/lMdtwSz5lOpDE67srw/HYe35f1z3fDQw+3txg7gNtWw== +body-parser@1.20.2, body-parser@^1.17.2: + version "1.20.2" + resolved "https://registry.yarnpkg.com/body-parser/-/body-parser-1.20.2.tgz#6feb0e21c4724d06de7ff38da36dad4f57a747fd; + integrity sha512-ml9pReCu3M61kGlqoTm2umSXTlRTuGTx0bfYj+uIUKKYycG5NtSbeetV3faSU6R7ajOPw0g/J1PvK4qNy7s5bA== dependencies: bytes "3.1.2" -content-type "~1.0.4" +content-type "~1.0.5" debug "2.6.9" depd "2.0.0" destroy "1.2.0" @@ -5084,7 +5084,7 @@ body-parser@1.20.1, body-parser@^1.17.2: iconv-lite "0.4.24" on-finished "2.4.1" qs "6.11.0" -raw-body "2.5.1" +raw-body "2.5.2" type-is "~1.6.18" unpipe "1.0.0" @@ -5884,10 +5884,10 @@ content-disposition@0.5.4: dependencies: safe-buffer "5.2.1" -content-type@~1.0.4: - version "1.0.4" - resolved "https://registry.yarnpkg.com/content-type/-/content-type-1.0.4.tgz#e138cc75e040c727b1966fe5e5f8c9aee256fe3b; - integrity sha512-hIP3EEPs8tB9AT1L+NUqtwOAps4mk2Zob89MWXMHjHWg9milF/j4osnnQLXBCBFBk/tvIG/tUc9mOUJiPBhPXA== +content-type@~1.0.4, content-type@~1.0.5: + version "1.0.5" + resolved "https://registry.yarnpkg.com/content-type/-/content-type-1.0.5.tgz#8b773162656d1d1086784c8f23a54ce6d73d7918; + integrity sha512-nTjqfcBFEipKdXCv4YDQWCfmcLZKm81ldF0pAopTvyrFGVbcR6P/VAAd5G7N+0tTr8QqiU0tFadD6FK4NtJwOA== continuation-local-storage@^3.2.1: version "3.2.1" @@ -5912,10 +5912,10 @@ cookie@0.4.2: resolved "https://registry.yarnpkg.com/cookie/-/cookie-0.4.2.tgz#0e41f24de5ecf317947c82fc789e06a884824432; integrity sha512-aSWTXFzaKWkvHO1Ny/s+ePFpvKsPnjc551iI41v3ny/ow6tBG5Vd+FuqGNhh1LxOmVzOlGUriIlOaokOvhaStA== -cookie@0.5.0: - version "0.5.0" - resolved "https://registry.yarnpkg.com/cookie/-/cookie-0.5.0.tgz#d1f5d71adec6558c58f389987c366aa47e994f8b; - integrity sha512-YZ3GUyn/o8gfKJlnlX7g7xq4gyO6OSuhGPKaaGssGB2qgDUS0gPgtTvoyZLTt9Ab6dC4hfc9dV5arkvc/OCmrw== +cookie@0.6.0: + version "0.6.0" + resolved "https://registry.yarnpkg.com/cookie/-/cookie-0.6.0.tgz#2798b04b071b0ecbff0dbb62a505a8efa4e19051; + integrity sha512-U71cyTamuh1CRNCfpGY6to28lxvNwPG4Guz/EVjgf3Jmzv0vlDp1atT9eS5dDjMYHucpHbWns6Lwf3BKz6svdw
(helix) branch master updated: [apache/helix] -- Enable JDK-8 Build alsong with JDK-11 (#2775)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new 224997721 [apache/helix] -- Enable JDK-8 Build alsong with JDK-11 (#2775) 224997721 is described below commit 224997721d6a9975bee2be16eeb7594899c03623 Author: Himanshu Kandwal AuthorDate: Tue Mar 26 11:17:00 2024 -0700 [apache/helix] -- Enable JDK-8 Build alsong with JDK-11 (#2775) We would like to provide a backward compatible support to our consumers where they also have an option to use JDK-8 compiled helix-core jar, if they have such a requirement. By default we will generate JDK-11 jars and JDK-8 jars using a classifier. --- helix-core/pom.xml | 38 ++ 1 file changed, 38 insertions(+) diff --git a/helix-core/pom.xml b/helix-core/pom.xml index d3934f579..ba3d7b7e5 100644 --- a/helix-core/pom.xml +++ b/helix-core/pom.xml @@ -179,14 +179,52 @@ + +org.apache.maven.plugins +maven-compiler-plugin +3.12.1 + + +JDK 8 +compile + + compile + + + ${project.build.outputDirectory}_jdk8 + 8 + true + + + +JDK 11 +compile + + compile + + + 11 + true + + + + org.apache.maven.plugins maven-jar-plugin +3.3.0 +default-package-jdk11 +package + jar test-jar + + ${project.build.outputDirectory}_jdk8 + jdk8 +
(helix) branch master updated: Do not start the server if user uses the default SECRET_TOKEN env value (#2783)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new aef8c0ee8 Do not start the server if user uses the default SECRET_TOKEN env value (#2783) aef8c0ee8 is described below commit aef8c0ee848c5cbc4a8b3202b578b2fb01279edc Author: Vivo AuthorDate: Thu Mar 21 13:17:03 2024 -0700 Do not start the server if user uses the default SECRET_TOKEN env value (#2783) * secure SECRET_TOKEN * strict the node version to 14 * format --- .github/workflows/helix-front.yml | 10 +- helix-front/package.json | 4 helix-front/server/app.ts | 16 ++-- 3 files changed, 23 insertions(+), 7 deletions(-) diff --git a/.github/workflows/helix-front.yml b/.github/workflows/helix-front.yml index 961aa0a4a..9b67dc33e 100644 --- a/.github/workflows/helix-front.yml +++ b/.github/workflows/helix-front.yml @@ -1,9 +1,9 @@ name: Helix Front CI on: pull_request: -branches: [ master ] +branches: [master] paths: - - 'helix-front/**' + - "helix-front/**" jobs: CI: @@ -16,9 +16,9 @@ jobs: - name: Setup Node environment uses: actions/setup-node@v3 with: - node-version: '16.x' - cache: 'yarn' - cache-dependency-path: 'helix-front/yarn.lock' + node-version: "14.x" + cache: "yarn" + cache-dependency-path: "helix-front/yarn.lock" - name: Install dependencies run: yarn diff --git a/helix-front/package.json b/helix-front/package.json index e151010ae..b7256f6be 100644 --- a/helix-front/package.json +++ b/helix-front/package.json @@ -164,5 +164,9 @@ "typescript": "4.6.4", "util": "^0.12.4", "webpack": "5" + }, + "engines": { +"npm": ">=6.0.0 <7.0.0", +"node": ">=14.0.0 <15.0.0" } } diff --git a/helix-front/server/app.ts b/helix-front/server/app.ts index 3560ed010..f34688711 100644 --- a/helix-front/server/app.ts +++ b/helix-front/server/app.ts @@ -18,6 +18,7 @@ import { } from './config'; import setRoutes from './routes'; +const isProd = process.env.NODE_ENV === 'production'; const httpsProxyAgent = PROXY_URL ? new ProxyAgent(PROXY_URL) : null; if (APP_INSIGHTS_CONNECTION_STRING) { @@ -35,7 +36,7 @@ if (APP_INSIGHTS_CONNECTION_STRING) { .start(); } -if (httpsProxyAgent && process.env.NODE_ENV === 'production') { +if (httpsProxyAgent && isProd) { // NOTES: // // - `defaultClient` property on `appInsights` doesn't exist @@ -52,13 +53,24 @@ const server = http.createServer(app); dotenv.load({ path: '.env' }); app.set('port', process.env.PORT || 4200); +const secretToken = process.env.SECRET_TOKEN; +if (!secretToken || secretToken === 'promiseyouwillchangeit') { + if (isProd) { +throw new Error('Please change your SECRET_TOKEN env'); + } else { +console.warn( + 'Remember to change your SECRET_TOKEN env before deploying to PROD' +); + } +} + app.use('/', express.static(path.join(__dirname, '../public'))); app.use(bodyParser.json()); app.use(bodyParser.urlencoded({ extended: true })); app.use( session({ store: SESSION_STORE, -secret: process.env.SECRET_TOKEN, +secret: secretToken, resave: true, saveUninitialized: true, cookie: { expires: new Date(2147483647000) },
(helix) branch master updated: Delete expected version (#2759)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new 0da9b4245 Delete expected version (#2759) 0da9b4245 is described below commit 0da9b42452b80db64e4698d091f21fe421086e5e Author: Grant Paláu Spencer AuthorDate: Tue Mar 19 15:51:29 2024 -0700 Delete expected version (#2759) Add delete with expected version API to zk client and data accessor --- .../java/org/apache/helix/BaseDataAccessor.java| 11 +++ .../helix/manager/zk/ZkBaseDataAccessor.java | 21 .../helix/manager/zk/ZkCacheBaseDataAccessor.java | 25 +++ .../helix/store/zk/AutoFallbackPropertyStore.java | 8 + .../helix/manager/zk/TestZkBaseDataAccessor.java | 37 ++ .../apache/helix/mock/MockBaseDataAccessor.java| 18 +++ .../zookeeper/api/client/RealmAwareZkClient.java | 2 ++ .../zookeeper/impl/client/DedicatedZkClient.java | 7 +++- .../zookeeper/impl/client/FederatedZkClient.java | 7 +++- .../zookeeper/impl/client/SharedZkClient.java | 7 +++- .../helix/zookeeper/zkclient/IZkConnection.java| 2 ++ .../apache/helix/zookeeper/zkclient/ZkClient.java | 14 +++- .../helix/zookeeper/zkclient/ZkConnection.java | 5 +++ .../client/RealmAwareZkClientFactoryTestBase.java | 24 ++ .../impl/client/TestFederatedZkClient.java | 26 ++- 15 files changed, 209 insertions(+), 5 deletions(-) diff --git a/helix-core/src/main/java/org/apache/helix/BaseDataAccessor.java b/helix-core/src/main/java/org/apache/helix/BaseDataAccessor.java index 162ffb05d..4fafb4271 100644 --- a/helix-core/src/main/java/org/apache/helix/BaseDataAccessor.java +++ b/helix-core/src/main/java/org/apache/helix/BaseDataAccessor.java @@ -112,6 +112,17 @@ public interface BaseDataAccessor { */ boolean remove(String path, int options); + /** + * This will remove the ZNode, if the ZNode's version matches the provided expectedVersion. This + * operation will fail if the node has any children. + * @param path Path to the ZNode to update + * @param options Set the type of ZNode see the valid values in {@link AccessOption} + * @param expectedVersion the expected version of the node to be removed, -1 means match any + * version + * @return true if the removal succeeded, false otherwise + */ + boolean removeWithExpectedVersion(String path, int options, int expectedVersion); + /** * Use it when creating children under a parent node. This will use async api for better * performance. If the child already exists it will return false. diff --git a/helix-core/src/main/java/org/apache/helix/manager/zk/ZkBaseDataAccessor.java b/helix-core/src/main/java/org/apache/helix/manager/zk/ZkBaseDataAccessor.java index e074a22e6..06ea97fb9 100644 --- a/helix-core/src/main/java/org/apache/helix/manager/zk/ZkBaseDataAccessor.java +++ b/helix-core/src/main/java/org/apache/helix/manager/zk/ZkBaseDataAccessor.java @@ -788,6 +788,27 @@ public class ZkBaseDataAccessor implements BaseDataAccessor { return true; } + /** + * Sync remove with expected version. It tries to remove the ZNode if the ZNode's version matches + * the provided expectedVersion. This operation will FAIL if the node has any children. Node does + * not exist is regarded as success. If expectedVersion is set to -1, then the ZNode version + * match is not enforced. + */ + @Override + public boolean removeWithExpectedVersion(String path, int options, int expectedVersion) { +try { + // operation will not throw exception when path successfully deleted or does not exist + // despite real error, operation will throw exception when path not empty, and in this + // case, we do NOT try to delete recursively + _zkClient.delete(path, expectedVersion); +} catch (ZkException zkException) { + LOG.error("Failed to delete {} with opts {} and expected version {}.", path, options, + expectedVersion, zkException); + return false; +} +return true; + } + /** * async create. give up on error other than NONODE */ diff --git a/helix-core/src/main/java/org/apache/helix/manager/zk/ZkCacheBaseDataAccessor.java b/helix-core/src/main/java/org/apache/helix/manager/zk/ZkCacheBaseDataAccessor.java index 6a635a5da..2be1a4f76 100644 --- a/helix-core/src/main/java/org/apache/helix/manager/zk/ZkCacheBaseDataAccessor.java +++ b/helix-core/src/main/java/org/apache/helix/manager/zk/ZkCacheBaseDataAccessor.java @@ -356,6 +356,31 @@ public class ZkCacheBaseDataAccessor implements HelixPropertyStore { return _baseAccessor.remove(serverPath, options); } + @Override + public boolean removeWithExpectedVersion(String path, int options, int expectedVersion) { +String clientP
(helix) branch master updated: [apache/helix] -- Fix PreferenceList Ordering Changes during Maintenance Mode (#2778)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new ce33735dd [apache/helix] -- Fix PreferenceList Ordering Changes during Maintenance Mode (#2778) ce33735dd is described below commit ce33735dda1dd8614cb8d84094119de1a4b30484 Author: Himanshu Kandwal AuthorDate: Tue Mar 19 15:49:55 2024 -0700 [apache/helix] -- Fix PreferenceList Ordering Changes during Maintenance Mode (#2778) Fixed PreferenceList Ordering Changes (indeterminism) during Maintenance Mode. --- .../rebalancer/MaintenanceRebalancer.java | 31 ++- .../rebalancer/TestMaintenanceRebalancer.java | 60 + ...MaintenanceRebalancer.ComputeNewIdealState.json | 98 ++ 3 files changed, 187 insertions(+), 2 deletions(-) diff --git a/helix-core/src/main/java/org/apache/helix/controller/rebalancer/MaintenanceRebalancer.java b/helix-core/src/main/java/org/apache/helix/controller/rebalancer/MaintenanceRebalancer.java index 9f04de535..9b5b4f929 100644 --- a/helix-core/src/main/java/org/apache/helix/controller/rebalancer/MaintenanceRebalancer.java +++ b/helix-core/src/main/java/org/apache/helix/controller/rebalancer/MaintenanceRebalancer.java @@ -28,6 +28,7 @@ import org.apache.helix.controller.dataproviders.ResourceControllerDataProvider; import org.apache.helix.controller.stages.CurrentStateOutput; import org.apache.helix.model.IdealState; import org.apache.helix.model.Partition; +import org.apache.helix.model.StateModelDefinition; import org.slf4j.Logger; import org.slf4j.LoggerFactory; @@ -55,12 +56,38 @@ public class MaintenanceRebalancer extends SemiAutoRebalancer OFFLINE and OFFLINE -> DROP state transitions. // Derived preference list from current state with state priority +StateModelDefinition stateModelDef = clusterData.getStateModelDef(currentIdealState.getStateModelDefRef()); + for (Partition partition : currentStateMap.keySet()) { Map stateMap = currentStateMap.get(partition); List preferenceList = new ArrayList<>(stateMap.keySet()); + + /** + * This sorting preserves the ordering of current state hosts in the order of current IS pref list + * Example: + * ideal state pref-list: [A, B, C] + * current-state: { + * A: FOLLOWER, + * B: LEADER, + * C: FOLLOWER + * } + * Lets say newPrefList = new ArrayList<>(current-state.keySet()) => [C, B, A] + * + * Sort 1: Sort based on preference-list order: + * + * newPrefList = [C, B, A] => [A, B, C] + */ Collections.sort(preferenceList, new PreferenceListNodeComparator(stateMap, - clusterData.getStateModelDef(currentIdealState.getStateModelDefRef()), - Collections.emptyList())); + stateModelDef, currentIdealState.getPreferenceList(partition.getPartitionName(; + + /** + * Sort 2: Sort based on state-priority order: + * + * newPrefList = [A, B, C] => [B, A, C] + * Here, A will be 2nd and C will be third always as both have same priority so original (pref-list) order will be maintained. + */ + preferenceList.sort(new StatePriorityComparator(stateMap, stateModelDef)); + currentIdealState.setPreferenceList(partition.getPartitionName(), preferenceList); } LOG.info(String diff --git a/helix-core/src/test/java/org/apache/helix/controller/rebalancer/TestMaintenanceRebalancer.java b/helix-core/src/test/java/org/apache/helix/controller/rebalancer/TestMaintenanceRebalancer.java new file mode 100644 index 0..50cdd418f --- /dev/null +++ b/helix-core/src/test/java/org/apache/helix/controller/rebalancer/TestMaintenanceRebalancer.java @@ -0,0 +1,60 @@ +package org.apache.helix.controller.rebalancer; + +import java.util.List; +import java.util.Map; + +import org.apache.helix.controller.dataproviders.ResourceControllerDataProvider; +import org.apache.helix.controller.stages.CurrentStateOutput; +import org.apache.helix.model.IdealState; +import org.apache.helix.model.MasterSlaveSMD; +import org.apache.helix.model.Partition; +import org.apache.helix.util.TestInputLoader; +import org.testng.Assert; +import org.testng.annotations.DataProvider; +import org.testng.annotations.Test; + +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; + +public class TestMaintenanceRebalancer { + + private static final String RESOURCE_NAME = "testResource"; + private static final String PARTITION_NAME = "testResourcePartition"; + + @Test(dataProvider = "TestComputeIdealStateInput") + public void testComputeIdealState(String comment, S
(helix) branch master updated: Fix BestPossibleExternalViewVerifier to use a ZkClient that has the serializer set to ByteArraySerializer (#2776)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new 7c4301b11 Fix BestPossibleExternalViewVerifier to use a ZkClient that has the serializer set to ByteArraySerializer (#2776) 7c4301b11 is described below commit 7c4301b115bbad44a09b199a84abcc2c238858a0 Author: Zachary Pinto AuthorDate: Wed Mar 13 20:00:32 2024 -0700 Fix BestPossibleExternalViewVerifier to use a ZkClient that has the serializer set to ByteArraySerializer (#2776) * Fix BestPossibleExternalViewVerifier to use a ZkClient that has the serializer set to ByteArraySerializer so it can read the assignment meta store best possible state. Fix BestPossibleExternalViewVerifier to actually calculate BEST_POSSIBLE instead of returning last persisted to ZK because we now need to consider handleDelayedRebalanceMinActiveReplica not being persisted to ZK(#2447). Fix handleDelayedRebalanceMinActiveReplica modifying in-memory _bestPossibleState in the _assignmen [...] --- .../rebalancer/waged/AssignmentManager.java| 8 ++- .../BestPossibleExternalViewVerifier.java | 27 +- .../rebalancer/TestInstanceOperation.java | 5 +--- 3 files changed, 19 insertions(+), 21 deletions(-) diff --git a/helix-core/src/main/java/org/apache/helix/controller/rebalancer/waged/AssignmentManager.java b/helix-core/src/main/java/org/apache/helix/controller/rebalancer/waged/AssignmentManager.java index 475e8aad1..8cb089cb9 100644 --- a/helix-core/src/main/java/org/apache/helix/controller/rebalancer/waged/AssignmentManager.java +++ b/helix-core/src/main/java/org/apache/helix/controller/rebalancer/waged/AssignmentManager.java @@ -23,6 +23,8 @@ import java.util.HashMap; import java.util.HashSet; import java.util.Map; import java.util.Set; +import java.util.stream.Collectors; + import org.apache.helix.HelixRebalanceException; import org.apache.helix.controller.stages.CurrentStateOutput; import org.apache.helix.model.ResourceAssignment; @@ -86,7 +88,11 @@ class AssignmentManager { if (assignmentMetadataStore != null) { try { _stateReadLatency.startMeasuringLatency(); -currentBestAssignment = new HashMap<>(assignmentMetadataStore.getBestPossibleAssignment()); +currentBestAssignment = + assignmentMetadataStore.getBestPossibleAssignment().entrySet().stream().collect( +Collectors.toMap(Map.Entry::getKey, +entry -> new ResourceAssignment(entry.getValue().getRecord(; +; _stateReadLatency.endMeasuringLatency(); } catch (Exception ex) { throw new HelixRebalanceException( diff --git a/helix-core/src/main/java/org/apache/helix/tools/ClusterVerifiers/BestPossibleExternalViewVerifier.java b/helix-core/src/main/java/org/apache/helix/tools/ClusterVerifiers/BestPossibleExternalViewVerifier.java index 1997bea06..0b0926dda 100644 --- a/helix-core/src/main/java/org/apache/helix/tools/ClusterVerifiers/BestPossibleExternalViewVerifier.java +++ b/helix-core/src/main/java/org/apache/helix/tools/ClusterVerifiers/BestPossibleExternalViewVerifier.java @@ -30,19 +30,16 @@ import java.util.Map; import java.util.Set; import org.apache.helix.HelixDefinedState; -import org.apache.helix.HelixRebalanceException; import org.apache.helix.PropertyKey; import org.apache.helix.controller.common.PartitionStateMap; import org.apache.helix.controller.dataproviders.ResourceControllerDataProvider; import org.apache.helix.controller.rebalancer.waged.ReadOnlyWagedRebalancer; -import org.apache.helix.controller.rebalancer.waged.RebalanceAlgorithm; import org.apache.helix.controller.stages.AttributeName; import org.apache.helix.controller.stages.BestPossibleStateCalcStage; import org.apache.helix.controller.stages.BestPossibleStateOutput; import org.apache.helix.controller.stages.ClusterEvent; import org.apache.helix.controller.stages.ClusterEventType; import org.apache.helix.controller.stages.CurrentStateComputationStage; -import org.apache.helix.controller.stages.CurrentStateOutput; import org.apache.helix.controller.stages.ResourceComputationStage; import org.apache.helix.manager.zk.ZkBucketDataAccessor; import org.apache.helix.model.ClusterConfig; @@ -50,7 +47,6 @@ import org.apache.helix.model.ExternalView; import org.apache.helix.model.IdealState; import org.apache.helix.model.Partition; import org.apache.helix.model.Resource; -import org.apache.helix.model.ResourceAssignment; import org.apache.helix.model.StateModelDefinition; import org.apache.helix.task.TaskConstants; import org.apache.helix.util.RebalanceUtil; @@ -59,8 +55,13 @@ import org.slf4j.Logger; import org.slf4j.LoggerFactory; /** - * verifier that the ExternalViews of given resources (or all resources in the cluster)
(helix) branch master updated: Mark HelixCustomeCodeRunner Deprecated (#2774)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new d1aa2fe61 Mark HelixCustomeCodeRunner Deprecated (#2774) d1aa2fe61 is described below commit d1aa2fe61cc03c98e79ea3ca87a5036c9a9639cf Author: Komal Desai AuthorDate: Tue Mar 12 12:09:56 2024 -0700 Mark HelixCustomeCodeRunner Deprecated (#2774) --- .../main/java/org/apache/helix/participant/HelixCustomCodeRunner.java| 1 + 1 file changed, 1 insertion(+) diff --git a/helix-core/src/main/java/org/apache/helix/participant/HelixCustomCodeRunner.java b/helix-core/src/main/java/org/apache/helix/participant/HelixCustomCodeRunner.java index 9afdef648..cfe5926f4 100644 --- a/helix-core/src/main/java/org/apache/helix/participant/HelixCustomCodeRunner.java +++ b/helix-core/src/main/java/org/apache/helix/participant/HelixCustomCodeRunner.java @@ -56,6 +56,7 @@ import org.slf4j.LoggerFactory; * .start() * */ +@Deprecated public class HelixCustomCodeRunner { private static final String LEADER_STANDBY = "LeaderStandby"; private static Logger LOG = LoggerFactory.getLogger(HelixCustomCodeRunner.class);
(helix) branch master updated: jackson-databind 2.12.7.1 (#2766)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new 4a77e86da jackson-databind 2.12.7.1 (#2766) 4a77e86da is described below commit 4a77e86dad31d6230fe46375b153525c2c186645 Author: PJ Fanning AuthorDate: Fri Mar 1 23:20:37 2024 +0100 jackson-databind 2.12.7.1 (#2766) --- helix-admin-webapp/pom.xml | 2 +- helix-core/pom.xml | 4 ++-- helix-rest/pom.xml | 4 ++-- helix-view-aggregator/pom.xml | 2 +- metadata-store-directory-common/pom.xml | 2 +- zookeeper-api/pom.xml | 2 +- 6 files changed, 8 insertions(+), 8 deletions(-) diff --git a/helix-admin-webapp/pom.xml b/helix-admin-webapp/pom.xml index c862d287f..4529b0807 100644 --- a/helix-admin-webapp/pom.xml +++ b/helix-admin-webapp/pom.xml @@ -100,7 +100,7 @@ com.fasterxml.jackson.core jackson-core - 2.12.6 + 2.12.7 commons-cli diff --git a/helix-core/pom.xml b/helix-core/pom.xml index 87869626f..d3934f579 100644 --- a/helix-core/pom.xml +++ b/helix-core/pom.xml @@ -86,12 +86,12 @@ com.fasterxml.jackson.core jackson-databind - 2.12.6.1 + 2.12.7.1 com.fasterxml.jackson.core jackson-core - 2.12.6 + 2.12.7 commons-io diff --git a/helix-rest/pom.xml b/helix-rest/pom.xml index 202f93b8f..0fe0963a3 100644 --- a/helix-rest/pom.xml +++ b/helix-rest/pom.xml @@ -126,12 +126,12 @@ com.fasterxml.jackson.core jackson-databind - 2.12.6.1 + 2.12.7.1 com.fasterxml.jackson.core jackson-annotations - 2.12.6 + 2.12.7 commons-cli diff --git a/helix-view-aggregator/pom.xml b/helix-view-aggregator/pom.xml index 9b9a5daba..bd7ff0e5b 100644 --- a/helix-view-aggregator/pom.xml +++ b/helix-view-aggregator/pom.xml @@ -84,7 +84,7 @@ under the License. com.fasterxml.jackson.core jackson-core - 2.12.6 + 2.12.7 commons-cli diff --git a/metadata-store-directory-common/pom.xml b/metadata-store-directory-common/pom.xml index 50ce7cceb..f173397cc 100644 --- a/metadata-store-directory-common/pom.xml +++ b/metadata-store-directory-common/pom.xml @@ -62,7 +62,7 @@ com.fasterxml.jackson.core jackson-databind - 2.12.6.1 + 2.12.7.1 org.slf4j diff --git a/zookeeper-api/pom.xml b/zookeeper-api/pom.xml index dd5fd065c..d44160fdb 100644 --- a/zookeeper-api/pom.xml +++ b/zookeeper-api/pom.xml @@ -88,7 +88,7 @@ com.fasterxml.jackson.core jackson-databind - 2.12.6.1 + 2.12.7.1 commons-codec
(helix) branch master updated: Fix flaky tests (#2645) (#2666)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new 7ba774fa2 Fix flaky tests (#2645) (#2666) 7ba774fa2 is described below commit 7ba774fa264cdf929791f830563542c6b3df4674 Author: Simon Hofbauer <61789910+ho...@users.noreply.github.com> AuthorDate: Tue Feb 27 19:14:31 2024 +0100 Fix flaky tests (#2645) (#2666) in org.apache.helix.rest.metadatastore. TestZkMetadataStoreDirectory Co-authored-by: simonh5 --- .../rest/metadatastore/TestZkMetadataStoreDirectory.java | 12 +--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/helix-rest/src/test/java/org/apache/helix/rest/metadatastore/TestZkMetadataStoreDirectory.java b/helix-rest/src/test/java/org/apache/helix/rest/metadatastore/TestZkMetadataStoreDirectory.java index 64e04df9f..38ddfba87 100644 --- a/helix-rest/src/test/java/org/apache/helix/rest/metadatastore/TestZkMetadataStoreDirectory.java +++ b/helix-rest/src/test/java/org/apache/helix/rest/metadatastore/TestZkMetadataStoreDirectory.java @@ -21,6 +21,7 @@ package org.apache.helix.rest.metadatastore; import java.util.ArrayList; import java.util.Arrays; +import java.util.Collection; import java.util.HashMap; import java.util.HashSet; import java.util.LinkedHashMap; @@ -31,7 +32,6 @@ import java.util.Set; import org.apache.helix.TestHelper; import org.apache.helix.msdcommon.constant.MetadataStoreRoutingConstants; -import org.apache.helix.msdcommon.exception.InvalidRoutingDataException; import org.apache.helix.rest.server.AbstractTestClass; import org.apache.helix.zookeeper.datamodel.ZNRecord; import org.apache.helix.zookeeper.datamodel.serializer.ZNRecordSerializer; @@ -136,7 +136,7 @@ public class TestZkMetadataStoreDirectory extends AbstractTestClass { realms.add(TEST_REALM_2); for (String namespace : _routingZkAddrMap.keySet()) { - Assert.assertEquals(_metadataStoreDirectory.getAllMetadataStoreRealms(namespace), realms); + assertCollectionsContainSameElementsIgnoringOrder(_metadataStoreDirectory.getAllMetadataStoreRealms(namespace), realms); } } @@ -147,7 +147,7 @@ public class TestZkMetadataStoreDirectory extends AbstractTestClass { allShardingKeys.addAll(TEST_SHARDING_KEYS_2); for (String namespace : _routingZkAddrMap.keySet()) { - Assert.assertEquals(_metadataStoreDirectory.getAllShardingKeys(namespace), allShardingKeys); + assertCollectionsContainSameElementsIgnoringOrder(_metadataStoreDirectory.getAllShardingKeys(namespace),allShardingKeys); } } @@ -375,4 +375,10 @@ public class TestZkMetadataStoreDirectory extends AbstractTestClass { return true; }, TestHelper.WAIT_DURATION), "Routing data path should be deleted after the tests."); } + private void assertCollectionsContainSameElementsIgnoringOrder(Collection collection1, + Collection collection2) { + Assert.assertEquals(collection2.size(), collection1.size()); + Assert.assertTrue(collection2.containsAll(collection1)); + Assert.assertTrue(collection1.containsAll(collection2)); + } }
(helix) branch master updated: Fix flaky test assertions (#2640) (#2663)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new 448e7a0cf Fix flaky test assertions (#2640) (#2663) 448e7a0cf is described below commit 448e7a0cf8c608178cab0b422c8594c6e3399d9f Author: Simon Hofbauer <61789910+ho...@users.noreply.github.com> AuthorDate: Tue Feb 27 18:32:26 2024 +0100 Fix flaky test assertions (#2640) (#2663) in test TestResourceAssignmentOptimizerAccessor #testComputePartitionAssignment Co-authored-by: simonh5 --- helix-rest/pom.xml | 6 .../TestResourceAssignmentOptimizerAccessor.java | 40 +++--- 2 files changed, 19 insertions(+), 27 deletions(-) diff --git a/helix-rest/pom.xml b/helix-rest/pom.xml index d9257cf03..202f93b8f 100644 --- a/helix-rest/pom.xml +++ b/helix-rest/pom.xml @@ -184,6 +184,12 @@ swagger-models 1.6.4 + + org.skyscreamer + jsonassert + 1.5.1 + test + diff --git a/helix-rest/src/test/java/org/apache/helix/rest/server/TestResourceAssignmentOptimizerAccessor.java b/helix-rest/src/test/java/org/apache/helix/rest/server/TestResourceAssignmentOptimizerAccessor.java index ad4142c67..899256619 100644 --- a/helix-rest/src/test/java/org/apache/helix/rest/server/TestResourceAssignmentOptimizerAccessor.java +++ b/helix-rest/src/test/java/org/apache/helix/rest/server/TestResourceAssignmentOptimizerAccessor.java @@ -38,6 +38,8 @@ import org.apache.helix.manager.zk.ZKHelixDataAccessor; import org.apache.helix.model.IdealState; import org.apache.helix.model.InstanceConfig; import org.apache.helix.rest.server.resources.helix.ResourceAssignmentOptimizerAccessor; +import org.json.JSONException; +import org.skyscreamer.jsonassert.JSONAssert; import org.testng.Assert; import org.testng.annotations.AfterClass; import org.testng.annotations.BeforeClass; @@ -99,7 +101,7 @@ public class TestResourceAssignmentOptimizerAccessor extends AbstractTestClass { } @Test - public void testComputePartitionAssignment() throws IOException { + public void testComputePartitionAssignment() throws IOException, JSONException { System.out.println("Start test :" + TestHelper.getTestMethodName()); // Test AddInstances, RemoveInstances and SwapInstances @@ -121,8 +123,7 @@ public class TestResourceAssignmentOptimizerAccessor extends AbstractTestClass { Assert.assertTrue(headers.containsKey(ResourceAssignmentOptimizerAccessor.RESPONSE_HEADER_KEY)); Assert.assertFalse( headers.get(ResourceAssignmentOptimizerAccessor.RESPONSE_HEADER_KEY).isEmpty()); - Assert.assertEquals(headers.get(ResourceAssignmentOptimizerAccessor.RESPONSE_HEADER_KEY).get(0), -"{instanceFilter=[], resourceFilter=[], returnFormat=IdealStateFormat}"); +JSONAssert.assertEquals((String) headers.get(ResourceAssignmentOptimizerAccessor.RESPONSE_HEADER_KEY).get(0), "{instanceFilter=[], resourceFilter=[], returnFormat=IdealStateFormat}", false); // Test partitionAssignment InstanceFilter String payload2 = "{\"Options\" : { \"InstanceFilter\" : [\"" + liveInstances.get(0) + "\" , \"" @@ -146,14 +147,10 @@ public class TestResourceAssignmentOptimizerAccessor extends AbstractTestClass { headers2.get(ResourceAssignmentOptimizerAccessor.RESPONSE_HEADER_KEY); Assert.assertFalse( headers2.get(ResourceAssignmentOptimizerAccessor.RESPONSE_HEADER_KEY).isEmpty()); -Assert.assertTrue( -partitionAssignmentMetadata2.get(0).equals( -"{instanceFilter=[" + liveInstances.get(0) + ", " + liveInstances.get(1) -+ "], resourceFilter=[], returnFormat=IdealStateFormat}") || -partitionAssignmentMetadata2.get(0).equals( -"{instanceFilter=[" + liveInstances.get(1) + ", " + liveInstances.get(0) -+ "], resourceFilter=[], returnFormat=IdealStateFormat}"), -partitionAssignmentMetadata2.get(0).toString()); +JSONAssert.assertEquals("{instanceFilter=[" + liveInstances.get(0) + ", " + liveInstances.get(1) + "], " ++ "resourceFilter=[], " + "returnFormat=IdealStateFormat}", partitionAssignmentMetadata2.get(0).toString(), false); +JSONAssert.assertEquals("{instanceFilter=[" + liveInstances.get(0) + ", " + liveInstances.get(1) + "], " ++ "resourceFilter=[], " + "returnFormat=IdealStateFormat}", ((String) partitionAssignmentMetadata2.get(0)), false); // Test partitionAssignment ResourceFilter String payload3 = @@ -177,14 +174,8 @@ public class TestResour
(helix) branch master updated: Fix flaky test assertion #2644 (#2665)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new ebda6a4c4 Fix flaky test assertion #2644 (#2665) ebda6a4c4 is described below commit ebda6a4c4c3a6103f3f7a8b080afeba17dd3d9fe Author: Simon Hofbauer <61789910+ho...@users.noreply.github.com> AuthorDate: Tue Feb 27 18:33:05 2024 +0100 Fix flaky test assertion #2644 (#2665) in org.apache.helix.rest.server. TestInstancesAccessor#testGetAllInstances Co-authored-by: simonh5 --- .../java/org/apache/helix/rest/server/TestInstancesAccessor.java | 8 ++-- 1 file changed, 6 insertions(+), 2 deletions(-) diff --git a/helix-rest/src/test/java/org/apache/helix/rest/server/TestInstancesAccessor.java b/helix-rest/src/test/java/org/apache/helix/rest/server/TestInstancesAccessor.java index 5cfab76a0..93722f05a 100644 --- a/helix-rest/src/test/java/org/apache/helix/rest/server/TestInstancesAccessor.java +++ b/helix-rest/src/test/java/org/apache/helix/rest/server/TestInstancesAccessor.java @@ -405,8 +405,12 @@ public class TestInstancesAccessor extends AbstractTestClass { Set instances = OBJECT_MAPPER.readValue(instancesStr, OBJECT_MAPPER.getTypeFactory().constructCollectionType(Set.class, String.class)); -Assert.assertEquals(instances, _instancesMap.get(CLUSTER_NAME), "Instances from response: " -+ instances + " vs instances actually: " + _instancesMap.get(CLUSTER_NAME)); +Assert.assertEquals(instances.size(), _instancesMap.get(CLUSTER_NAME).size(), "Different amount of elements in " ++ "the sets: " + instances.size() + " vs: " + _instancesMap.get(CLUSTER_NAME).size()); +Assert.assertTrue(instances.containsAll(_instancesMap.get(CLUSTER_NAME)), "instances set does not contain all " ++ "elements of _instanceMap"); +Assert.assertTrue(_instancesMap.get(CLUSTER_NAME).containsAll(instances), "_instanceMap set does not contain all " ++ "elements of instances"); System.out.println("End test :" + TestHelper.getTestMethodName()); }
(helix) branch master updated: Fix flaky test (#2643) (#2664)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new 1eea25388 Fix flaky test (#2643) (#2664) 1eea25388 is described below commit 1eea2538859c2ba572eeb2f51cd94ab91bea61bd Author: Simon Hofbauer <61789910+ho...@users.noreply.github.com> AuthorDate: Tue Feb 27 18:32:39 2024 +0100 Fix flaky test (#2643) (#2664) in org.apache.helix.rest.server. TestResourceAccessor#testGetResources Co-authored-by: simonh5 --- .../java/org/apache/helix/rest/server/TestResourceAccessor.java | 6 -- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/helix-rest/src/test/java/org/apache/helix/rest/server/TestResourceAccessor.java b/helix-rest/src/test/java/org/apache/helix/rest/server/TestResourceAccessor.java index 683bc0b1c..2bb91d416 100644 --- a/helix-rest/src/test/java/org/apache/helix/rest/server/TestResourceAccessor.java +++ b/helix-rest/src/test/java/org/apache/helix/rest/server/TestResourceAccessor.java @@ -75,8 +75,10 @@ public class TestResourceAccessor extends AbstractTestClass { Set resources = OBJECT_MAPPER.readValue(idealStates, OBJECT_MAPPER.getTypeFactory().constructCollectionType(Set.class, String.class)); -Assert.assertEquals(resources, _resourcesMap.get("TestCluster_0"), "Resources from response: " -+ resources + " vs clusters actually: " + _resourcesMap.get("TestCluster_0")); +Assert.assertTrue(resources.size() == _resourcesMap.get("TestCluster_0").size() +&& resources.containsAll(_resourcesMap.get("TestCluster_0")) +&& _resourcesMap.get("TestCluster_0").containsAll(resources), +"Sets are not equal. Resources from response: " + resources + " vs clusters actually: " + _resourcesMap.get("TestCluster_0")); System.out.println("End test :" + TestHelper.getTestMethodName()); }
(helix) branch master updated: fix flaky test assertion #2667 (#2682)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new 442be1122 fix flaky test assertion #2667 (#2682) 442be1122 is described below commit 442be1122d68b6d356e3f918b7bdd34d465bf94b Author: Simon Hofbauer <61789910+ho...@users.noreply.github.com> AuthorDate: Tue Feb 27 18:32:08 2024 +0100 fix flaky test assertion #2667 (#2682) fix flaky test assertion in org.apache.helix .rest.server.TestPerInstanceAccessor.testGetAllMessages Co-authored-by: simonh5 --- .../java/org/apache/helix/rest/server/TestPerInstanceAccessor.java | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/helix-rest/src/test/java/org/apache/helix/rest/server/TestPerInstanceAccessor.java b/helix-rest/src/test/java/org/apache/helix/rest/server/TestPerInstanceAccessor.java index 355d5d101..f9e48ca1f 100644 --- a/helix-rest/src/test/java/org/apache/helix/rest/server/TestPerInstanceAccessor.java +++ b/helix-rest/src/test/java/org/apache/helix/rest/server/TestPerInstanceAccessor.java @@ -331,8 +331,11 @@ public class TestPerInstanceAccessor extends AbstractTestClass { Set instances = OBJECT_MAPPER.readValue(instancesStr, OBJECT_MAPPER.getTypeFactory().constructCollectionType(Set.class, String.class)); -Assert.assertEquals(instances, _instancesMap.get(CLUSTER_NAME), "Instances from response: " -+ instances + " vs instances actually: " + _instancesMap.get(CLUSTER_NAME)); +String errorMessage = "Instances from response: "+ instances + " vs instances actually: " ++ _instancesMap.get(CLUSTER_NAME); +Assert.assertEquals(instances.size(), _instancesMap.get(CLUSTER_NAME).size(), errorMessage); +Assert.assertTrue(instances.containsAll(_instancesMap.get(CLUSTER_NAME)), errorMessage); +Assert.assertTrue(_instancesMap.get(CLUSTER_NAME).containsAll(instances), errorMessage); System.out.println("End test :" + TestHelper.getTestMethodName()); }
(helix) branch master updated: Bump es5-ext from 0.10.62 to 0.10.63 in /helix-front (#2768)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new e8490f9b5 Bump es5-ext from 0.10.62 to 0.10.63 in /helix-front (#2768) e8490f9b5 is described below commit e8490f9b580b046b58d50d9e2bf80391286ef0f6 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> AuthorDate: Tue Feb 27 09:31:14 2024 -0800 Bump es5-ext from 0.10.62 to 0.10.63 in /helix-front (#2768) Bumps [es5-ext](https://github.com/medikoo/es5-ext) from 0.10.62 to 0.10.63. - [Release notes](https://github.com/medikoo/es5-ext/releases) - [Changelog](https://github.com/medikoo/es5-ext/blob/main/CHANGELOG.md) - [Commits](https://github.com/medikoo/es5-ext/compare/v0.10.62...v0.10.63) --- updated-dependencies: - dependency-name: es5-ext dependency-type: indirect ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- helix-front/yarn.lock | 17 ++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/helix-front/yarn.lock b/helix-front/yarn.lock index 1ed04b7f8..7e7fd2227 100644 --- a/helix-front/yarn.lock +++ b/helix-front/yarn.lock @@ -7131,12 +7131,13 @@ es-to-primitive@^1.2.1: is-symbol "^1.0.2" es5-ext@^0.10.35, es5-ext@^0.10.50, es5-ext@^0.10.62, es5-ext@~0.10.14: - version "0.10.62" - resolved "https://registry.yarnpkg.com/es5-ext/-/es5-ext-0.10.62.tgz#5e6adc19a6da524bf3d1e02bbc8960e5eb49a9a5; - integrity sha512-BHLqn0klhEpnOKSrzn/Xsz2UIW8j+cGmo9JLzr8BiUapV8hPL9+FliFqjwr9ngW7jWdnxv6eO+/LqyhJVqgrjA== + version "0.10.63" + resolved "https://registry.yarnpkg.com/es5-ext/-/es5-ext-0.10.63.tgz#9c222a63b6a332ac80b1e373b426af723b895bd6; + integrity sha512-hUCZd2Byj/mNKjfP9jXrdVZ62B8KuA/VoK7X8nUh5qT+AxDmcbvZz041oDVZdbIN1qW6XY9VDNwzkvKnZvK2TQ== dependencies: es6-iterator "^2.0.3" es6-symbol "^3.1.3" +esniff "^2.0.1" next-tick "^1.1.0" es5-shim@^4.5.13: @@ -7762,6 +7763,16 @@ eslint@^8.12.0: strip-json-comments "^3.1.0" text-table "^0.2.0" +esniff@^2.0.1: + version "2.0.1" + resolved "https://registry.yarnpkg.com/esniff/-/esniff-2.0.1.tgz#a4d4b43a5c71c7ec51c51098c1d8a29081f9b308; + integrity sha512-kTUIGKQ/mDPFoJ0oVfcmyJn4iBDRptjNVIzwIFR7tqWXdVI9xfA2RMwY/gbSpJG3lkdWNEjLap/NqVHZiJsdfg== + dependencies: +d "^1.0.1" +es5-ext "^0.10.62" +event-emitter "^0.3.5" +type "^2.7.2" + espree@^9.4.0: version "9.4.0" resolved "https://registry.yarnpkg.com/espree/-/espree-9.4.0.tgz#cd4bc3d6e9336c433265fc0aa016fc1aaf182f8a;
(helix) branch master updated: Expose addListener and support listening on single instance config (#2752)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new 6bdc88804 Expose addListener and support listening on single instance config (#2752) 6bdc88804 is described below commit 6bdc888042daefa7e75d93df44270db6d4a26849 Author: Zachary Pinto AuthorDate: Mon Feb 26 17:47:22 2024 -0800 Expose addListener and support listening on single instance config (#2752) Expose addListener method in HelixManager to allow users to register any listeners that implement one of the Listener interfaces. Add support in CallbackHandler for InstanceConfigChangeListener to be used on a single instance config. --- .../main/java/org/apache/helix/HelixManager.java | 14 +++- .../apache/helix/manager/zk/CallbackHandler.java | 10 ++- .../apache/helix/manager/zk/ZKHelixManager.java| 3 +- .../event/MockCloudEventAwareHelixManager.java | 7 ++ .../controller/stages/DummyClusterManager.java | 7 ++ .../rebalancer/TestInstanceOperation.java | 74 ++ .../java/org/apache/helix/mock/MockManager.java| 8 ++- .../helix/participant/MockZKHelixManager.java | 7 ++ 8 files changed, 113 insertions(+), 17 deletions(-) diff --git a/helix-core/src/main/java/org/apache/helix/HelixManager.java b/helix-core/src/main/java/org/apache/helix/HelixManager.java index b5c2a6448..c1d2ad18c 100644 --- a/helix-core/src/main/java/org/apache/helix/HelixManager.java +++ b/helix-core/src/main/java/org/apache/helix/HelixManager.java @@ -51,7 +51,7 @@ import org.apache.helix.participant.StateMachineEngine; import org.apache.helix.spectator.RoutingTableProvider; import org.apache.helix.store.zk.ZkHelixPropertyStore; import org.apache.helix.zookeeper.datamodel.ZNRecord; - +import org.apache.zookeeper.Watcher; /** * Class that represents the Helix Agent. @@ -108,6 +108,18 @@ public interface HelixManager { */ void disconnect(); + /** + * Add a change listener on the specified propertyKey for the specified + * changeType and eventTypes. + * @see org.apache.helix.api.listeners for the list of available listeners + * @param listener the listener to add + * @param propertyKey the property key to listen to + * @param changeType the type of change to listen to + * @param eventType the event type to listen for + */ + void addListener(Object listener, PropertyKey propertyKey, HelixConstants.ChangeType changeType, + Watcher.Event.EventType[] eventType); + /** * @see IdealStateChangeListener#onIdealStateChange(List, NotificationContext) * @param listener diff --git a/helix-core/src/main/java/org/apache/helix/manager/zk/CallbackHandler.java b/helix-core/src/main/java/org/apache/helix/manager/zk/CallbackHandler.java index 2a19a7a4e..c2809620e 100644 --- a/helix-core/src/main/java/org/apache/helix/manager/zk/CallbackHandler.java +++ b/helix-core/src/main/java/org/apache/helix/manager/zk/CallbackHandler.java @@ -358,7 +358,15 @@ public class CallbackHandler implements IZkChildListener, IZkDataListener { configChangeListener.onConfigChange(configs, changeContext); } else if (_listener instanceof InstanceConfigChangeListener) { InstanceConfigChangeListener listener = (InstanceConfigChangeListener) _listener; - List configs = preFetch(_propertyKey); + List configs = Collections.emptyList(); + if (_propertyKey.getParams().length > 2 && _preFetchEnabled) { +// If there are more than 2 params, that means the property key is for a specific instance +// and will not have children. +InstanceConfig config = _accessor.getProperty(_propertyKey); +configs = config != null ? Collections.singletonList(config) : Collections.emptyList(); + } else { +configs = preFetch(_propertyKey); + } listener.onInstanceConfigChange(configs, changeContext); } } else if (_changeType == RESOURCE_CONFIG) { diff --git a/helix-core/src/main/java/org/apache/helix/manager/zk/ZKHelixManager.java b/helix-core/src/main/java/org/apache/helix/manager/zk/ZKHelixManager.java index 7a62d1923..3473bed08 100644 --- a/helix-core/src/main/java/org/apache/helix/manager/zk/ZKHelixManager.java +++ b/helix-core/src/main/java/org/apache/helix/manager/zk/ZKHelixManager.java @@ -419,7 +419,8 @@ public class ZKHelixManager implements HelixManager, IZkStateListener { } } - void addListener(Object listener, PropertyKey propertyKey, ChangeType changeType, + @Override + public void addListener(Object listener, PropertyKey propertyKey, ChangeType changeType, EventType[] eventType) { checkConnected(_waitForConnectedTimeout); diff --git a/helix-core/src/test/java/org/apache/helix/cloud/event/MockCloudEventAwareHelixM
(helix) branch master updated: Fix TestClusterAccessor test (#2767)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new c0fa6dca1 Fix TestClusterAccessor test (#2767) c0fa6dca1 is described below commit c0fa6dca1df11f63521bdd0f1ed5c3a5f969c813 Author: Junkai Xue AuthorDate: Mon Feb 26 17:45:36 2024 -0800 Fix TestClusterAccessor test (#2767) * Fix TestClusterAccessor * change format --- .../java/org/apache/helix/rest/server/TestClusterAccessor.java | 7 ++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/helix-rest/src/test/java/org/apache/helix/rest/server/TestClusterAccessor.java b/helix-rest/src/test/java/org/apache/helix/rest/server/TestClusterAccessor.java index d10ef9781..e882cf0ce 100644 --- a/helix-rest/src/test/java/org/apache/helix/rest/server/TestClusterAccessor.java +++ b/helix-rest/src/test/java/org/apache/helix/rest/server/TestClusterAccessor.java @@ -99,7 +99,7 @@ public class TestClusterAccessor extends AbstractTestClass { Set clusters = OBJECT_MAPPER.readValue(clustersStr, OBJECT_MAPPER.getTypeFactory().constructCollectionType(Set.class, String.class)); -Assert.assertEquals(clusters, _clusters); +Assert.assertTrue(isSame(clusters, _clusters)); validateAuditLogSize(1); AuditLog auditLog = _auditLogger.getAuditLogs().get(0); @@ -1528,6 +1528,11 @@ public class TestClusterAccessor extends AbstractTestClass { Response.Status.OK.getStatusCode(), null); } + private boolean isSame(Set result, Set expected) { +return result.size() == expected.size() && result.containsAll(expected) && expected.containsAll( +result); + } + private void validateAuditLogSize(int expected) { Assert.assertEquals(_auditLogger.getAuditLogs().size(), expected, "AuditLog:" + _auditLogger.getAuditLogs().toString());
(helix) branch master updated: upgrade xstream to 1.4.20 to pick up fixes for 2 CVEs (#2763)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new 53b588982 upgrade xstream to 1.4.20 to pick up fixes for 2 CVEs (#2763) 53b588982 is described below commit 53b588982c281009085076b480a3ce58cccbe618 Author: PJ Fanning AuthorDate: Mon Feb 26 19:55:27 2024 +0100 upgrade xstream to 1.4.20 to pick up fixes for 2 CVEs (#2763) --- helix-admin-webapp/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/helix-admin-webapp/pom.xml b/helix-admin-webapp/pom.xml index c4e9ea135..c862d287f 100644 --- a/helix-admin-webapp/pom.xml +++ b/helix-admin-webapp/pom.xml @@ -90,7 +90,7 @@ com.thoughtworks.xstream xstream - 1.4.19 + 1.4.20 com.fasterxml.jackson.core
(helix) branch master updated: Fix build and disable helix-front build
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new 710715271 Fix build and disable helix-front build 710715271 is described below commit 710715271b626fb5d9b01aeb31e15b4e8d8f45e4 Author: Junkai Xue AuthorDate: Fri Feb 23 14:25:56 2024 -0800 Fix build and disable helix-front build --- pom.xml| 2 +- recipes/rabbitmq-consumer-group/pom.xml| 2 +- .../helix/recipes/rabbitmq/ConsumerThread.java | 37 +++--- 3 files changed, 27 insertions(+), 14 deletions(-) diff --git a/pom.xml b/pom.xml index 55a356530..1827b8e1e 100644 --- a/pom.xml +++ b/pom.xml @@ -316,7 +316,7 @@ helix-rest helix-lock helix-agent -helix-front + recipes helix-view-aggregator meta-client diff --git a/recipes/rabbitmq-consumer-group/pom.xml b/recipes/rabbitmq-consumer-group/pom.xml index d87467c43..4ebf02f97 100644 --- a/recipes/rabbitmq-consumer-group/pom.xml +++ b/recipes/rabbitmq-consumer-group/pom.xml @@ -84,7 +84,7 @@ com.rabbitmq amqp-client - 5.18.0 + 5.20.0 com.101tec diff --git a/recipes/rabbitmq-consumer-group/src/main/java/org/apache/helix/recipes/rabbitmq/ConsumerThread.java b/recipes/rabbitmq-consumer-group/src/main/java/org/apache/helix/recipes/rabbitmq/ConsumerThread.java index f2a5a2e47..1c61b4c92 100644 --- a/recipes/rabbitmq-consumer-group/src/main/java/org/apache/helix/recipes/rabbitmq/ConsumerThread.java +++ b/recipes/rabbitmq-consumer-group/src/main/java/org/apache/helix/recipes/rabbitmq/ConsumerThread.java @@ -21,10 +21,14 @@ package org.apache.helix.recipes.rabbitmq; import java.io.IOException; +import com.rabbitmq.client.AMQP; import com.rabbitmq.client.Channel; import com.rabbitmq.client.Connection; import com.rabbitmq.client.ConnectionFactory; -import com.rabbitmq.client.QueueingConsumer; +import com.rabbitmq.client.Consumer; +import com.rabbitmq.client.DefaultConsumer; +import com.rabbitmq.client.Delivery; +import com.rabbitmq.client.Envelope; public class ConsumerThread extends Thread { private static final String EXCHANGE_NAME = "topic_logs"; @@ -56,19 +60,14 @@ public class ConsumerThread extends Thread { System.out.println(" [*] " + _consumerId + " Waiting for messages on " + bindingKey + ". To exit press CTRL+C"); - QueueingConsumer consumer = new QueueingConsumer(channel); + Consumer consumer = new MyConsumer(channel); channel.basicConsume(queueName, true, consumer); - while (true) { -QueueingConsumer.Delivery delivery = consumer.nextDelivery(); -String message = new String(delivery.getBody()); -String routingKey = delivery.getEnvelope().getRoutingKey(); - -System.out.println(" [x] " + _consumerId + " Received '" + routingKey + "':'" + message -+ "'"); - } -} catch (InterruptedException e) { - System.err.println(" [-] " + _consumerId + " on " + _partition + " is interrupted ..."); + String consumerTag = channel.basicConsume(queueName, false, consumer); + System.out.println("press any key to terminate"); + System.in.read(); + channel.basicCancel(consumerTag); + channel.close(); } catch (Exception e) { e.printStackTrace(); } finally { @@ -82,4 +81,18 @@ public class ConsumerThread extends Thread { } } } + + private static class MyConsumer extends DefaultConsumer { + +public MyConsumer(Channel channel) { + super(channel); +} + +@Override +public void handleDelivery(String consumerTag, Envelope envelope, +AMQP.BasicProperties properties, byte[] body) { + System.out.println( + " [x] Received '" + envelope.getRoutingKey() + "':'" + new String(body) + "'"); +} + } }
(helix) branch master updated: force datattable scroll width width to 100% (#2652)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new 1a4da73a7 force datattable scroll width width to 100% (#2652) 1a4da73a7 is described below commit 1a4da73a7449be44810f3f5342bcb093b2bc86d8 Author: Grant Paláu Spencer AuthorDate: Thu Feb 22 15:50:34 2024 -0800 force datattable scroll width width to 100% (#2652) Force resource view to use 100% parent container width --- helix-front/src/styles.scss | 4 1 file changed, 4 insertions(+) diff --git a/helix-front/src/styles.scss b/helix-front/src/styles.scss index 32836f635..9d84945ce 100644 --- a/helix-front/src/styles.scss +++ b/helix-front/src/styles.scss @@ -37,6 +37,10 @@ // } } +.datatable-scroll { + width: 100% !important; +} + body { font-family: Roboto, Arial, sans-serif; margin: 0;
(helix) branch master updated: Bump guava from 31.1-jre to 32.0.0-jre in /helix-core (#2537)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new cd1607036 Bump guava from 31.1-jre to 32.0.0-jre in /helix-core (#2537) cd1607036 is described below commit cd1607036578fa819d3023bafe7cae2fea332a13 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> AuthorDate: Thu Feb 22 15:50:13 2024 -0800 Bump guava from 31.1-jre to 32.0.0-jre in /helix-core (#2537) Bumps [guava](https://github.com/google/guava) from 31.1-jre to 32.0.0-jre. - [Release notes](https://github.com/google/guava/releases) - [Commits](https://github.com/google/guava/commits) --- updated-dependencies: - dependency-name: com.google.guava:guava dependency-type: direct:production ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- helix-core/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/helix-core/pom.xml b/helix-core/pom.xml index 0315e1448..87869626f 100644 --- a/helix-core/pom.xml +++ b/helix-core/pom.xml @@ -137,7 +137,7 @@ com.google.guava guava - 31.1-jre + 32.0.0-jre org.yaml
(helix) branch master updated: Bump ip from 1.1.8 to 1.1.9 in /helix-front (#2762)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new 3a3deb44a Bump ip from 1.1.8 to 1.1.9 in /helix-front (#2762) 3a3deb44a is described below commit 3a3deb44ac21d9ed2bc0e066588cd6aec71508af Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> AuthorDate: Thu Feb 22 15:49:42 2024 -0800 Bump ip from 1.1.8 to 1.1.9 in /helix-front (#2762) Bumps [ip](https://github.com/indutny/node-ip) from 1.1.8 to 1.1.9. - [Commits](https://github.com/indutny/node-ip/compare/v1.1.8...v1.1.9) --- updated-dependencies: - dependency-name: ip dependency-type: indirect ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- helix-front/yarn.lock | 12 ++-- 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/helix-front/yarn.lock b/helix-front/yarn.lock index 3f09cba0d..1ed04b7f8 100644 --- a/helix-front/yarn.lock +++ b/helix-front/yarn.lock @@ -9463,14 +9463,14 @@ interpret@^2.2.0: integrity sha512-Ju0Bz/cEia55xDwUWEa8+olFpCiQoypjnQySseKtmjNrnps3P+xfpUmGr90T7yjlVJmOtybRvPXhKMbHr+fWnw== ip@^1.1.0, ip@^1.1.5: - version "1.1.8" - resolved "https://registry.yarnpkg.com/ip/-/ip-1.1.8.tgz#ae05948f6b075435ed3307acce04629da8cdbf48; - integrity sha512-PuExPYUiu6qMBQb4l06ecm6T6ujzhmh+MeJcW9wa89PoAz5pvd4zPgN5WJV104mb6S2T1AwNIAaB70JNrLQWhg== + version "1.1.9" + resolved "https://registry.yarnpkg.com/ip/-/ip-1.1.9.tgz#8dfbcc99a754d07f425310b86a99546b1151e396; + integrity sha512-cyRxvOEpNHNtchU3Ln9KC/auJgup87llfQpQ+t5ghoC/UhL16SWzbueiCsdTnWmqAWl7LadfuwhlqmtOaqMHdQ== ip@^2.0.0: - version "2.0.0" - resolved "https://registry.yarnpkg.com/ip/-/ip-2.0.0.tgz#4cf4ab182fee2314c75ede1276f8c80b479936da; - integrity sha512-WKa+XuLG1A1R0UWhl2+1XQSi+fZWMsYKffMZTTYsiZaUD8k2yDAj5atimTUD2TZkyCkNEeYE5NhFZmupOGtjYQ== + version "2.0.1" + resolved "https://registry.yarnpkg.com/ip/-/ip-2.0.1.tgz#e8f3595d33a3ea66490204234b77636965307105; + integrity sha512-lJUL9imLTNi1ZfXT+DU6rBBdbiKGBuay9B6xGSPVjUeQwaH1RIGqef8RZkUtHioLmSNpPR5M4HVKJGm1j8FWVQ== ipaddr.js@1.9.1: version "1.9.1"
(helix) branch master updated: Bump com.rabbitmq:amqp-client in /recipes/rabbitmq-consumer-group (#2679)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new 3962e490f Bump com.rabbitmq:amqp-client in /recipes/rabbitmq-consumer-group (#2679) 3962e490f is described below commit 3962e490fe64a288cd03475a38d1f6ce829d465a Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> AuthorDate: Thu Feb 22 15:49:25 2024 -0800 Bump com.rabbitmq:amqp-client in /recipes/rabbitmq-consumer-group (#2679) Bumps [com.rabbitmq:amqp-client](https://github.com/rabbitmq/rabbitmq-java-client) from 4.12.0 to 5.18.0. - [Release notes](https://github.com/rabbitmq/rabbitmq-java-client/releases) - [Commits](https://github.com/rabbitmq/rabbitmq-java-client/compare/v4.12.0...v5.18.0) --- updated-dependencies: - dependency-name: com.rabbitmq:amqp-client dependency-type: direct:production ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- recipes/rabbitmq-consumer-group/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/recipes/rabbitmq-consumer-group/pom.xml b/recipes/rabbitmq-consumer-group/pom.xml index ccabef656..d87467c43 100644 --- a/recipes/rabbitmq-consumer-group/pom.xml +++ b/recipes/rabbitmq-consumer-group/pom.xml @@ -84,7 +84,7 @@ com.rabbitmq amqp-client - 4.12.0 + 5.18.0 com.101tec
(helix) branch master updated: Bump org.xerial.snappy:snappy-java in /zookeeper-api (#2633)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new 0c4fb6e8f Bump org.xerial.snappy:snappy-java in /zookeeper-api (#2633) 0c4fb6e8f is described below commit 0c4fb6e8fb458bd3a2be6b8eb404e299e901f85a Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> AuthorDate: Thu Feb 22 15:49:07 2024 -0800 Bump org.xerial.snappy:snappy-java in /zookeeper-api (#2633) Bumps [org.xerial.snappy:snappy-java](https://github.com/xerial/snappy-java) from 1.1.7 to 1.1.10.4. - [Release notes](https://github.com/xerial/snappy-java/releases) - [Commits](https://github.com/xerial/snappy-java/compare/1.1.7...v1.1.10.4) --- updated-dependencies: - dependency-name: org.xerial.snappy:snappy-java dependency-type: direct:development ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- zookeeper-api/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/zookeeper-api/pom.xml b/zookeeper-api/pom.xml index befc1b319..dd5fd065c 100644 --- a/zookeeper-api/pom.xml +++ b/zookeeper-api/pom.xml @@ -119,7 +119,7 @@ org.xerial.snappy snappy-java - 1.1.7 + 1.1.10.4 test
(helix) branch master updated: Bump org.xerial.snappy:snappy-java in /meta-client (#2631)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new c59bb5cd6 Bump org.xerial.snappy:snappy-java in /meta-client (#2631) c59bb5cd6 is described below commit c59bb5cd67eb4e831fc89b355dd0fb237824738f Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> AuthorDate: Thu Feb 22 15:48:14 2024 -0800 Bump org.xerial.snappy:snappy-java in /meta-client (#2631) Bumps [org.xerial.snappy:snappy-java](https://github.com/xerial/snappy-java) from 1.1.7 to 1.1.10.4. - [Release notes](https://github.com/xerial/snappy-java/releases) - [Commits](https://github.com/xerial/snappy-java/compare/1.1.7...v1.1.10.4) --- updated-dependencies: - dependency-name: org.xerial.snappy:snappy-java dependency-type: direct:development ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- meta-client/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta-client/pom.xml b/meta-client/pom.xml index 4dcae95bd..29092ef1e 100644 --- a/meta-client/pom.xml +++ b/meta-client/pom.xml @@ -70,7 +70,7 @@ under the License. org.xerial.snappy snappy-java - 1.1.7 + 1.1.10.4 test
(helix) branch master updated: Bump guava in /metadata-store-directory-common (#2536)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new 9a1dd4d86 Bump guava in /metadata-store-directory-common (#2536) 9a1dd4d86 is described below commit 9a1dd4d8624ec10ea2d11403457cd594aa08918e Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> AuthorDate: Thu Feb 22 15:47:28 2024 -0800 Bump guava in /metadata-store-directory-common (#2536) Bumps [guava](https://github.com/google/guava) from 31.1-jre to 32.0.0-jre. - [Release notes](https://github.com/google/guava/releases) - [Commits](https://github.com/google/guava/commits) --- updated-dependencies: - dependency-name: com.google.guava:guava dependency-type: direct:production ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- metadata-store-directory-common/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/metadata-store-directory-common/pom.xml b/metadata-store-directory-common/pom.xml index 114d901d1..50ce7cceb 100644 --- a/metadata-store-directory-common/pom.xml +++ b/metadata-store-directory-common/pom.xml @@ -87,7 +87,7 @@ com.google.guava guava - 31.1-jre + 32.0.0-jre compile
(helix) branch master updated: Bump jetty-server in /helix-rest (#2534)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new b3094060e Bump jetty-server in /helix-rest (#2534) b3094060e is described below commit b3094060e5113c5be0eaea5726e5b1c417a474ef Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> AuthorDate: Thu Feb 22 15:47:00 2024 -0800 Bump jetty-server in /helix-rest (#2534) Bumps [jetty-server](https://github.com/eclipse/jetty.project) from 8.1.8.v20121106 to 9.4.51.v20230217. - [Release notes](https://github.com/eclipse/jetty.project/releases) - [Commits](https://github.com/eclipse/jetty.project/compare/jetty-8.1.8.v20121106...jetty-9.4.51.v20230217) --- updated-dependencies: - dependency-name: org.eclipse.jetty:jetty-server dependency-type: direct:production ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
(helix) branch master updated (1dafdde9b -> 7c9db10cb)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/helix.git from 1dafdde9b Bump org.xerial.snappy:snappy-java from 1.1.7 to 1.1.10.4 in /helix-core (#2632) add 7c9db10cb Bump jetty-server from 8.1.8.v20121106 to 9.4.51.v20230217 (#2533) No new revisions were added by this update. Summary of changes: helix-rest/pom.xml | 2 +- pom.xml| 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
(helix) branch master updated: Bump org.xerial.snappy:snappy-java from 1.1.7 to 1.1.10.4 in /helix-core (#2632)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new 1dafdde9b Bump org.xerial.snappy:snappy-java from 1.1.7 to 1.1.10.4 in /helix-core (#2632) 1dafdde9b is described below commit 1dafdde9b85b6337ee50717b256a5939a977f378 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> AuthorDate: Thu Feb 22 15:45:28 2024 -0800 Bump org.xerial.snappy:snappy-java from 1.1.7 to 1.1.10.4 in /helix-core (#2632) Bumps [org.xerial.snappy:snappy-java](https://github.com/xerial/snappy-java) from 1.1.7 to 1.1.10.4. - [Release notes](https://github.com/xerial/snappy-java/releases) - [Commits](https://github.com/xerial/snappy-java/compare/1.1.7...v1.1.10.4) --- updated-dependencies: - dependency-name: org.xerial.snappy:snappy-java dependency-type: direct:production ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- helix-core/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/helix-core/pom.xml b/helix-core/pom.xml index 7f4f082e3..0315e1448 100644 --- a/helix-core/pom.xml +++ b/helix-core/pom.xml @@ -162,7 +162,7 @@ org.xerial.snappy snappy-java - 1.1.7 + 1.1.10.4
(helix) branch master updated: Bump @babel/traverse from 7.19.6 to 7.23.2 in /helix-front (#2671)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new b0d0be860 Bump @babel/traverse from 7.19.6 to 7.23.2 in /helix-front (#2671) b0d0be860 is described below commit b0d0be860b191658b6a1f7b3d6d1cbc3b4fffb4c Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> AuthorDate: Thu Feb 22 15:43:40 2024 -0800 Bump @babel/traverse from 7.19.6 to 7.23.2 in /helix-front (#2671) Bumps [@babel/traverse](https://github.com/babel/babel/tree/HEAD/packages/babel-traverse) from 7.19.6 to 7.23.2. - [Release notes](https://github.com/babel/babel/releases) - [Changelog](https://github.com/babel/babel/blob/main/CHANGELOG.md) - [Commits](https://github.com/babel/babel/commits/v7.23.2/packages/babel-traverse) --- updated-dependencies: - dependency-name: "@babel/traverse" dependency-type: indirect ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- helix-front/yarn.lock | 131 +- 1 file changed, 118 insertions(+), 13 deletions(-) diff --git a/helix-front/yarn.lock b/helix-front/yarn.lock index 37dd8b95f..3f09cba0d 100644 --- a/helix-front/yarn.lock +++ b/helix-front/yarn.lock @@ -440,6 +440,14 @@ dependencies: "@babel/highlight" "^7.18.6" +"@babel/code-frame@^7.22.13": + version "7.22.13" + resolved "https://registry.yarnpkg.com/@babel/code-frame/-/code-frame-7.22.13.tgz#e3c1c099402598483b7a8c46a721d1038803755e; + integrity sha512-XktuhWlJ5g+3TJXc5upd9Ks1HutSArik6jf2eAjYFyIOf4ej3RN+184cZbzDvbPnuTJIUhPKKJE3cIsYTiAT3w== + dependencies: +"@babel/highlight" "^7.22.13" +chalk "^2.4.2" + "@babel/compat-data@^7.16.8", "@babel/compat-data@^7.17.7", "@babel/compat-data@^7.19.3", "@babel/compat-data@^7.19.4": version "7.19.4" resolved "https://registry.yarnpkg.com/@babel/compat-data/-/compat-data-7.19.4.tgz#95c86de137bf0317f3a570e1b6e996b427299747; @@ -527,6 +535,16 @@ "@jridgewell/gen-mapping" "^0.3.2" jsesc "^2.5.1" +"@babel/generator@^7.23.0": + version "7.23.0" + resolved "https://registry.yarnpkg.com/@babel/generator/-/generator-7.23.0.tgz#df5c386e2218be505b34837acbcb874d7a983420; + integrity sha512-lN85QRR+5IbYrMWM6Y4pE/noaQtg4pNiqeNGX60eqOfo6gtEj6uw/JagelB8vVztSd7R6M5n1+PQkDbHbBRU4g== + dependencies: +"@babel/types" "^7.23.0" +"@jridgewell/gen-mapping" "^0.3.2" +"@jridgewell/trace-mapping" "^0.3.17" +jsesc "^2.5.1" + "@babel/helper-annotate-as-pure@7.16.7": version "7.16.7" resolved "https://registry.yarnpkg.com/@babel/helper-annotate-as-pure/-/helper-annotate-as-pure-7.16.7.tgz#bb2339a7534a9c128e3102024c60760a3a7f3862; @@ -611,6 +629,11 @@ resolved "https://registry.yarnpkg.com/@babel/helper-environment-visitor/-/helper-environment-visitor-7.18.9.tgz#0c0cee9b35d2ca190478756865bb3528422f51be; integrity sha512-3r/aACDJ3fhQ/EVgFy0hpj8oHyHpQc+LPtJoY9SzTThAsStm4Ptegq92vqKoE3vD706ZVFWITnMnxucw+S9Ipg== +"@babel/helper-environment-visitor@^7.22.20": + version "7.22.20" + resolved "https://registry.yarnpkg.com/@babel/helper-environment-visitor/-/helper-environment-visitor-7.22.20.tgz#96159db61d34a29dba454c959f5ae4a649ba9167; + integrity sha512-zfedSIzFhat/gFhWfHtgWvlec0nqB9YEIVrpuwjruLlXfUSnA8cJB0miHKwqDnQ7d32aKo2xt88/xZptwxbfhA== + "@babel/helper-explode-assignable-expression@^7.18.6": version "7.18.6" resolved "https://registry.yarnpkg.com/@babel/helper-explode-assignable-expression/-/helper-explode-assignable-expression-7.18.6.tgz#41f8228ef0a6f1a036b8dfdfec7ce94f9a6bc096; @@ -626,6 +649,14 @@ "@babel/template" "^7.18.10" "@babel/types" "^7.19.0" +"@babel/helper-function-name@^7.23.0": + version "7.23.0" + resolved "https://registry.yarnpkg.com/@babel/helper-function-name/-/helper-function-name-7.23.0.tgz#1f9a3cdbd5b2698a670c30d2735f9af95ed52759; + integrity sha512-OErEqsrxjZTJciZ4Oo+eoZqeW9UIiOcuYKRJA4ZAgV9myA+pOXhhmpfNCKjEH/auVfEYVFJ6y1Tc4r0eIApqiw== + dependencies: +"@babel/template" "^7.22.15" +"@babel/types" "^7.23.0" + "@babel/helper-hoist-variables@^7.18.6": version "7.18.6" resolved "https://registry.yarnpkg.com/@babel/helper-hoist-variables/-/helper-hoist-variables-7.18.6.tgz#d4d2c8fb4baeaa5c68b99cc8245c56554f926678; @
(helix) branch master updated (20c409743 -> 4bc2abf8a)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/helix.git from 20c409743 Bump follow-redirects from 1.15.2 to 1.15.4 in /helix-front (#2732) add 4bc2abf8a Bump org.codehaus.plexus:plexus-utils from 2.1 to 3.0.24 in /helix-front (#2634) No new revisions were added by this update. Summary of changes: helix-front/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
(helix) branch master updated: Bump crypto-js from 4.1.1 to 4.2.0 in /helix-front (#2681)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new d51c11e8c Bump crypto-js from 4.1.1 to 4.2.0 in /helix-front (#2681) d51c11e8c is described below commit d51c11e8c965e0801bcd1ae0c179aaa1296a8834 Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> AuthorDate: Thu Feb 22 15:43:25 2024 -0800 Bump crypto-js from 4.1.1 to 4.2.0 in /helix-front (#2681) Bumps [crypto-js](https://github.com/brix/crypto-js) from 4.1.1 to 4.2.0. - [Commits](https://github.com/brix/crypto-js/compare/4.1.1...4.2.0) --- updated-dependencies: - dependency-name: crypto-js dependency-type: indirect ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- helix-front/yarn.lock | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/helix-front/yarn.lock b/helix-front/yarn.lock index 06629e1e7..37dd8b95f 100644 --- a/helix-front/yarn.lock +++ b/helix-front/yarn.lock @@ -5950,9 +5950,9 @@ cross-spawn@^7.0.0, cross-spawn@^7.0.2, cross-spawn@^7.0.3: which "^2.0.1" crypto-js@^4.0.0: - version "4.1.1" - resolved "https://registry.yarnpkg.com/crypto-js/-/crypto-js-4.1.1.tgz#9e485bcf03521041bd85844786b83fb7619736cf; - integrity sha512-o2JlM7ydqd3Qk9CA0L4NL6mTzU2sdx96a+oOfPu8Mkl/PK51vSyoi8/rQ8NknZtk44vq15lmhAj9CIAGwgeWKw== + version "4.2.0" + resolved "https://registry.yarnpkg.com/crypto-js/-/crypto-js-4.2.0.tgz#4d931639ecdfd12ff80e8186dba6af2c2e856631; + integrity sha512-KALDyEYgpY+Rlob/iriUtjV6d5Eq+Y191A5g4UqLAi8CyGP9N1+FdVbkc1SxKc2r4YAYqG8JzO2KGL+AizD70Q== css-blank-pseudo@^3.0.2: version "3.0.3"
(helix) branch master updated: Bump follow-redirects from 1.15.2 to 1.15.4 in /helix-front (#2732)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new 20c409743 Bump follow-redirects from 1.15.2 to 1.15.4 in /helix-front (#2732) 20c409743 is described below commit 20c4097432238c3eb120d7d06f6e7c81d005342c Author: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> AuthorDate: Thu Feb 22 15:37:17 2024 -0800 Bump follow-redirects from 1.15.2 to 1.15.4 in /helix-front (#2732) Bumps [follow-redirects](https://github.com/follow-redirects/follow-redirects) from 1.15.2 to 1.15.4. - [Release notes](https://github.com/follow-redirects/follow-redirects/releases) - [Commits](https://github.com/follow-redirects/follow-redirects/compare/v1.15.2...v1.15.4) --- updated-dependencies: - dependency-name: follow-redirects dependency-type: indirect ... Signed-off-by: dependabot[bot] Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> --- helix-front/yarn.lock | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/helix-front/yarn.lock b/helix-front/yarn.lock index e6bd49f21..06629e1e7 100644 --- a/helix-front/yarn.lock +++ b/helix-front/yarn.lock @@ -8209,9 +8209,9 @@ flatted@^3.1.0: integrity sha512-5nqDSxl8nn5BSNxyR3n4I6eDmbolI6WT+QqR547RwxQapgjQBmtktdP+HTBb/a/zLsbzERTONyUB5pefh5TtjQ== follow-redirects@^1.0.0: - version "1.15.2" - resolved "https://registry.yarnpkg.com/follow-redirects/-/follow-redirects-1.15.2.tgz#b460864144ba63f2681096f274c4e57026da2c13; - integrity sha512-VQLG33o04KaQ8uYi2tVNbdrWp1QWxNNea+nmIB4EVM28v0hmP17z7aG1+wAkNzVq4KeXTq3221ye5qTJP91JwA== + version "1.15.4" + resolved "https://registry.yarnpkg.com/follow-redirects/-/follow-redirects-1.15.4.tgz#cdc7d308bf6493126b17ea2191ea0ccf3e535adf; + integrity sha512-Cr4D/5wlrb0z9dgERpUL3LrmPKVDsETIJhaCMeDfuFYcqa5bldGV6wBsAN6X/vxlXQtFBMrXdXxdL8CbDTGniw== for-each@^0.3.3: version "0.3.3"
(helix) branch master updated: [apache/helix] -- Updated Maps.of to Sets.of for Java-8 build complaince (#2756)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new b027a8d74 [apache/helix] -- Updated Maps.of to Sets.of for Java-8 build complaince (#2756) b027a8d74 is described below commit b027a8d740a25739d91c24f7841ccceef5a2ed45 Author: Himanshu Kandwal AuthorDate: Thu Feb 22 15:33:48 2024 -0800 [apache/helix] -- Updated Maps.of to Sets.of for Java-8 build complaince (#2756) Final Commit Message: Updated Helix codebase to be build compatible with JDK-8. Open source release will be based on Java 11. --- .../rebalancer/TestInstanceOperation.java | 27 +++--- .../resources/helix/PerInstanceAccessor.java | 7 +++--- .../rest/server/TestPartitionAssignmentAPI.java| 3 ++- 3 files changed, 20 insertions(+), 17 deletions(-) diff --git a/helix-core/src/test/java/org/apache/helix/integration/rebalancer/TestInstanceOperation.java b/helix-core/src/test/java/org/apache/helix/integration/rebalancer/TestInstanceOperation.java index 765add7fd..276757741 100644 --- a/helix-core/src/test/java/org/apache/helix/integration/rebalancer/TestInstanceOperation.java +++ b/helix-core/src/test/java/org/apache/helix/integration/rebalancer/TestInstanceOperation.java @@ -14,6 +14,7 @@ import java.util.UUID; import java.util.concurrent.atomic.AtomicInteger; import java.util.stream.Collectors; +import com.google.common.collect.ImmutableMap; import com.google.common.collect.ImmutableSet; import org.apache.helix.ConfigAccessor; import org.apache.helix.HelixAdmin; @@ -558,7 +559,7 @@ public class TestInstanceOperation extends ZkTestBase { // but none of them are in a top state. Assert.assertTrue(_bestPossibleClusterVerifier.verifyByPolling()); validateEVsCorrect(getEVs(), originalEVs, swapOutInstancesToSwapInInstances, -Set.of(instanceToSwapInName), Collections.emptySet()); +ImmutableSet.of(instanceToSwapInName), Collections.emptySet()); // Assert canSwapBeCompleted is true Assert.assertTrue(_gSetupTool.getClusterManagementTool() @@ -585,7 +586,7 @@ public class TestInstanceOperation extends ZkTestBase { // Validate that the SWAP_IN instance has the same partitions the SWAP_OUT instance had before // swap was completed. verifier(() -> (validateEVsCorrect(getEVs(), originalEVs, swapOutInstancesToSwapInInstances, -Collections.emptySet(), Set.of(instanceToSwapInName))), TIMEOUT); +Collections.emptySet(), ImmutableSet.of(instanceToSwapInName))), TIMEOUT); } @Test(dependsOnMethods = "testNodeSwap") @@ -624,7 +625,7 @@ public class TestInstanceOperation extends ZkTestBase { // but none of them are in a top state. Assert.assertTrue(_bestPossibleClusterVerifier.verifyByPolling()); validateEVsCorrect(getEVs(), originalEVs, swapOutInstancesToSwapInInstances, -Set.of(instanceToSwapInName), Collections.emptySet()); +ImmutableSet.of(instanceToSwapInName), Collections.emptySet()); // Assert canSwapBeCompleted is true Assert.assertTrue(_gSetupTool.getClusterManagementTool() @@ -676,7 +677,7 @@ public class TestInstanceOperation extends ZkTestBase { // Validate that the SWAP_IN instance has the same partitions the SWAP_OUT instance had before // swap was completed. verifier(() -> (validateEVsCorrect(getEVs(), originalEVs, swapOutInstancesToSwapInInstances, -Collections.emptySet(), Set.of(instanceToSwapInName))), TIMEOUT); +Collections.emptySet(), ImmutableSet.of(instanceToSwapInName))), TIMEOUT); } @Test(dependsOnMethods = "testNodeSwapDisableAndReenable") @@ -723,7 +724,7 @@ public class TestInstanceOperation extends ZkTestBase { // but none of them are in a top state. Assert.assertTrue(_bestPossibleClusterVerifier.verifyByPolling()); validateEVsCorrect(getEVs(), originalEVs, swapOutInstancesToSwapInInstances, -Set.of(instanceToSwapInName), Collections.emptySet()); +ImmutableSet.of(instanceToSwapInName), Collections.emptySet()); // Validate that the SWAP_OUT instance is in routing tables and SWAP_IN is not. validateRoutingTablesInstance(getEVs(), instanceToSwapOutName, true); @@ -745,7 +746,7 @@ public class TestInstanceOperation extends ZkTestBase { // Validate that the SWAP_IN instance has the same partitions the SWAP_OUT instance had before // swap was completed. verifier(() -> (validateEVsCorrect(getEVs(), originalEVs, swapOutInstancesToSwapInInstances, -Collections.emptySet(), Set.of(instanceToSwapInName))), TIMEOUT); +Collections.emptySet(), ImmutableSet.of(instanceToSwapInName))), TIMEOUT); } @Test(dependsOnMethods = "testNodeSwapSwapInNodeNoInstanceOperationDisabled") @@ -786,7 +787,7 @@ public class T
(helix) branch master updated: fix waged design doc url (#2725)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new de05ad2c7 fix waged design doc url (#2725) de05ad2c7 is described below commit de05ad2c77ab02475aea1b07656a6bca18232ba2 Author: Grant Paláu Spencer AuthorDate: Thu Feb 22 10:13:28 2024 -0800 fix waged design doc url (#2725) Fix waged doc URL in docstrings --- .../org/apache/helix/controller/rebalancer/waged/WagedRebalancer.java | 2 +- helix-core/src/main/java/org/apache/helix/model/ClusterConfig.java| 4 ++-- helix-core/src/main/java/org/apache/helix/model/InstanceConfig.java | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-) diff --git a/helix-core/src/main/java/org/apache/helix/controller/rebalancer/waged/WagedRebalancer.java b/helix-core/src/main/java/org/apache/helix/controller/rebalancer/waged/WagedRebalancer.java index b0527e5e8..ae7e49a1d 100644 --- a/helix-core/src/main/java/org/apache/helix/controller/rebalancer/waged/WagedRebalancer.java +++ b/helix-core/src/main/java/org/apache/helix/controller/rebalancer/waged/WagedRebalancer.java @@ -62,7 +62,7 @@ import org.slf4j.LoggerFactory; /** * Weight-Aware Globally-Even Distribute Rebalancer. * @see https://github.com/apache/helix/wiki/Design-Proposal---Weight-Aware-Globally-Even-Distribute-Rebalancer;> + * href="https://github.com/apache/helix/wiki/Weight-Aware-Globally-Even-Distribute-Rebalancer;> * Design Document * */ diff --git a/helix-core/src/main/java/org/apache/helix/model/ClusterConfig.java b/helix-core/src/main/java/org/apache/helix/model/ClusterConfig.java index 750bbe612..fdc05ac9b 100644 --- a/helix-core/src/main/java/org/apache/helix/model/ClusterConfig.java +++ b/helix-core/src/main/java/org/apache/helix/model/ClusterConfig.java @@ -922,7 +922,7 @@ public class ClusterConfig extends HelixProperty { * Set the default instance capacity information with an Integer mapping. * This information is required by the global rebalancer. * @see - * https://github.com/apache/helix/wiki/Design-Proposal---Weight-Aware-Globally-Even-Distribute-Rebalancer#rebalance-algorithm-adapter + * https://github.com/apache/helix/wiki/Weight-Aware-Globally-Even-Distribute-Rebalancer#rebalance-algorithm-adapter * * If the instance capacity is not configured in either Instance Config nor Cluster Config, the * cluster topology is considered invalid. So the rebalancer may stop working. @@ -948,7 +948,7 @@ public class ClusterConfig extends HelixProperty { * Set the default partition weight information with an Integer mapping. * This information is required by the global rebalancer. * @see - * https://github.com/apache/helix/wiki/Design-Proposal---Weight-Aware-Globally-Even-Distribute-Rebalancer#rebalance-algorithm-adapter + * https://github.com/apache/helix/wiki/Weight-Aware-Globally-Even-Distribute-Rebalancer#rebalance-algorithm-adapter * * If the partition weight is not configured in either Resource Config nor Cluster Config, the * cluster topology is considered invalid. So the rebalancer may stop working. diff --git a/helix-core/src/main/java/org/apache/helix/model/InstanceConfig.java b/helix-core/src/main/java/org/apache/helix/model/InstanceConfig.java index 98da7340f..2f3da1456 100644 --- a/helix-core/src/main/java/org/apache/helix/model/InstanceConfig.java +++ b/helix-core/src/main/java/org/apache/helix/model/InstanceConfig.java @@ -654,7 +654,7 @@ public class InstanceConfig extends HelixProperty { * * This information is required by the global rebalancer. * @see - * https://github.com/apache/helix/wiki/Design-Proposal---Weight-Aware-Globally-Even-Distribute-Rebalancer#rebalance-algorithm-adapter + * https://github.com/apache/helix/wiki/Weight-Aware-Globally-Even-Distribute-Rebalancer#rebalance-algorithm-adapter * * If the instance capacity is not configured in neither Instance Config nor Cluster Config, the * cluster topology is considered invalid. So the rebalancer may stop working.
(helix) branch master updated: minor version bump to 1.7.0 (#2676)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new 7c86f4a5c minor version bump to 1.7.0 (#2676) 7c86f4a5c is described below commit 7c86f4a5cf6eab730c4d265bc82d92262f820335 Author: Grant Paláu Spencer AuthorDate: Tue Feb 13 15:50:16 2024 -0800 minor version bump to 1.7.0 (#2676) --- helix-front/package.json | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/helix-front/package.json b/helix-front/package.json index 5f23f2ef0..e151010ae 100644 --- a/helix-front/package.json +++ b/helix-front/package.json @@ -1,6 +1,6 @@ { "name": "helix-front", - "version": "1.6.1", + "version": "1.7.0", "main": "/dist/server/app.js", "author": "Vivo Xu ", "contributors": [
(helix) branch master updated: Support Simple Greedy Rebalance Strategy (#2758)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new 424aacc5e Support Simple Greedy Rebalance Strategy (#2758) 424aacc5e is described below commit 424aacc5e67dd4d7ede5d45950777cdd3432 Author: Junkai Xue AuthorDate: Tue Feb 13 09:31:23 2024 -0800 Support Simple Greedy Rebalance Strategy (#2758) * Support Simple Greedy Rebalance Strategy Support Simple Greedy Rebalance Strategy for RoundRobin assignment. It also supported with global limitation on how many partition assigned for each of the instance. * Add integration tests and fix issues --- .../helix/controller/common/CapacityNode.java | 91 ++ .../ResourceControllerDataProvider.java| 30 ++ .../strategy/GreedyRebalanceStrategy.java | 103 + .../java/org/apache/helix/model/ClusterConfig.java | 24 + .../rebalancer/TestGreedyRebalanceStrategy.java| 85 + ...balanceWithGlobalPerInstancePartitionLimit.java | 82 6 files changed, 415 insertions(+) diff --git a/helix-core/src/main/java/org/apache/helix/controller/common/CapacityNode.java b/helix-core/src/main/java/org/apache/helix/controller/common/CapacityNode.java new file mode 100644 index 0..fa5068e13 --- /dev/null +++ b/helix-core/src/main/java/org/apache/helix/controller/common/CapacityNode.java @@ -0,0 +1,91 @@ +package org.apache.helix.controller.common; + +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import java.util.HashMap; +import java.util.HashSet; +import java.util.Map; +import java.util.Set; + +/** + * A Node is an entity that can serve capacity recording purpose. It has a capacity and knowledge + * of partitions assigned to it, so it can decide if it can receive additional partitions. + */ +public class CapacityNode { + private int _currentlyAssigned; + private int _capacity; + private final String _id; + private final Map> _partitionMap; + + public CapacityNode(String id) { +_partitionMap = new HashMap<>(); +_currentlyAssigned = 0; +this._id = id; + } + + /** + * Check if this replica can be legally added to this node + * + * @param resource The resource to assign + * @param partition The partition to assign + * @return true if the assignment can be made, false otherwise + */ + public boolean canAdd(String resource, String partition) { +if (_currentlyAssigned >= _capacity || (_partitionMap.containsKey(resource) +&& _partitionMap.get(resource).contains(partition))) { + return false; +} +_partitionMap.computeIfAbsent(resource, k -> new HashSet<>()).add(partition); +_currentlyAssigned++; +return true; + } + + /** + * Set the capacity of this node + * @param capacity The capacity to set + */ + public void setCapacity(int capacity) { +_capacity = capacity; + } + + /** + * Get the ID of this node + * @return The ID of this node + */ + public String getId() { +return _id; + } + + /** + * Get number of partitions currently assigned to this node + * @return The number of partitions currently assigned to this node + */ + public int getCurrentlyAssigned() { +return _currentlyAssigned; + } + + @Override + public String toString() { +StringBuilder sb = new StringBuilder(); + sb.append("##\nname=").append(_id).append("\nassigned:").append(_currentlyAssigned) +.append("\ncapacity:").append(_capacity); +return sb.toString(); + } +} diff --git a/helix-core/src/main/java/org/apache/helix/controller/dataproviders/ResourceControllerDataProvider.java b/helix-core/src/main/java/org/apache/helix/controller/dataproviders/ResourceControllerDataProvider.java index 9ca6de4f7..bc0cbf530 100644 --- a/helix-core/src/main/java/org/apache/helix/controller/dataproviders/ResourceControllerDataProvider.java +++ b/helix-core/src/main/java/org/apache/helix/controller/dataprovid
(helix) branch master updated: Update Java Doc for RebalanceStrategy
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new 8eb55b290 Update Java Doc for RebalanceStrategy 8eb55b290 is described below commit 8eb55b290e3ef0f167845300866498053669757a Author: Junkai Xue AuthorDate: Mon Jan 8 11:05:41 2024 -0800 Update Java Doc for RebalanceStrategy --- .../rebalancer/strategy/RebalanceStrategy.java | 20 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/helix-core/src/main/java/org/apache/helix/controller/rebalancer/strategy/RebalanceStrategy.java b/helix-core/src/main/java/org/apache/helix/controller/rebalancer/strategy/RebalanceStrategy.java index 39a1010b5..99c593485 100644 --- a/helix-core/src/main/java/org/apache/helix/controller/rebalancer/strategy/RebalanceStrategy.java +++ b/helix-core/src/main/java/org/apache/helix/controller/rebalancer/strategy/RebalanceStrategy.java @@ -35,10 +35,11 @@ public interface RebalanceStrategy { /** * Perform the necessary initialization for the rebalance strategy object. * - * @param resourceName - * @param partitions - * @param states - * @param maximumPerNode + * @param resourceName The resource for assignment mapping computation. + * @param partitionsThe partition names of the resource. + * @param statesThe state -> required state count mapping for the resource, ordered by + * priority. + * @param maximumPerNodeThe maximum number of partitions that can be assigned to a node. */ void init(String resourceName, final List partitions, final LinkedHashMap states, int maximumPerNode); @@ -46,10 +47,13 @@ public interface RebalanceStrategy { /** * Compute the preference lists and (optional partition-state mapping) for the given resource. * - * @param liveNodes - * @param currentMapping - * @param allNodes - * @return + * @param liveNodes The list of live nodes names. + * @param currentMapping The current mapping of partition->state->instance. + * @param allNodes The list of all node names. This could be filtered for evacuation + * and swap operations. + * @param clusterDataThe cache of the cluster data snapshot from Zookeeper. + * @return The computed IdealState with ListFields for preference lists (optional) + * and MapFields for partition->state->instance mapping (required). */ ZNRecord computePartitionAssignment(final List allNodes, final List liveNodes, final Map> currentMapping,
(helix) branch metaclient updated: Update Java Doc for RebalanceStrategy
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch metaclient in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/metaclient by this push: new 58a021cb9 Update Java Doc for RebalanceStrategy 58a021cb9 is described below commit 58a021cb9874e01d1153a28295e0e90e0580ee57 Author: Junkai Xue AuthorDate: Mon Jan 8 11:05:41 2024 -0800 Update Java Doc for RebalanceStrategy --- .../rebalancer/strategy/RebalanceStrategy.java | 20 1 file changed, 12 insertions(+), 8 deletions(-) diff --git a/helix-core/src/main/java/org/apache/helix/controller/rebalancer/strategy/RebalanceStrategy.java b/helix-core/src/main/java/org/apache/helix/controller/rebalancer/strategy/RebalanceStrategy.java index 39a1010b5..99c593485 100644 --- a/helix-core/src/main/java/org/apache/helix/controller/rebalancer/strategy/RebalanceStrategy.java +++ b/helix-core/src/main/java/org/apache/helix/controller/rebalancer/strategy/RebalanceStrategy.java @@ -35,10 +35,11 @@ public interface RebalanceStrategy { /** * Perform the necessary initialization for the rebalance strategy object. * - * @param resourceName - * @param partitions - * @param states - * @param maximumPerNode + * @param resourceName The resource for assignment mapping computation. + * @param partitionsThe partition names of the resource. + * @param statesThe state -> required state count mapping for the resource, ordered by + * priority. + * @param maximumPerNodeThe maximum number of partitions that can be assigned to a node. */ void init(String resourceName, final List partitions, final LinkedHashMap states, int maximumPerNode); @@ -46,10 +47,13 @@ public interface RebalanceStrategy { /** * Compute the preference lists and (optional partition-state mapping) for the given resource. * - * @param liveNodes - * @param currentMapping - * @param allNodes - * @return + * @param liveNodes The list of live nodes names. + * @param currentMapping The current mapping of partition->state->instance. + * @param allNodes The list of all node names. This could be filtered for evacuation + * and swap operations. + * @param clusterDataThe cache of the cluster data snapshot from Zookeeper. + * @return The computed IdealState with ListFields for preference lists (optional) + * and MapFields for partition->state->instance mapping (required). */ ZNRecord computePartitionAssignment(final List allNodes, final List liveNodes, final Map> currentMapping,
(helix) branch revert-2279-dependabot/maven/helix-admin-webapp/com.fasterxml.jackson.core-jackson-databind-2.12.7.1 deleted (was 0b154cfa2)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a change to branch revert-2279-dependabot/maven/helix-admin-webapp/com.fasterxml.jackson.core-jackson-databind-2.12.7.1 in repository https://gitbox.apache.org/repos/asf/helix.git was 0b154cfa2 Revert "Bump jackson-databind from 2.12.6.1 to 2.12.7.1 in /helix-admin-webapp (#2279)" This change permanently discards the following revisions: discard 0b154cfa2 Revert "Bump jackson-databind from 2.12.6.1 to 2.12.7.1 in /helix-admin-webapp (#2279)"
(helix) branch ivy_removal created (now 7f2a88df1)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a change to branch ivy_removal in repository https://gitbox.apache.org/repos/asf/helix.git at 7f2a88df1 Fix condition to check if DOMAIN has all required TOPOLOGY keys and allows for extra kv pairs, to reduce noisy logs. (#2688) No new revisions were added by this update.
[helix] 02/02: Bump snapshot
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git commit 6241bcbf5f9fab1726c87c20b5e948daa95b7bbf Author: Junkai Xue AuthorDate: Thu Sep 28 23:31:16 2023 -0700 Bump snapshot --- 1-SNAPSHOT.ivy => helix-admin-webapp-1.3.2-SNAPSHOT.ivy} | 4 ++-- ...ent-1.3.1-SNAPSHOT.ivy => helix-agent-1.3.2-SNAPSHOT.ivy} | 12 ++-- ...on-1.3.1-SNAPSHOT.ivy => helix-common-1.3.2-SNAPSHOT.ivy} | 2 +- ...core-1.3.1-SNAPSHOT.ivy => helix-core-1.3.2-SNAPSHOT.ivy} | 2 +- ...lock-1.3.1-SNAPSHOT.ivy => helix-lock-1.3.2-SNAPSHOT.ivy} | 4 ++-- ...rest-1.3.1-SNAPSHOT.ivy => helix-rest-1.3.2-SNAPSHOT.ivy} | 4 ++-- ...SNAPSHOT.ivy => helix-view-aggregator-1.3.2-SNAPSHOT.ivy} | 4 ++-- ...ent-1.3.1-SNAPSHOT.ivy => meta-client-1.3.2-SNAPSHOT.ivy} | 4 ++-- ...vy => metadata-store-directory-common-1.3.2-SNAPSHOT.ivy} | 4 ++-- ...-1.3.1-SNAPSHOT.ivy => metrics-common-1.3.2-SNAPSHOT.ivy} | 2 +- ...i-1.3.1-SNAPSHOT.ivy => zookeeper-api-1.3.2-SNAPSHOT.ivy} | 2 +- 11 files changed, 22 insertions(+), 22 deletions(-) diff --git a/helix-admin-webapp/helix-admin-webapp-1.3.1-SNAPSHOT.ivy b/helix-admin-webapp/helix-admin-webapp-1.3.2-SNAPSHOT.ivy similarity index 97% rename from helix-admin-webapp/helix-admin-webapp-1.3.1-SNAPSHOT.ivy rename to helix-admin-webapp/helix-admin-webapp-1.3.2-SNAPSHOT.ivy index a75110435..eb3ea3207 100644 --- a/helix-admin-webapp/helix-admin-webapp-1.3.1-SNAPSHOT.ivy +++ b/helix-admin-webapp/helix-admin-webapp-1.3.2-SNAPSHOT.ivy @@ -20,7 +20,7 @@ under the License. @@ -43,7 +43,7 @@ under the License. - + diff --git a/helix-agent/helix-agent-1.3.1-SNAPSHOT.ivy b/helix-agent/helix-agent-1.3.2-SNAPSHOT.ivy similarity index 96% rename from helix-agent/helix-agent-1.3.1-SNAPSHOT.ivy rename to helix-agent/helix-agent-1.3.2-SNAPSHOT.ivy index 15889f8d6..463ba63d1 100644 --- a/helix-agent/helix-agent-1.3.1-SNAPSHOT.ivy +++ b/helix-agent/helix-agent-1.3.2-SNAPSHOT.ivy @@ -21,7 +21,7 @@ under the License. http://ant.apache.org/ivy/maven;> @@ -31,7 +31,7 @@ under the License. org.apache.helix.agent*;version=${project.version};-noimport:=true svn false - 1.3.1-SNAPSHOT + 1.3.2-SNAPSHOT org.apache.helix*, org.apache.commons.cli;version=[1.2,2), @@ -46,7 +46,7 @@ under the License. https://repository.apache.org/content/repositories/snapshots true true - 1.3.1-SNAPSHOT + 1.3.2-SNAPSHOT UTF-8 6.0.1 clean install @@ -77,12 +77,12 @@ under the License. - + - + @@ -92,7 +92,7 @@ under the License. - + diff --git a/helix-common/helix-common-1.3.1-SNAPSHOT.ivy b/helix-common/helix-common-1.3.2-SNAPSHOT.ivy similarity index 99% rename from helix-common/helix-common-1.3.1-SNAPSHOT.ivy rename to helix-common/helix-common-1.3.2-SNAPSHOT.ivy index 4c953cdd2..4245996c4 100644 --- a/helix-common/helix-common-1.3.1-SNAPSHOT.ivy +++ b/helix-common/helix-common-1.3.2-SNAPSHOT.ivy @@ -20,7 +20,7 @@ under the License. diff --git a/helix-core/helix-core-1.3.1-SNAPSHOT.ivy b/helix-core/helix-core-1.3.2-SNAPSHOT.ivy similarity index 99% rename from helix-core/helix-core-1.3.1-SNAPSHOT.ivy rename to helix-core/helix-core-1.3.2-SNAPSHOT.ivy index 7bb0a921e..8368c2fd2 100755 --- a/helix-core/helix-core-1.3.1-SNAPSHOT.ivy +++ b/helix-core/helix-core-1.3.2-SNAPSHOT.ivy @@ -20,7 +20,7 @@ under the License. http://ant.apache.org/ivy/maven;> diff --git a/helix-lock/helix-lock-1.3.1-SNAPSHOT.ivy b/helix-lock/helix-lock-1.3.2-SNAPSHOT.ivy similarity index 97% rename from helix-lock/helix-lock-1.3.1-SNAPSHOT.ivy rename to helix-lock/helix-lock-1.3.2-SNAPSHOT.ivy index 46c90c9f1..217c6702e 100644 --- a/helix-lock/helix-lock-1.3.1-SNAPSHOT.ivy +++ b/helix-lock/helix-lock-1.3.2-SNAPSHOT.ivy @@ -20,7 +20,7 @@ under the License. @@ -43,6 +43,6 @@ under the License. - + diff --git a/helix-rest/helix-rest-1.3.1-SNAPSHOT.ivy b/helix-rest/helix-rest-1.3.2-SNAPSHOT.ivy similarity index 97% rename from helix-rest/helix-rest-1.3.1-SNAPSHOT.ivy rename to helix-rest/helix-rest-1.3.2-SNAPSHOT.ivy index 40604bf14..742bf14a3 100644 --- a/helix-rest/helix-rest-1.3.1-SNAPSHOT.ivy +++ b/helix-rest/helix
[helix] 01/02: Update menu bar
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git commit 4a464b8b9d943652b4d8a89120c0aff7eecda96f Author: Junkai Xue AuthorDate: Thu Sep 28 23:24:07 2023 -0700 Update menu bar --- website/src/site/markdown/index.md | 10 +- website/src/site/resources/.htaccess | 4 ++-- website/src/site/site.xml| 14 +++--- 3 files changed, 14 insertions(+), 14 deletions(-) diff --git a/website/src/site/markdown/index.md b/website/src/site/markdown/index.md index 741ace628..5c64b63ee 100644 --- a/website/src/site/markdown/index.md +++ b/website/src/site/markdown/index.md @@ -77,9 +77,9 @@ Modeling a distributed system as a state machine with constraints on states and Apache Helix has a new release: -* [1.3.0](./1.3.0-docs/index.html) - A release containing new generic metadata store client and CVE fixes +* [1.3.1](./1.3.1-docs/index.html) - A release containing fix and cloud support -[\[Quick Start\]](./1.3.0-docs/Quickstart.html) [\[Release Notes\]](./1.3.0-docs/releasenotes/release-1.3.0.html) +[\[Quick Start\]](./1.3.1-docs/Quickstart.html) [\[Release Notes\]](./1.3.1-docs/releasenotes/release-1.3.1.html) * [0.9.9](./0.9.9-docs/index.html) - A release with several critical bug fixing for 0.9 series release @@ -87,7 +87,7 @@ Apache Helix has a new release: ### Download -1.3.0 +1.3.1 0.9.9 @@ -97,7 +97,7 @@ Apache Helix has a new release: org.apache.helix helix-core - 1.3.0 + 1.3.1 ``` @@ -110,6 +110,6 @@ Requirements: JDK 11, Maven 3.6.0+ for 1.2.0+ ``` git clone https://github.com/apache/helix.git cd helix -git checkout helix-1.3.0 +git checkout helix-1.3.1 mvn install package -DskipTests ``` diff --git a/website/src/site/resources/.htaccess b/website/src/site/resources/.htaccess index ed335ad69..fa23c9cb6 100644 --- a/website/src/site/resources/.htaccess +++ b/website/src/site/resources/.htaccess @@ -19,5 +19,5 @@ Redirect /download.html /download.cgi Redirect /0.9.9-docs/download.html /0.9.9-docs/download.cgi -Redirect /1.2.0-docs/download.html /1.2.0-docs/download.cgi -Redirect /1.3.0-docs/download.html /1.3.0-docs/download.cgi \ No newline at end of file +Redirect /1.3.0-docs/download.html /1.3.0-docs/download.cgi +Redirect /1.3.1-docs/download.html /1.3.1-docs/download.cgi \ No newline at end of file diff --git a/website/src/site/site.xml b/website/src/site/site.xml index 6b52bc3a6..fe24596c1 100644 --- a/website/src/site/site.xml +++ b/website/src/site/site.xml @@ -56,17 +56,17 @@ + - - - - - - + + + + + @@ -140,7 +140,7 @@ A cluster management framework for partitioned and replicated distributed resources Learn -Documentation|Helix 1.3.0|Helix 0.9.10 +Documentation|Helix 1.3.1|Helix 0.9.10 Get Involved ASF
[helix] branch master updated (47c1718c9 -> 6241bcbf5)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/helix.git from 47c1718c9 Remove 1.2.0 docs new 4a464b8b9 Update menu bar new 6241bcbf5 Bump snapshot The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: ...-SNAPSHOT.ivy => helix-admin-webapp-1.3.2-SNAPSHOT.ivy} | 4 ++-- ...t-1.3.1-SNAPSHOT.ivy => helix-agent-1.3.2-SNAPSHOT.ivy} | 12 ++-- ...-1.3.1-SNAPSHOT.ivy => helix-common-1.3.2-SNAPSHOT.ivy} | 2 +- ...re-1.3.1-SNAPSHOT.ivy => helix-core-1.3.2-SNAPSHOT.ivy} | 2 +- ...ck-1.3.1-SNAPSHOT.ivy => helix-lock-1.3.2-SNAPSHOT.ivy} | 4 ++-- ...st-1.3.1-SNAPSHOT.ivy => helix-rest-1.3.2-SNAPSHOT.ivy} | 4 ++-- ...APSHOT.ivy => helix-view-aggregator-1.3.2-SNAPSHOT.ivy} | 4 ++-- ...t-1.3.1-SNAPSHOT.ivy => meta-client-1.3.2-SNAPSHOT.ivy} | 4 ++-- ... => metadata-store-directory-common-1.3.2-SNAPSHOT.ivy} | 4 ++-- 3.1-SNAPSHOT.ivy => metrics-common-1.3.2-SNAPSHOT.ivy} | 2 +- website/src/site/markdown/index.md | 10 +- website/src/site/resources/.htaccess | 4 ++-- website/src/site/site.xml | 14 +++--- ...1.3.1-SNAPSHOT.ivy => zookeeper-api-1.3.2-SNAPSHOT.ivy} | 2 +- 14 files changed, 36 insertions(+), 36 deletions(-) rename helix-admin-webapp/{helix-admin-webapp-1.3.1-SNAPSHOT.ivy => helix-admin-webapp-1.3.2-SNAPSHOT.ivy} (97%) rename helix-agent/{helix-agent-1.3.1-SNAPSHOT.ivy => helix-agent-1.3.2-SNAPSHOT.ivy} (96%) rename helix-common/{helix-common-1.3.1-SNAPSHOT.ivy => helix-common-1.3.2-SNAPSHOT.ivy} (99%) rename helix-core/{helix-core-1.3.1-SNAPSHOT.ivy => helix-core-1.3.2-SNAPSHOT.ivy} (99%) rename helix-lock/{helix-lock-1.3.1-SNAPSHOT.ivy => helix-lock-1.3.2-SNAPSHOT.ivy} (97%) rename helix-rest/{helix-rest-1.3.1-SNAPSHOT.ivy => helix-rest-1.3.2-SNAPSHOT.ivy} (97%) rename helix-view-aggregator/{helix-view-aggregator-1.3.1-SNAPSHOT.ivy => helix-view-aggregator-1.3.2-SNAPSHOT.ivy} (97%) rename meta-client/{meta-client-1.3.1-SNAPSHOT.ivy => meta-client-1.3.2-SNAPSHOT.ivy} (97%) rename metadata-store-directory-common/{metadata-store-directory-common-1.3.1-SNAPSHOT.ivy => metadata-store-directory-common-1.3.2-SNAPSHOT.ivy} (97%) rename metrics-common/{metrics-common-1.3.1-SNAPSHOT.ivy => metrics-common-1.3.2-SNAPSHOT.ivy} (99%) rename zookeeper-api/{zookeeper-api-1.3.1-SNAPSHOT.ivy => zookeeper-api-1.3.2-SNAPSHOT.ivy} (99%)
svn commit: r1912597 [1/2] - in /helix/site-content: ./ 0.9.9-docs/ 1.3.0-docs/ 1.3.1-docs/ 1.3.1-docs/recipes/ 1.3.1-docs/releasenotes/
Author: jxue Date: Fri Sep 29 06:24:52 2023 New Revision: 1912597 URL: http://svn.apache.org/viewvc?rev=1912597=rev Log: Site checkin for project Apache Helix :: Website Modified: helix/site-content/.htaccess helix/site-content/0.9.9-docs/dependencies.html helix/site-content/0.9.9-docs/dependency-info.html helix/site-content/0.9.9-docs/dependency-management.html helix/site-content/0.9.9-docs/distribution-management.html helix/site-content/0.9.9-docs/integration.html helix/site-content/0.9.9-docs/issue-tracking.html helix/site-content/0.9.9-docs/license.html helix/site-content/0.9.9-docs/mail-lists.html helix/site-content/0.9.9-docs/plugin-management.html helix/site-content/0.9.9-docs/plugins.html helix/site-content/0.9.9-docs/project-info.html helix/site-content/0.9.9-docs/project-summary.html helix/site-content/0.9.9-docs/source-repository.html helix/site-content/0.9.9-docs/team-list.html helix/site-content/1.3.0-docs/dependencies.html helix/site-content/1.3.0-docs/dependency-info.html helix/site-content/1.3.0-docs/dependency-management.html helix/site-content/1.3.0-docs/distribution-management.html helix/site-content/1.3.0-docs/integration.html helix/site-content/1.3.0-docs/issue-tracking.html helix/site-content/1.3.0-docs/license.html helix/site-content/1.3.0-docs/mail-lists.html helix/site-content/1.3.0-docs/plugin-management.html helix/site-content/1.3.0-docs/plugins.html helix/site-content/1.3.0-docs/project-info.html helix/site-content/1.3.0-docs/project-summary.html helix/site-content/1.3.0-docs/source-repository.html helix/site-content/1.3.0-docs/team-list.html helix/site-content/1.3.1-docs/Building.html helix/site-content/1.3.1-docs/Features.html helix/site-content/1.3.1-docs/Metrics.html helix/site-content/1.3.1-docs/Quickstart.html helix/site-content/1.3.1-docs/Tutorial.html helix/site-content/1.3.1-docs/auto_exit_maintenance.html helix/site-content/1.3.1-docs/dependencies.html helix/site-content/1.3.1-docs/dependency-info.html helix/site-content/1.3.1-docs/dependency-management.html helix/site-content/1.3.1-docs/design_crushed.html helix/site-content/1.3.1-docs/distribution-management.html helix/site-content/1.3.1-docs/download.html helix/site-content/1.3.1-docs/index.html helix/site-content/1.3.1-docs/integration.html helix/site-content/1.3.1-docs/issue-tracking.html helix/site-content/1.3.1-docs/license.html helix/site-content/1.3.1-docs/mail-lists.html helix/site-content/1.3.1-docs/plugin-management.html helix/site-content/1.3.1-docs/plugins.html helix/site-content/1.3.1-docs/privacy-policy.html helix/site-content/1.3.1-docs/project-info.html helix/site-content/1.3.1-docs/project-summary.html helix/site-content/1.3.1-docs/quota_scheduling.html helix/site-content/1.3.1-docs/recipes/lock_manager.html helix/site-content/1.3.1-docs/recipes/rabbitmq_consumer_group.html helix/site-content/1.3.1-docs/recipes/rsync_replicated_file_store.html helix/site-content/1.3.1-docs/recipes/service_discovery.html helix/site-content/1.3.1-docs/recipes/task_dag_execution.html helix/site-content/1.3.1-docs/releasenotes/release-1.3.1.html helix/site-content/1.3.1-docs/source-repository.html helix/site-content/1.3.1-docs/team-list.html helix/site-content/1.3.1-docs/tutorial_admin.html helix/site-content/1.3.1-docs/tutorial_agent.html helix/site-content/1.3.1-docs/tutorial_cloud_support.html helix/site-content/1.3.1-docs/tutorial_controller.html helix/site-content/1.3.1-docs/tutorial_customized_view.html helix/site-content/1.3.1-docs/tutorial_distributed_lock.html helix/site-content/1.3.1-docs/tutorial_health.html helix/site-content/1.3.1-docs/tutorial_messaging.html helix/site-content/1.3.1-docs/tutorial_participant.html helix/site-content/1.3.1-docs/tutorial_propstore.html helix/site-content/1.3.1-docs/tutorial_rebalance.html helix/site-content/1.3.1-docs/tutorial_rest_service.html helix/site-content/1.3.1-docs/tutorial_spectator.html helix/site-content/1.3.1-docs/tutorial_state.html helix/site-content/1.3.1-docs/tutorial_task_framework.html helix/site-content/1.3.1-docs/tutorial_task_throttling.html helix/site-content/1.3.1-docs/tutorial_throttling.html helix/site-content/1.3.1-docs/tutorial_ui.html helix/site-content/1.3.1-docs/tutorial_user_content_store.html helix/site-content/1.3.1-docs/tutorial_user_def_rebalancer.html helix/site-content/1.3.1-docs/tutorial_yaml.html helix/site-content/dependency-convergence.html helix/site-content/dependency-info.html helix/site-content/dependency-management.html helix/site-content/distribution-management.html helix/site-content/index.html helix/site-content/integration.html helix/site-content/issue-tracking.html helix
svn commit: r1912597 [2/2] - in /helix/site-content: ./ 0.9.9-docs/ 1.3.0-docs/ 1.3.1-docs/ 1.3.1-docs/recipes/ 1.3.1-docs/releasenotes/
Modified: helix/site-content/1.3.1-docs/tutorial_throttling.html URL: http://svn.apache.org/viewvc/helix/site-content/1.3.1-docs/tutorial_throttling.html?rev=1912597=1912596=1912597=diff == --- helix/site-content/1.3.1-docs/tutorial_throttling.html (original) +++ helix/site-content/1.3.1-docs/tutorial_throttling.html Fri Sep 29 06:24:52 2023 @@ -7,7 +7,7 @@ @@ -130,7 +130,7 @@ Release 1.3.1 / Tutorial - Throttling - Last Published: 2023-09-24 + Last Published: 2023-09-28 Modified: helix/site-content/1.3.1-docs/tutorial_ui.html URL: http://svn.apache.org/viewvc/helix/site-content/1.3.1-docs/tutorial_ui.html?rev=1912597=1912596=1912597=diff == --- helix/site-content/1.3.1-docs/tutorial_ui.html (original) +++ helix/site-content/1.3.1-docs/tutorial_ui.html Fri Sep 29 06:24:52 2023 @@ -7,7 +7,7 @@ @@ -130,7 +130,7 @@ Release 1.3.1 / Tutorial - Helix UI Setup - Last Published: 2023-09-24 + Last Published: 2023-09-28 Modified: helix/site-content/1.3.1-docs/tutorial_user_content_store.html URL: http://svn.apache.org/viewvc/helix/site-content/1.3.1-docs/tutorial_user_content_store.html?rev=1912597=1912596=1912597=diff == --- helix/site-content/1.3.1-docs/tutorial_user_content_store.html (original) +++ helix/site-content/1.3.1-docs/tutorial_user_content_store.html Fri Sep 29 06:24:52 2023 @@ -7,7 +7,7 @@ @@ -130,7 +130,7 @@ Release 1.3.1 / Tutorial - User Defined Content Store for Tasks - Last Published: 2023-09-24 + Last Published: 2023-09-28 Modified: helix/site-content/1.3.1-docs/tutorial_user_def_rebalancer.html URL: http://svn.apache.org/viewvc/helix/site-content/1.3.1-docs/tutorial_user_def_rebalancer.html?rev=1912597=1912596=1912597=diff == --- helix/site-content/1.3.1-docs/tutorial_user_def_rebalancer.html (original) +++ helix/site-content/1.3.1-docs/tutorial_user_def_rebalancer.html Fri Sep 29 06:24:52 2023 @@ -7,7 +7,7 @@ @@ -130,7 +130,7 @@ Release 1.3.1 / Tutorial - User-Defined Rebalancing - Last Published: 2023-09-24 + Last Published: 2023-09-28 Modified: helix/site-content/1.3.1-docs/tutorial_yaml.html URL: http://svn.apache.org/viewvc/helix/site-content/1.3.1-docs/tutorial_yaml.html?rev=1912597=1912596=1912597=diff == --- helix/site-content/1.3.1-docs/tutorial_yaml.html (original) +++ helix/site-content/1.3.1-docs/tutorial_yaml.html Fri Sep 29 06:24:52 2023 @@ -7,7 +7,7 @@ @@ -130,7 +130,7 @@ Release 1.3.1 / Tutorial - YAML Cluster Setup - Last Published: 2023-09-24 + Last Published: 2023-09-28 Modified: helix/site-content/dependency-convergence.html URL: http://svn.apache.org/viewvc/helix/site-content/dependency-convergence.html?rev=1912597=1912596=1912597=diff == --- helix/site-content/dependency-convergence.html (original) +++ helix/site-content/dependency-convergence.html Fri Sep 29 06:24:52 2023 @@ -83,19 +83,19 @@ Documentation + 1.3.1 1.3.0 - 1.2.0 0.9.10 (0.9.9) trunk
[helix] branch master updated (84d89efaf -> 47c1718c9)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/helix.git from 84d89efaf Fix CurrentStateComputationStage when mis configure(#2635) new 0db2f3ae3 Release note 1.3.1 new 47c1718c9 Remove 1.2.0 docs The 2 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: website/1.2.0/src/site/xdoc/download.xml.vm| 226 - website/{1.2.0 => 1.3.1}/pom.xml | 4 +- .../src/site/apt/privacy-policy.apt| 0 .../src/site/apt/releasenotes/release-1.3.1.apt} | 39 ++-- .../{1.2.0 => 1.3.1}/src/site/markdown/Building.md | 4 +- .../{1.2.0 => 1.3.1}/src/site/markdown/Features.md | 0 .../{1.2.0 => 1.3.1}/src/site/markdown/Metrics.md | 0 .../src/site/markdown/Quickstart.md| 6 +- .../{1.2.0 => 1.3.1}/src/site/markdown/Tutorial.md | 0 .../src/site/markdown/auto_exit_maintenance.md | 0 .../src/site/markdown/design_crushed.md| 0 .../{1.2.0 => 1.3.1}/src/site/markdown/index.md| 6 +- .../src/site/markdown/quota_scheduling.md | 0 .../src/site/markdown/recipes/lock_manager.md | 2 +- .../markdown/recipes/rabbitmq_consumer_group.md| 2 +- .../recipes/rsync_replicated_file_store.md | 2 +- .../src/site/markdown/recipes/service_discovery.md | 0 .../site/markdown/recipes/task_dag_execution.md| 2 +- .../src/site/markdown/tutorial_admin.md| 6 +- .../src/site/markdown/tutorial_agent.md| 0 .../src/site/markdown/tutorial_cloud_support.md| 0 .../src/site/markdown/tutorial_controller.md | 0 .../src/site/markdown/tutorial_customized_view.md | 0 .../src/site/markdown/tutorial_distributed_lock.md | 0 .../src/site/markdown/tutorial_health.md | 0 .../src/site/markdown/tutorial_messaging.md| 2 +- .../src/site/markdown/tutorial_participant.md | 0 .../src/site/markdown/tutorial_propstore.md| 2 +- .../src/site/markdown/tutorial_rebalance.md| 0 .../src/site/markdown/tutorial_rest_service.md | 6 +- .../src/site/markdown/tutorial_spectator.md| 0 .../src/site/markdown/tutorial_state.md| 0 .../src/site/markdown/tutorial_task_framework.md | 0 .../src/site/markdown/tutorial_task_throttling.md | 0 .../src/site/markdown/tutorial_throttling.md | 0 .../src/site/markdown/tutorial_ui.md | 2 +- .../site/markdown/tutorial_user_content_store.md | 0 .../site/markdown/tutorial_user_def_rebalancer.md | 0 .../src/site/markdown/tutorial_yaml.md | 0 .../{1.2.0 => 1.3.1}/src/site/resources/.htaccess | 0 .../src/site/resources/JobExample.json | 0 .../src/site/resources/WorkflowExample.json| 0 .../resources/css/bootstrap-responsive.min.css | 0 .../src/site/resources/css/bootstrap.min.css | 0 .../src/site/resources/download.cgi| 0 .../images/CustomizedViewSystemArchitecture.jpeg | Bin .../images/HelixPriorityLockWorkflow.jpeg | Bin .../src/site/resources/images/PFS-Generic.png | Bin .../images/ParticipantAutoRegistrationLogic.png| Bin .../src/site/resources/images/RSYNC_BASED_PFS.png | Bin .../site/resources/images/TaskFrameworkLayers.png | Bin .../src/site/resources/images/UIScreenshot.png | Bin .../src/site/resources/images/UIScreenshot2.png| Bin .../resources/images/auto-exit-maintenance.jpg | Bin .../images/design/crushed/after-using-crushed.png | Bin .../images/design/crushed/before-using-crush.png | Bin .../resources/images/design/crushed/classes.png| Bin .../images/design/crushed/crushed-master-dist.png | Bin .../design/crushed/crushed-partition-dist.png | Bin .../images/design/crushed/cursh-master-dist.png| Bin .../images/design/crushed/cursh-partition-dist.png | Bin .../crushed/example-cluster-master-dist-after.png | Bin .../design/crushed/example-cluster-master-dist.png | Bin .../crushed/example-cluster-partition-dist.png | Bin .../crushed/example-movement-on-expansion.png | Bin .../design/crushed/node-down-master-move.png | Bin .../design/crushed/node-down-partition-move.png| Bin .../images/design/crushed/performance.png | Bin .../images/quota_InstanceCapacityManager.jpeg | Bin .../src/site/resources/images/quota_intro.png | Bin .../src/site/resources/js/bootstrap.min.js | 0 .../src/site/resources/js/jquery.min.js| 0 website/{1.2.0 => 1.3.1}/src/site/site.xml | 8 +- .../{1.3.0 => 1.3.1}/src/site/xdoc/download.xml.vm | 0 website/{
svn commit: r64254 - /dev/helix/1.3.1/ /release/helix/1.3.1/
Author: jxue Date: Fri Sep 29 05:07:24 2023 New Revision: 64254 Log: Release 1.3.1 Added: release/helix/1.3.1/ - copied from r64253, dev/helix/1.3.1/ Removed: dev/helix/1.3.1/
svn commit: r1912533 [13/15] - in /helix/site-content: ./ 0.9.9-docs/ 1.2.0-docs/ 1.3.0-docs/ 1.3.1-docs/ 1.3.1-docs/css/ 1.3.1-docs/images/ 1.3.1-docs/images/design/ 1.3.1-docs/images/design/crushed/
Added: helix/site-content/1.3.1-docs/tutorial_rest_service.html URL: http://svn.apache.org/viewvc/helix/site-content/1.3.1-docs/tutorial_rest_service.html?rev=1912533=auto == --- helix/site-content/1.3.1-docs/tutorial_rest_service.html (added) +++ helix/site-content/1.3.1-docs/tutorial_rest_service.html Mon Sep 25 03:26:49 2023 @@ -0,0 +1,1232 @@ + + + + + + + + + + + + + + Apache Helix Tutorial - REST Service 2.0 + + + + + + + + + + + + + + + + + + + + + + + + + +var _paq = window._paq = window._paq || []; +_paq.push(['disableCookies']); +_paq.push(['trackPageView']); +_paq.push(['enableLinkTracking']); + +(function() { +var u="https://analytics.apache.org"; +_paq.push(['setTrackerUrl', u+'/matomo.php']); +_paq.push(['setSiteId', '15']); +var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; +g.async=true; g.src=u+'/matomo.js'; s.parentNode.insertBefore(g,s); +})(); + + + + + + + + + + + + + + + + http://helix.apache.org;>Apache Helix + + + Helix 1.3.1 + + Get Helix + + Download + Building + Release Notes + + + + Hands-On + + Quick Start + Tutorial + Javadocs + + + + Recipes + + Distributed lock manager + Rabbit MQ consumer group + Rsync replicated file store + Service discovery + Distributed task DAG execution + + + + + + + + + + + + + + + + + + http://helix.apache.org/; id="bannerLeft"> + A cluster management framework for partitioned and replicated distributed resources + + + http://www.apache.org/; id="bannerRight"> + + + + + + + Apache Helix + / + Release 1.3.1 + / + Tutorial - REST Service 2.0 + Last Published:
svn commit: r1912533 [10/15] - in /helix/site-content: ./ 0.9.9-docs/ 1.2.0-docs/ 1.3.0-docs/ 1.3.1-docs/ 1.3.1-docs/css/ 1.3.1-docs/images/ 1.3.1-docs/images/design/ 1.3.1-docs/images/design/crushed/
Added: helix/site-content/1.3.1-docs/tutorial_admin.html URL: http://svn.apache.org/viewvc/helix/site-content/1.3.1-docs/tutorial_admin.html?rev=1912533=auto == --- helix/site-content/1.3.1-docs/tutorial_admin.html (added) +++ helix/site-content/1.3.1-docs/tutorial_admin.html Mon Sep 25 03:26:49 2023 @@ -0,0 +1,808 @@ + + + + + + + + + + + + + + Apache Helix Tutorial - Admin Operations + + + + + + + + + + + + + + + + + + + + + + + + + +var _paq = window._paq = window._paq || []; +_paq.push(['disableCookies']); +_paq.push(['trackPageView']); +_paq.push(['enableLinkTracking']); + +(function() { +var u="https://analytics.apache.org"; +_paq.push(['setTrackerUrl', u+'/matomo.php']); +_paq.push(['setSiteId', '15']); +var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; +g.async=true; g.src=u+'/matomo.js'; s.parentNode.insertBefore(g,s); +})(); + + + + + + + + + + + + + + + + http://helix.apache.org;>Apache Helix + + + Helix 1.3.1 + + Get Helix + + Download + Building + Release Notes + + + + Hands-On + + Quick Start + Tutorial + Javadocs + + + + Recipes + + Distributed lock manager + Rabbit MQ consumer group + Rsync replicated file store + Service discovery + Distributed task DAG execution + + + + + + + + + + + + + + + + + + http://helix.apache.org/; id="bannerLeft"> + A cluster management framework for partitioned and replicated distributed resources + + + http://www.apache.org/; id="bannerRight"> + + + + + + + Apache Helix + / + Release 1.3.1 + / + Tutorial - Admin Operations + Last Published: 2023-09-24 +
svn commit: r1912533 [11/15] - in /helix/site-content: ./ 0.9.9-docs/ 1.2.0-docs/ 1.3.0-docs/ 1.3.1-docs/ 1.3.1-docs/css/ 1.3.1-docs/images/ 1.3.1-docs/images/design/ 1.3.1-docs/images/design/crushed/
Added: helix/site-content/1.3.1-docs/tutorial_controller.html URL: http://svn.apache.org/viewvc/helix/site-content/1.3.1-docs/tutorial_controller.html?rev=1912533=auto == --- helix/site-content/1.3.1-docs/tutorial_controller.html (added) +++ helix/site-content/1.3.1-docs/tutorial_controller.html Mon Sep 25 03:26:49 2023 @@ -0,0 +1,383 @@ + + + + + + + + + + + + + + Apache Helix Tutorial - Controller + + + + + + + + + + + + + + + + + + + + + + + + + +var _paq = window._paq = window._paq || []; +_paq.push(['disableCookies']); +_paq.push(['trackPageView']); +_paq.push(['enableLinkTracking']); + +(function() { +var u="https://analytics.apache.org"; +_paq.push(['setTrackerUrl', u+'/matomo.php']); +_paq.push(['setSiteId', '15']); +var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; +g.async=true; g.src=u+'/matomo.js'; s.parentNode.insertBefore(g,s); +})(); + + + + + + + + + + + + + + + + http://helix.apache.org;>Apache Helix + + + Helix 1.3.1 + + Get Helix + + Download + Building + Release Notes + + + + Hands-On + + Quick Start + Tutorial + Javadocs + + + + Recipes + + Distributed lock manager + Rabbit MQ consumer group + Rsync replicated file store + Service discovery + Distributed task DAG execution + + + + + + + + + + + + + + + + + + http://helix.apache.org/; id="bannerLeft"> + A cluster management framework for partitioned and replicated distributed resources + + + http://www.apache.org/; id="bannerRight"> + + + + + + + Apache Helix + / + Release 1.3.1 + / + Tutorial - Controller + Last Published: 2023-09-24 +
svn commit: r1912533 [9/15] - in /helix/site-content: ./ 0.9.9-docs/ 1.2.0-docs/ 1.3.0-docs/ 1.3.1-docs/ 1.3.1-docs/css/ 1.3.1-docs/images/ 1.3.1-docs/images/design/ 1.3.1-docs/images/design/crushed/
Added: helix/site-content/1.3.1-docs/recipes/service_discovery.html URL: http://svn.apache.org/viewvc/helix/site-content/1.3.1-docs/recipes/service_discovery.html?rev=1912533=auto == --- helix/site-content/1.3.1-docs/recipes/service_discovery.html (added) +++ helix/site-content/1.3.1-docs/recipes/service_discovery.html Mon Sep 25 03:26:49 2023 @@ -0,0 +1,414 @@ + + + + + + + + + + + + + + Apache Helix Service Discovery + + + + + + + + + + + + + + + + + + + + + + + + + +var _paq = window._paq = window._paq || []; +_paq.push(['disableCookies']); +_paq.push(['trackPageView']); +_paq.push(['enableLinkTracking']); + +(function() { +var u="https://analytics.apache.org"; +_paq.push(['setTrackerUrl', u+'/matomo.php']); +_paq.push(['setSiteId', '15']); +var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; +g.async=true; g.src=u+'/matomo.js'; s.parentNode.insertBefore(g,s); +})(); + + + + + + + + + + + + + + + + http://helix.apache.org;>Apache Helix + + + Helix 1.3.1 + + Get Helix + + Download + Building + Release Notes + + + + Hands-On + + Quick Start + Tutorial + Javadocs + + + + Recipes + + Distributed lock manager + Rabbit MQ consumer group + Rsync replicated file store + Service discovery + Distributed task DAG execution + + + + + + + + + + + + + + + + + + http://helix.apache.org/; id="bannerLeft"> + A cluster management framework for partitioned and replicated distributed resources + + + http://www.apache.org/; id="bannerRight"> + + + + + + + Apache Helix + / + Release 1.3.1 + / + Service Discovery + Last Published: 2023-09-24 +
svn commit: r1912533 [14/15] - in /helix/site-content: ./ 0.9.9-docs/ 1.2.0-docs/ 1.3.0-docs/ 1.3.1-docs/ 1.3.1-docs/css/ 1.3.1-docs/images/ 1.3.1-docs/images/design/ 1.3.1-docs/images/design/crushed/
Added: helix/site-content/1.3.1-docs/tutorial_task_framework.html URL: http://svn.apache.org/viewvc/helix/site-content/1.3.1-docs/tutorial_task_framework.html?rev=1912533=auto == --- helix/site-content/1.3.1-docs/tutorial_task_framework.html (added) +++ helix/site-content/1.3.1-docs/tutorial_task_framework.html Mon Sep 25 03:26:49 2023 @@ -0,0 +1,715 @@ + + + + + + + + + + + + + + Apache Helix Tutorial - Task Framework + + + + + + + + + + + + + + + + + + + + + + + + + +var _paq = window._paq = window._paq || []; +_paq.push(['disableCookies']); +_paq.push(['trackPageView']); +_paq.push(['enableLinkTracking']); + +(function() { +var u="https://analytics.apache.org"; +_paq.push(['setTrackerUrl', u+'/matomo.php']); +_paq.push(['setSiteId', '15']); +var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; +g.async=true; g.src=u+'/matomo.js'; s.parentNode.insertBefore(g,s); +})(); + + + + + + + + + + + + + + + + http://helix.apache.org;>Apache Helix + + + Helix 1.3.1 + + Get Helix + + Download + Building + Release Notes + + + + Hands-On + + Quick Start + Tutorial + Javadocs + + + + Recipes + + Distributed lock manager + Rabbit MQ consumer group + Rsync replicated file store + Service discovery + Distributed task DAG execution + + + + + + + + + + + + + + + + + + http://helix.apache.org/; id="bannerLeft"> + A cluster management framework for partitioned and replicated distributed resources + + + http://www.apache.org/; id="bannerRight"> + + + + + + + Apache Helix + / + Release 1.3.1 + / + Tutorial - Task Framework + Last Published:
svn commit: r1912533 [8/15] - in /helix/site-content: ./ 0.9.9-docs/ 1.2.0-docs/ 1.3.0-docs/ 1.3.1-docs/ 1.3.1-docs/css/ 1.3.1-docs/images/ 1.3.1-docs/images/design/ 1.3.1-docs/images/design/crushed/
Added: helix/site-content/1.3.1-docs/quota_scheduling.html URL: http://svn.apache.org/viewvc/helix/site-content/1.3.1-docs/quota_scheduling.html?rev=1912533=auto == --- helix/site-content/1.3.1-docs/quota_scheduling.html (added) +++ helix/site-content/1.3.1-docs/quota_scheduling.html Mon Sep 25 03:26:49 2023 @@ -0,0 +1,442 @@ + + + + + + + + + + + + + + Apache Helix Quota-based Task Scheduling + + + + + + + + + + + + + + + + + + + + + + + + + +var _paq = window._paq = window._paq || []; +_paq.push(['disableCookies']); +_paq.push(['trackPageView']); +_paq.push(['enableLinkTracking']); + +(function() { +var u="https://analytics.apache.org"; +_paq.push(['setTrackerUrl', u+'/matomo.php']); +_paq.push(['setSiteId', '15']); +var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; +g.async=true; g.src=u+'/matomo.js'; s.parentNode.insertBefore(g,s); +})(); + + + + + + + + + + + + + + + + http://helix.apache.org;>Apache Helix + + + Helix 1.3.1 + + Get Helix + + Download + Building + Release Notes + + + + Hands-On + + Quick Start + Tutorial + Javadocs + + + + Recipes + + Distributed lock manager + Rabbit MQ consumer group + Rsync replicated file store + Service discovery + Distributed task DAG execution + + + + + + + + + + + + + + + + + + http://helix.apache.org/; id="bannerLeft"> + A cluster management framework for partitioned and replicated distributed resources + + + http://www.apache.org/; id="bannerRight"> + + + + + + + Apache Helix + / + Release 1.3.1 + / + Quota-based Task Scheduling + Last Published: 2023-09-24 +
svn commit: r1912533 [15/15] - in /helix/site-content: ./ 0.9.9-docs/ 1.2.0-docs/ 1.3.0-docs/ 1.3.1-docs/ 1.3.1-docs/css/ 1.3.1-docs/images/ 1.3.1-docs/images/design/ 1.3.1-docs/images/design/crushed/
Added: helix/site-content/1.3.1-docs/tutorial_user_def_rebalancer.html URL: http://svn.apache.org/viewvc/helix/site-content/1.3.1-docs/tutorial_user_def_rebalancer.html?rev=1912533=auto == --- helix/site-content/1.3.1-docs/tutorial_user_def_rebalancer.html (added) +++ helix/site-content/1.3.1-docs/tutorial_user_def_rebalancer.html Mon Sep 25 03:26:49 2023 @@ -0,0 +1,400 @@ + + + + + + + + + + + + + + Apache Helix Tutorial - User-Defined Rebalancing + + + + + + + + + + + + + + + + + + + + + + + + + +var _paq = window._paq = window._paq || []; +_paq.push(['disableCookies']); +_paq.push(['trackPageView']); +_paq.push(['enableLinkTracking']); + +(function() { +var u="https://analytics.apache.org"; +_paq.push(['setTrackerUrl', u+'/matomo.php']); +_paq.push(['setSiteId', '15']); +var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; +g.async=true; g.src=u+'/matomo.js'; s.parentNode.insertBefore(g,s); +})(); + + + + + + + + + + + + + + + + http://helix.apache.org;>Apache Helix + + + Helix 1.3.1 + + Get Helix + + Download + Building + Release Notes + + + + Hands-On + + Quick Start + Tutorial + Javadocs + + + + Recipes + + Distributed lock manager + Rabbit MQ consumer group + Rsync replicated file store + Service discovery + Distributed task DAG execution + + + + + + + + + + + + + + + + + + http://helix.apache.org/; id="bannerLeft"> + A cluster management framework for partitioned and replicated distributed resources + + + http://www.apache.org/; id="bannerRight"> + + + + + + + Apache Helix + / + Release 1.3.1 + / + Tutorial - User-Defined Rebalancing +
svn commit: r1912533 [4/15] - in /helix/site-content: ./ 0.9.9-docs/ 1.2.0-docs/ 1.3.0-docs/ 1.3.1-docs/ 1.3.1-docs/css/ 1.3.1-docs/images/ 1.3.1-docs/images/design/ 1.3.1-docs/images/design/crushed/
Added: helix/site-content/1.3.1-docs/css/bootstrap-responsive.min.css URL: http://svn.apache.org/viewvc/helix/site-content/1.3.1-docs/css/bootstrap-responsive.min.css?rev=1912533=auto == --- helix/site-content/1.3.1-docs/css/bootstrap-responsive.min.css (added) +++ helix/site-content/1.3.1-docs/css/bootstrap-responsive.min.css Mon Sep 25 03:26:49 2023 @@ -0,0 +1,9 @@ +/*! + * Bootstrap Responsive v2.2.2 + * + * Copyright 2012 Twitter, Inc + * Licensed under the Apache License v2.0 + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Designed and built with all the love in the world @twitter by @mdo and @fat. + */@-ms-viewport{width:device-width}.clearfix{*zoom:1}.clearfix:before,.clearfix:after{display:table;line-height:0;content:""}.clearfix:after{clear:both}.hide-text{font:0/0 a;color:transparent;text-shadow:none;background-color:transparent;border:0}.input-block-level{display:block;width:100%;min-height:30px;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}.hidden{display:none;visibility:hidden}.visible-phone{display:none!important}.visible-tablet{display:none!important}.hidden-desktop{display:none!important}.visible-desktop{display:inherit!important}@media(min-width:768px) and (max-width:979px){.hidden-desktop{display:inherit!important}.visible-desktop{display:none!important}.visible-tablet{display:inherit!important}.hidden-tablet{display:none!important}}@media(max-width:767px){.hidden-desktop{display:inherit!important}.visible-desktop{display:none!important}.visible-phone{display:inherit!important}.hidden-phone{display:none!important}}@media(min-width:1 200px){.row{margin-left:-30px;*zoom:1}.row:before,.row:after{display:table;line-height:0;content:""}.row:after{clear:both}[class*="span"]{float:left;min-height:1px;margin-left:30px}.container,.navbar-static-top .container,.navbar-fixed-top .container,.navbar-fixed-bottom .container{width:1170px}.span12{width:1170px}.span11{width:1070px}.span10{width:970px}.span9{width:870px}.span8{width:770px}.span7{width:670px}.span6{width:570px}.span5{width:470px}.span4{width:370px}.span3{width:270px}.span2{width:170px}.span1{width:70px}.offset12{margin-left:1230px}.offset11{margin-left:1130px}.offset10{margin-left:1030px}.offset9{margin-left:930px}.offset8{margin-left:830px}.offset7{margin-left:730px}.offset6{margin-left:630px}.offset5{margin-left:530px}.offset4{margin-left:430px}.offset3{margin-left:330px}.offset2{margin-left:230px}.offset1{margin-left:130px}.row-fluid{width:100%;*zoom:1}.row-fluid:before,.row-fluid:after{display:table;line-height:0;content:""}.row-fluid:after{clear:both}.row-fl uid [class*="span"]{display:block;float:left;width:100%;min-height:30px;margin-left:2.564102564102564%;*margin-left:2.5109110747408616%;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}.row-fluid [class*="span"]:first-child{margin-left:0}.row-fluid .controls-row [class*="span"]+[class*="span"]{margin-left:2.564102564102564%}.row-fluid .span12{width:100%;*width:99.94680851063829%}.row-fluid .span11{width:91.45299145299145%;*width:91.39979996362975%}.row-fluid .span10{width:82.90598290598291%;*width:82.8527914166212%}.row-fluid .span9{width:74.35897435897436%;*width:74.30578286961266%}.row-fluid .span8{width:65.81196581196582%;*width:65.75877432260411%}.row-fluid .span7{width:57.26495726495726%;*width:57.21176577559556%}.row-fluid .span6{width:48.717948717948715%;*width:48.664757228587014%}.row-fluid .span5{width:40.17094017094017%;*width:40.11774868157847%}.row-fluid .span4{width:31.623931623931625%;*width:31.570740134569924%}.row-fluid .span3{width:23.0 76923076923077%;*width:23.023731587561375%}.row-fluid .span2{width:14.52991452991453%;*width:14.476723040552828%}.row-fluid .span1{width:5.982905982905983%;*width:5.929714493544281%}.row-fluid .offset12{margin-left:105.12820512820512%;*margin-left:105.02182214948171%}.row-fluid .offset12:first-child{margin-left:102.56410256410257%;*margin-left:102.45771958537915%}.row-fluid .offset11{margin-left:96.58119658119658%;*margin-left:96.47481360247316%}.row-fluid .offset11:first-child{margin-left:94.01709401709402%;*margin-left:93.91071103837061%}.row-fluid .offset10{margin-left:88.03418803418803%;*margin-left:87.92780505546462%}.row-fluid .offset10:first-child{margin-left:85.47008547008548%;*margin-left:85.36370249136206%}.row-fluid .offset9{margin-left:79.48717948717949%;*margin-left:79.38079650845607%}.row-fluid .offset9:first-child{margin-left:76.92307692307693%;*margin-left:76.81669394435352%}.row-fluid .offset8{margin-left:70.94017094017094%;*margin-left:70.83378796144753%}.row-fluid .offset8:first-child{margin-left:68.37606837606839%;*margin-left:68.26968539734497%}.row-fluid .offset7{margin-left:62.393162393162385%;*margin-left:62.28677941443899%}.row-fluid
svn commit: r1912533 [3/15] - in /helix/site-content: ./ 0.9.9-docs/ 1.2.0-docs/ 1.3.0-docs/ 1.3.1-docs/ 1.3.1-docs/css/ 1.3.1-docs/images/ 1.3.1-docs/images/design/ 1.3.1-docs/images/design/crushed/
Added: helix/site-content/1.3.1-docs/Quickstart.html URL: http://svn.apache.org/viewvc/helix/site-content/1.3.1-docs/Quickstart.html?rev=1912533=auto == --- helix/site-content/1.3.1-docs/Quickstart.html (added) +++ helix/site-content/1.3.1-docs/Quickstart.html Mon Sep 25 03:26:49 2023 @@ -0,0 +1,869 @@ + + + + + + + + + + + + + + Apache Helix Quickstart + + + + + + + + + + + + + + + + + + + + + + + + + +var _paq = window._paq = window._paq || []; +_paq.push(['disableCookies']); +_paq.push(['trackPageView']); +_paq.push(['enableLinkTracking']); + +(function() { +var u="https://analytics.apache.org"; +_paq.push(['setTrackerUrl', u+'/matomo.php']); +_paq.push(['setSiteId', '15']); +var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; +g.async=true; g.src=u+'/matomo.js'; s.parentNode.insertBefore(g,s); +})(); + + + + + + + + + + + + + + + + http://helix.apache.org;>Apache Helix + + + Helix 1.3.1 + + Get Helix + + Download + Building + Release Notes + + + + Hands-On + + Quick Start + Tutorial + Javadocs + + + + Recipes + + Distributed lock manager + Rabbit MQ consumer group + Rsync replicated file store + Service discovery + Distributed task DAG execution + + + + + + + + + + + + + + + + + + http://helix.apache.org/; id="bannerLeft"> + A cluster management framework for partitioned and replicated distributed resources + + + http://www.apache.org/; id="bannerRight"> + + + + + + + Apache Helix + / + Release 1.3.1 + / + Quickstart + Last Published: 2023-09-24 + + + + + + +
svn commit: r1912533 [12/15] - in /helix/site-content: ./ 0.9.9-docs/ 1.2.0-docs/ 1.3.0-docs/ 1.3.1-docs/ 1.3.1-docs/css/ 1.3.1-docs/images/ 1.3.1-docs/images/design/ 1.3.1-docs/images/design/crushed/
Added: helix/site-content/1.3.1-docs/tutorial_messaging.html URL: http://svn.apache.org/viewvc/helix/site-content/1.3.1-docs/tutorial_messaging.html?rev=1912533=auto == --- helix/site-content/1.3.1-docs/tutorial_messaging.html (added) +++ helix/site-content/1.3.1-docs/tutorial_messaging.html Mon Sep 25 03:26:49 2023 @@ -0,0 +1,295 @@ + + + + + + + + + + + + + + Apache Helix Tutorial - Messaging + + + + + + + + + + + + + + + + + + + + + + + + + +var _paq = window._paq = window._paq || []; +_paq.push(['disableCookies']); +_paq.push(['trackPageView']); +_paq.push(['enableLinkTracking']); + +(function() { +var u="https://analytics.apache.org"; +_paq.push(['setTrackerUrl', u+'/matomo.php']); +_paq.push(['setSiteId', '15']); +var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; +g.async=true; g.src=u+'/matomo.js'; s.parentNode.insertBefore(g,s); +})(); + + + + + + + + + + + + + + + + http://helix.apache.org;>Apache Helix + + + Helix 1.3.1 + + Get Helix + + Download + Building + Release Notes + + + + Hands-On + + Quick Start + Tutorial + Javadocs + + + + Recipes + + Distributed lock manager + Rabbit MQ consumer group + Rsync replicated file store + Service discovery + Distributed task DAG execution + + + + + + + + + + + + + + + + + + http://helix.apache.org/; id="bannerLeft"> + A cluster management framework for partitioned and replicated distributed resources + + + http://www.apache.org/; id="bannerRight"> + + + + + + + Apache Helix + / + Release 1.3.1 + / + Tutorial - Messaging + Last Published: 2023-09-24 +
svn commit: r1912533 [6/15] - in /helix/site-content: ./ 0.9.9-docs/ 1.2.0-docs/ 1.3.0-docs/ 1.3.1-docs/ 1.3.1-docs/css/ 1.3.1-docs/images/ 1.3.1-docs/images/design/ 1.3.1-docs/images/design/crushed/
Added: helix/site-content/1.3.1-docs/issue-tracking.html URL: http://svn.apache.org/viewvc/helix/site-content/1.3.1-docs/issue-tracking.html?rev=1912533=auto == --- helix/site-content/1.3.1-docs/issue-tracking.html (added) +++ helix/site-content/1.3.1-docs/issue-tracking.html Mon Sep 25 03:26:49 2023 @@ -0,0 +1,245 @@ + + + + + + + + + + + + + + Apache Helix Issue Management + + + + + + + + + + + + + + + + + + + + + + + + + +var _paq = window._paq = window._paq || []; +_paq.push(['disableCookies']); +_paq.push(['trackPageView']); +_paq.push(['enableLinkTracking']); + +(function() { +var u="https://analytics.apache.org"; +_paq.push(['setTrackerUrl', u+'/matomo.php']); +_paq.push(['setSiteId', '15']); +var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; +g.async=true; g.src=u+'/matomo.js'; s.parentNode.insertBefore(g,s); +})(); + + + + + + + + + + + + + + + + http://helix.apache.org;>Apache Helix + + + Helix 1.3.1 + + Get Helix + + Download + Building + Release Notes + + + + Hands-On + + Quick Start + Tutorial + Javadocs + + + + Recipes + + Distributed lock manager + Rabbit MQ consumer group + Rsync replicated file store + Service discovery + Distributed task DAG execution + + + + + + + + + + + + + + + + + + http://helix.apache.org/; id="bannerLeft"> + A cluster management framework for partitioned and replicated distributed resources + + + http://www.apache.org/; id="bannerRight"> + + + + + + + Apache Helix + / + Release 1.3.1 + / + Issue Management + Last Published: 2023-09-24 + + +
svn commit: r1912533 [1/15] - in /helix/site-content: ./ 0.9.9-docs/ 1.2.0-docs/ 1.3.0-docs/ 1.3.1-docs/ 1.3.1-docs/css/ 1.3.1-docs/images/ 1.3.1-docs/images/design/ 1.3.1-docs/images/design/crushed/
Author: jxue Date: Mon Sep 25 03:26:49 2023 New Revision: 1912533 URL: http://svn.apache.org/viewvc?rev=1912533=rev Log: Site checkin for project Apache Helix :: Website Added: helix/site-content/1.3.1-docs/ helix/site-content/1.3.1-docs/.htaccess helix/site-content/1.3.1-docs/Building.html helix/site-content/1.3.1-docs/Features.html helix/site-content/1.3.1-docs/JobExample.json helix/site-content/1.3.1-docs/Metrics.html helix/site-content/1.3.1-docs/Quickstart.html helix/site-content/1.3.1-docs/Tutorial.html helix/site-content/1.3.1-docs/WorkflowExample.json helix/site-content/1.3.1-docs/auto_exit_maintenance.html helix/site-content/1.3.1-docs/css/ helix/site-content/1.3.1-docs/css/bootstrap-responsive.min.css helix/site-content/1.3.1-docs/css/bootstrap.min.css helix/site-content/1.3.1-docs/css/bootswatch.css helix/site-content/1.3.1-docs/css/docs.css helix/site-content/1.3.1-docs/css/lightbox.css helix/site-content/1.3.1-docs/css/print.css helix/site-content/1.3.1-docs/css/reflow-skin.css helix/site-content/1.3.1-docs/css/site.css helix/site-content/1.3.1-docs/dependencies.html helix/site-content/1.3.1-docs/dependency-info.html helix/site-content/1.3.1-docs/dependency-management.html helix/site-content/1.3.1-docs/design_crushed.html helix/site-content/1.3.1-docs/distribution-management.html helix/site-content/1.3.1-docs/download.cgi helix/site-content/1.3.1-docs/download.html helix/site-content/1.3.1-docs/images/ helix/site-content/1.3.1-docs/images/CustomizedViewSystemArchitecture.jpeg (with props) helix/site-content/1.3.1-docs/images/HelixPriorityLockWorkflow.jpeg (with props) helix/site-content/1.3.1-docs/images/PFS-Generic.png (with props) helix/site-content/1.3.1-docs/images/ParticipantAutoRegistrationLogic.png (with props) helix/site-content/1.3.1-docs/images/RSYNC_BASED_PFS.png (with props) helix/site-content/1.3.1-docs/images/TaskFrameworkLayers.png (with props) helix/site-content/1.3.1-docs/images/UIScreenshot.png (with props) helix/site-content/1.3.1-docs/images/UIScreenshot2.png (with props) helix/site-content/1.3.1-docs/images/auto-exit-maintenance.jpg (with props) helix/site-content/1.3.1-docs/images/close.gif (with props) helix/site-content/1.3.1-docs/images/design/ helix/site-content/1.3.1-docs/images/design/crushed/ helix/site-content/1.3.1-docs/images/design/crushed/after-using-crushed.png (with props) helix/site-content/1.3.1-docs/images/design/crushed/before-using-crush.png (with props) helix/site-content/1.3.1-docs/images/design/crushed/classes.png (with props) helix/site-content/1.3.1-docs/images/design/crushed/crushed-master-dist.png (with props) helix/site-content/1.3.1-docs/images/design/crushed/crushed-partition-dist.png (with props) helix/site-content/1.3.1-docs/images/design/crushed/cursh-master-dist.png (with props) helix/site-content/1.3.1-docs/images/design/crushed/cursh-partition-dist.png (with props) helix/site-content/1.3.1-docs/images/design/crushed/example-cluster-master-dist-after.png (with props) helix/site-content/1.3.1-docs/images/design/crushed/example-cluster-master-dist.png (with props) helix/site-content/1.3.1-docs/images/design/crushed/example-cluster-partition-dist.png (with props) helix/site-content/1.3.1-docs/images/design/crushed/example-movement-on-expansion.png (with props) helix/site-content/1.3.1-docs/images/design/crushed/node-down-master-move.png (with props) helix/site-content/1.3.1-docs/images/design/crushed/node-down-partition-move.png (with props) helix/site-content/1.3.1-docs/images/design/crushed/performance.png (with props) helix/site-content/1.3.1-docs/images/quota_InstanceCapacityManager.jpeg (with props) helix/site-content/1.3.1-docs/images/quota_intro.png (with props) helix/site-content/1.3.1-docs/img/ helix/site-content/1.3.1-docs/img/close.png (with props) helix/site-content/1.3.1-docs/img/glyphicons-halflings-white.png (with props) helix/site-content/1.3.1-docs/img/glyphicons-halflings.png (with props) helix/site-content/1.3.1-docs/img/loading.gif (with props) helix/site-content/1.3.1-docs/img/next.png (with props) helix/site-content/1.3.1-docs/img/prev.png (with props) helix/site-content/1.3.1-docs/index.html helix/site-content/1.3.1-docs/integration.html helix/site-content/1.3.1-docs/issue-tracking.html helix/site-content/1.3.1-docs/js/ helix/site-content/1.3.1-docs/js/bootstrap.min.js helix/site-content/1.3.1-docs/js/jquery.min.js helix/site-content/1.3.1-docs/js/lightbox.min.js helix/site-content/1.3.1-docs/js/reflow-scroll.js helix/site-content/1.3.1-docs/js/reflow-skin.js helix/site-content/1.3.1-docs/license.html helix/site-content/1.3.1-docs/mail
svn commit: r1912533 [5/15] - in /helix/site-content: ./ 0.9.9-docs/ 1.2.0-docs/ 1.3.0-docs/ 1.3.1-docs/ 1.3.1-docs/css/ 1.3.1-docs/images/ 1.3.1-docs/images/design/ 1.3.1-docs/images/design/crushed/
Added: helix/site-content/1.3.1-docs/design_crushed.html URL: http://svn.apache.org/viewvc/helix/site-content/1.3.1-docs/design_crushed.html?rev=1912533=auto == --- helix/site-content/1.3.1-docs/design_crushed.html (added) +++ helix/site-content/1.3.1-docs/design_crushed.html Mon Sep 25 03:26:49 2023 @@ -0,0 +1,368 @@ + + + + + + + + + + + + + + Apache Helix CrushED (Crush-based rebalancer with Even Distribution) + + + + + + + + + + + + + + + + + + + + + + + + + +var _paq = window._paq = window._paq || []; +_paq.push(['disableCookies']); +_paq.push(['trackPageView']); +_paq.push(['enableLinkTracking']); + +(function() { +var u="https://analytics.apache.org"; +_paq.push(['setTrackerUrl', u+'/matomo.php']); +_paq.push(['setSiteId', '15']); +var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; +g.async=true; g.src=u+'/matomo.js'; s.parentNode.insertBefore(g,s); +})(); + + + + + + + + + + + + + + + + http://helix.apache.org;>Apache Helix + + + Helix 1.3.1 + + Get Helix + + Download + Building + Release Notes + + + + Hands-On + + Quick Start + Tutorial + Javadocs + + + + Recipes + + Distributed lock manager + Rabbit MQ consumer group + Rsync replicated file store + Service discovery + Distributed task DAG execution + + + + + + + + + + + + + + + + + + http://helix.apache.org/; id="bannerLeft"> + A cluster management framework for partitioned and replicated distributed resources + + + http://www.apache.org/; id="bannerRight"> + + + + + + + Apache Helix + / + Release 1.3.1 + / + CrushED (Crush-based rebalancer with Even Distribution) +
svn commit: r1912533 [7/15] - in /helix/site-content: ./ 0.9.9-docs/ 1.2.0-docs/ 1.3.0-docs/ 1.3.1-docs/ 1.3.1-docs/css/ 1.3.1-docs/images/ 1.3.1-docs/images/design/ 1.3.1-docs/images/design/crushed/
Added: helix/site-content/1.3.1-docs/plugin-management.html URL: http://svn.apache.org/viewvc/helix/site-content/1.3.1-docs/plugin-management.html?rev=1912533=auto == --- helix/site-content/1.3.1-docs/plugin-management.html (added) +++ helix/site-content/1.3.1-docs/plugin-management.html Mon Sep 25 03:26:49 2023 @@ -0,0 +1,411 @@ + + + + + + + + + + + + + + Apache Helix Project Plugin Management + + + + + + + + + + + + + + + + + + + + + + + + + +var _paq = window._paq = window._paq || []; +_paq.push(['disableCookies']); +_paq.push(['trackPageView']); +_paq.push(['enableLinkTracking']); + +(function() { +var u="https://analytics.apache.org"; +_paq.push(['setTrackerUrl', u+'/matomo.php']); +_paq.push(['setSiteId', '15']); +var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; +g.async=true; g.src=u+'/matomo.js'; s.parentNode.insertBefore(g,s); +})(); + + + + + + + + + + + + + + + + http://helix.apache.org;>Apache Helix + + + Helix 1.3.1 + + Get Helix + + Download + Building + Release Notes + + + + Hands-On + + Quick Start + Tutorial + Javadocs + + + + Recipes + + Distributed lock manager + Rabbit MQ consumer group + Rsync replicated file store + Service discovery + Distributed task DAG execution + + + + + + + + + + + + + + + + + + http://helix.apache.org/; id="bannerLeft"> + A cluster management framework for partitioned and replicated distributed resources + + + http://www.apache.org/; id="bannerRight"> + + + + + + + Apache Helix + / + Release 1.3.1 + / + Project Plugin Management + Last Published: 2023-09-24 +
svn commit: r1912533 [2/15] - in /helix/site-content: ./ 0.9.9-docs/ 1.2.0-docs/ 1.3.0-docs/ 1.3.1-docs/ 1.3.1-docs/css/ 1.3.1-docs/images/ 1.3.1-docs/images/design/ 1.3.1-docs/images/design/crushed/
Added: helix/site-content/1.3.1-docs/Features.html URL: http://svn.apache.org/viewvc/helix/site-content/1.3.1-docs/Features.html?rev=1912533=auto == --- helix/site-content/1.3.1-docs/Features.html (added) +++ helix/site-content/1.3.1-docs/Features.html Mon Sep 25 03:26:49 2023 @@ -0,0 +1,485 @@ + + + + + + + + + + + + + + Apache Helix Features + + + + + + + + + + + + + + + + + + + + + + + + + +var _paq = window._paq = window._paq || []; +_paq.push(['disableCookies']); +_paq.push(['trackPageView']); +_paq.push(['enableLinkTracking']); + +(function() { +var u="https://analytics.apache.org"; +_paq.push(['setTrackerUrl', u+'/matomo.php']); +_paq.push(['setSiteId', '15']); +var d=document, g=d.createElement('script'), s=d.getElementsByTagName('script')[0]; +g.async=true; g.src=u+'/matomo.js'; s.parentNode.insertBefore(g,s); +})(); + + + + + + + + + + + + + + + + http://helix.apache.org;>Apache Helix + + + Helix 1.3.1 + + Get Helix + + Download + Building + Release Notes + + + + Hands-On + + Quick Start + Tutorial + Javadocs + + + + Recipes + + Distributed lock manager + Rabbit MQ consumer group + Rsync replicated file store + Service discovery + Distributed task DAG execution + + + + + + + + + + + + + + + + + + http://helix.apache.org/; id="bannerLeft"> + A cluster management framework for partitioned and replicated distributed resources + + + http://www.apache.org/; id="bannerRight"> + + + + + + + Apache Helix + / + Release 1.3.1 + / + Features + Last Published: 2023-09-24 + + + + + + +
svn commit: r64175 - in /dev/helix/1.3.1: ./ binaries/ src/
Author: jxue Date: Mon Sep 25 03:11:12 2023 New Revision: 64175 Log: Helix 1.3.1 release Added: dev/helix/1.3.1/ dev/helix/1.3.1/binaries/ dev/helix/1.3.1/binaries/helix-admin-webapp-1.3.1-pkg.tar (with props) dev/helix/1.3.1/binaries/helix-admin-webapp-1.3.1-pkg.tar.asc dev/helix/1.3.1/binaries/helix-admin-webapp-1.3.1-pkg.tar.md5 dev/helix/1.3.1/binaries/helix-admin-webapp-1.3.1-pkg.tar.sha512 dev/helix/1.3.1/binaries/helix-agent-1.3.1-pkg.tar (with props) dev/helix/1.3.1/binaries/helix-agent-1.3.1-pkg.tar.asc dev/helix/1.3.1/binaries/helix-agent-1.3.1-pkg.tar.md5 dev/helix/1.3.1/binaries/helix-agent-1.3.1-pkg.tar.sha512 dev/helix/1.3.1/binaries/helix-core-1.3.1-pkg.tar (with props) dev/helix/1.3.1/binaries/helix-core-1.3.1-pkg.tar.asc dev/helix/1.3.1/binaries/helix-core-1.3.1-pkg.tar.md5 dev/helix/1.3.1/binaries/helix-core-1.3.1-pkg.tar.sha512 dev/helix/1.3.1/binaries/helix-front-1.3.1-pkg.tar (with props) dev/helix/1.3.1/binaries/helix-front-1.3.1-pkg.tar.asc dev/helix/1.3.1/binaries/helix-front-1.3.1-pkg.tar.md5 dev/helix/1.3.1/binaries/helix-front-1.3.1-pkg.tar.sha512 dev/helix/1.3.1/binaries/helix-lock-1.3.1-pkg.tar (with props) dev/helix/1.3.1/binaries/helix-lock-1.3.1-pkg.tar.asc dev/helix/1.3.1/binaries/helix-lock-1.3.1-pkg.tar.md5 dev/helix/1.3.1/binaries/helix-lock-1.3.1-pkg.tar.sha512 dev/helix/1.3.1/binaries/helix-rest-1.3.1-pkg.tar (with props) dev/helix/1.3.1/binaries/helix-rest-1.3.1-pkg.tar.asc dev/helix/1.3.1/binaries/helix-rest-1.3.1-pkg.tar.md5 dev/helix/1.3.1/binaries/helix-rest-1.3.1-pkg.tar.sha512 dev/helix/1.3.1/binaries/helix-view-aggregator-1.3.1-pkg.tar (with props) dev/helix/1.3.1/binaries/helix-view-aggregator-1.3.1-pkg.tar.asc dev/helix/1.3.1/binaries/helix-view-aggregator-1.3.1-pkg.tar.md5 dev/helix/1.3.1/binaries/helix-view-aggregator-1.3.1-pkg.tar.sha512 dev/helix/1.3.1/binaries/zookeeper-api-1.3.1-pkg.tar (with props) dev/helix/1.3.1/binaries/zookeeper-api-1.3.1-pkg.tar.asc dev/helix/1.3.1/binaries/zookeeper-api-1.3.1-pkg.tar.md5 dev/helix/1.3.1/binaries/zookeeper-api-1.3.1-pkg.tar.sha512 dev/helix/1.3.1/src/ dev/helix/1.3.1/src/helix-1.3.1-src.zip (with props) dev/helix/1.3.1/src/helix-1.3.1-src.zip.asc dev/helix/1.3.1/src/helix-1.3.1-src.zip.md5 dev/helix/1.3.1/src/helix-1.3.1-src.zip.sha512 Added: dev/helix/1.3.1/binaries/helix-admin-webapp-1.3.1-pkg.tar == Binary file - no diff available. Propchange: dev/helix/1.3.1/binaries/helix-admin-webapp-1.3.1-pkg.tar -- svn:mime-type = application/octet-stream Added: dev/helix/1.3.1/binaries/helix-admin-webapp-1.3.1-pkg.tar.asc == --- dev/helix/1.3.1/binaries/helix-admin-webapp-1.3.1-pkg.tar.asc (added) +++ dev/helix/1.3.1/binaries/helix-admin-webapp-1.3.1-pkg.tar.asc Mon Sep 25 03:11:12 2023 @@ -0,0 +1,16 @@ +-BEGIN PGP SIGNATURE- + +iQIzBAABCgAdFiEE72WW8pPA5nF2SiWlmNmvYXFla5QFAmUQ93gACgkQmNmvYXFl +a5QVuxAAulOt2aKnfK0N6yZ/2b0RhXn175jmAy4IqRMPq/ZBD3QTkwpWEHO3YLwQ +8nLBTfhK4lYqeOAbB5EbM4tUFl4S1+09rGE8YTQ8U8yFo8/N1j72RctFrgzz8diI +ViBJ4jqxupNskWfUMLJK2pfENMPYOql/WJNIVcDxKmTNjt5AJIXimj4WRi07qnqz +Wkq7X+psu0Zf0gUruXaAFvalbXlW0o3+InRkiuS9z4LPlCMDes+ul2C53NY0Mx7s +v8P06iw/wn7yAXmO/O/A5+VAL0pzTDq2wsLV6yS6Ej81NW+bnuog+ie8cJkop3ox +pO0iGvJ1HTMpdxVLq4VnJI+ENkNLzc6XLJDLW3h3/rcRc3y/AkfWo5YUOVuUhudV +S57qg6nK/SiWxfDGt+lkmuZjwoofKyXXiZlDi/Qmfjrc7wBkP1cHpyAk9faXGH5N +B3H+qDra8PhfdvdO6oG27ii3ApACqDZJdGcBmVT/jqMFbXBndMOoX0jEGbjGSEVb +iMKVOL7qXhaHUNu0W4glmxXJPqWmJdtgZQkB2kTUym5xWrQzxUSzS084Ma2IJefj +5WEwSGIoC0jsjEbehsEFyRztJCBHMVoU3sSumqsYrrN0hZJYc3acUPt+4NKe+JWV +q0PwHjWlwS90f53Vk+3/pU9NC4OWjl8I6NdW/CuTjcLYiphgLmU= +=BjO5 +-END PGP SIGNATURE- Added: dev/helix/1.3.1/binaries/helix-admin-webapp-1.3.1-pkg.tar.md5 == --- dev/helix/1.3.1/binaries/helix-admin-webapp-1.3.1-pkg.tar.md5 (added) +++ dev/helix/1.3.1/binaries/helix-admin-webapp-1.3.1-pkg.tar.md5 Mon Sep 25 03:11:12 2023 @@ -0,0 +1 @@ +a8c674c081991540cb5ce40be884e00a \ No newline at end of file Added: dev/helix/1.3.1/binaries/helix-admin-webapp-1.3.1-pkg.tar.sha512 == --- dev/helix/1.3.1/binaries/helix-admin-webapp-1.3.1-pkg.tar.sha512 (added) +++ dev/helix/1.3.1/binaries/helix-admin-webapp-1.3.1-pkg.tar.sha512 Mon Sep 25 03:11:12 2023 @@ -0,0 +1 @@ +d741933d8ee0a66f993089cb3256ba1c1811147c847587fd5fa0163336bf06854d4c6290f4d7aed0b03405e8cb2a45936f769140bdb8800235c6c4be73277327 helix-admin-webapp-1.3.1-pkg.tar Added: dev/helix/1.3.1/binaries/helix-agent-1.3.1-pkg.tar
[helix] branch master updated: [maven-release-plugin] prepare for next development iteration
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new 38ac277c4 [maven-release-plugin] prepare for next development iteration 38ac277c4 is described below commit 38ac277c46db4d06d845caa6d2633a8e882b2e03 Author: Junkai Xue AuthorDate: Sun Sep 24 09:50:57 2023 -0700 [maven-release-plugin] prepare for next development iteration --- helix-admin-webapp/pom.xml | 2 +- helix-agent/pom.xml | 2 +- helix-common/pom.xml | 2 +- helix-core/pom.xml | 2 +- helix-front/pom.xml | 2 +- helix-lock/pom.xml | 2 +- helix-rest/pom.xml | 4 ++-- helix-view-aggregator/pom.xml| 2 +- meta-client/pom.xml | 2 +- metadata-store-directory-common/pom.xml | 2 +- metrics-common/pom.xml | 2 +- pom.xml | 4 ++-- recipes/distributed-lock-manager/pom.xml | 2 +- recipes/pom.xml | 2 +- recipes/rabbitmq-consumer-group/pom.xml | 2 +- recipes/rsync-replicated-file-system/pom.xml | 2 +- recipes/service-discovery/pom.xml| 2 +- recipes/task-execution/pom.xml | 2 +- zookeeper-api/pom.xml| 2 +- 19 files changed, 21 insertions(+), 21 deletions(-) diff --git a/helix-admin-webapp/pom.xml b/helix-admin-webapp/pom.xml index fd9bf3468..c4e9ea135 100644 --- a/helix-admin-webapp/pom.xml +++ b/helix-admin-webapp/pom.xml @@ -21,7 +21,7 @@ org.apache.helix helix -1.3.1 +1.3.2-SNAPSHOT 4.0.0 diff --git a/helix-agent/pom.xml b/helix-agent/pom.xml index 86b42b4a9..bf8ad1cb4 100644 --- a/helix-agent/pom.xml +++ b/helix-agent/pom.xml @@ -22,7 +22,7 @@ org.apache.helix helix -1.3.1 +1.3.2-SNAPSHOT helix-agent bundle diff --git a/helix-common/pom.xml b/helix-common/pom.xml index 9725a9491..40f64043b 100644 --- a/helix-common/pom.xml +++ b/helix-common/pom.xml @@ -21,7 +21,7 @@ org.apache.helix helix -1.3.1 +1.3.2-SNAPSHOT 4.0.0 diff --git a/helix-core/pom.xml b/helix-core/pom.xml index 17e6b132b..7f4f082e3 100644 --- a/helix-core/pom.xml +++ b/helix-core/pom.xml @@ -21,7 +21,7 @@ org.apache.helix helix -1.3.1 +1.3.2-SNAPSHOT 4.0.0 diff --git a/helix-front/pom.xml b/helix-front/pom.xml index 6d5207597..910ada0e6 100644 --- a/helix-front/pom.xml +++ b/helix-front/pom.xml @@ -21,7 +21,7 @@ org.apache.helix helix -1.3.1 +1.3.2-SNAPSHOT 4.0.0 diff --git a/helix-lock/pom.xml b/helix-lock/pom.xml index a405354d5..f3c740bd6 100644 --- a/helix-lock/pom.xml +++ b/helix-lock/pom.xml @@ -21,7 +21,7 @@ org.apache.helix helix -1.3.1 +1.3.2-SNAPSHOT 4.0.0 diff --git a/helix-rest/pom.xml b/helix-rest/pom.xml index a2cb8fa41..ac986da97 100644 --- a/helix-rest/pom.xml +++ b/helix-rest/pom.xml @@ -21,7 +21,7 @@ org.apache.helix helix -1.3.1 +1.3.2-SNAPSHOT 4.0.0 @@ -43,7 +43,7 @@ -1.3.1 +1.3.2-SNAPSHOT org.apache.helix*, org.apache.commons.cli*, diff --git a/helix-view-aggregator/pom.xml b/helix-view-aggregator/pom.xml index 86d440254..9b9a5daba 100644 --- a/helix-view-aggregator/pom.xml +++ b/helix-view-aggregator/pom.xml @@ -21,7 +21,7 @@ under the License. org.apache.helix helix -1.3.1 +1.3.2-SNAPSHOT 4.0.0 diff --git a/meta-client/pom.xml b/meta-client/pom.xml index f920c7ad1..4dcae95bd 100644 --- a/meta-client/pom.xml +++ b/meta-client/pom.xml @@ -21,7 +21,7 @@ under the License. org.apache.helix helix -1.3.1 +1.3.2-SNAPSHOT 4.0.0 diff --git a/metadata-store-directory-common/pom.xml b/metadata-store-directory-common/pom.xml index cccfb49dd..114d901d1 100644 --- a/metadata-store-directory-common/pom.xml +++ b/metadata-store-directory-common/pom.xml @@ -21,7 +21,7 @@ org.apache.helix helix -1.3.1 +1.3.2-SNAPSHOT 4.0.0 diff --git a/metrics-common/pom.xml b/metrics-common/pom.xml index 7e8c153fd..2dbe016cb 100644 --- a/metrics-common/pom.xml +++ b/metrics-common/pom.xml @@ -21,7 +21,7 @@ org.apache.helix helix -1.3.1 +1.3.2-SNAPSHOT 4.0.0 diff --git a/pom.xml b/pom.xml index 6a4c62b77..dc3cd286a 100644 --- a/pom.xml +++ b/pom.xml @@ -29,7 +29,7 @@ org.apache.helix helix - 1.3.1 + 1.3.2-SNAPSHOT pom Apache Helix @@ -431,7 +431,7 @@ 1.1.1-SNAPSHOT UTF-8 -1695573552 +1695574257 https://analysis.apache.org/ diff --git a/recipes/distributed-lock-manager/pom.xml b/recipes/distributed-lock-manager
[helix] annotated tag helix-1.3.1 created (now e362db5e3)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a change to annotated tag helix-1.3.1 in repository https://gitbox.apache.org/repos/asf/helix.git at e362db5e3 (tag) tagging fd6a039c9ea9e9ca1a561d0a57eebe70494cade7 (commit) replaces helix-1.3.0 by Junkai Xue on Sun Sep 24 09:50:52 2023 -0700 - Log - [maven-release-plugin] copy for tag helix-1.3.1 --- No new revisions were added by this update.
[helix] branch master updated: [maven-release-plugin] prepare release helix-1.3.1
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new fd6a039c9 [maven-release-plugin] prepare release helix-1.3.1 fd6a039c9 is described below commit fd6a039c9ea9e9ca1a561d0a57eebe70494cade7 Author: Junkai Xue AuthorDate: Sun Sep 24 09:50:47 2023 -0700 [maven-release-plugin] prepare release helix-1.3.1 --- helix-admin-webapp/pom.xml | 2 +- helix-agent/pom.xml | 2 +- helix-common/pom.xml | 2 +- helix-core/pom.xml | 2 +- helix-front/pom.xml | 2 +- helix-lock/pom.xml | 2 +- helix-rest/pom.xml | 4 ++-- helix-view-aggregator/pom.xml| 2 +- meta-client/pom.xml | 2 +- metadata-store-directory-common/pom.xml | 2 +- metrics-common/pom.xml | 2 +- pom.xml | 6 +++--- recipes/distributed-lock-manager/pom.xml | 2 +- recipes/pom.xml | 2 +- recipes/rabbitmq-consumer-group/pom.xml | 2 +- recipes/rsync-replicated-file-system/pom.xml | 2 +- recipes/service-discovery/pom.xml| 2 +- recipes/task-execution/pom.xml | 2 +- zookeeper-api/pom.xml| 2 +- 19 files changed, 22 insertions(+), 22 deletions(-) diff --git a/helix-admin-webapp/pom.xml b/helix-admin-webapp/pom.xml index db67505ac..fd9bf3468 100644 --- a/helix-admin-webapp/pom.xml +++ b/helix-admin-webapp/pom.xml @@ -21,7 +21,7 @@ org.apache.helix helix -1.3.1-SNAPSHOT +1.3.1 4.0.0 diff --git a/helix-agent/pom.xml b/helix-agent/pom.xml index b5882226f..86b42b4a9 100644 --- a/helix-agent/pom.xml +++ b/helix-agent/pom.xml @@ -22,7 +22,7 @@ org.apache.helix helix -1.3.1-SNAPSHOT +1.3.1 helix-agent bundle diff --git a/helix-common/pom.xml b/helix-common/pom.xml index a4efbd5f7..9725a9491 100644 --- a/helix-common/pom.xml +++ b/helix-common/pom.xml @@ -21,7 +21,7 @@ org.apache.helix helix -1.3.1-SNAPSHOT +1.3.1 4.0.0 diff --git a/helix-core/pom.xml b/helix-core/pom.xml index 9a3c303e7..17e6b132b 100644 --- a/helix-core/pom.xml +++ b/helix-core/pom.xml @@ -21,7 +21,7 @@ org.apache.helix helix -1.3.1-SNAPSHOT +1.3.1 4.0.0 diff --git a/helix-front/pom.xml b/helix-front/pom.xml index 38922a4cd..6d5207597 100644 --- a/helix-front/pom.xml +++ b/helix-front/pom.xml @@ -21,7 +21,7 @@ org.apache.helix helix -1.3.1-SNAPSHOT +1.3.1 4.0.0 diff --git a/helix-lock/pom.xml b/helix-lock/pom.xml index 40ab862d5..a405354d5 100644 --- a/helix-lock/pom.xml +++ b/helix-lock/pom.xml @@ -21,7 +21,7 @@ org.apache.helix helix -1.3.1-SNAPSHOT +1.3.1 4.0.0 diff --git a/helix-rest/pom.xml b/helix-rest/pom.xml index 30a6c6865..a2cb8fa41 100644 --- a/helix-rest/pom.xml +++ b/helix-rest/pom.xml @@ -21,7 +21,7 @@ org.apache.helix helix -1.3.1-SNAPSHOT +1.3.1 4.0.0 @@ -43,7 +43,7 @@ -1.3.1-SNAPSHOT +1.3.1 org.apache.helix*, org.apache.commons.cli*, diff --git a/helix-view-aggregator/pom.xml b/helix-view-aggregator/pom.xml index 56e908312..86d440254 100644 --- a/helix-view-aggregator/pom.xml +++ b/helix-view-aggregator/pom.xml @@ -21,7 +21,7 @@ under the License. org.apache.helix helix -1.3.1-SNAPSHOT +1.3.1 4.0.0 diff --git a/meta-client/pom.xml b/meta-client/pom.xml index 083a7b699..f920c7ad1 100644 --- a/meta-client/pom.xml +++ b/meta-client/pom.xml @@ -21,7 +21,7 @@ under the License. org.apache.helix helix -1.3.1-SNAPSHOT +1.3.1 4.0.0 diff --git a/metadata-store-directory-common/pom.xml b/metadata-store-directory-common/pom.xml index 32466a690..cccfb49dd 100644 --- a/metadata-store-directory-common/pom.xml +++ b/metadata-store-directory-common/pom.xml @@ -21,7 +21,7 @@ org.apache.helix helix -1.3.1-SNAPSHOT +1.3.1 4.0.0 diff --git a/metrics-common/pom.xml b/metrics-common/pom.xml index eb6685959..7e8c153fd 100644 --- a/metrics-common/pom.xml +++ b/metrics-common/pom.xml @@ -21,7 +21,7 @@ org.apache.helix helix -1.3.1-SNAPSHOT +1.3.1 4.0.0 diff --git a/pom.xml b/pom.xml index cb0f21e9f..6a4c62b77 100644 --- a/pom.xml +++ b/pom.xml @@ -29,7 +29,7 @@ org.apache.helix helix - 1.3.1-SNAPSHOT + 1.3.1 pom Apache Helix @@ -417,7 +417,7 @@ scm:git:https://gitbox.apache.org/repos/asf/helix.git scm:git:https://gitbox.apache.org/repos/asf/helix.git https://gitbox.apache.org/repos/asf?p=helix.git;a=summary -helix-1.3.0 +helix-1.3.1
[helix] branch master updated (c9402ea1e -> 279c3f39e)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/helix.git from c9402ea1e Fix the 'BadVersion' exception handling in doSet() code path. (#2614) add 279c3f39e Bump jackson-databind in /helix-view-aggregator (#2280) No new revisions were added by this update. Summary of changes: helix-view-aggregator/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[helix] branch master updated: Change TopStateHandoffReportStage to be an Async Stage as it is the slowest stage in the pipeline. It can be run async since it does not add to ClusterEvent and simply c
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git The following commit(s) were added to refs/heads/master by this push: new fbefa6490 Change TopStateHandoffReportStage to be an Async Stage as it is the slowest stage in the pipeline. It can be run async since it does not add to ClusterEvent and simply computes and reports metrics. No following stages depend on it. (#2610) fbefa6490 is described below commit fbefa64909781d479ec486a93d22bba416322fb8 Author: Zachary Pinto AuthorDate: Thu Aug 31 20:10:13 2023 -0700 Change TopStateHandoffReportStage to be an Async Stage as it is the slowest stage in the pipeline. It can be run async since it does not add to ClusterEvent and simply computes and reports metrics. No following stages depend on it. (#2610) In some cases TopStateHandoffReportStage is taking a very long portion of the total pipeline execution time. In order to speed up the total pipeline execution time, we will make TopStateHandoffReportStage async since it is simply computing metrics and reporting without adding to ClusterEvent. --- .../helix/controller/pipeline/AsyncWorkerType.java | 1 + .../stages/TopStateHandoffReportStage.java | 22 +++--- .../helix/controller/stages/BaseStageTest.java | 11 --- 3 files changed, 24 insertions(+), 10 deletions(-) diff --git a/helix-core/src/main/java/org/apache/helix/controller/pipeline/AsyncWorkerType.java b/helix-core/src/main/java/org/apache/helix/controller/pipeline/AsyncWorkerType.java index fcbf03f6b..a1afb95f2 100644 --- a/helix-core/src/main/java/org/apache/helix/controller/pipeline/AsyncWorkerType.java +++ b/helix-core/src/main/java/org/apache/helix/controller/pipeline/AsyncWorkerType.java @@ -27,6 +27,7 @@ package org.apache.helix.controller.pipeline; */ public enum AsyncWorkerType { + TopStateHandoffReportWorker, TargetExternalViewCalcWorker, PersistAssignmentWorker, ExternalViewComputeWorker, diff --git a/helix-core/src/main/java/org/apache/helix/controller/stages/TopStateHandoffReportStage.java b/helix-core/src/main/java/org/apache/helix/controller/stages/TopStateHandoffReportStage.java index 77a84a448..aec55aae5 100644 --- a/helix-core/src/main/java/org/apache/helix/controller/stages/TopStateHandoffReportStage.java +++ b/helix-core/src/main/java/org/apache/helix/controller/stages/TopStateHandoffReportStage.java @@ -26,7 +26,8 @@ import org.apache.helix.controller.LogUtil; import org.apache.helix.controller.dataproviders.BaseControllerDataProvider; import org.apache.helix.controller.dataproviders.ResourceControllerDataProvider; import org.apache.helix.controller.dataproviders.WorkflowControllerDataProvider; -import org.apache.helix.controller.pipeline.AbstractBaseStage; +import org.apache.helix.controller.pipeline.AbstractAsyncBaseStage; +import org.apache.helix.controller.pipeline.AsyncWorkerType; import org.apache.helix.controller.pipeline.StageException; import org.apache.helix.model.CurrentState; import org.apache.helix.model.LiveInstance; @@ -42,20 +43,27 @@ import org.slf4j.LoggerFactory; /** * Observe top state handoff and report latency */ -public class TopStateHandoffReportStage extends AbstractBaseStage { +public class TopStateHandoffReportStage extends AbstractAsyncBaseStage { private static final long DEFAULT_HANDOFF_USER_LATENCY = 0L; private static Logger LOG = LoggerFactory.getLogger(TopStateHandoffReportStage.class); public static final long TIMESTAMP_NOT_RECORDED = -1L; @Override - public void process(ClusterEvent event) throws Exception { + public AsyncWorkerType getAsyncWorkerType() { +return AsyncWorkerType.TopStateHandoffReportWorker; + } + + @Override + public void execute(final ClusterEvent event) throws Exception { _eventId = event.getEventId(); -final BaseControllerDataProvider cache = event.getAttribute(AttributeName.ControllerDataProvider.name()); -final Long lastPipelineFinishTimestamp = event - .getAttributeWithDefault(AttributeName.LastRebalanceFinishTimeStamp.name(), +final BaseControllerDataProvider cache = +event.getAttribute(AttributeName.ControllerDataProvider.name()); +final Long lastPipelineFinishTimestamp = + event.getAttributeWithDefault(AttributeName.LastRebalanceFinishTimeStamp.name(), TIMESTAMP_NOT_RECORDED); final Map resourceMap = event.getAttribute(AttributeName.RESOURCES.name()); -final CurrentStateOutput currentStateOutput = event.getAttribute(AttributeName.CURRENT_STATE.name()); +final CurrentStateOutput currentStateOutput = +event.getAttribute(AttributeName.CURRENT_STATE.name()); final ClusterStatusMonitor clusterStatusMonitor = event.getAttribute(AttributeName.clusterStatusMonitor.name()); diff --git a/helix-core/src/test/java/org/apache/helix/controller
[helix] 14/21: Metaclient leader election - basics (#2558)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git commit 36231faebaf85559c39578b1ce1bab71903d35e4 Author: xyuanlu AuthorDate: Tue Jul 18 09:41:55 2023 -0700 Metaclient leader election - basics (#2558) Co-authored-by: Xiaoyuan Lu --- .../org/apache/helix/metaclient/api/OpResult.java | 2 +- .../helix/metaclient/datamodel/DataRecord.java | 5 + .../helix/metaclient/impl/zk/ZkMetaClient.java | 50 +++-- .../metaclient/impl/zk/util/ZkMetaClientUtil.java | 2 +- .../leaderelection/LeaderElectionClient.java | 204 ++--- .../recipes/leaderelection/LeaderInfo.java | 34 +++- .../leaderelection/LeaderInfoSerializer.java | 42 + .../java/org/apache/helix/metaclient/TestUtil.java | 22 +++ .../recipes/leaderelection/TestLeaderElection.java | 64 +++ .../datamodel/serializer/ZNRecordSerializer.java | 2 +- .../helix/zookeeper/zkclient/ZkConnection.java | 1 + 11 files changed, 366 insertions(+), 62 deletions(-) diff --git a/meta-client/src/main/java/org/apache/helix/metaclient/api/OpResult.java b/meta-client/src/main/java/org/apache/helix/metaclient/api/OpResult.java index effed8543..e3621190a 100644 --- a/meta-client/src/main/java/org/apache/helix/metaclient/api/OpResult.java +++ b/meta-client/src/main/java/org/apache/helix/metaclient/api/OpResult.java @@ -26,7 +26,7 @@ import java.util.List; */ public class OpResult { - enum Type { + public enum Type { ERRORRESULT, GETDATARESULT, GETCHILDRENRESULT, diff --git a/meta-client/src/main/java/org/apache/helix/metaclient/datamodel/DataRecord.java b/meta-client/src/main/java/org/apache/helix/metaclient/datamodel/DataRecord.java index 3ed6928f1..91e5409ca 100644 --- a/meta-client/src/main/java/org/apache/helix/metaclient/datamodel/DataRecord.java +++ b/meta-client/src/main/java/org/apache/helix/metaclient/datamodel/DataRecord.java @@ -20,6 +20,7 @@ package org.apache.helix.metaclient.datamodel; */ import com.fasterxml.jackson.databind.annotation.JsonSerialize; +import org.apache.helix.metaclient.recipes.leaderelection.LeaderInfo; import org.apache.helix.zookeeper.datamodel.ZNRecord; /** @@ -35,4 +36,8 @@ public class DataRecord extends ZNRecord { public DataRecord(ZNRecord record) { super(record); } + + public DataRecord(DataRecord record, String id) { +super(record, id); + } } diff --git a/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/ZkMetaClient.java b/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/ZkMetaClient.java index a1b6eb1ad..7f68ec9ca 100644 --- a/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/ZkMetaClient.java +++ b/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/ZkMetaClient.java @@ -83,17 +83,15 @@ public class ZkMetaClient implements MetaClientInterface, AutoCloseable { // Lock all activities related to ZkClient connection private ReentrantLock _zkClientConnectionMutex = new ReentrantLock(); - public ZkMetaClient(ZkMetaClientConfig config) { _initConnectionTimeout = config.getConnectionInitTimeoutInMillis(); _reconnectTimeout = config.getMetaClientReconnectPolicy().getAutoReconnectTimeout(); // TODO: Right new ZkClient reconnect using exp backoff with fixed max backoff interval. We should // Allow user to config reconnect policy -_zkClient = new ZkClient( -new ZkConnection(config.getConnectionAddress(), (int) config.getSessionTimeoutInMillis()), +_zkClient = new ZkClient(new ZkConnection(config.getConnectionAddress(), (int) config.getSessionTimeoutInMillis()), (int) _initConnectionTimeout, _reconnectTimeout /*use reconnect timeout for retry timeout*/, -config.getZkSerializer(), config.getMonitorType(), config.getMonitorKey(), -config.getMonitorInstanceName(), config.getMonitorRootPathOnly(), false, true); +config.getZkSerializer(), config.getMonitorType(), config.getMonitorKey(), config.getMonitorInstanceName(), +config.getMonitorRootPathOnly(), false, true); _zkClientReconnectMonitor = Executors.newSingleThreadScheduledExecutor(); _reconnectStateChangeListener = new ReconnectStateChangeListener(); } @@ -102,6 +100,8 @@ public class ZkMetaClient implements MetaClientInterface, AutoCloseable { public void create(String key, Object data) { try { create(key, data, EntryMode.PERSISTENT); +} catch (ZkException e) { + throw ZkMetaClientUtil.translateZkExceptionToMetaclientException(e); } catch (Exception e) { throw new MetaClientException(e); } @@ -110,16 +110,18 @@ public class ZkMetaClient implements MetaClientInterface, AutoCloseable { @Override public void create(String key, Object data, MetaClientInterface.EntryMode mode) { -try{ +try
[helix] 03/21: tests for get, create, set, update, and data change listeners
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git commit b93f89a0603e32faf7e498cda9433c9b5fe3c8e4 Author: Grant Palau Spencer AuthorDate: Thu Jun 1 15:17:59 2023 -0700 tests for get, create, set, update, and data change listeners --- .../metaclient/impl/zk/TestStressZkClient.java | 406 - 1 file changed, 388 insertions(+), 18 deletions(-) diff --git a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestStressZkClient.java b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestStressZkClient.java index 9306891d6..3e7140f64 100644 --- a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestStressZkClient.java +++ b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestStressZkClient.java @@ -1,24 +1,394 @@ package org.apache.helix.metaclient.impl.zk; /* -* Licensed to the Apache Software Foundation (ASF) under one -* or more contributor license agreements. See the NOTICE file -* distributed with this work for additional information -* regarding copyright ownership. The ASF licenses this file -* to you under the Apache License, Version 2.0 (the -* "License"); you may not use this file except in compliance -* with the License. You may obtain a copy of the License at -* -* http://www.apache.org/licenses/LICENSE-2.0 -* -* Unless required by applicable law or agreed to in writing, -* software distributed under the License is distributed on an -* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY -* KIND, either express or implied. See the License for the -* specific language governing permissions and limitations -* under the License. -*/ + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import java.util.HashMap; +import java.util.HashSet; +import java.util.Map; +import java.util.Set; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.TimeUnit; +import java.util.concurrent.atomic.AtomicBoolean; +import org.apache.helix.metaclient.api.DataChangeListener; +import org.apache.helix.metaclient.api.DataUpdater; +import org.apache.helix.metaclient.api.MetaClientInterface; +import org.apache.helix.metaclient.exception.MetaClientException; +import org.apache.helix.metaclient.impl.zk.factory.ZkMetaClientConfig; +import org.apache.helix.zookeeper.exception.ZkClientException; +import org.testng.Assert; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.testng.annotations.Test; +import static org.apache.helix.metaclient.api.MetaClientInterface.EntryMode.*; public class TestStressZkClient extends ZkMetaClientTestBase { - // TODO: add tests + + private static final long TEST_ITERATION_COUNT = 1000; + private static final Logger LOG = LoggerFactory.getLogger(TestStressZkClient.class); + + private void cleanUpTestNodes(String parentZnodeKey) { +try (ZkMetaClient zkMetaClient = createZkMetaClient()) { + zkMetaClient.connect(); + int retryCount = 0; + while (zkMetaClient.countDirectChildren(parentZnodeKey) > 0) { +try { + retryCount++; + zkMetaClient.recursiveDelete(parentZnodeKey); +} catch (ZkClientException e) { + if (retryCount >= 3) { +throw new ZkClientException("Failed to clean up test nodes after 3 tries", e); + } +} + } + Assert.assertEquals(zkMetaClient.countDirectChildren(parentZnodeKey), 0); +} + } + + @Test + public void testCreate() { +String zkParentKey = "/stressZk_testCreate"; + +try (ZkMetaClient zkMetaClient = createZkMetaClient()) { + zkMetaClient.connect(); + zkMetaClient.create(zkParentKey, "parent_node"); + + for (int i = 0; i < TEST_ITERATION_COUNT; i++) { +zkMetaClient.create(zkParentKey + "/" + i, i); + } + + for (int i = 0; i < TEST_ITERATION_COUNT; i++) { +Assert.assertEquals(String.valueOf(zkMetaClient.get(zkParentKey + "/" + i)
[helix] 18/21: Lattice LockClient LockInfoSerializer Implementation(#2580)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git commit 26f319608cae205e66bc6ab3cbf3b882b4980ae9 Author: Marcos Rico Peng <55266861+marcosr...@users.noreply.github.com> AuthorDate: Wed Aug 2 18:41:59 2023 +0200 Lattice LockClient LockInfoSerializer Implementation(#2580) - Co-authored-by: mapeng --- .../helix/metaclient/recipes/lock/LockClient.java | 9 ++--- .../helix/metaclient/recipes/lock/LockInfo.java| 30 - .../recipes/lock/LockInfoSerializer.java | 39 ++ .../metaclient/recipes/lock/LockClientTest.java| 36 4 files changed, 85 insertions(+), 29 deletions(-) diff --git a/meta-client/src/main/java/org/apache/helix/metaclient/recipes/lock/LockClient.java b/meta-client/src/main/java/org/apache/helix/metaclient/recipes/lock/LockClient.java index 480e87c30..a186f3d81 100644 --- a/meta-client/src/main/java/org/apache/helix/metaclient/recipes/lock/LockClient.java +++ b/meta-client/src/main/java/org/apache/helix/metaclient/recipes/lock/LockClient.java @@ -48,8 +48,7 @@ public class LockClient implements LockClientInterface, AutoCloseable { if (MetaClientConfig.StoreType.ZOOKEEPER.equals(config.getStoreType())) { ZkMetaClientConfig zkMetaClientConfig = new ZkMetaClientConfig.ZkMetaClientConfigBuilder(). setConnectionAddress(config.getConnectionAddress()) - // Currently only support ZNRecordSerializer. TODO: make this configurable - .setZkSerializer((new ZNRecordSerializer())) + .setZkSerializer((new LockInfoSerializer())) .build(); _metaClient = new ZkMetaClientFactory().getMetaClient(zkMetaClientConfig); _metaClient.connect(); @@ -110,11 +109,7 @@ public class LockClient implements LockClientInterface, AutoCloseable { if (stat == null) { return null; } -//Create a new DataRecord from underlying record -DataRecord dataRecord = new DataRecord(_metaClient.get(key)); -//Create a new LockInfo from DataRecord -LockInfo lockInfo = new LockInfo(dataRecord, stat); -return lockInfo; +return new LockInfo(_metaClient.get(key), stat); } @Override diff --git a/meta-client/src/main/java/org/apache/helix/metaclient/recipes/lock/LockInfo.java b/meta-client/src/main/java/org/apache/helix/metaclient/recipes/lock/LockInfo.java index 26736f1b0..2a08c3754 100644 --- a/meta-client/src/main/java/org/apache/helix/metaclient/recipes/lock/LockInfo.java +++ b/meta-client/src/main/java/org/apache/helix/metaclient/recipes/lock/LockInfo.java @@ -39,7 +39,6 @@ public class LockInfo extends DataRecord { public static final long DEFAULT_LAST_RENEWED_AT_LONG = -1L; public static final long DEFAULT_TIMEOUT_DURATION = -1L; private static final String DEFAULT_LOCK_INFO = "lockInfo."; - private DataRecord _dataRecord; /** * The keys to lock information @@ -59,7 +58,6 @@ public class LockInfo extends DataRecord { */ public LockInfo() { super(DEFAULT_LOCK_INFO); -_dataRecord = new DataRecord(DEFAULT_LOCK_INFO); setLockInfoFields(DEFAULT_LOCK_ID_TEXT, DEFAULT_OWNER_ID_TEXT, DEFAULT_CLIENT_ID_TEXT, DEFAULT_CLIENT_DATA, DEFAULT_GRANTED_AT_LONG, DEFAULT_LAST_RENEWED_AT_LONG, DEFAULT_TIMEOUT_DURATION); } @@ -140,7 +138,7 @@ public class LockInfo extends DataRecord { * It is created by the lockClient and a new one is created for each time the lock is acquired. */ public void setLockId(String lockId) { -_dataRecord.setSimpleField(LockInfoAttribute.LOCK_ID.name(), lockId == null ? DEFAULT_LOCK_ID_TEXT : lockId); +setSimpleField(LockInfoAttribute.LOCK_ID.name(), lockId == null ? DEFAULT_LOCK_ID_TEXT : lockId); } /** @@ -150,7 +148,7 @@ public class LockInfo extends DataRecord { *by the same owner. */ public void setOwnerId(String ownerId) { -_dataRecord.setSimpleField(LockInfoAttribute.OWNER_ID.name(), ownerId == null ? DEFAULT_OWNER_ID_TEXT : ownerId); +setSimpleField(LockInfoAttribute.OWNER_ID.name(), ownerId == null ? DEFAULT_OWNER_ID_TEXT : ownerId); } /** @@ -158,7 +156,7 @@ public class LockInfo extends DataRecord { * @param clientId Unique identifier that represents who will get the lock (the client). */ public void setClientId(String clientId) { -_dataRecord.setSimpleField(LockInfoAttribute.CLIENT_ID.name(), clientId == null ? DEFAULT_CLIENT_ID_TEXT : clientId); +setSimpleField(LockInfoAttribute.CLIENT_ID.name(), clientId == null ? DEFAULT_CLIENT_ID_TEXT : clientId); } /** @@ -166,7 +164,7 @@ public class LockInfo extends DataRecord { * @param clientData String representing the serialized data object */ public void setClientData(String clientData) { -_dataRecor
[helix] 13/21: MultiThreading Stress Test Lattice - Puppy Logic and Skeleton (Part 1) (#2549)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git commit 5389b173945b6f82cd5691a7c0457b1ccb98988a Author: Marcos Rico Peng <55266861+marcosr...@users.noreply.github.com> AuthorDate: Thu Jul 13 17:16:30 2023 -0400 MultiThreading Stress Test Lattice - Puppy Logic and Skeleton (Part 1) (#2549) - Co-authored-by: mapeng --- .../helix/metaclient/puppy/AbstractPuppy.java | 95 ++ .../apache/helix/metaclient/puppy/ExecDelay.java | 63 ++ .../helix/metaclient/puppy/PuppyManager.java | 64 +++ .../apache/helix/metaclient/puppy/PuppyMode.java | 28 +++ .../apache/helix/metaclient/puppy/PuppySpec.java | 53 5 files changed, 303 insertions(+) diff --git a/meta-client/src/test/java/org/apache/helix/metaclient/puppy/AbstractPuppy.java b/meta-client/src/test/java/org/apache/helix/metaclient/puppy/AbstractPuppy.java new file mode 100644 index 0..85137fc17 --- /dev/null +++ b/meta-client/src/test/java/org/apache/helix/metaclient/puppy/AbstractPuppy.java @@ -0,0 +1,95 @@ +package org.apache.helix.metaclient.puppy; + +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import org.apache.helix.metaclient.api.MetaClientInterface; +import java.util.HashMap; + +/** + * AbstractPuppy object contains interfaces to implement puppy and main logics to manage puppy life cycle + */ +public abstract class AbstractPuppy implements Runnable { + + protected MetaClientInterface _metaclient; + protected PuppySpec _puppySpec; + public HashMap _eventChangeCounterMap; + protected int _unhandledErrorCounter; + + public AbstractPuppy(MetaClientInterface metaclient, PuppySpec puppySpec) { +_metaclient = metaclient; +_puppySpec = puppySpec; +_eventChangeCounterMap = new HashMap<>(); + } + + /** + * Implements puppy's main logic. Puppy needs to implement its chaos logic, recovery logic based on + * errorRate, recoverDelay. For OneOff puppy, it will bark once with execDelay in spec, and for + * Repeat puppy, it will bark forever, with execDelay between 2 barks + */ + protected abstract void bark() throws Exception; + + /** + * Implements puppy's final cleanup logic - it will be called only once right before the puppy terminates. + * Before the puppy terminates, it needs to recover from all chaos it created. + */ + protected abstract void cleanup(); + + @Override + public void run() { +try { + while (true) { +try { + bark(); +} catch (Exception e) { + incrementUnhandledErrorCounter(); + e.printStackTrace(); +} + +if (getPuppySpec().getMode() == PuppyMode.ONE_OFF) { + cleanup(); + break; +} else { + try { +Thread.sleep(getPuppySpec().getExecDelay().getNextDelay()); + } catch (InterruptedException e) { +// Handle interruption if necessary + } +} + } +} catch (Exception e) { + e.printStackTrace(); +} + } + + public PuppySpec getPuppySpec() { +return _puppySpec; + } + + public int getUnhandledErrorCounter() { +return _unhandledErrorCounter; + } + + private void incrementUnhandledErrorCounter() { +_unhandledErrorCounter++; + } +} + + + diff --git a/meta-client/src/test/java/org/apache/helix/metaclient/puppy/ExecDelay.java b/meta-client/src/test/java/org/apache/helix/metaclient/puppy/ExecDelay.java new file mode 100644 index 0..da0bc529a --- /dev/null +++ b/meta-client/src/test/java/org/apache/helix/metaclient/puppy/ExecDelay.java @@ -0,0 +1,63 @@ +package org.apache.helix.metaclient.puppy; + +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the Lice
[helix] 21/21: Leader election client - Proactively send leader gone event when disconnect from ZK (#2585)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git commit 2e53694f77df0360d8092fe2b576eb4faa433ab2 Author: xyuanlu AuthorDate: Tue Aug 15 17:15:27 2023 -0700 Leader election client - Proactively send leader gone event when disconnect from ZK (#2585) --- .../leaderelection/LeaderElectionClient.java | 64 -- .../LeaderElectionListenerInterfaceAdapter.java| 29 ++- .../zk/TestConnectStateChangeListenerAndRetry.java | 32 +-- .../metaclient/impl/zk/TestStressZkClient.java | 3 +- .../apache/helix/metaclient/impl/zk/TestUtil.java | 24 ++ .../leaderelection/LeaderElectionPuppy.java| 5 + .../recipes/leaderelection/TestLeaderElection.java | 247 ++--- .../TestMultiClientLeaderElection.java | 3 + 8 files changed, 273 insertions(+), 134 deletions(-) diff --git a/meta-client/src/main/java/org/apache/helix/metaclient/recipes/leaderelection/LeaderElectionClient.java b/meta-client/src/main/java/org/apache/helix/metaclient/recipes/leaderelection/LeaderElectionClient.java index 7b13778c0..3bcf09ceb 100644 --- a/meta-client/src/main/java/org/apache/helix/metaclient/recipes/leaderelection/LeaderElectionClient.java +++ b/meta-client/src/main/java/org/apache/helix/metaclient/recipes/leaderelection/LeaderElectionClient.java @@ -33,6 +33,7 @@ import org.apache.helix.metaclient.api.DataChangeListener; import org.apache.helix.metaclient.api.MetaClientInterface; import org.apache.helix.metaclient.api.Op; import org.apache.helix.metaclient.api.OpResult; +import org.apache.helix.metaclient.exception.MetaClientBadVersionException; import org.apache.helix.metaclient.exception.MetaClientException; import org.apache.helix.metaclient.exception.MetaClientNoNodeException; import org.apache.helix.metaclient.exception.MetaClientNodeExistsException; @@ -83,7 +84,7 @@ public class LeaderElectionClient implements AutoCloseable { /** * Construct a LeaderElectionClient using a user passed in leaderElectionConfig. It creates a MetaClient * instance underneath. - * When MetaClient is auto closed be cause of being disconnected and auto retry connection timed out, A new + * When MetaClient is auto closed because of being disconnected and auto retry connection timed out, A new * MetaClient instance will be created and keeps retry connection. * * @param metaClientConfig The config used to create an metaclient. @@ -257,20 +258,24 @@ public class LeaderElectionClient implements AutoCloseable { } // check if current participant is the leader // read data and stats, check, and multi check + delete -ImmutablePair tup = _metaClient.getDataAndStat(key); -if (tup.left.getLeaderName().equalsIgnoreCase(_participant)) { - int expectedVersion = tup.right.getVersion(); - List ops = Arrays.asList(Op.check(key, expectedVersion), Op.delete(key, expectedVersion)); - //Execute transactional support on operations - List opResults = _metaClient.transactionOP(ops); - if (opResults.get(0).getType() == ERRORRESULT) { -if (isLeader(leaderPath)) { - // Participant re-elected as leader. - throw new ConcurrentModificationException("Concurrent operation, please retry"); -} else { - LOG.info("Someone else is already leader"); +try { + ImmutablePair tup = _metaClient.getDataAndStat(key); + if (tup.left.getLeaderName().equalsIgnoreCase(_participant)) { +int expectedVersion = tup.right.getVersion(); +List ops = Arrays.asList(Op.check(key, expectedVersion), Op.delete(key, expectedVersion)); +//Execute transactional support on operations +List opResults = _metaClient.transactionOP(ops); +if (opResults.get(0).getType() == ERRORRESULT) { + if (isLeader(leaderPath)) { +// Participant re-elected as leader. +throw new ConcurrentModificationException("Concurrent operation, please retry"); + } else { +LOG.info("Someone else is already leader"); + } } } +} catch (MetaClientNoNodeException ex) { + LOG.info("No Leader for participant pool {} when exit the pool", leaderPath); } } @@ -334,8 +339,10 @@ public class LeaderElectionClient implements AutoCloseable { * @return A boolean value indicating if registration is success. */ public boolean subscribeLeadershipChanges(String leaderPath, LeaderElectionListenerInterface listener) { -_metaClient.subscribeDataChange(leaderPath + LEADER_ENTRY_KEY, new LeaderElectionListenerInterfaceAdapter(listener), -false); +LeaderElectionListenerInterfaceAdapter adapter = new LeaderElectionListenerInterfaceAdapter(leaderPath, listener); +_metaClient.subscribeDataChange(leaderPath + LEADER_ENTRY_KEY,
[helix] 12/21: ZkClient - only register one time watcher for read data when not using persist listener. (#2555)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git commit 6aa7c2cce8c6202e80df63ef11a1ee1af085b7dc Author: xyuanlu AuthorDate: Thu Jul 13 10:25:10 2023 -0700 ZkClient - only register one time watcher for read data when not using persist listener. (#2555) - Co-authored-by: Xiaoyuan Lu --- .../helix/metaclient/impl/zk/TestZkMetaClient.java | 22 +++ .../apache/helix/zookeeper/zkclient/ZkClient.java | 2 +- .../zkclient/TestZkClientPersistWatcher.java | 75 +- 3 files changed, 67 insertions(+), 32 deletions(-) diff --git a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestZkMetaClient.java b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestZkMetaClient.java index 8aca150b0..19f21977b 100644 --- a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestZkMetaClient.java +++ b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestZkMetaClient.java @@ -504,6 +504,28 @@ public class TestZkMetaClient extends ZkMetaClientTestBase{ } } + @Test + public void testChangeListener() throws Exception { +final String basePath = "/TestZkMetaClient_ChangeListener"; +final int count = 100; +try (ZkMetaClient zkMetaClient = createZkMetaClient()) { + zkMetaClient.connect(); + DataChangeListener listener = new DataChangeListener() { + +@Override +public void handleDataChange(String key, Object data, ChangeType changeType) +throws Exception { +} + }; + zkMetaClient.subscribeDataChange(basePath, listener, false); + zkMetaClient.create(basePath, ""); + zkMetaClient.get(basePath); + zkMetaClient.exists(basePath); + zkMetaClient.getDataAndStat(basePath); + zkMetaClient.getDirectChildrenKeys(basePath); +} + } + /** * Transactional op calls zk.multi() with a set of ops (operations) * and the return values are converted into metaclient opResults. diff --git a/zookeeper-api/src/main/java/org/apache/helix/zookeeper/zkclient/ZkClient.java b/zookeeper-api/src/main/java/org/apache/helix/zookeeper/zkclient/ZkClient.java index 064f6b494..2a06158d0 100644 --- a/zookeeper-api/src/main/java/org/apache/helix/zookeeper/zkclient/ZkClient.java +++ b/zookeeper-api/src/main/java/org/apache/helix/zookeeper/zkclient/ZkClient.java @@ -2232,7 +2232,7 @@ public class ZkClient implements Watcher { @SuppressWarnings("unchecked") public T readData(String path, Stat stat) { -return (T) readData(path, stat, hasChildOrDataListeners(path)); +return (T) readData(path, stat, (!_usePersistWatcher) && hasChildOrDataListeners(path)); } @SuppressWarnings("unchecked") diff --git a/zookeeper-api/src/test/java/org/apache/helix/zookeeper/zkclient/TestZkClientPersistWatcher.java b/zookeeper-api/src/test/java/org/apache/helix/zookeeper/zkclient/TestZkClientPersistWatcher.java index 76f5352c9..c54bca1ef 100644 --- a/zookeeper-api/src/test/java/org/apache/helix/zookeeper/zkclient/TestZkClientPersistWatcher.java +++ b/zookeeper-api/src/test/java/org/apache/helix/zookeeper/zkclient/TestZkClientPersistWatcher.java @@ -41,8 +41,7 @@ public class TestZkClientPersistWatcher extends ZkTestBase { void testZkClientDataChange() throws Exception { org.apache.helix.zookeeper.impl.client.ZkClient.Builder builder = new org.apache.helix.zookeeper.impl.client.ZkClient.Builder(); -builder.setZkServer(ZkTestBase.ZK_ADDR).setMonitorRootPathOnly(false) -.setUsePersistWatcher(true); + builder.setZkServer(ZkTestBase.ZK_ADDR).setMonitorRootPathOnly(false).setUsePersistWatcher(true); org.apache.helix.zookeeper.impl.client.ZkClient zkClient = builder.build(); zkClient.setZkSerializer(new BasicZkSerializer(new SerializableSerializer())); int count = 1000; @@ -63,8 +62,8 @@ public class TestZkClientPersistWatcher extends ZkTestBase { zkClient.subscribeDataChanges(path, dataListener); zkClient.create(path, "datat", CreateMode.PERSISTENT); -for(int i=0; i currentChilds) - throws Exception { + public void handleChildChange(String parentPath, List currentChilds) throws Exception { countDownLatch1.countDown(); -event_count[0]++ ; +event_count[0]++; } }; IZkChildListener childListener2 = new IZkChildListener() { @Override - public void handleChildChange(String parentPath, List currentChilds) - throws Exception { + public void handleChildChange(String parentPath, List currentChilds) throws Exception { countDownLatch2.countDown(); event_count[0]++; } @@ -108,6 +104,7 @@ public class TestZkClientPersistWatcher extends ZkTestBase { } Assert.assertTrue(countDownLatch1.aw
[helix] 19/21: Lattice Puppy Stress Test Race Condition Fix and Code Cleanup (#2579)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git commit c3afa7f2714c44e73fe8f0210a624f88b382a270 Author: Marcos Rico Peng <55266861+marcosr...@users.noreply.github.com> AuthorDate: Wed Aug 2 18:46:26 2023 +0200 Lattice Puppy Stress Test Race Condition Fix and Code Cleanup (#2579) - Co-authored-by: mapeng --- .../apache/helix/metaclient/MetaClientTestUtil.java | 2 +- .../zk/TestMultiThreadStressTest/CreatePuppy.java | 3 +-- .../zk/TestMultiThreadStressTest/DeletePuppy.java | 2 +- .../impl/zk/TestMultiThreadStressTest/GetPuppy.java | 1 - .../impl/zk/TestMultiThreadStressTest/SetPuppy.java | 1 - .../TestMultiThreadStressZKClient.java | 6 ++ .../zk/TestMultiThreadStressTest/UpdatePuppy.java | 1 - .../helix/metaclient/puppy/AbstractPuppy.java | 21 +++-- 8 files changed, 12 insertions(+), 25 deletions(-) diff --git a/meta-client/src/test/java/org/apache/helix/metaclient/MetaClientTestUtil.java b/meta-client/src/test/java/org/apache/helix/metaclient/MetaClientTestUtil.java index 2c7543e31..f0a14c4a4 100644 --- a/meta-client/src/test/java/org/apache/helix/metaclient/MetaClientTestUtil.java +++ b/meta-client/src/test/java/org/apache/helix/metaclient/MetaClientTestUtil.java @@ -4,7 +4,7 @@ import java.util.concurrent.TimeUnit; public class MetaClientTestUtil { - public static final long WAIT_DURATION = TimeUnit.MINUTES.toMicros(1); + public static final long WAIT_DURATION = TimeUnit.MINUTES.toMillis(1); public interface Verifier { boolean verify() throws Exception; diff --git a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/CreatePuppy.java b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/CreatePuppy.java index 3e28df06b..3940f79ab 100644 --- a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/CreatePuppy.java +++ b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/CreatePuppy.java @@ -64,8 +64,7 @@ public class CreatePuppy extends AbstractPuppy { @Override protected void cleanup() { -// Implement the recovery logic by deleting the created documents -_metaclient.recursiveDelete(_parentPath); +// Cleanup logic in test case } private boolean shouldIntroduceError() { diff --git a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/DeletePuppy.java b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/DeletePuppy.java index e0e1b7b5c..1aa9d4d72 100644 --- a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/DeletePuppy.java +++ b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/DeletePuppy.java @@ -58,7 +58,7 @@ public class DeletePuppy extends AbstractPuppy { @Override protected void cleanup() { -_metaclient.recursiveDelete(_parentPath); +// Do nothing } private boolean shouldIntroduceError() { diff --git a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/GetPuppy.java b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/GetPuppy.java index fe24b2bd3..4af1c4df3 100644 --- a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/GetPuppy.java +++ b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/GetPuppy.java @@ -59,7 +59,6 @@ public class GetPuppy extends AbstractPuppy { @Override protected void cleanup() { -_metaclient.recursiveDelete(_parentPath); } private boolean shouldIntroduceError() { diff --git a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/SetPuppy.java b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/SetPuppy.java index c0de4ece7..3385b8673 100644 --- a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/SetPuppy.java +++ b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/SetPuppy.java @@ -61,7 +61,6 @@ public class SetPuppy extends AbstractPuppy { @Override protected void cleanup() { -_metaclient.recursiveDelete(_parentPath); } private boolean shouldIntroduceError() { diff --git a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/TestMultiThreadStressZKClient.java b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/TestMultiThreadStressZKClient.java index 5ee026bc1..6a01fffa7 100644 --- a/meta-client/src/test/java/org/apache/helix/metaclient/i
[helix] branch master updated (792f21162 -> 2e53694f7)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/helix.git from 792f21162 [apache/helix] -- Fixes #2590 Removed resetting of the missingTopStatePartitionsBeyondThresholdGauge to avoid incorrect accounting. new ff4c5cadd DataRecord Serializer new df1eb6ae9 Inline new b93f89a06 tests for get, create, set, update, and data change listeners new 1f80503b6 deserialize test new b45361ba3 refactor, update with dataRecord new 17f8ab3fd set test iteration count to 500 new 09649b34c [helix/metaclient] ZkMetaClient Stress Tests (Part 2) (#2523) new 0e917599c MetaClient Distributed Semaphore API and Permit object (#2520) new 53f9fa7c3 Add API for getDataAndStat (#2535) new c7a0d28fc Lattice MetaClient Distributed Semaphore Implementation (#2515) new f41a2c606 MetaClient LockClient and Semaphore Creation Edge Cases and Added Exception(#2550) new 6aa7c2cce ZkClient - only register one time watcher for read data when not using persist listener. (#2555) new 5389b1739 MultiThreading Stress Test Lattice - Puppy Logic and Skeleton (Part 1) (#2549) new 36231faeb Metaclient leader election - basics (#2558) new b1f59e6f8 Multithreading stress test lattice - CRUD puppies and Listener Tests (#2548) new 52fe73818 Metaclient - Leader election - Track participants and add participant info (#2562) new 4eacde331 Metaclient - Leader election - leader change event notification (#2560) new 26f319608 Lattice LockClient LockInfoSerializer Implementation(#2580) new c3afa7f27 Lattice Puppy Stress Test Race Condition Fix and Code Cleanup (#2579) new 7a96d9ba8 Add stress test for Metaclient leader election (#2574) new 2e53694f7 Leader election client - Proactively send leader gone event when disconnect from ZK (#2585) The 21 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: .../helix/metaclient/api/MetaClientInterface.java | 19 +- .../org/apache/helix/metaclient/api/OpResult.java | 2 +- .../helix/metaclient/datamodel/DataRecord.java | 5 + ...ion.java => MetaClientNodeExistsException.java} | 11 +- .../helix/metaclient/impl/zk/ZkMetaClient.java | 73 ++-- .../impl/zk/adapter/DataListenerAdapter.java | 1 + .../metaclient/impl/zk/util/ZkMetaClientUtil.java | 12 + .../leaderelection/LeaderElectionClient.java | 318 -- .../LeaderElectionListenerInterface.java | 15 +- .../LeaderElectionListenerInterfaceAdapter.java| 64 +++ .../recipes/leaderelection/LeaderInfo.java | 34 +- .../leaderelection/LeaderInfoSerializer.java | 42 ++ .../recipes/lock/DataRecordSerializer.java | 19 +- .../recipes/lock/DistributedSemaphore.java | 248 +++ .../helix/metaclient/recipes/lock/LockClient.java | 19 +- .../helix/metaclient/recipes/lock/LockInfo.java| 30 +- .../recipes/lock/LockInfoSerializer.java | 26 +- .../helix/metaclient/recipes/lock/Permit.java | 87 .../helix/metaclient/MetaClientTestUtil.java | 25 ++ .../zk/TestConnectStateChangeListenerAndRetry.java | 42 +- .../zk/TestMultiThreadStressTest/CreatePuppy.java | 77 .../zk/TestMultiThreadStressTest/DeletePuppy.java | 67 +++ .../zk/TestMultiThreadStressTest/GetPuppy.java | 67 +++ .../zk/TestMultiThreadStressTest/SetPuppy.java | 70 .../TestMultiThreadStressZKClient.java | 350 .../zk/TestMultiThreadStressTest/UpdatePuppy.java | 73 .../metaclient/impl/zk/TestStressZkClient.java | 455 - .../apache/helix/metaclient/impl/zk/TestUtil.java | 92 + .../helix/metaclient/impl/zk/TestZkMetaClient.java | 50 +++ .../metaclient/impl/zk/ZkMetaClientTestBase.java | 2 + .../helix/metaclient/puppy/AbstractPuppy.java | 91 + .../apache/helix/metaclient/puppy/ExecDelay.java | 63 +++ .../helix/metaclient/puppy/PuppyManager.java | 64 +++ .../apache/helix/metaclient/puppy/PuppyMode.java | 10 +- .../apache/helix/metaclient/puppy/PuppySpec.java | 42 +- .../leaderelection/LeaderElectionPuppy.java| 92 + .../recipes/leaderelection/TestLeaderElection.java | 361 .../TestMultiClientLeaderElection.java | 85 .../recipes/lock/DistributedSemaphoreTest.java | 105 + .../metaclient/recipes/lock/LockClientTest.java| 36 +- .../datamodel/serializer/ZNRecordSerializer.java | 2 +- .../apache/helix/zookeeper/zkclient/ZkClient.java | 2 +- .../zkclient/TestZkClientPersistWatcher.java | 75 ++-- 43 files changed, 3184 insertions(+), 239 deletions(-) cop
[helix] 20/21: Add stress test for Metaclient leader election (#2574)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git commit 7a96d9ba8c7dd671f6847bb8f01f50dbf41d7ca9 Author: xyuanlu AuthorDate: Mon Aug 7 09:56:18 2023 -0700 Add stress test for Metaclient leader election (#2574) --- .../zk/TestConnectStateChangeListenerAndRetry.java | 11 ++- .../metaclient/impl/zk/ZkMetaClientTestBase.java | 2 + .../helix/metaclient/puppy/AbstractPuppy.java | 2 +- .../leaderelection/LeaderElectionPuppy.java| 87 ++ .../recipes/leaderelection/TestLeaderElection.java | 55 -- .../TestMultiClientLeaderElection.java | 82 6 files changed, 214 insertions(+), 25 deletions(-) diff --git a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestConnectStateChangeListenerAndRetry.java b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestConnectStateChangeListenerAndRetry.java index c74b7d7ef..086db51c7 100644 --- a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestConnectStateChangeListenerAndRetry.java +++ b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestConnectStateChangeListenerAndRetry.java @@ -19,6 +19,8 @@ package org.apache.helix.metaclient.impl.zk; * under the License. */ +import java.io.File; +import java.io.IOException; import java.util.Date; import java.util.concurrent.CountDownLatch; import java.util.concurrent.Executors; @@ -27,17 +29,21 @@ import java.util.concurrent.locks.Condition; import java.util.concurrent.locks.Lock; import java.util.concurrent.locks.ReentrantLock; +import org.apache.commons.io.FileUtils; import org.apache.helix.metaclient.api.ConnectStateChangeListener; import org.apache.helix.metaclient.api.MetaClientInterface; import org.apache.helix.metaclient.impl.zk.factory.ZkMetaClientConfig; import org.apache.helix.metaclient.policy.ExponentialBackoffReconnectPolicy; +import org.apache.helix.zookeeper.zkclient.IDefaultNameSpace; import org.apache.helix.zookeeper.zkclient.ZkClient; import org.apache.helix.zookeeper.zkclient.ZkServer; import org.apache.zookeeper.WatchedEvent; import org.apache.zookeeper.Watcher; import org.testng.Assert; import org.testng.annotations.AfterSuite; +import org.testng.annotations.AfterTest; import org.testng.annotations.BeforeSuite; +import org.testng.annotations.BeforeTest; import org.testng.annotations.Test; import static org.apache.helix.metaclient.constants.MetaClientConstants.DEFAULT_INIT_EXP_BACKOFF_RETRY_INTERVAL_MS; @@ -69,14 +75,14 @@ public class TestConnectStateChangeListenerAndRetry { zkClient.process(event); } - @BeforeSuite + @BeforeTest public void prepare() { System.out.println("START TestConnectStateChangeListenerAndRetry at " + new Date(System.currentTimeMillis())); // start local zookeeper server _zkServer = ZkMetaClientTestBase.startZkServer(ZK_ADDR); } - @AfterSuite + @AfterTest public void cleanUp() { System.out.println("END TestConnectStateChangeListenerAndRetry at " + new Date(System.currentTimeMillis())); } @@ -162,7 +168,6 @@ public class TestConnectStateChangeListenerAndRetry { zkMetaClient.create("/key", "value"); Assert.fail("Create call after close should throw IllegalStateException"); } catch (Exception ex) { -System.out.println("ex " + ex); Assert.assertTrue(ex instanceof IllegalStateException); } } diff --git a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/ZkMetaClientTestBase.java b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/ZkMetaClientTestBase.java index 2a5f4b97b..eade017c1 100644 --- a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/ZkMetaClientTestBase.java +++ b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/ZkMetaClientTestBase.java @@ -49,6 +49,7 @@ public abstract class ZkMetaClientTestBase { */ @BeforeSuite public void prepare() { +System.out.println("ZkMetaClientTestBase start "); // Enable extended types and create a ZkClient System.setProperty("zookeeper.extendedTypesEnabled", "true"); // start local zookeeper server @@ -57,6 +58,7 @@ public abstract class ZkMetaClientTestBase { @AfterSuite public void cleanUp() { +System.out.println("ZkMetaClientTestBase shut down"); _zkServer.shutdown(); } diff --git a/meta-client/src/test/java/org/apache/helix/metaclient/puppy/AbstractPuppy.java b/meta-client/src/test/java/org/apache/helix/metaclient/puppy/AbstractPuppy.java index 9ce21fc15..b16f78615 100644 --- a/meta-client/src/test/java/org/apache/helix/metaclient/puppy/AbstractPuppy.java +++ b/meta-client/src/test/java/org/apache/helix/meta
[helix] 16/21: Metaclient - Leader election - Track participants and add participant info (#2562)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git commit 52fe73818d9be5b493888943da58d2e763c8a5be Author: xyuanlu AuthorDate: Sat Jul 22 21:45:29 2023 -0700 Metaclient - Leader election - Track participants and add participant info (#2562) Co-authored-by: Xiaoyuan Lu --- .../helix/metaclient/impl/zk/ZkMetaClient.java | 2 - .../leaderelection/LeaderElectionClient.java | 132 - .../{TestUtil.java => MetaClientTestUtil.java} | 7 +- .../apache/helix/metaclient/impl/zk/TestUtil.java | 68 +++ .../recipes/leaderelection/TestLeaderElection.java | 104 ++-- 5 files changed, 266 insertions(+), 47 deletions(-) diff --git a/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/ZkMetaClient.java b/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/ZkMetaClient.java index 691f31cde..8753747f3 100644 --- a/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/ZkMetaClient.java +++ b/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/ZkMetaClient.java @@ -57,14 +57,12 @@ import org.apache.helix.zookeeper.impl.client.ZkClient; import org.apache.helix.zookeeper.zkclient.IZkStateListener; import org.apache.helix.zookeeper.zkclient.ZkConnection; import org.apache.helix.zookeeper.zkclient.exception.ZkException; -import org.apache.helix.zookeeper.zkclient.exception.ZkInterruptedException; import org.apache.zookeeper.CreateMode; import org.apache.zookeeper.KeeperException; import org.apache.zookeeper.Watcher; import org.slf4j.Logger; import org.slf4j.LoggerFactory; -import static org.apache.helix.metaclient.impl.zk.util.ZkMetaClientUtil.convertZkEntryModeToMetaClientEntryMode; import static org.apache.helix.metaclient.impl.zk.util.ZkMetaClientUtil.translateZkExceptionToMetaclientException; diff --git a/meta-client/src/main/java/org/apache/helix/metaclient/recipes/leaderelection/LeaderElectionClient.java b/meta-client/src/main/java/org/apache/helix/metaclient/recipes/leaderelection/LeaderElectionClient.java index d5bdb735a..39233e979 100644 --- a/meta-client/src/main/java/org/apache/helix/metaclient/recipes/leaderelection/LeaderElectionClient.java +++ b/meta-client/src/main/java/org/apache/helix/metaclient/recipes/leaderelection/LeaderElectionClient.java @@ -21,11 +21,14 @@ package org.apache.helix.metaclient.recipes.leaderelection; import java.util.Arrays; import java.util.ConcurrentModificationException; +import java.util.HashMap; import java.util.HashSet; import java.util.List; +import java.util.Map; import java.util.Set; import org.apache.commons.lang3.tuple.ImmutablePair; +import org.apache.helix.metaclient.api.ConnectStateChangeListener; import org.apache.helix.metaclient.api.DataChangeListener; import org.apache.helix.metaclient.api.MetaClientInterface; import org.apache.helix.metaclient.api.Op; @@ -34,6 +37,7 @@ import org.apache.helix.metaclient.exception.MetaClientException; import org.apache.helix.metaclient.exception.MetaClientNoNodeException; import org.apache.helix.metaclient.exception.MetaClientNodeExistsException; import org.apache.helix.metaclient.factories.MetaClientConfig; +import org.apache.helix.metaclient.impl.zk.ZkMetaClient; import org.apache.helix.metaclient.impl.zk.factory.ZkMetaClientConfig; import org.apache.helix.metaclient.impl.zk.factory.ZkMetaClientFactory; import org.slf4j.Logger; @@ -68,8 +72,13 @@ public class LeaderElectionClient implements AutoCloseable { // A list of leader election group that this client joins. private Set _leaderGroups = new HashSet<>(); + private Map _participantInfos = new HashMap<>(); + private final static String LEADER_ENTRY_KEY = "/LEADER"; + private final static String PARTICIPANTS_ENTRY_KEY = "/PARTICIPANTS"; + private final static String PARTICIPANTS_ENTRY_PARENT = "/PARTICIPANTS/"; ReElectListener _reElectListener = new ReElectListener(); + ConnectStateListener _connectStateListener = new ConnectStateListener(); /** * Construct a LeaderElectionClient using a user passed in leaderElectionConfig. It creates a MetaClient @@ -90,6 +99,7 @@ public class LeaderElectionClient implements AutoCloseable { metaClientConfig.getConnectionAddress()).setZkSerializer((new LeaderInfoSerializer())).build(); _metaClient = new ZkMetaClientFactory().getMetaClient(zkMetaClientConfig); _metaClient.connect(); + _metaClient.subscribeStateChanges(_connectStateListener); } else { throw new MetaClientException("Unsupported store type: " + metaClientConfig.getStoreType()); } @@ -121,8 +131,8 @@ public class LeaderElectionClient implements AutoCloseable { * @throws RuntimeException if the operation is not succeeded. */ public void joinLeaderElectionParticipantPoo
[helix] 15/21: Multithreading stress test lattice - CRUD puppies and Listener Tests (#2548)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git commit b1f59e6f8669633df47523f7e36e989afd496fa2 Author: Marcos Rico Peng <55266861+marcosr...@users.noreply.github.com> AuthorDate: Thu Jul 20 13:05:37 2023 -0400 Multithreading stress test lattice - CRUD puppies and Listener Tests (#2548) - Co-authored-by: mapeng --- .../helix/metaclient/impl/zk/ZkMetaClient.java | 6 +- .../zk/TestMultiThreadStressTest/CreatePuppy.java | 78 + .../zk/TestMultiThreadStressTest/DeletePuppy.java | 67 .../zk/TestMultiThreadStressTest/GetPuppy.java | 68 .../zk/TestMultiThreadStressTest/SetPuppy.java | 71 + .../TestMultiThreadStressZKClient.java | 352 + .../zk/TestMultiThreadStressTest/UpdatePuppy.java | 74 + .../helix/metaclient/puppy/AbstractPuppy.java | 7 +- 8 files changed, 717 insertions(+), 6 deletions(-) diff --git a/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/ZkMetaClient.java b/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/ZkMetaClient.java index 7f68ec9ca..691f31cde 100644 --- a/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/ZkMetaClient.java +++ b/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/ZkMetaClient.java @@ -101,9 +101,7 @@ public class ZkMetaClient implements MetaClientInterface, AutoCloseable { try { create(key, data, EntryMode.PERSISTENT); } catch (ZkException e) { - throw ZkMetaClientUtil.translateZkExceptionToMetaclientException(e); -} catch (Exception e) { - throw new MetaClientException(e); + throw translateZkExceptionToMetaclientException(e); } } @@ -113,7 +111,7 @@ public class ZkMetaClient implements MetaClientInterface, AutoCloseable { try { _zkClient.create(key, data, ZkMetaClientUtil.convertMetaClientMode(mode)); } catch (ZkException e) { - throw ZkMetaClientUtil.translateZkExceptionToMetaclientException(e); + throw translateZkExceptionToMetaclientException(e); } catch (KeeperException e) { throw new MetaClientException(e); } diff --git a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/CreatePuppy.java b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/CreatePuppy.java new file mode 100644 index 0..3e28df06b --- /dev/null +++ b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/CreatePuppy.java @@ -0,0 +1,78 @@ +package org.apache.helix.metaclient.impl.zk.TestMultiThreadStressTest; + +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import org.apache.helix.metaclient.api.MetaClientInterface; +import org.apache.helix.metaclient.exception.MetaClientNodeExistsException; +import org.apache.helix.metaclient.puppy.AbstractPuppy; +import org.apache.helix.metaclient.puppy.PuppySpec; + +import java.util.Random; + +public class CreatePuppy extends AbstractPuppy { + + private final Random _random; + private final String _parentPath = "/test"; + + public CreatePuppy(MetaClientInterface metaclient, PuppySpec puppySpec) { +super(metaclient, puppySpec); +_random = new Random(); + } + + @Override + protected void bark() { +// Implement the chaos logic for creating nodes +int randomNumber = _random.nextInt(_puppySpec.getNumberDiffPaths()); +if (shouldIntroduceError()) { + try { +// Simulate an error by creating an invalid path +_metaclient.create("invalid", "test"); + } catch (IllegalArgumentException e) { // Catch invalid exception +System.out.println(Thread.currentThread().getName() + " tried to create an invalid path" + " at time: " + System.currentTimeMillis()); +// Expected exception + } +} else { + // Normal behavior - create a new node + try { +System.out.println(Thread.currentThread().getName() + " is attempting to create node: " +
[helix] 05/21: refactor, update with dataRecord
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git commit b45361ba373bdf8c7167d02164e4442f423ed7e2 Author: Grant Palau Spencer AuthorDate: Sat Jun 3 11:00:33 2023 -0700 refactor, update with dataRecord --- .../metaclient/impl/zk/TestStressZkClient.java | 406 ++--- 1 file changed, 183 insertions(+), 223 deletions(-) diff --git a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestStressZkClient.java b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestStressZkClient.java index 4b541ba74..5706dec7d 100644 --- a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestStressZkClient.java +++ b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestStressZkClient.java @@ -32,295 +32,254 @@ import org.apache.helix.metaclient.api.MetaClientInterface; import org.apache.helix.metaclient.datamodel.DataRecord; import org.apache.helix.metaclient.exception.MetaClientException; import org.apache.helix.metaclient.impl.zk.factory.ZkMetaClientConfig; +import org.apache.helix.metaclient.recipes.lock.DataRecordSerializer; import org.apache.helix.zookeeper.datamodel.ZNRecord; import org.apache.helix.zookeeper.datamodel.serializer.ZNRecordSerializer; import org.apache.helix.zookeeper.exception.ZkClientException; +import org.junit.Before; import org.testng.Assert; import org.slf4j.Logger; import org.slf4j.LoggerFactory; +import org.testng.annotations.AfterTest; +import org.testng.annotations.BeforeTest; import org.testng.annotations.Test; import static org.apache.helix.metaclient.api.MetaClientInterface.EntryMode.*; public class TestStressZkClient extends ZkMetaClientTestBase { - private static final long TEST_ITERATION_COUNT = 1000; - private static final Logger LOG = LoggerFactory.getLogger(TestStressZkClient.class); + private ZkMetaClient _zkMetaClient; + private static final long TEST_ITERATION_COUNT = 10; + @BeforeTest + private void setUp() { +this._zkMetaClient = createZkMetaClient(); +this._zkMetaClient.connect(); + } + + @AfterTest + private void tearDown() { +this._zkMetaClient.close(); + } @Test public void testCreate() { String zkParentKey = "/stressZk_testCreate"; -try (ZkMetaClient zkMetaClient = createZkMetaClient()) { - zkMetaClient.connect(); - zkMetaClient.create(zkParentKey, "parent_node"); +_zkMetaClient.create(zkParentKey, "parent_node"); - for (int i = 0; i < TEST_ITERATION_COUNT; i++) { -zkMetaClient.create(zkParentKey + "/" + i, i); - } +for (int i = 0; i < TEST_ITERATION_COUNT; i++) { + _zkMetaClient.create(zkParentKey + "/" + i, i); +} - for (int i = 0; i < TEST_ITERATION_COUNT; i++) { -Assert.assertEquals(String.valueOf(zkMetaClient.get(zkParentKey + "/" + i)), String.valueOf(i)); - } +for (int i = 0; i < TEST_ITERATION_COUNT; i++) { + Assert.assertEquals(String.valueOf(_zkMetaClient.get(zkParentKey + "/" + i)), String.valueOf(i)); +} - for (int i = 0; i < TEST_ITERATION_COUNT; i++) { -try { - zkMetaClient.create("/a/b/c", "invalid_path"); - Assert.fail("Should have failed with incorrect path."); -} catch (MetaClientException ignoredException) { -} +for (int i = 0; i < TEST_ITERATION_COUNT; i++) { + try { +_zkMetaClient.create("/a/b/c", "invalid_path"); +Assert.fail("Should have failed with incorrect path."); + } catch (MetaClientException ignoredException) { } +} - for (int i = 0; i < TEST_ITERATION_COUNT; i++) { -try { - zkMetaClient.create("a/b/c", "invalid_path"); - Assert.fail("Should have failed with invalid path - no leading /."); -} catch (Exception ignoredException) { -} +for (int i = 0; i < TEST_ITERATION_COUNT; i++) { + try { +_zkMetaClient.create("a/b/c", "invalid_path"); +Assert.fail("Should have failed with invalid path - no leading /."); + } catch (Exception ignoredException) { } // cleanup - zkMetaClient.recursiveDelete(zkParentKey); - Assert.assertEquals(zkMetaClient.countDirectChildren(zkParentKey), 0); + _zkMetaClient.recursiveDelete(zkParentKey); + Assert.assertEquals(_zkMetaClient.countDirectChildren(zkParentKey), 0); } } @Test public void testCreateContainer() { final String zkParentKey = "/stressZk_testCreateContainer"; -try (ZkMetaClient zkMetaClient = createZkMetaClient()) { - zkMetaClient.connect(); - zkMetaClient.create(zkParentKey, "parent_node"); +_z
[helix] 01/21: DataRecord Serializer
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git commit ff4c5cadd476726198daaee0e832de25a10539e2 Author: mapeng AuthorDate: Fri Jun 2 13:53:21 2023 -0400 DataRecord Serializer --- .../recipes/lock/DataRecordSerializer.java | 32 ++ 1 file changed, 32 insertions(+) diff --git a/meta-client/src/main/java/org/apache/helix/metaclient/recipes/lock/DataRecordSerializer.java b/meta-client/src/main/java/org/apache/helix/metaclient/recipes/lock/DataRecordSerializer.java new file mode 100644 index 0..6fee7bdea --- /dev/null +++ b/meta-client/src/main/java/org/apache/helix/metaclient/recipes/lock/DataRecordSerializer.java @@ -0,0 +1,32 @@ +package org.apache.helix.metaclient.recipes.lock; + +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import org.apache.helix.metaclient.datamodel.DataRecord; +import org.apache.helix.zookeeper.datamodel.ZNRecord; +import org.apache.helix.zookeeper.datamodel.serializer.ZNRecordSerializer; + +public class DataRecordSerializer extends ZNRecordSerializer { + @Override + public Object deserialize(byte[] bytes) { +Object o = super.deserialize(bytes); +return new DataRecord((ZNRecord) o); + } +}
[helix] 17/21: Metaclient - Leader election - leader change event notification (#2560)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git commit 4eacde3312265e5184708f59641451bccb649ca0 Author: xyuanlu AuthorDate: Sat Jul 22 22:39:37 2023 -0700 Metaclient - Leader election - leader change event notification (#2560) Co-authored-by: Xiaoyuan Lu --- .../impl/zk/adapter/DataListenerAdapter.java | 1 + .../leaderelection/LeaderElectionClient.java | 4 +- .../LeaderElectionListenerInterface.java | 15 ++- .../LeaderElectionListenerInterfaceAdapter.java| 43 +++ .../zk/TestConnectStateChangeListenerAndRetry.java | 3 +- .../recipes/leaderelection/TestLeaderElection.java | 135 - .../helix/zookeeper/zkclient/ZkConnection.java | 1 - 7 files changed, 193 insertions(+), 9 deletions(-) diff --git a/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/adapter/DataListenerAdapter.java b/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/adapter/DataListenerAdapter.java index 94ae198ce..748b6ed3f 100644 --- a/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/adapter/DataListenerAdapter.java +++ b/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/adapter/DataListenerAdapter.java @@ -21,6 +21,7 @@ package org.apache.helix.metaclient.impl.zk.adapter; import org.apache.helix.metaclient.api.DataChangeListener; import org.apache.helix.zookeeper.zkclient.IZkDataListener; +import org.apache.helix.zookeeper.zkclient.annotation.PreFetchChangedData; import org.apache.zookeeper.Watcher; diff --git a/meta-client/src/main/java/org/apache/helix/metaclient/recipes/leaderelection/LeaderElectionClient.java b/meta-client/src/main/java/org/apache/helix/metaclient/recipes/leaderelection/LeaderElectionClient.java index 39233e979..7b13778c0 100644 --- a/meta-client/src/main/java/org/apache/helix/metaclient/recipes/leaderelection/LeaderElectionClient.java +++ b/meta-client/src/main/java/org/apache/helix/metaclient/recipes/leaderelection/LeaderElectionClient.java @@ -334,7 +334,8 @@ public class LeaderElectionClient implements AutoCloseable { * @return A boolean value indicating if registration is success. */ public boolean subscribeLeadershipChanges(String leaderPath, LeaderElectionListenerInterface listener) { -//TODO: add converter class for LeaderElectionListenerInterface +_metaClient.subscribeDataChange(leaderPath + LEADER_ENTRY_KEY, new LeaderElectionListenerInterfaceAdapter(listener), +false); return false; } @@ -343,6 +344,7 @@ public class LeaderElectionClient implements AutoCloseable { * @param listener An implementation of LeaderElectionListenerInterface */ public void unsubscribeLeadershipChanges(String leaderPath, LeaderElectionListenerInterface listener) { +_metaClient.unsubscribeDataChange(leaderPath + LEADER_ENTRY_KEY, new LeaderElectionListenerInterfaceAdapter(listener)); } @Override diff --git a/meta-client/src/main/java/org/apache/helix/metaclient/recipes/leaderelection/LeaderElectionListenerInterface.java b/meta-client/src/main/java/org/apache/helix/metaclient/recipes/leaderelection/LeaderElectionListenerInterface.java index 0436e1eb0..230fc2af1 100644 --- a/meta-client/src/main/java/org/apache/helix/metaclient/recipes/leaderelection/LeaderElectionListenerInterface.java +++ b/meta-client/src/main/java/org/apache/helix/metaclient/recipes/leaderelection/LeaderElectionListenerInterface.java @@ -24,9 +24,16 @@ package org.apache.helix.metaclient.recipes.leaderelection; * leader node is deleted. */ public interface LeaderElectionListenerInterface { + enum ChangeType { +LEADER_ACQUIRED, +LEADER_LOST + } + // When new leader is elected: - // noLeader (null) ->has leader (new leader name) - // When existing leader not leader anymore: - // has Leader (prevleader name) ->no leader (null) - public void onLeadershipChange(String leaderPath, String prevLeader, String curLeader); + // ChangeType == NEW_LEADER_ELECTED, curLeader is the new leader name + // When no leader anymore: + // ChangeType == LEADER_GONE, curLeader is an empty string + // In ZK implementation, since notification does not include changed data and metaclient fetches + // the entry when event comes, it is possible that + public void onLeadershipChange(String leaderPath, ChangeType type, String curLeader); } \ No newline at end of file diff --git a/meta-client/src/main/java/org/apache/helix/metaclient/recipes/leaderelection/LeaderElectionListenerInterfaceAdapter.java b/meta-client/src/main/java/org/apache/helix/metaclient/recipes/leaderelection/LeaderElectionListenerInterfaceAdapter.java new file mode 100644 index 0..5c64d6790 --- /dev/null +++
[helix] 09/21: Add API for getDataAndStat (#2535)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git commit 53f9fa7c32c0b11f8cf00353478f50d3c98dead5 Author: xyuanlu AuthorDate: Wed Jun 21 09:05:02 2023 -0700 Add API for getDataAndStat (#2535) Add API for getDataAndStat --- .../helix/metaclient/api/MetaClientInterface.java | 13 -- .../helix/metaclient/impl/zk/ZkMetaClient.java | 16 +++-- .../metaclient/impl/zk/util/ZkMetaClientUtil.java | 8 +++ .../helix/metaclient/impl/zk/TestZkMetaClient.java | 28 ++ 4 files changed, 61 insertions(+), 4 deletions(-) diff --git a/meta-client/src/main/java/org/apache/helix/metaclient/api/MetaClientInterface.java b/meta-client/src/main/java/org/apache/helix/metaclient/api/MetaClientInterface.java index 375c0814c..b4bfaac2e 100644 --- a/meta-client/src/main/java/org/apache/helix/metaclient/api/MetaClientInterface.java +++ b/meta-client/src/main/java/org/apache/helix/metaclient/api/MetaClientInterface.java @@ -23,7 +23,9 @@ import java.util.List; import java.util.concurrent.TimeUnit; import org.apache.commons.lang3.NotImplementedException; +import org.apache.commons.lang3.tuple.ImmutablePair; import org.apache.helix.metaclient.exception.MetaClientInterruptException; +import org.apache.helix.metaclient.exception.MetaClientNoNodeException; import org.apache.helix.metaclient.exception.MetaClientTimeoutException; @@ -195,12 +197,19 @@ public interface MetaClientInterface { /** * Fetch the data for a given key. - * TODO: define exception type when key does not exist * @param key key to identify the entry - * @return Return data of the entry + * @return Return data of the entry. Return null if data does not exists. */ T get(final String key); + /** + * Fetch the data and stat for a given key. + * @param key key to identify the entry + * @return Return an ImmutablePair of data and stat for the entry. + * @throws MetaClientNoNodeException if no such entry + */ + ImmutablePair getDataAndStat(final String key); + /** * API for transaction. The list of operation will be executed as an atomic operation. * @param ops a list of operations. These operations will all be executed or none of them. diff --git a/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/ZkMetaClient.java b/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/ZkMetaClient.java index 4398e8e3a..fc8998e95 100644 --- a/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/ZkMetaClient.java +++ b/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/ZkMetaClient.java @@ -28,6 +28,7 @@ import java.util.concurrent.locks.ReentrantLock; import com.google.common.annotations.VisibleForTesting; import org.apache.commons.lang3.NotImplementedException; +import org.apache.commons.lang3.tuple.ImmutablePair; import org.apache.helix.metaclient.api.AsyncCallback; import org.apache.helix.metaclient.api.ChildChangeListener; import org.apache.helix.metaclient.api.ConnectStateChangeListener; @@ -166,8 +167,7 @@ public class ZkMetaClient implements MetaClientInterface, AutoCloseable { if (zkStats == null) { return null; } - return new Stat(convertZkEntryModeToMetaClientEntryMode(zkStats.getEphemeralOwner()), - zkStats.getVersion(), zkStats.getCtime(), zkStats.getMtime(), -1); + return ZkMetaClientUtil.convertZkStatToStat(zkStats); } catch (ZkException e) { throw translateZkExceptionToMetaclientException(e); } @@ -178,6 +178,18 @@ public class ZkMetaClient implements MetaClientInterface, AutoCloseable { return _zkClient.readData(key, true); } + + @Override + public ImmutablePair getDataAndStat(final String key) { +try { + org.apache.zookeeper.data.Stat zkStat = new org.apache.zookeeper.data.Stat(); + T data = _zkClient.readData(key, zkStat); + return ImmutablePair.of(data, ZkMetaClientUtil.convertZkStatToStat(zkStat)); +} catch (ZkException e) { + throw translateZkExceptionToMetaclientException(e); +} + } + @Override public List getDirectChildrenKeys(String key) { try { diff --git a/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/util/ZkMetaClientUtil.java b/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/util/ZkMetaClientUtil.java index f21a883f3..6bf0ce74b 100644 --- a/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/util/ZkMetaClientUtil.java +++ b/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/util/ZkMetaClientUtil.java @@ -262,6 +262,14 @@ public class ZkMetaClientUtil { } } + public static MetaClientInterface.Stat convertZkStatToStat( + org.apache.zookeeper.data.Stat zkStat) { +return new MetaClientInterface.Stat( +convertZkEntryModeToMetaClientEntryMode(zkStat.getEphemeralOwner
[helix] 11/21: MetaClient LockClient and Semaphore Creation Edge Cases and Added Exception(#2550)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git commit f41a2c606c715a5da436c47f5063978e893aee68 Author: Marcos Rico Peng <55266861+marcosr...@users.noreply.github.com> AuthorDate: Wed Jul 12 13:07:21 2023 -0400 MetaClient LockClient and Semaphore Creation Edge Cases and Added Exception(#2550) Co-authored-by: mapeng --- .../helix/metaclient/api/MetaClientInterface.java | 6 .../exception/MetaClientNodeExistsException.java | 38 ++ .../helix/metaclient/impl/zk/ZkMetaClient.java | 5 +++ .../metaclient/impl/zk/util/ZkMetaClientUtil.java | 4 +++ .../recipes/lock/DistributedSemaphore.java | 7 ++-- .../helix/metaclient/recipes/lock/LockClient.java | 6 ++-- 6 files changed, 62 insertions(+), 4 deletions(-) diff --git a/meta-client/src/main/java/org/apache/helix/metaclient/api/MetaClientInterface.java b/meta-client/src/main/java/org/apache/helix/metaclient/api/MetaClientInterface.java index b4bfaac2e..5b26896a9 100644 --- a/meta-client/src/main/java/org/apache/helix/metaclient/api/MetaClientInterface.java +++ b/meta-client/src/main/java/org/apache/helix/metaclient/api/MetaClientInterface.java @@ -429,6 +429,12 @@ public interface MetaClientInterface { */ void disconnect(); + /** + * Check whether client is closed + * @return true if client is closed, false otherwise + */ + boolean isClosed(); + /** * @return client current connection state with metadata service. */ diff --git a/meta-client/src/main/java/org/apache/helix/metaclient/exception/MetaClientNodeExistsException.java b/meta-client/src/main/java/org/apache/helix/metaclient/exception/MetaClientNodeExistsException.java new file mode 100644 index 0..74e6d4f0f --- /dev/null +++ b/meta-client/src/main/java/org/apache/helix/metaclient/exception/MetaClientNodeExistsException.java @@ -0,0 +1,38 @@ +package org.apache.helix.metaclient.exception; + +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +public final class MetaClientNodeExistsException extends MetaClientException { + public MetaClientNodeExistsException() { +super(); + } + + public MetaClientNodeExistsException(String message, Throwable cause) { +super(message, cause); + } + + public MetaClientNodeExistsException(String message) { +super(message); + } + + public MetaClientNodeExistsException(Throwable cause) { +super(cause); + } +} diff --git a/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/ZkMetaClient.java b/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/ZkMetaClient.java index fc8998e95..a1b6eb1ad 100644 --- a/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/ZkMetaClient.java +++ b/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/ZkMetaClient.java @@ -420,6 +420,11 @@ public class ZkMetaClient implements MetaClientInterface, AutoCloseable { disconnect(); } + @Override + public boolean isClosed() { +return _zkClient.isClosed(); + } + @Override public List transactionOP(Iterable ops) { // Convert list of MetaClient Ops to Zk Ops diff --git a/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/util/ZkMetaClientUtil.java b/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/util/ZkMetaClientUtil.java index 6bf0ce74b..dec8711cf 100644 --- a/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/util/ZkMetaClientUtil.java +++ b/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/util/ZkMetaClientUtil.java @@ -34,11 +34,13 @@ import org.apache.helix.metaclient.exception.MetaClientException; import org.apache.helix.metaclient.exception.MetaClientInterruptException; import org.apache.helix.metaclient.exception.MetaClientNoNodeException; import org.apache.helix.metaclient.exception.MetaClientTimeoutException; +import org.apache.helix.metaclient.exception.MetaClientNodeExistsException; import org.apache.helix.zookeeper.zkclient.exception.ZkBadVersionException; import org.apache.helix.zookeeper.zkclient.exce
[helix] 10/21: Lattice MetaClient Distributed Semaphore Implementation (#2515)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git commit c7a0d28fc605f521c8597622a6eea442b34b3195 Author: Marcos Rico Peng <55266861+marcosr...@users.noreply.github.com> AuthorDate: Fri Jun 23 19:12:23 2023 -0400 Lattice MetaClient Distributed Semaphore Implementation (#2515) Co-authored-by: mapeng --- .../recipes/lock/DistributedSemaphore.java | 123 ++--- .../helix/metaclient/recipes/lock/LockClient.java | 8 +- .../recipes/lock/DistributedSemaphoreTest.java | 105 ++ 3 files changed, 221 insertions(+), 15 deletions(-) diff --git a/meta-client/src/main/java/org/apache/helix/metaclient/recipes/lock/DistributedSemaphore.java b/meta-client/src/main/java/org/apache/helix/metaclient/recipes/lock/DistributedSemaphore.java index 0cccb130f..7b16e78fd 100644 --- a/meta-client/src/main/java/org/apache/helix/metaclient/recipes/lock/DistributedSemaphore.java +++ b/meta-client/src/main/java/org/apache/helix/metaclient/recipes/lock/DistributedSemaphore.java @@ -22,18 +22,45 @@ package org.apache.helix.metaclient.recipes.lock; import org.apache.commons.lang3.NotImplementedException; import org.apache.helix.metaclient.api.MetaClientInterface; import org.apache.helix.metaclient.datamodel.DataRecord; +import org.apache.helix.metaclient.exception.MetaClientException; import org.apache.helix.metaclient.factories.MetaClientConfig; +import org.apache.helix.metaclient.impl.zk.factory.ZkMetaClientConfig; +import org.apache.helix.metaclient.impl.zk.factory.ZkMetaClientFactory; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.util.ArrayList; import java.util.Collection; import java.util.concurrent.TimeUnit; public class DistributedSemaphore { + private final MetaClientInterface _metaClient; + private String _path; + private static final String INITIAL_CAPACITY_NAME = "INITIAL_CAPACITY"; + private static final String REMAINING_CAPACITY_NAME = "REMAINING_CAPACITY"; + private static final long DEFAULT_REMAINING_CAPACITY = -1; + private static final Logger LOG = LoggerFactory.getLogger(DistributedSemaphore.class); /** * Create a distributed semaphore client with the given configuration. * @param config configuration of the client */ public DistributedSemaphore(MetaClientConfig config) { -throw new NotImplementedException("Not implemented yet."); +if (config == null) { + throw new MetaClientException("Configuration cannot be null"); +} +LOG.info("Creating DistributedSemaphore Client"); +if (MetaClientConfig.StoreType.ZOOKEEPER.equals(config.getStoreType())) { + ZkMetaClientConfig zkMetaClientConfig = new ZkMetaClientConfig.ZkMetaClientConfigBuilder() + .setConnectionAddress(config.getConnectionAddress()) + .setZkSerializer(new DataRecordSerializer()) // Currently only support ZNRecordSerializer. + // Setting DataRecordSerializer as DataRecord extends ZNRecord. + .build(); + _metaClient = new ZkMetaClientFactory().getMetaClient(zkMetaClientConfig); + _metaClient.connect(); +} else { + throw new MetaClientException("Unsupported store type: " + config.getStoreType()); +} } /** @@ -41,7 +68,17 @@ public class DistributedSemaphore { * @param client client to connect to */ public DistributedSemaphore(MetaClientInterface client) { -throw new NotImplementedException("Not implemented yet."); +if (client == null) { + throw new MetaClientException("Client cannot be null"); +} +LOG.info("Connecting to existing DistributedSemaphore Client"); +_metaClient = client; +try { + _metaClient.connect(); + // TODO: Differentiate exception catch between already connected and already closed. +} catch (IllegalStateException e) { + // Ignore as it either has already been connected or already been closed. +} } /** @@ -50,7 +87,22 @@ public class DistributedSemaphore { * @param capacity capacity of the semaphore */ public void createSemaphore(String path, int capacity) { -throw new NotImplementedException("Not implemented yet."); +if (capacity <= 0) { + throw new MetaClientException("Capacity must be positive"); +} +if (path == null || path.isEmpty()) { + throw new MetaClientException("Invalid path to create semaphore"); +} +if (_metaClient.exists(path) != null) { + throw new MetaClientException("Semaphore already exists"); +} +if (_metaClient.exists(path) == null) { + DataRecord dataRecord = new DataRecord(path); + dataRecord.setLongField(INITIAL_CAPACITY_NAME, capacity); + dataRecord.setLongField(REMAINING_CAPACITY_NAME, capacity); +
[helix] 07/21: [helix/metaclient] ZkMetaClient Stress Tests (Part 2) (#2523)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git commit 09649b34c4941adfab951defa52975683393bb80 Author: Marcos Rico Peng <55266861+marcosr...@users.noreply.github.com> AuthorDate: Wed Jun 7 17:36:51 2023 -0400 [helix/metaclient] ZkMetaClient Stress Tests (Part 2) (#2523) ZkMetaClient Stress Tests (Part 2) - Co-authored-by: mapeng --- .../metaclient/impl/zk/TestStressZkClient.java | 96 +++--- 1 file changed, 83 insertions(+), 13 deletions(-) diff --git a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestStressZkClient.java b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestStressZkClient.java index 393a17f1b..8a3f9c115 100644 --- a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestStressZkClient.java +++ b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestStressZkClient.java @@ -19,27 +19,18 @@ package org.apache.helix.metaclient.impl.zk; * under the License. */ -import java.util.HashMap; -import java.util.HashSet; -import java.util.Map; -import java.util.Set; +import java.util.List; +import java.util.ArrayList; import java.util.concurrent.CountDownLatch; import java.util.concurrent.TimeUnit; import java.util.concurrent.atomic.AtomicBoolean; -import org.apache.helix.metaclient.api.DataChangeListener; -import org.apache.helix.metaclient.api.DataUpdater; -import org.apache.helix.metaclient.api.MetaClientInterface; + +import org.apache.helix.metaclient.api.*; import org.apache.helix.metaclient.datamodel.DataRecord; import org.apache.helix.metaclient.exception.MetaClientException; import org.apache.helix.metaclient.impl.zk.factory.ZkMetaClientConfig; import org.apache.helix.metaclient.recipes.lock.DataRecordSerializer; -import org.apache.helix.zookeeper.datamodel.ZNRecord; -import org.apache.helix.zookeeper.datamodel.serializer.ZNRecordSerializer; -import org.apache.helix.zookeeper.exception.ZkClientException; -import org.junit.Before; import org.testng.Assert; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; import org.testng.annotations.AfterTest; import org.testng.annotations.BeforeTest; import org.testng.annotations.Test; @@ -114,6 +105,29 @@ public class TestStressZkClient extends ZkMetaClientTestBase { Assert.assertEquals(_zkMetaClient.countDirectChildren(zkParentKey), 0); } + @Test + public void testCreateAndRenewTTL() { +final String zkParentKey = "/stressZk_testCreateAndRenewTTL"; +_zkMetaClient.create(zkParentKey, ENTRY_STRING_VALUE); +for (int i = 0; i < TEST_ITERATION_COUNT; i++) { + _zkMetaClient.createWithTTL(zkParentKey + i, ENTRY_STRING_VALUE, 1); +} +for (int i = 0; i < TEST_ITERATION_COUNT; i++) { + Assert.assertNotNull(_zkMetaClient.exists(zkParentKey)); +} +for (int i = 0; i < TEST_ITERATION_COUNT; i++) { + _zkMetaClient.renewTTLNode(zkParentKey + i); +} +MetaClientInterface.Stat stat; +for (int i = 0; i < TEST_ITERATION_COUNT; i++) { + stat = _zkMetaClient.exists(zkParentKey + i); + Assert.assertNotSame(stat.getCreationTime(), stat.getModifiedTime()); +} +// cleanup +_zkMetaClient.recursiveDelete(zkParentKey); +Assert.assertEquals(_zkMetaClient.countDirectChildren(zkParentKey), 0); + } + @Test public void testGet() { final String zkParentKey = "/stressZk_testGet"; @@ -369,4 +383,60 @@ public class TestStressZkClient extends ZkMetaClientTestBase { _zkMetaClient.recursiveDelete(zkParentKey); Assert.assertEquals(_zkMetaClient.countDirectChildren(zkParentKey), 0); } + + @Test + public void testTransactionOps() { +String zkParentKey = "/stressZk_testTransactionOp"; +_zkMetaClient.create(zkParentKey, "parent_node"); + +// Transaction Create +List ops = new ArrayList<>(); +for (int i = 0; i < TEST_ITERATION_COUNT; i++) { + ops.add(Op.create(zkParentKey + "/" + i, new byte[0], PERSISTENT)); +} +List opResults = _zkMetaClient.transactionOP(ops); +for (int i = 0; i < TEST_ITERATION_COUNT; i++) { + Assert.assertTrue(opResults.get(i) instanceof OpResult.CreateResult); +} +for (int i = 0; i < TEST_ITERATION_COUNT; i++) { + Assert.assertNotNull(_zkMetaClient.exists(zkParentKey + "/" + i)); +} + +// Transaction Set +List ops_set = new ArrayList<>(); +for (int i = 0; i < TEST_ITERATION_COUNT; i++) { + ops_set.add(Op.set(zkParentKey + "/" + i, new byte[0], -1)); +} +List opsResultSet = _zkMetaClient.transactionOP(ops_set); +for (int i = 0; i < TEST_ITERATION_COUNT; i++) { + Assert.assertTrue(opsResultSet.get(i) instanceof OpResult.SetDataResult); +} + +// Transaction Delete +List op
[helix] 04/21: deserialize test
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git commit 1f80503b675f0b4bb916ccd29e9f4bc84c8ed5b0 Author: Grant Palau Spencer AuthorDate: Sat Jun 3 10:06:19 2023 -0700 deserialize test --- .../metaclient/impl/zk/TestStressZkClient.java | 140 - 1 file changed, 79 insertions(+), 61 deletions(-) diff --git a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestStressZkClient.java b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestStressZkClient.java index 3e7140f64..4b541ba74 100644 --- a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestStressZkClient.java +++ b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestStressZkClient.java @@ -29,8 +29,11 @@ import java.util.concurrent.atomic.AtomicBoolean; import org.apache.helix.metaclient.api.DataChangeListener; import org.apache.helix.metaclient.api.DataUpdater; import org.apache.helix.metaclient.api.MetaClientInterface; +import org.apache.helix.metaclient.datamodel.DataRecord; import org.apache.helix.metaclient.exception.MetaClientException; import org.apache.helix.metaclient.impl.zk.factory.ZkMetaClientConfig; +import org.apache.helix.zookeeper.datamodel.ZNRecord; +import org.apache.helix.zookeeper.datamodel.serializer.ZNRecordSerializer; import org.apache.helix.zookeeper.exception.ZkClientException; import org.testng.Assert; import org.slf4j.Logger; @@ -43,23 +46,6 @@ public class TestStressZkClient extends ZkMetaClientTestBase { private static final long TEST_ITERATION_COUNT = 1000; private static final Logger LOG = LoggerFactory.getLogger(TestStressZkClient.class); - private void cleanUpTestNodes(String parentZnodeKey) { -try (ZkMetaClient zkMetaClient = createZkMetaClient()) { - zkMetaClient.connect(); - int retryCount = 0; - while (zkMetaClient.countDirectChildren(parentZnodeKey) > 0) { -try { - retryCount++; - zkMetaClient.recursiveDelete(parentZnodeKey); -} catch (ZkClientException e) { - if (retryCount >= 3) { -throw new ZkClientException("Failed to clean up test nodes after 3 tries", e); - } -} - } - Assert.assertEquals(zkMetaClient.countDirectChildren(parentZnodeKey), 0); -} - } @Test public void testCreate() { @@ -92,7 +78,10 @@ public class TestStressZkClient extends ZkMetaClientTestBase { } catch (Exception ignoredException) { } } - cleanUpTestNodes(zkParentKey); + + // cleanup + zkMetaClient.recursiveDelete(zkParentKey); + Assert.assertEquals(zkMetaClient.countDirectChildren(zkParentKey), 0); } } @@ -111,7 +100,9 @@ public class TestStressZkClient extends ZkMetaClientTestBase { Assert.assertEquals(String.valueOf(zkMetaClient.get(zkParentKey + "/" + i)), String.valueOf(i)); } - cleanUpTestNodes(zkParentKey); + // cleanup + zkMetaClient.recursiveDelete(zkParentKey); + Assert.assertEquals(zkMetaClient.countDirectChildren(zkParentKey), 0); } } @@ -142,8 +133,11 @@ public class TestStressZkClient extends ZkMetaClientTestBase { } catch (Exception ignoredException) { } } + + // cleanup + zkMetaClient.recursiveDelete(zkParentKey); + Assert.assertEquals(zkMetaClient.countDirectChildren(zkParentKey), 0); } -cleanUpTestNodes(zkParentKey); } @Test @@ -212,9 +206,11 @@ public class TestStressZkClient extends ZkMetaClientTestBase { } catch (Exception ignoredException) { } } -} -cleanUpTestNodes(zkParentKey); + // cleanup + zkMetaClient.recursiveDelete(zkParentKey); + Assert.assertEquals(zkMetaClient.countDirectChildren(zkParentKey), 0); +} } @Test @@ -251,8 +247,11 @@ public class TestStressZkClient extends ZkMetaClientTestBase { Assert.assertEquals(zkMetaClient.get(childKey), String.valueOf(i)); Assert.assertEquals(entryStat.getVersion(), 2); } + + // cleanup + zkMetaClient.recursiveDelete(zkParentKey); + Assert.assertEquals(zkMetaClient.countDirectChildren(zkParentKey), 0); } -cleanUpTestNodes(zkParentKey); } @Test @@ -260,59 +259,72 @@ public class TestStressZkClient extends ZkMetaClientTestBase { final String zkParentKey = "/stressZk_testUpdateSingleNode"; ZkMetaClientConfig config = new ZkMetaClientConfig.ZkMetaClientConfigBuilder().setConnectionAddress(ZK_ADDR).build(); -try (ZkMetaClient zkMetaClient = new ZkMetaClient<>(config)) { +try (ZkMetaClient zkMetaClient = new ZkMetaClient<>(config)) { zkMetaClient.connect(); - zkMetaClient.create(zkParentKey, 0); + ZNRecord record = new ZNRecord("foo"); + record.setIntF
[helix] 06/21: set test iteration count to 500
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git commit 17f8ab3fd167023e3b3e7e8c8bcffaf8c7e3e243 Author: Grant Palau Spencer AuthorDate: Mon Jun 5 10:52:30 2023 -0700 set test iteration count to 500 --- .../java/org/apache/helix/metaclient/impl/zk/TestStressZkClient.java| 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestStressZkClient.java b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestStressZkClient.java index 5706dec7d..393a17f1b 100644 --- a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestStressZkClient.java +++ b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestStressZkClient.java @@ -48,7 +48,7 @@ import static org.apache.helix.metaclient.api.MetaClientInterface.EntryMode.*; public class TestStressZkClient extends ZkMetaClientTestBase { private ZkMetaClient _zkMetaClient; - private static final long TEST_ITERATION_COUNT = 10; + private static final long TEST_ITERATION_COUNT = 500; @BeforeTest private void setUp() {
[helix] 08/21: MetaClient Distributed Semaphore API and Permit object (#2520)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git commit 0e917599cce521b1ca5fdaf60a13b8dc5cf9d1d0 Author: Marcos Rico Peng <55266861+marcosr...@users.noreply.github.com> AuthorDate: Sun Jun 11 00:53:46 2023 -0400 MetaClient Distributed Semaphore API and Permit object (#2520) MetaClient Distributed Semaphore API and Permit object - Co-authored-by: mapeng --- .../recipes/lock/DistributedSemaphore.java | 148 + .../helix/metaclient/recipes/lock/Permit.java | 87 2 files changed, 235 insertions(+) diff --git a/meta-client/src/main/java/org/apache/helix/metaclient/recipes/lock/DistributedSemaphore.java b/meta-client/src/main/java/org/apache/helix/metaclient/recipes/lock/DistributedSemaphore.java new file mode 100644 index 0..0cccb130f --- /dev/null +++ b/meta-client/src/main/java/org/apache/helix/metaclient/recipes/lock/DistributedSemaphore.java @@ -0,0 +1,148 @@ +package org.apache.helix.metaclient.recipes.lock; + +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import org.apache.commons.lang3.NotImplementedException; +import org.apache.helix.metaclient.api.MetaClientInterface; +import org.apache.helix.metaclient.datamodel.DataRecord; +import org.apache.helix.metaclient.factories.MetaClientConfig; +import java.util.Collection; +import java.util.concurrent.TimeUnit; + +public class DistributedSemaphore { + + /** + * Create a distributed semaphore client with the given configuration. + * @param config configuration of the client + */ + public DistributedSemaphore(MetaClientConfig config) { +throw new NotImplementedException("Not implemented yet."); + } + + /** + * Connect to an existing distributed semaphore client. + * @param client client to connect to + */ + public DistributedSemaphore(MetaClientInterface client) { +throw new NotImplementedException("Not implemented yet."); + } + + /** + * Create a distributed semaphore with the given path and capacity. + * @param path path of the semaphore + * @param capacity capacity of the semaphore + */ + public void createSemaphore(String path, int capacity) { +throw new NotImplementedException("Not implemented yet."); + } + + /** + * Connect to an existing distributed semaphore. + * @param path path of the semaphore + */ + public void connectSemaphore(String path) { +throw new NotImplementedException("Not implemented yet."); + } + + /** + * Acquire a permit. If no permit is available, log error and return null. + * @return a permit + */ + public Permit acquire() { +throw new NotImplementedException("Not implemented yet."); + } + + + /** + * Try to acquire multiple permits. If not enough permits are available, log error and return null. + * @param count number of permits to acquire + * @return a collection of permits + */ + public Collection acquire(int count) { +throw new NotImplementedException("Not implemented yet."); + } + + /** + * Try to acquire a permit. If no enough permit is available, wait for a specific time or return when it was able to acquire. + * If timeout <=0, then return immediately when not able to acquire. + * @param count number of permits to acquire + * @param timeout time to wait + * @param unit time unit + * @return a collection of permits + */ + public Collection acquire(int count, long timeout, TimeUnit unit) { +throw new NotImplementedException("Not implemented yet."); + } + + /** + * Get the remaining capacity of the semaphore + * @return remaining capacity + */ + public long getRemainingCapacity() { +throw new NotImplementedException("Not implemented yet."); + } + + /** + * Get the semaphore data record + * @return semaphore data record + */ + private DataRecord getSemaphore() { +throw new NotImplementedException("Not implemented yet."); + } + + /** + * Return a permit. If the permit is already returned, log and r
[helix] 02/21: Inline
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/helix.git commit df1eb6ae92e086676d22adda2f39ed43c2fbf8c7 Author: mapeng AuthorDate: Fri Jun 2 13:59:35 2023 -0400 Inline --- .../org/apache/helix/metaclient/recipes/lock/DataRecordSerializer.java | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/meta-client/src/main/java/org/apache/helix/metaclient/recipes/lock/DataRecordSerializer.java b/meta-client/src/main/java/org/apache/helix/metaclient/recipes/lock/DataRecordSerializer.java index 6fee7bdea..bf7806327 100644 --- a/meta-client/src/main/java/org/apache/helix/metaclient/recipes/lock/DataRecordSerializer.java +++ b/meta-client/src/main/java/org/apache/helix/metaclient/recipes/lock/DataRecordSerializer.java @@ -26,7 +26,6 @@ import org.apache.helix.zookeeper.datamodel.serializer.ZNRecordSerializer; public class DataRecordSerializer extends ZNRecordSerializer { @Override public Object deserialize(byte[] bytes) { -Object o = super.deserialize(bytes); -return new DataRecord((ZNRecord) o); +return new DataRecord((ZNRecord) super.deserialize(bytes)); } }
[helix] 19/21: Lattice Puppy Stress Test Race Condition Fix and Code Cleanup (#2579)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch metaclient in repository https://gitbox.apache.org/repos/asf/helix.git commit a1fa502835b80cb4928cb68c85b3a4d24247487a Author: Marcos Rico Peng <55266861+marcosr...@users.noreply.github.com> AuthorDate: Wed Aug 2 18:46:26 2023 +0200 Lattice Puppy Stress Test Race Condition Fix and Code Cleanup (#2579) - Co-authored-by: mapeng --- .../apache/helix/metaclient/MetaClientTestUtil.java | 2 +- .../zk/TestMultiThreadStressTest/CreatePuppy.java | 3 +-- .../zk/TestMultiThreadStressTest/DeletePuppy.java | 2 +- .../impl/zk/TestMultiThreadStressTest/GetPuppy.java | 1 - .../impl/zk/TestMultiThreadStressTest/SetPuppy.java | 1 - .../TestMultiThreadStressZKClient.java | 6 ++ .../zk/TestMultiThreadStressTest/UpdatePuppy.java | 1 - .../helix/metaclient/puppy/AbstractPuppy.java | 21 +++-- 8 files changed, 12 insertions(+), 25 deletions(-) diff --git a/meta-client/src/test/java/org/apache/helix/metaclient/MetaClientTestUtil.java b/meta-client/src/test/java/org/apache/helix/metaclient/MetaClientTestUtil.java index 2c7543e31..f0a14c4a4 100644 --- a/meta-client/src/test/java/org/apache/helix/metaclient/MetaClientTestUtil.java +++ b/meta-client/src/test/java/org/apache/helix/metaclient/MetaClientTestUtil.java @@ -4,7 +4,7 @@ import java.util.concurrent.TimeUnit; public class MetaClientTestUtil { - public static final long WAIT_DURATION = TimeUnit.MINUTES.toMicros(1); + public static final long WAIT_DURATION = TimeUnit.MINUTES.toMillis(1); public interface Verifier { boolean verify() throws Exception; diff --git a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/CreatePuppy.java b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/CreatePuppy.java index 3e28df06b..3940f79ab 100644 --- a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/CreatePuppy.java +++ b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/CreatePuppy.java @@ -64,8 +64,7 @@ public class CreatePuppy extends AbstractPuppy { @Override protected void cleanup() { -// Implement the recovery logic by deleting the created documents -_metaclient.recursiveDelete(_parentPath); +// Cleanup logic in test case } private boolean shouldIntroduceError() { diff --git a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/DeletePuppy.java b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/DeletePuppy.java index e0e1b7b5c..1aa9d4d72 100644 --- a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/DeletePuppy.java +++ b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/DeletePuppy.java @@ -58,7 +58,7 @@ public class DeletePuppy extends AbstractPuppy { @Override protected void cleanup() { -_metaclient.recursiveDelete(_parentPath); +// Do nothing } private boolean shouldIntroduceError() { diff --git a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/GetPuppy.java b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/GetPuppy.java index fe24b2bd3..4af1c4df3 100644 --- a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/GetPuppy.java +++ b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/GetPuppy.java @@ -59,7 +59,6 @@ public class GetPuppy extends AbstractPuppy { @Override protected void cleanup() { -_metaclient.recursiveDelete(_parentPath); } private boolean shouldIntroduceError() { diff --git a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/SetPuppy.java b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/SetPuppy.java index c0de4ece7..3385b8673 100644 --- a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/SetPuppy.java +++ b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/SetPuppy.java @@ -61,7 +61,6 @@ public class SetPuppy extends AbstractPuppy { @Override protected void cleanup() { -_metaclient.recursiveDelete(_parentPath); } private boolean shouldIntroduceError() { diff --git a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/TestMultiThreadStressZKClient.java b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/TestMultiThreadStressZKClient.java index 5ee026bc1..6a01fffa7 100644 --- a/meta-client/src/test/java/org/apache/helix/metaclient/i
[helix] 12/21: ZkClient - only register one time watcher for read data when not using persist listener. (#2555)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch metaclient in repository https://gitbox.apache.org/repos/asf/helix.git commit def0bb6d6b22789c0c40716a9233a3258082eb76 Author: xyuanlu AuthorDate: Thu Jul 13 10:25:10 2023 -0700 ZkClient - only register one time watcher for read data when not using persist listener. (#2555) - Co-authored-by: Xiaoyuan Lu --- .../helix/metaclient/impl/zk/TestZkMetaClient.java | 22 +++ .../apache/helix/zookeeper/zkclient/ZkClient.java | 2 +- .../zkclient/TestZkClientPersistWatcher.java | 75 +- 3 files changed, 67 insertions(+), 32 deletions(-) diff --git a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestZkMetaClient.java b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestZkMetaClient.java index 8aca150b0..19f21977b 100644 --- a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestZkMetaClient.java +++ b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestZkMetaClient.java @@ -504,6 +504,28 @@ public class TestZkMetaClient extends ZkMetaClientTestBase{ } } + @Test + public void testChangeListener() throws Exception { +final String basePath = "/TestZkMetaClient_ChangeListener"; +final int count = 100; +try (ZkMetaClient zkMetaClient = createZkMetaClient()) { + zkMetaClient.connect(); + DataChangeListener listener = new DataChangeListener() { + +@Override +public void handleDataChange(String key, Object data, ChangeType changeType) +throws Exception { +} + }; + zkMetaClient.subscribeDataChange(basePath, listener, false); + zkMetaClient.create(basePath, ""); + zkMetaClient.get(basePath); + zkMetaClient.exists(basePath); + zkMetaClient.getDataAndStat(basePath); + zkMetaClient.getDirectChildrenKeys(basePath); +} + } + /** * Transactional op calls zk.multi() with a set of ops (operations) * and the return values are converted into metaclient opResults. diff --git a/zookeeper-api/src/main/java/org/apache/helix/zookeeper/zkclient/ZkClient.java b/zookeeper-api/src/main/java/org/apache/helix/zookeeper/zkclient/ZkClient.java index 064f6b494..2a06158d0 100644 --- a/zookeeper-api/src/main/java/org/apache/helix/zookeeper/zkclient/ZkClient.java +++ b/zookeeper-api/src/main/java/org/apache/helix/zookeeper/zkclient/ZkClient.java @@ -2232,7 +2232,7 @@ public class ZkClient implements Watcher { @SuppressWarnings("unchecked") public T readData(String path, Stat stat) { -return (T) readData(path, stat, hasChildOrDataListeners(path)); +return (T) readData(path, stat, (!_usePersistWatcher) && hasChildOrDataListeners(path)); } @SuppressWarnings("unchecked") diff --git a/zookeeper-api/src/test/java/org/apache/helix/zookeeper/zkclient/TestZkClientPersistWatcher.java b/zookeeper-api/src/test/java/org/apache/helix/zookeeper/zkclient/TestZkClientPersistWatcher.java index 76f5352c9..c54bca1ef 100644 --- a/zookeeper-api/src/test/java/org/apache/helix/zookeeper/zkclient/TestZkClientPersistWatcher.java +++ b/zookeeper-api/src/test/java/org/apache/helix/zookeeper/zkclient/TestZkClientPersistWatcher.java @@ -41,8 +41,7 @@ public class TestZkClientPersistWatcher extends ZkTestBase { void testZkClientDataChange() throws Exception { org.apache.helix.zookeeper.impl.client.ZkClient.Builder builder = new org.apache.helix.zookeeper.impl.client.ZkClient.Builder(); -builder.setZkServer(ZkTestBase.ZK_ADDR).setMonitorRootPathOnly(false) -.setUsePersistWatcher(true); + builder.setZkServer(ZkTestBase.ZK_ADDR).setMonitorRootPathOnly(false).setUsePersistWatcher(true); org.apache.helix.zookeeper.impl.client.ZkClient zkClient = builder.build(); zkClient.setZkSerializer(new BasicZkSerializer(new SerializableSerializer())); int count = 1000; @@ -63,8 +62,8 @@ public class TestZkClientPersistWatcher extends ZkTestBase { zkClient.subscribeDataChanges(path, dataListener); zkClient.create(path, "datat", CreateMode.PERSISTENT); -for(int i=0; i currentChilds) - throws Exception { + public void handleChildChange(String parentPath, List currentChilds) throws Exception { countDownLatch1.countDown(); -event_count[0]++ ; +event_count[0]++; } }; IZkChildListener childListener2 = new IZkChildListener() { @Override - public void handleChildChange(String parentPath, List currentChilds) - throws Exception { + public void handleChildChange(String parentPath, List currentChilds) throws Exception { countDownLatch2.countDown(); event_count[0]++; } @@ -108,6 +104,7 @@ public class TestZkClientPersistWatcher extends ZkTestBase { } Assert.ass
[helix] 15/21: Multithreading stress test lattice - CRUD puppies and Listener Tests (#2548)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch metaclient in repository https://gitbox.apache.org/repos/asf/helix.git commit 785c61ad5bfbf94d384d335972a16d8030cc18ce Author: Marcos Rico Peng <55266861+marcosr...@users.noreply.github.com> AuthorDate: Thu Jul 20 13:05:37 2023 -0400 Multithreading stress test lattice - CRUD puppies and Listener Tests (#2548) - Co-authored-by: mapeng --- .../helix/metaclient/impl/zk/ZkMetaClient.java | 6 +- .../zk/TestMultiThreadStressTest/CreatePuppy.java | 78 + .../zk/TestMultiThreadStressTest/DeletePuppy.java | 67 .../zk/TestMultiThreadStressTest/GetPuppy.java | 68 .../zk/TestMultiThreadStressTest/SetPuppy.java | 71 + .../TestMultiThreadStressZKClient.java | 352 + .../zk/TestMultiThreadStressTest/UpdatePuppy.java | 74 + .../helix/metaclient/puppy/AbstractPuppy.java | 7 +- 8 files changed, 717 insertions(+), 6 deletions(-) diff --git a/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/ZkMetaClient.java b/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/ZkMetaClient.java index 7f68ec9ca..691f31cde 100644 --- a/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/ZkMetaClient.java +++ b/meta-client/src/main/java/org/apache/helix/metaclient/impl/zk/ZkMetaClient.java @@ -101,9 +101,7 @@ public class ZkMetaClient implements MetaClientInterface, AutoCloseable { try { create(key, data, EntryMode.PERSISTENT); } catch (ZkException e) { - throw ZkMetaClientUtil.translateZkExceptionToMetaclientException(e); -} catch (Exception e) { - throw new MetaClientException(e); + throw translateZkExceptionToMetaclientException(e); } } @@ -113,7 +111,7 @@ public class ZkMetaClient implements MetaClientInterface, AutoCloseable { try { _zkClient.create(key, data, ZkMetaClientUtil.convertMetaClientMode(mode)); } catch (ZkException e) { - throw ZkMetaClientUtil.translateZkExceptionToMetaclientException(e); + throw translateZkExceptionToMetaclientException(e); } catch (KeeperException e) { throw new MetaClientException(e); } diff --git a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/CreatePuppy.java b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/CreatePuppy.java new file mode 100644 index 0..3e28df06b --- /dev/null +++ b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestMultiThreadStressTest/CreatePuppy.java @@ -0,0 +1,78 @@ +package org.apache.helix.metaclient.impl.zk.TestMultiThreadStressTest; + +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, + * software distributed under the License is distributed on an + * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY + * KIND, either express or implied. See the License for the + * specific language governing permissions and limitations + * under the License. + */ + +import org.apache.helix.metaclient.api.MetaClientInterface; +import org.apache.helix.metaclient.exception.MetaClientNodeExistsException; +import org.apache.helix.metaclient.puppy.AbstractPuppy; +import org.apache.helix.metaclient.puppy.PuppySpec; + +import java.util.Random; + +public class CreatePuppy extends AbstractPuppy { + + private final Random _random; + private final String _parentPath = "/test"; + + public CreatePuppy(MetaClientInterface metaclient, PuppySpec puppySpec) { +super(metaclient, puppySpec); +_random = new Random(); + } + + @Override + protected void bark() { +// Implement the chaos logic for creating nodes +int randomNumber = _random.nextInt(_puppySpec.getNumberDiffPaths()); +if (shouldIntroduceError()) { + try { +// Simulate an error by creating an invalid path +_metaclient.create("invalid", "test"); + } catch (IllegalArgumentException e) { // Catch invalid exception +System.out.println(Thread.currentThread().getName() + " tried to create an invalid path" + " at time: " + System.currentTimeMillis()); +// Expected exception + } +} else { + // Normal behavior - create a new node + try { +System.out.println(Thread.currentThread().getName() + " is attem
[helix] 07/21: [helix/metaclient] ZkMetaClient Stress Tests (Part 2) (#2523)
This is an automated email from the ASF dual-hosted git repository. jxue pushed a commit to branch metaclient in repository https://gitbox.apache.org/repos/asf/helix.git commit 1e9b19ba64af840a5d844eac15bedad54f81 Author: Marcos Rico Peng <55266861+marcosr...@users.noreply.github.com> AuthorDate: Wed Jun 7 17:36:51 2023 -0400 [helix/metaclient] ZkMetaClient Stress Tests (Part 2) (#2523) ZkMetaClient Stress Tests (Part 2) - Co-authored-by: mapeng --- .../metaclient/impl/zk/TestStressZkClient.java | 96 +++--- 1 file changed, 83 insertions(+), 13 deletions(-) diff --git a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestStressZkClient.java b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestStressZkClient.java index 393a17f1b..8a3f9c115 100644 --- a/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestStressZkClient.java +++ b/meta-client/src/test/java/org/apache/helix/metaclient/impl/zk/TestStressZkClient.java @@ -19,27 +19,18 @@ package org.apache.helix.metaclient.impl.zk; * under the License. */ -import java.util.HashMap; -import java.util.HashSet; -import java.util.Map; -import java.util.Set; +import java.util.List; +import java.util.ArrayList; import java.util.concurrent.CountDownLatch; import java.util.concurrent.TimeUnit; import java.util.concurrent.atomic.AtomicBoolean; -import org.apache.helix.metaclient.api.DataChangeListener; -import org.apache.helix.metaclient.api.DataUpdater; -import org.apache.helix.metaclient.api.MetaClientInterface; + +import org.apache.helix.metaclient.api.*; import org.apache.helix.metaclient.datamodel.DataRecord; import org.apache.helix.metaclient.exception.MetaClientException; import org.apache.helix.metaclient.impl.zk.factory.ZkMetaClientConfig; import org.apache.helix.metaclient.recipes.lock.DataRecordSerializer; -import org.apache.helix.zookeeper.datamodel.ZNRecord; -import org.apache.helix.zookeeper.datamodel.serializer.ZNRecordSerializer; -import org.apache.helix.zookeeper.exception.ZkClientException; -import org.junit.Before; import org.testng.Assert; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; import org.testng.annotations.AfterTest; import org.testng.annotations.BeforeTest; import org.testng.annotations.Test; @@ -114,6 +105,29 @@ public class TestStressZkClient extends ZkMetaClientTestBase { Assert.assertEquals(_zkMetaClient.countDirectChildren(zkParentKey), 0); } + @Test + public void testCreateAndRenewTTL() { +final String zkParentKey = "/stressZk_testCreateAndRenewTTL"; +_zkMetaClient.create(zkParentKey, ENTRY_STRING_VALUE); +for (int i = 0; i < TEST_ITERATION_COUNT; i++) { + _zkMetaClient.createWithTTL(zkParentKey + i, ENTRY_STRING_VALUE, 1); +} +for (int i = 0; i < TEST_ITERATION_COUNT; i++) { + Assert.assertNotNull(_zkMetaClient.exists(zkParentKey)); +} +for (int i = 0; i < TEST_ITERATION_COUNT; i++) { + _zkMetaClient.renewTTLNode(zkParentKey + i); +} +MetaClientInterface.Stat stat; +for (int i = 0; i < TEST_ITERATION_COUNT; i++) { + stat = _zkMetaClient.exists(zkParentKey + i); + Assert.assertNotSame(stat.getCreationTime(), stat.getModifiedTime()); +} +// cleanup +_zkMetaClient.recursiveDelete(zkParentKey); +Assert.assertEquals(_zkMetaClient.countDirectChildren(zkParentKey), 0); + } + @Test public void testGet() { final String zkParentKey = "/stressZk_testGet"; @@ -369,4 +383,60 @@ public class TestStressZkClient extends ZkMetaClientTestBase { _zkMetaClient.recursiveDelete(zkParentKey); Assert.assertEquals(_zkMetaClient.countDirectChildren(zkParentKey), 0); } + + @Test + public void testTransactionOps() { +String zkParentKey = "/stressZk_testTransactionOp"; +_zkMetaClient.create(zkParentKey, "parent_node"); + +// Transaction Create +List ops = new ArrayList<>(); +for (int i = 0; i < TEST_ITERATION_COUNT; i++) { + ops.add(Op.create(zkParentKey + "/" + i, new byte[0], PERSISTENT)); +} +List opResults = _zkMetaClient.transactionOP(ops); +for (int i = 0; i < TEST_ITERATION_COUNT; i++) { + Assert.assertTrue(opResults.get(i) instanceof OpResult.CreateResult); +} +for (int i = 0; i < TEST_ITERATION_COUNT; i++) { + Assert.assertNotNull(_zkMetaClient.exists(zkParentKey + "/" + i)); +} + +// Transaction Set +List ops_set = new ArrayList<>(); +for (int i = 0; i < TEST_ITERATION_COUNT; i++) { + ops_set.add(Op.set(zkParentKey + "/" + i, new byte[0], -1)); +} +List opsResultSet = _zkMetaClient.transactionOP(ops_set); +for (int i = 0; i < TEST_ITERATION_COUNT; i++) { + Assert.assertTrue(opsResultSet.get(i) instanceof OpResult.SetDataResult); +} + +// Transaction Delete +List op