GitHub user jiajunwang opened a pull request:
https://github.com/apache/helix/pull/89
[HELIX-655] Helix per-participant concurrent task throttling
Add per participant concurrent task throttling.
1. Add a participant configuration item "MAX_CONCURRENT_TASK" for
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/89#discussion_r116554013
--- Diff:
helix-core/src/main/java/org/apache/helix/controller/stages/BestPossibleStateCalcStage.java
---
@@ -90,60 +96,86 @@ private
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/89#discussion_r116554664
--- Diff:
helix-core/src/main/java/org/apache/helix/controller/stages/BestPossibleStateCalcStage.java
---
@@ -204,4 +236,28 @@ private MappingCalculator
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/89#discussion_r116555244
--- Diff:
helix-core/src/main/java/org/apache/helix/controller/stages/CurrentStateOutput.java
---
@@ -260,13 +260,72 @@ public Message getPendingState
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/89#discussion_r116556982
--- Diff:
helix-core/src/main/java/org/apache/helix/controller/stages/CurrentStateOutput.java
---
@@ -260,13 +260,72 @@ public Message getPendingState
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/89#discussion_r116557426
--- Diff: helix-core/src/main/java/org/apache/helix/task/JobRebalancer.java
---
@@ -57,6 +57,7 @@
new FixedTargetTaskAssignmentCalculator
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/89#discussion_r116559458
--- Diff: helix-core/src/main/java/org/apache/helix/task/JobRebalancer.java
---
@@ -704,4 +712,30 @@ private PartitionAssignment(String instance, String
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/89#discussion_r116564901
--- Diff: helix-core/src/main/java/org/apache/helix/task/JobRebalancer.java
---
@@ -424,9 +425,15 @@ private ResourceAssignment
computeResourceMapping
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/89#discussion_r116571005
--- Diff:
helix-core/src/main/java/org/apache/helix/controller/stages/BestPossibleStateCalcStage.java
---
@@ -90,60 +96,86 @@ private
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/89#discussion_r116576058
--- Diff:
helix-core/src/main/java/org/apache/helix/controller/stages/CurrentStateOutput.java
---
@@ -260,13 +260,72 @@ public Message getPendingState
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/88#discussion_r117135837
--- Diff: helix-core/src/main/java/org/apache/helix/task/JobRebalancer.java
---
@@ -420,6 +411,14 @@ private ResourceAssignment
computeResourceMapping
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/88#discussion_r117137720
--- Diff: helix-core/src/main/java/org/apache/helix/task/JobRebalancer.java
---
@@ -455,6 +454,44 @@ private ResourceAssignment
computeResourceMapping
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/88#discussion_r117420922
--- Diff: helix-core/src/main/java/org/apache/helix/task/JobRebalancer.java
---
@@ -455,6 +454,44 @@ private ResourceAssignment
computeResourceMapping
GitHub user jiajunwang opened a pull request:
https://github.com/apache/helix/pull/95
[HELIX-657] Fix unexpected idealstate overwrite when persist assignmeâ¦
â¦nt is on.
1. Change persist method from set to update in PersistAssignmentStage.
The new updater only
Github user jiajunwang commented on the issue:
https://github.com/apache/helix/pull/89
Rebase to latest change and apply one additional change as suggested by
lei-xia.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/89#discussion_r119465769
--- Diff:
helix-core/src/main/java/org/apache/helix/controller/stages/BestPossibleStateCalcStage.java
---
@@ -90,61 +96,84 @@ private
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/89#discussion_r119465719
--- Diff:
helix-core/src/main/java/org/apache/helix/model/InstanceConfig.java ---
@@ -408,6 +410,18 @@ public void setInstanceEnabledForPartition(String
Github user jiajunwang commented on the issue:
https://github.com/apache/helix/pull/89
Closed since master branch updated.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
Github user jiajunwang closed the pull request at:
https://github.com/apache/helix/pull/89
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is
GitHub user jiajunwang opened a pull request:
https://github.com/apache/helix/pull/100
[HELIX-655] Helix per-participant concurrent task throttling
Add per participant concurrent task throttling.
Add a participant configuration item "MAX_CONCURRENT_TASK" for
GitHub user jiajunwang opened a pull request:
https://github.com/apache/helix/pull/101
Add task execution throttling manual into TaskFramework document.
Add task execution throttling manual into TaskFramework document.
Also update version to enable "mvn site".
You
GitHub user jiajunwang opened a pull request:
https://github.com/apache/helix/pull/102
Fix test TestSwapInstance.
Ensure resources have normal state before disabling an instance in the test.
This can avoid possible unpredictable resource partition assignment if the
node is
GitHub user jiajunwang opened a pull request:
https://github.com/apache/helix/pull/103
Fix disconnected zkConnection issue.
One issue is found that when zkConnection is using an invalid zookeeper
object (null), the callers will get NPE error.
Affected Helix components are
GitHub user jiajunwang opened a pull request:
https://github.com/apache/helix/pull/104
Record workflow scheduling history in recurrent workflows.
Add records of scheduling history.
When deleting a recurrent workflow, also remove all scheduled workflows
that are finished
GitHub user jiajunwang opened a pull request:
https://github.com/apache/helix/pull/131
Bump up ZOOKEEPER version to 3.4.11.
There is a zk connection related bug (ZOOKEEPER-2775) fixed in 3.4.11. Bump
up version to get the fix.
You can merge this pull request into a Git repository
GitHub user jiajunwang opened a pull request:
https://github.com/apache/helix/pull/137
[HELIX-674] Introducing constraints based rebalancing mechanism.
Constraint can be customized by application to restrict how rebalancing is
processed.
The change also includes examples to
Github user jiajunwang commented on the issue:
https://github.com/apache/helix/pull/137
@guoyuepeng Thanks for the comment. Fixed.
Also updated all TODOs, and refactored the demo features with test cases.
---
GitHub user jiajunwang opened a pull request:
https://github.com/apache/helix/pull/142
[HELIX-675] Refactor controller start/cleanup logic to ensure monitorâ¦
⦠register/reset is handled in any event orders.
Due to different possible event process order, controller init
GitHub user jiajunwang opened a pull request:
https://github.com/apache/helix/pull/143
[HELIX-676] Fix the issue that the controller keep updating idealstates
when there is no real diff.
The causes of the problem are that:
1.A previous issue introduced into
GitHub user jiajunwang opened a pull request:
https://github.com/apache/helix/pull/144
[HELIX-677] Change error log to info/warning when replica count cannoâ¦
â¦t be read in ResourceMonitor.
Some string such as ALL_LIVEINSTANCES in the replica field cannot be
processed
GitHub user jiajunwang opened a pull request:
https://github.com/apache/helix/pull/145
[HELIX-674] Introducing constraints based rebalancing mechanism.
Constraint can be customized by application to restrict how rebalancing is
processed.
The change also includes examples to
Github user jiajunwang closed the pull request at:
https://github.com/apache/helix/pull/137
---
Github user jiajunwang commented on the issue:
https://github.com/apache/helix/pull/137
Changed to another branch instead of using master.
https://github.com/apache/helix/pull/145
---
GitHub user jiajunwang opened a pull request:
https://github.com/apache/helix/pull/147
[Helix 678] Clear controller event queue when it is shutdown or no longer
the leader
Clear controller event queue when it is shutdown or no longer the leader
You can merge this pull request
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/145#discussion_r174323138
--- Diff:
helix-core/src/main/java/org/apache/helix/api/rebalancer/constraint/dataprovider/CapacityProvider.java
---
@@ -0,0 +1,38 @@
+package
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/145#discussion_r174323213
--- Diff:
helix-core/src/main/java/org/apache/helix/api/rebalancer/constraint/dataprovider/PartitionQuotaProvider.java
---
@@ -0,0 +1,33 @@
+package
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/145#discussion_r174323126
--- Diff:
helix-core/src/main/java/org/apache/helix/api/rebalancer/constraint/dataprovider/CapacityProvider.java
---
@@ -0,0 +1,38 @@
+package
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/145#discussion_r174323268
--- Diff:
helix-core/src/main/java/org/apache/helix/controller/rebalancer/constraint/PartitionWeightAwareEvennessConstraint.java
---
@@ -0,0 +1,94
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/145#discussion_r174324068
--- Diff:
helix-core/src/main/java/org/apache/helix/api/rebalancer/constraint/dataprovider/PartitionQuotaProvider.java
---
@@ -0,0 +1,33 @@
+package
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/145#discussion_r174324850
--- Diff:
helix-core/src/main/java/org/apache/helix/util/WeightAwareRebalanceUtil.java ---
@@ -0,0 +1,141 @@
+package org.apache.helix.util
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/145#discussion_r174326382
--- Diff:
helix-core/src/main/java/org/apache/helix/util/WeightAwareRebalanceUtil.java ---
@@ -0,0 +1,141 @@
+package org.apache.helix.util
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/145#discussion_r174326343
--- Diff:
helix-core/src/main/java/org/apache/helix/util/WeightAwareRebalanceUtil.java ---
@@ -0,0 +1,141 @@
+package org.apache.helix.util
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/145#discussion_r174326641
--- Diff:
helix-core/src/main/java/org/apache/helix/util/WeightAwareRebalanceUtil.java ---
@@ -0,0 +1,141 @@
+package org.apache.helix.util
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/145#discussion_r174576961
--- Diff:
helix-core/src/main/java/org/apache/helix/util/WeightAwareRebalanceUtil.java ---
@@ -0,0 +1,141 @@
+package org.apache.helix.util
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/145#discussion_r174576953
--- Diff:
helix-core/src/main/java/org/apache/helix/util/WeightAwareRebalanceUtil.java ---
@@ -0,0 +1,141 @@
+package org.apache.helix.util
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/145#discussion_r174576991
--- Diff:
helix-core/src/main/java/org/apache/helix/api/rebalancer/constraint/dataprovider/CapacityProvider.java
---
@@ -0,0 +1,38 @@
+package
Github user jiajunwang commented on the issue:
https://github.com/apache/helix/pull/145
PR code updated.
Added examples and a lot of comments for clarifying the usage and API
define.
---
GitHub user jiajunwang opened a pull request:
https://github.com/apache/helix/pull/151
Tutorial update for task framework new features in 0.8
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jiajunwang/helix tutorial
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/145#discussion_r175879053
--- Diff:
helix-core/src/main/java/org/apache/helix/api/rebalancer/constraint/AbstractRebalanceSoftConstraint.java
---
@@ -0,0 +1,56 @@
+package
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/145#discussion_r175879486
--- Diff:
helix-core/src/main/java/org/apache/helix/controller/rebalancer/constraint/TotalCapacityConstraint.java
---
@@ -0,0 +1,79 @@
+package
Github user jiajunwang closed the pull request at:
https://github.com/apache/helix/pull/151
---
GitHub user jiajunwang opened a pull request:
https://github.com/apache/helix/pull/199
Adding monitoring metrics document.
Adding monitoring metrics document.
Link task framework related sections to the tutorial.
You can merge this pull request into a Git repository by running
Github user jiajunwang commented on the issue:
https://github.com/apache/helix/pull/199
Snapshorts
![index](https://user-images.githubusercontent.com/1803880/39263645-ab6e7bd4-4877-11e8-9e02-23aca0e42986.png)
[Apache Helix - Helix Monitoring
Metrics.pdf](https
GitHub user jiajunwang opened a pull request:
https://github.com/apache/helix/pull/207
[HELIX-707] Fix topstate handoff metrics.
We've confirmed a bug in the logic that calculates topstate handoff
duration.
With this issue, if the previous master instance is offline, an
GitHub user jiajunwang opened a pull request:
https://github.com/apache/helix/pull/212
[HELIX-] Separate CardDealingAdjustmentAlgorithm for constraint based
rebalancâ¦
â¦e strategy.
For keeping CRUSHed stable, we decide to split the logic changes made for
constraint
GitHub user jiajunwang opened a pull request:
https://github.com/apache/helix/pull/233
[HELIX-726][HELIX-727] Helix monitor metrics improvement
Contains 2 changes that improve Helix monitoring.
You can merge this pull request into a Git repository by running:
$ git pull https
GitHub user jiajunwang opened a pull request:
https://github.com/apache/helix/pull/237
ZkClient related fix and DelayedAutoRebalancer fix.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jiajunwang/helix master
Alternatively
GitHub user jiajunwang opened a pull request:
https://github.com/apache/helix/pull/242
Adding more metrics into ClusterStatus about aggregated partition info
Also, include a ZK client change that fixes several potential issues we
noticed.
You can merge this pull request into a Git
GitHub user jiajunwang opened a pull request:
https://github.com/apache/helix/pull/253
Sync the local change about RoutingTableProvider and related metrics change.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/jiajunwang/helix
GitHub user jiajunwang opened a pull request:
https://github.com/apache/helix/pull/279
Check in the intermediate state calculate stage for best possible state.
Resource rebalance pipeline should continuously processing resource even
some resources cannot be calculated.
This is
GitHub user jiajunwang opened a pull request:
https://github.com/apache/helix/pull/297
ZkClient related improvments
We identify 2 potential issues that may cause a retrying ZK operation
failed unexpectedly. These commits fix the problem.
You can merge this pull request into a Git
Github user jiajunwang commented on the issue:
https://github.com/apache/helix/pull/297
I split the ZkConnection commit to separate logic change and file movement.
Just to ensure the history of change is clear.
---
Github user jiajunwang commented on a diff in the pull request:
https://github.com/apache/helix/pull/297#discussion_r251134740
--- Diff:
helix-core/src/main/java/org/apache/helix/manager/zk/zookeeper/ZkConnection.java
---
@@ -84,6 +83,25 @@ public void close() throws
Github user jiajunwang commented on the issue:
https://github.com/apache/helix/pull/300
I don't think we can simply make this thread daemon. Actually, it only
makes sense when all the handlers are finalized. If one or more handlers are
still active, the re-subscribe functi
64 matches
Mail list logo