This is an automated email from the ASF dual-hosted git repository.
jianglongtao pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/shardingsphere.git
The following commit(s) were added to refs/heads/master by this push:
new 829edd0 add Weight replica load-balance algorithm (#13943)
829edd0 is described below
commit 829edd0a4839e2aebbd18d2ebcb9dbd6b5f5d0a8
Author: cherubimLV <[email protected]>
AuthorDate: Thu Dec 9 15:04:40 2021 +0800
add Weight replica load-balance algorithm (#13943)
* add Weight replica load-balance algorithm
* Adjust the code style
* Solve license missing issues
* Solve license missing issues
* definition cumulative accuracy loss threshold
* definition cumulative accuracy loss threshold
* method of extraction
* delete field document
* Adjust the use of double
Adjust the use of lombok
* Update readwrite-splitting.en.md
add WeightReplicaLoadBalanceAlgorithm Documentation
* Update load-balance.en.md
Add weight algorithm documentation
* Update readwrite-splitting.cn.md
添加权重算法文档
* Update load-balance.cn.md
添加权重访问算法文档
* Add parameter range description
---
.../content/dev-manual/readwrite-splitting.cn.md | 1 +
.../content/dev-manual/readwrite-splitting.en.md | 1 +
.../builtin-algorithm/load-balance.cn.md | 12 ++
.../builtin-algorithm/load-balance.en.md | 13 +++
.../WeightReplicaLoadBalanceAlgorithm.java | 121 +++++++++++++++++++++
...dwritesplitting.spi.ReplicaLoadBalanceAlgorithm | 1 +
.../WeightReplicaLoadBalanceAlgorithmTest.java | 47 ++++++++
7 files changed, 196 insertions(+)
diff --git a/docs/document/content/dev-manual/readwrite-splitting.cn.md
b/docs/document/content/dev-manual/readwrite-splitting.cn.md
index 84f62a2..95aa3cf 100644
--- a/docs/document/content/dev-manual/readwrite-splitting.cn.md
+++ b/docs/document/content/dev-manual/readwrite-splitting.cn.md
@@ -15,3 +15,4 @@ chapter = true
| ----------------------------------------- | ----------------------- |
| RoundRobinReplicaLoadBalanceAlgorithm | 基于轮询的读库负载均衡算法 |
| RandomReplicaLoadBalanceAlgorithm | 基于随机的读库负载均衡算法 |
+| WeightReplicaLoadBalanceAlgorithm | 基于权重的读库负载均衡算法 |
diff --git a/docs/document/content/dev-manual/readwrite-splitting.en.md
b/docs/document/content/dev-manual/readwrite-splitting.en.md
index 508cf0d..a825405 100644
--- a/docs/document/content/dev-manual/readwrite-splitting.en.md
+++ b/docs/document/content/dev-manual/readwrite-splitting.en.md
@@ -15,3 +15,4 @@ chapter = true
| ------------------------------------- |
------------------------------------------------------- |
| RoundRobinReplicaLoadBalanceAlgorithm | Round robin load balance algorithm
of replica databases |
| RandomReplicaLoadBalanceAlgorithm | Random load balance algorithm of
replica databases |
+| WeightReplicaLoadBalanceAlgorithm | Weight load balance algorithm of
replica databases |
diff --git
a/docs/document/content/user-manual/shardingsphere-jdbc/builtin-algorithm/load-balance.cn.md
b/docs/document/content/user-manual/shardingsphere-jdbc/builtin-algorithm/load-balance.cn.md
index 6be0938..a3b2366 100644
---
a/docs/document/content/user-manual/shardingsphere-jdbc/builtin-algorithm/load-balance.cn.md
+++
b/docs/document/content/user-manual/shardingsphere-jdbc/builtin-algorithm/load-balance.cn.md
@@ -14,3 +14,15 @@ weight = 4
类型：RANDOM
可配置属性：无
+
+## 权重访问算法
+
+类型：WEIGHT
+
+可配置属性：
+
+> 使用中的读库都必须配置权重
+
+| *属性名称* | *数据类型* | *说明*
|
+| ------------------------------ | ---------- |
---------------------------------------------- |
+| \- <read-data_source-name> (+) | double |
属性名字使用读库名字，参数填写读库对应的权重值。权重参数范围最小值>0，合计<=Double.MAX_VALUE。 |
diff --git
a/docs/document/content/user-manual/shardingsphere-jdbc/builtin-algorithm/load-balance.en.md
b/docs/document/content/user-manual/shardingsphere-jdbc/builtin-algorithm/load-balance.en.md
index a75af6e..2c49203 100644
---
a/docs/document/content/user-manual/shardingsphere-jdbc/builtin-algorithm/load-balance.en.md
+++
b/docs/document/content/user-manual/shardingsphere-jdbc/builtin-algorithm/load-balance.en.md
@@ -14,3 +14,16 @@ Attributes: None
Type: RANDOM
Attributes: None
+
+## Weight Algorithm
+
+Type: WEIGHT
+
+Attributes:
+
+> All read data in use must be configured with weights
+
+| *Name* | *DataType* | *Description*
|
+| ---------------------------------- | ---------- |
---------------------------------------------- |
+| \- <read-data_source-name> (+) | double | The attribute name uses the
read database name, and the parameter fills in the weight value corresponding
to the read database.The minimum value of the weight parameter range>0,the
total <=Double.MAX_VALUE. |
+
diff --git
a/shardingsphere-features/shardingsphere-readwrite-splitting/shardingsphere-readwrite-splitting-core/src/main/java/org/apache/shardingsphere/readwritesplitting/algorithm/WeightReplicaLoadBalanceAlgorithm.java
b/shardingsphere-features/shardingsphere-readwrite-splitting/shardingsphere-readwrite-splitting-core/src/main/java/org/apache/shardingsphere/readwritesplitting/algorithm/WeightReplicaLoadBalanceAlgorithm.java
new file mode 100644
index 0000000..8465e46
--- /dev/null
+++
b/shardingsphere-features/shardingsphere-readwrite-splitting/shardingsphere-readwrite-splitting-core/src/main/java/org/apache/shardingsphere/readwritesplitting/algorithm/WeightReplicaLoadBalanceAlgorithm.java
@@ -0,0 +1,121 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.readwritesplitting.algorithm;
+
+import lombok.Getter;
+import lombok.Setter;
+import
org.apache.shardingsphere.readwritesplitting.spi.ReplicaLoadBalanceAlgorithm;
+
+import java.util.Arrays;
+import java.util.List;
+import java.util.Properties;
+import java.util.concurrent.ConcurrentHashMap;
+import java.util.concurrent.ThreadLocalRandom;
+
+/**
+ * Weight replica load-balance algorithm.
+ */
+@Getter
+@Setter
+public final class WeightReplicaLoadBalanceAlgorithm implements
ReplicaLoadBalanceAlgorithm {
+
+ private static final double ACCURACY_THRESHOLD = 0.0001;
+
+ private static final ConcurrentHashMap<String, double[]> WEIGHT_MAP = new
ConcurrentHashMap<>();
+
+ private Properties props = new Properties();
+
+ @Override
+ public String getType() {
+ return "WEIGHT";
+ }
+
+ @Override
+ public String getDataSource(final String name, final String
writeDataSourceName, final List<String> readDataSourceNames) {
+ double[] weight = WEIGHT_MAP.containsKey(name) ? WEIGHT_MAP.get(name)
: initWeight(readDataSourceNames);
+ WEIGHT_MAP.putIfAbsent(name, weight);
+ return getDataSourceName(readDataSourceNames, weight);
+ }
+
+ private String getDataSourceName(final List<String> readDataSourceNames,
final double[] weight) {
+ double randomWeight = ThreadLocalRandom.current().nextDouble(0, 1);
+ int index = Arrays.binarySearch(weight, randomWeight);
+ if (index < 0) {
+ index = -index - 1;
+ return index < weight.length && randomWeight < weight[index] ?
readDataSourceNames.get(index) :
readDataSourceNames.get(readDataSourceNames.size() - 1);
+ } else {
+ return readDataSourceNames.get(index);
+ }
+ }
+
+ private double[] initWeight(final List<String> readDataSourceNames) {
+ double[] weights = getWeights(readDataSourceNames);
+ if (weights.length != 0 && Math.abs(weights[weights.length - 1] -
1.0D) >= ACCURACY_THRESHOLD) {
+ throw new IllegalStateException("The cumulative weight is
calculated incorrectly, and the sum of the probabilities is not equal to 1.");
+ }
+ return weights;
+ }
+
+ private double[] getWeights(final List<String> readDataSourceNames) {
+ double[] exactWeights = new double[readDataSourceNames.size()];
+ int index = 0;
+ double sum = 0D;
+ for (String readDataSourceName : readDataSourceNames) {
+ double weight = getWeightValue(readDataSourceName);
+ exactWeights[index++] = weight;
+ sum += weight;
+ }
+ for (int i = 0; i < index; i++) {
+ if (exactWeights[i] <= 0) {
+ continue;
+ }
+ exactWeights[i] = exactWeights[i] / sum;
+ }
+ return calcWeight(exactWeights);
+ }
+
+ private double[] calcWeight(final double[] exactWeights) {
+ double[] weights = new double[exactWeights.length];
+ double randomRange = 0D;
+ for (int i = 0; i < weights.length; i++) {
+ weights[i] = randomRange + exactWeights[i];
+ randomRange += exactWeights[i];
+ }
+ return weights;
+ }
+
+ private double getWeightValue(final String readDataSourceName) {
+ Object weightObject = props.get(readDataSourceName);
+ if (weightObject == null) {
+ throw new IllegalStateException("Read database access weight is
not configured：" + readDataSourceName);
+ }
+ double weight;
+ try {
+ weight = Double.parseDouble(weightObject.toString());
+ } catch (NumberFormatException e) {
+ throw new NumberFormatException("Read database weight
configuration error, configuration parameters:" + weightObject.toString());
+ }
+ if (Double.isInfinite(weight)) {
+ weight = 10000.0D;
+ }
+ if (Double.isNaN(weight)) {
+ weight = 1.0D;
+ }
+ return weight;
+ }
+}
diff --git
a/shardingsphere-features/shardingsphere-readwrite-splitting/shardingsphere-readwrite-splitting-core/src/main/resources/META-INF/services/org.apache.shardingsphere.readwritesplitting.spi.ReplicaLoadBalanceAlgorithm
b/shardingsphere-features/shardingsphere-readwrite-splitting/shardingsphere-readwrite-splitting-core/src/main/resources/META-INF/services/org.apache.shardingsphere.readwritesplitting.spi.ReplicaLoadBalanceAlgorithm
index 432b695..7f2de5a 100644
---
a/shardingsphere-features/shardingsphere-readwrite-splitting/shardingsphere-readwrite-splitting-core/src/main/resources/META-INF/services/org.apache.shardingsphere.readwritesplitting.spi.ReplicaLoadBalanceAlgorithm
+++
b/shardingsphere-features/shardingsphere-readwrite-splitting/shardingsphere-readwrite-splitting-core/src/main/resources/META-INF/services/org.apache.shardingsphere.readwritesplitting.spi.ReplicaLoadBalanceAlgorithm
@@ -17,3 +17,4 @@
org.apache.shardingsphere.readwritesplitting.algorithm.RoundRobinReplicaLoadBalanceAlgorithm
org.apache.shardingsphere.readwritesplitting.algorithm.RandomReplicaLoadBalanceAlgorithm
+org.apache.shardingsphere.readwritesplitting.algorithm.WeightReplicaLoadBalanceAlgorithm
diff --git
a/shardingsphere-features/shardingsphere-readwrite-splitting/shardingsphere-readwrite-splitting-core/src/test/java/org/apache/shardingsphere/readwritesplitting/algorithm/WeightReplicaLoadBalanceAlgorithmTest.java
b/shardingsphere-features/shardingsphere-readwrite-splitting/shardingsphere-readwrite-splitting-core/src/test/java/org/apache/shardingsphere/readwritesplitting/algorithm/WeightReplicaLoadBalanceAlgorithmTest.java
new file mode 100644
index 0000000..fc64db7
--- /dev/null
+++
b/shardingsphere-features/shardingsphere-readwrite-splitting/shardingsphere-readwrite-splitting-core/src/test/java/org/apache/shardingsphere/readwritesplitting/algorithm/WeightReplicaLoadBalanceAlgorithmTest.java
@@ -0,0 +1,47 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.shardingsphere.readwritesplitting.algorithm;
+
+import org.junit.Test;
+
+import java.util.Arrays;
+import java.util.List;
+import java.util.Properties;
+
+import static org.hamcrest.CoreMatchers.notNullValue;
+import static org.junit.Assert.assertThat;
+
+public final class WeightReplicaLoadBalanceAlgorithmTest {
+
+ private final WeightReplicaLoadBalanceAlgorithm
weightReplicaLoadBalanceAlgorithm = new WeightReplicaLoadBalanceAlgorithm();
+
+ @Test
+ public void assertGetDataSource() {
+ final Properties properties = new Properties();
+ properties.setProperty("test_read_ds_1", "5");
+ properties.setProperty("test_read_ds_2", "5");
+ weightReplicaLoadBalanceAlgorithm.setProps(properties);
+ String writeDataSourceName = "test_write_ds";
+ String readDataSourceName1 = "test_read_ds_1";
+ String readDataSourceName2 = "test_read_ds_2";
+ List<String> readDataSourceNames = Arrays.asList(readDataSourceName1,
readDataSourceName2);
+ assertThat(weightReplicaLoadBalanceAlgorithm.getDataSource("ds",
writeDataSourceName, readDataSourceNames), notNullValue());
+ assertThat(weightReplicaLoadBalanceAlgorithm.getDataSource("ds",
writeDataSourceName, readDataSourceNames), notNullValue());
+ assertThat(weightReplicaLoadBalanceAlgorithm.getDataSource("ds",
writeDataSourceName, readDataSourceNames), notNullValue());
+ }
+}
