[GitHub] wu-sheng commented on a change in pull request #1347: support shardingjdbc database storage feature
wu-sheng commented on a change in pull request #1347: support shardingjdbc
database storage feature
URL:
https://github.com/apache/incubator-skywalking/pull/1347#discussion_r195410323
##
File path:
apm-collector/apm-collector-component/client-component/src/main/java/org/apache/skywalking/apm/collector/client/shardingjdbc/ShardingjdbcClient.java
##
@@ -0,0 +1,160 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+
+package org.apache.skywalking.apm.collector.client.shardingjdbc;
+
+import java.sql.Connection;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Properties;
+
+import javax.sql.DataSource;
+
+import org.apache.commons.dbcp.BasicDataSource;
+import org.apache.skywalking.apm.collector.client.Client;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import io.shardingjdbc.core.api.ShardingDataSourceFactory;
+import io.shardingjdbc.core.api.config.ShardingRuleConfiguration;
+
+/**
+ * @author linjiaqi
+ */
+public class ShardingjdbcClient implements Client {
+
+private final Logger logger =
LoggerFactory.getLogger(ShardingjdbcClient.class);
Review comment:
@chidaodezhongsheng You still miss this one.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] wu-sheng commented on a change in pull request #1347: support shardingjdbc database storage feature
wu-sheng commented on a change in pull request #1347: support shardingjdbc
database storage feature
URL:
https://github.com/apache/incubator-skywalking/pull/1347#discussion_r195410337
##
File path:
apm-collector/apm-collector-storage/collector-storage-shardingjdbc-provider/src/main/java/org/apache/skywalking/apm/collector/storage/shardingjdbc/base/define/ShardingjdbcStorageInstaller.java
##
@@ -0,0 +1,99 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.apm.collector.storage.shardingjdbc.base.define;
+
+import java.util.List;
+
+import org.apache.skywalking.apm.collector.client.Client;
+import
org.apache.skywalking.apm.collector.client.shardingjdbc.ShardingjdbcClient;
+import
org.apache.skywalking.apm.collector.client.shardingjdbc.ShardingjdbcClientException;
+import org.apache.skywalking.apm.collector.core.data.TableDefine;
+import org.apache.skywalking.apm.collector.core.util.Const;
+import org.apache.skywalking.apm.collector.storage.StorageException;
+import org.apache.skywalking.apm.collector.storage.StorageInstallException;
+import org.apache.skywalking.apm.collector.storage.StorageInstaller;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+/**
+ * @author linjiaqi
+ */
+public class ShardingjdbcStorageInstaller extends StorageInstaller {
+
+private final Logger logger =
LoggerFactory.getLogger(ShardingjdbcStorageInstaller.class);
+
+public ShardingjdbcStorageInstaller(boolean isHighPerformanceMode) {
+super(isHighPerformanceMode);
+}
+
+@Override protected void defineFilter(List tableDefines) {
+int size = tableDefines.size();
+for (int i = size - 1; i >= 0; i--) {
+if (!(tableDefines.get(i) instanceof ShardingjdbcTableDefine)) {
+tableDefines.remove(i);
+}
+}
+}
+
+@Override protected boolean isExists(Client client, TableDefine
tableDefine) throws StorageException {
+logger.info("check if table {} exist ", tableDefine.getName());
+return false;
+}
+
+@Override protected void columnCheck(Client client, TableDefine
tableDefine) throws StorageException {
Review comment:
@peng-yongsheng Besides 3.x(still preview), shardingJDBC doesn't support
read meta data of DB
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] wu-sheng commented on a change in pull request #1347: support shardingjdbc database storage feature
wu-sheng commented on a change in pull request #1347: support shardingjdbc
database storage feature
URL:
https://github.com/apache/incubator-skywalking/pull/1347#discussion_r195275008
##
File path:
apm-collector/apm-collector-component/client-component/src/main/java/org/apache/skywalking/apm/collector/client/shardingjdbc/ShardingjdbcClient.java
##
@@ -0,0 +1,160 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+
+package org.apache.skywalking.apm.collector.client.shardingjdbc;
+
+import java.sql.Connection;
+import java.sql.PreparedStatement;
+import java.sql.ResultSet;
+import java.sql.SQLException;
+import java.sql.Statement;
+import java.util.HashMap;
+import java.util.Map;
+import java.util.Properties;
+
+import javax.sql.DataSource;
+
+import org.apache.commons.dbcp.BasicDataSource;
+import org.apache.skywalking.apm.collector.client.Client;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import io.shardingjdbc.core.api.ShardingDataSourceFactory;
+import io.shardingjdbc.core.api.config.ShardingRuleConfiguration;
+
+/**
+ * @author linjiaqi
+ */
+public class ShardingjdbcClient implements Client {
+
+private final Logger logger =
LoggerFactory.getLogger(ShardingjdbcClient.class);
Review comment:
+1 The un-static field of logger may trigger big performance issue.
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
[GitHub] wu-sheng commented on a change in pull request #1347: support shardingjdbc database storage feature
wu-sheng commented on a change in pull request #1347: support shardingjdbc database storage feature URL: https://github.com/apache/incubator-skywalking/pull/1347#discussion_r194666240 ## File path: docs/en/Use-ShardingJDBC-as-storage-implementor.md ## @@ -0,0 +1,18 @@ +# Supported Database Sharding Storage +Beside the default Elasticsearch storage, it also support the database sharding storage, it allow the users to store data in multiple databases. +Note: it only supported MYSQL database sharding, and due to the license restrictions, the users need to import MYSQL Driver manually. + +## Requirement +- Manually import MySQL Driver package mysql-connector-java-5.1.36.jar to collector libs directory. +- In config/application.yml, close the elasticsearch configuration and open the shardingjdbc configuration, multiple data source configurations should be separated by Half Comma. +``` + storage: +shardingjdbc: + driverClass: com.mysql.jdbc.Driver + url: jdbc:mysql://ip1:port1/skywalking,jdbc:mysql://ip2:port2/skywalking + userName: admin,admin Review comment: Add comments for this: ``` # Usernames, which match the sequence of Datasource URLs ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] wu-sheng commented on a change in pull request #1347: support shardingjdbc database storage feature
wu-sheng commented on a change in pull request #1347: support shardingjdbc database storage feature URL: https://github.com/apache/incubator-skywalking/pull/1347#discussion_r194667780 ## File path: docs/cn/Use-ShardingJDBC-as-storage-implementor-CN.md ## @@ -0,0 +1,18 @@ +# 支持数据库分片存储 +除了默认的Elasticsearch存储外，还支持数据库分片存储，允许用户将数据存储在多个数据库中。 +注意：目前仅支持MYSQL数据库的分片存储，且由于license限制，需要用户手动引入mysql驱动包。 + +## 配置要求 +- 手工导入MYSQL的驱动包mysql-connector-java-5.1.36.jar到collector libs目录下。 +- config/application.yml中，关闭elasticsearch配置，打开shardingjdbc配置，多个数据源配置用半角逗号隔开。 +``` + storage: +shardingjdbc: + driverClass: com.mysql.jdbc.Driver + url: jdbc:mysql://ip1:port1/skywalking,jdbc:mysql://ip2:port2/skywalking + userName: admin,admin + password: 123456,123456 +``` + +## 参阅 +- [shardingsphere官网](http://shardingsphere.io) Review comment: Sync all in English. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] wu-sheng commented on a change in pull request #1347: support shardingjdbc database storage feature
wu-sheng commented on a change in pull request #1347: support shardingjdbc database storage feature URL: https://github.com/apache/incubator-skywalking/pull/1347#discussion_r194666713 ## File path: docs/en/Use-ShardingJDBC-as-storage-implementor.md ## @@ -0,0 +1,18 @@ +# Supported Database Sharding Storage +Beside the default Elasticsearch storage, it also support the database sharding storage, it allow the users to store data in multiple databases. +Note: it only supported MYSQL database sharding, and due to the license restrictions, the users need to import MYSQL Driver manually. + +## Requirement +- Manually import MySQL Driver package mysql-connector-java-5.1.36.jar to collector libs directory. +- In config/application.yml, close the elasticsearch configuration and open the shardingjdbc configuration, multiple data source configurations should be separated by Half Comma. +``` + storage: +shardingjdbc: + driverClass: com.mysql.jdbc.Driver + url: jdbc:mysql://ip1:port1/skywalking,jdbc:mysql://ip2:port2/skywalking + userName: admin,admin + password: 123456,123456 +``` + +## See also +- [shardingsphere website](http://shardingsphere.io) Review comment: ShardingJDBC now named as ShardingSphere since its 3.x, see [here](http://shardingsphere.io) This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] wu-sheng commented on a change in pull request #1347: support shardingjdbc database storage feature
wu-sheng commented on a change in pull request #1347: support shardingjdbc database storage feature URL: https://github.com/apache/incubator-skywalking/pull/1347#discussion_r194666412 ## File path: docs/en/Use-ShardingJDBC-as-storage-implementor.md ## @@ -0,0 +1,18 @@ +# Supported Database Sharding Storage +Beside the default Elasticsearch storage, it also support the database sharding storage, it allow the users to store data in multiple databases. +Note: it only supported MYSQL database sharding, and due to the license restrictions, the users need to import MYSQL Driver manually. + +## Requirement +- Manually import MySQL Driver package mysql-connector-java-5.1.36.jar to collector libs directory. +- In config/application.yml, close the elasticsearch configuration and open the shardingjdbc configuration, multiple data source configurations should be separated by Half Comma. +``` + storage: +shardingjdbc: + driverClass: com.mysql.jdbc.Driver + url: jdbc:mysql://ip1:port1/skywalking,jdbc:mysql://ip2:port2/skywalking + userName: admin,admin + password: 123456,123456 Review comment: Add comments for this: ``` # Passwords, which match the sequence of Datasource URLs ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] wu-sheng commented on a change in pull request #1347: support shardingjdbc database storage feature
wu-sheng commented on a change in pull request #1347: support shardingjdbc database storage feature URL: https://github.com/apache/incubator-skywalking/pull/1347#discussion_r194665655 ## File path: docs/en/Use-ShardingJDBC-as-storage-implementor.md ## @@ -0,0 +1,18 @@ +# Supported Database Sharding Storage +Beside the default Elasticsearch storage, it also support the database sharding storage, it allow the users to store data in multiple databases. +Note: it only supported MYSQL database sharding, and due to the license restrictions, the users need to import MYSQL Driver manually. + +## Requirement +- Manually import MySQL Driver package mysql-connector-java-5.1.36.jar to collector libs directory. +- In config/application.yml, close the elasticsearch configuration and open the shardingjdbc configuration, multiple data source configurations should be separated by Half Comma. +``` + storage: +shardingjdbc: + driverClass: com.mysql.jdbc.Driver + url: jdbc:mysql://ip1:port1/skywalking,jdbc:mysql://ip2:port2/skywalking Review comment: Add comments for this: ``` # JDBC Datasource connections for ShardingJDBC, multiple should be separated by comma ``` This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] wu-sheng commented on a change in pull request #1347: support shardingjdbc database storage feature
wu-sheng commented on a change in pull request #1347: support shardingjdbc database storage feature URL: https://github.com/apache/incubator-skywalking/pull/1347#discussion_r194665328 ## File path: docs/en/Use-ShardingJDBC-as-storage-implementor.md ## @@ -0,0 +1,18 @@ +# Supported Database Sharding Storage +Beside the default Elasticsearch storage, it also support the database sharding storage, it allow the users to store data in multiple databases. +Note: it only supported MYSQL database sharding, and due to the license restrictions, the users need to import MYSQL Driver manually. + +## Requirement +- Manually import MySQL Driver package mysql-connector-java-5.1.36.jar to collector libs directory. +- In config/application.yml, close the elasticsearch configuration and open the shardingjdbc configuration, multiple data source configurations should be separated by Half Comma. Review comment: `In config/application.yml, close the ElasticSearch configuration and open the shardingjdbc configuration` -> `In config/application.yml, delete the ElasticSearch configuration and add the following shardingjdbc configuration` Remove ~~multiple data source configurations should be separated by Half Comma.~~ This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] wu-sheng commented on a change in pull request #1347: support shardingjdbc database storage feature
wu-sheng commented on a change in pull request #1347: support shardingjdbc database storage feature URL: https://github.com/apache/incubator-skywalking/pull/1347#discussion_r194664591 ## File path: docs/en/Use-ShardingJDBC-as-storage-implementor.md ## @@ -0,0 +1,18 @@ +# Supported Database Sharding Storage +Beside the default Elasticsearch storage, it also support the database sharding storage, it allow the users to store data in multiple databases. Review comment: Beside the default Elasticsearch storage, the user can use shardingJDBC and MySQL as storage implementor. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] wu-sheng commented on a change in pull request #1347: support shardingjdbc database storage feature
wu-sheng commented on a change in pull request #1347: support shardingjdbc database storage feature URL: https://github.com/apache/incubator-skywalking/pull/1347#discussion_r194655271 ## File path: apm-collector/apm-collector-component/client-component/pom.xml ## @@ -101,5 +101,15 @@ + +io.shardingjdbc +sharding-jdbc-core +2.0.3 + + +commons-dbcp +commons-dbcp +1.4 + Review comment: dependency and version should be defined in collector root pom.xml See here: https://github.com/apache/incubator-skywalking/blob/master/apm-collector/pom.xml#L97-L240 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
