This is an automated email from the ASF dual-hosted git repository. shaofengshi pushed a commit to branch document in repository https://gitbox.apache.org/repos/asf/kylin.git
commit 94d03cd3e935eab10c8237eeb2d160e516a7a531 Author: shaofengshi <[email protected]> AuthorDate: Thu Jun 21 08:34:49 2018 +0800 Add superset and redash, move jdbc datasource to tutorial --- website/_data/docs23.yml | 5 +- website/_docs23/index.md | 3 ++ .../setup_jdbc_datasource.md} | 56 +++++++++++++++++++--- 3 files changed, 53 insertions(+), 11 deletions(-) diff --git a/website/_data/docs23.yml b/website/_data/docs23.yml index bb8a44b..40012c0 100644 --- a/website/_data/docs23.yml +++ b/website/_data/docs23.yml @@ -49,6 +49,7 @@ - tutorial/setup_systemcube - tutorial/use_cube_planner - tutorial/use_dashboard + - tutorial/setup_jdbc_datasource - title: Integration docs: @@ -78,7 +79,3 @@ - howto/howto_update_coprocessor - howto/howto_install_ranger_kylin_plugin - howto/howto_enable_zookeeper_acl - - howto/howto_setup_systemcube - - howto/howto_use_cube_planner - - howto/howto_use_dashboard - - howto/howto_setup_jdbc_datasource diff --git a/website/_docs23/index.md b/website/_docs23/index.md index 3b60dd7..390990a 100644 --- a/website/_docs23/index.md +++ b/website/_docs23/index.md @@ -42,6 +42,7 @@ Tutorial 10. [Setup System Cube](tutorial/setup_systemcube.html) 11. [Optimize with Cube Planner](tutorial/use_cube_planner.html) 12. [Use System Dashboard](tutorial/use_dashboard.html) +13. [Setup JDBC Data Source](tutorial/setup_jdbc_datasource.html) Connectivity and APIs @@ -59,6 +60,8 @@ Connectivity and APIs 11. [Connect from Apache Spark](tutorial/spark.html) 12. [Connect from Hue](tutorial/hue.html) 13. [Connect from Qlik Sense](tutorial/Qlik.html) +14. [Connect from Apache Superset](/blog/2018/01/01/kylin-and-superset/) +15. [Connect from Redash](/blog/2018/05/08/redash-kylin-plugin-strikingly/) Operations diff --git a/website/_docs23/howto/howto_setup_jdbc_datasource.md b/website/_docs23/tutorial/setup_jdbc_datasource.md similarity index 55% rename from website/_docs23/howto/howto_setup_jdbc_datasource.md rename to website/_docs23/tutorial/setup_jdbc_datasource.md index 7653360..881d76f 100644 --- a/website/_docs23/howto/howto_setup_jdbc_datasource.md +++ b/website/_docs23/tutorial/setup_jdbc_datasource.md @@ -2,7 +2,7 @@ layout: docs23 title: Setup JDBC Data Source categories: howto -permalink: /docs23/howto/howto_setup_jdbc_datasource.html +permalink: /docs23/tutorial/setup_jdbc_datasource.html --- > Available since Apache Kylin v2.3.x @@ -19,24 +19,66 @@ Kylin uses Apache Sqoop to load data from relational databases to HDFS. Download 2. Prepare JDBC driver - Copy Kylin JDBC driver from `$KYLIN_HOME/lib` to `$SQOOP_HOME/lib`. +You need to download the JDBC Driver of your database to the Kylin server. The JDBC driver jar should be added to `$KYLIN_HOME/ext` and `$SQOOP_HOME/lib` folder. 3. Configure Kylin -In `$KYLIN_HOME/conf/kylin.properties`, add the following configurations (assumming your MySQL is in the same host): +In `$KYLIN_HOME/conf/kylin.properties`, add the following configurations. + +MySQL sample: ``` kylin.source.default=8 -kylin.source.jdbc.connection-url=jdbc:mysql://localhost:3306/employees +kylin.source.jdbc.connection-url=jdbc:mysql://hostname:3306/employees kylin.source.jdbc.driver=com.mysql.jdbc.Driver kylin.source.jdbc.dialect=mysql -kylin.source.jdbc.user=your_mysql_username -kylin.source.jdbc.pass=your_mysql_password +kylin.source.jdbc.user=your_username +kylin.source.jdbc.pass=your_password kylin.source.jdbc.sqoop-home=/usr/hdp/current/sqoop-client/bin kylin.source.jdbc.filed-delimiter=| ``` -Please note, when configure these parameters in `conf/kylin.properties`, all your projects are using the JDBC as data source. If you need access different types of data source, you need configure these parameters at project level, this is the recommended way. +SQL Server sample: + +``` +kylin.source.default=8 +kylin.source.jdbc.connection-url=jdbc:sqlserver://hostname:1433;database=sample +kylin.source.jdbc.driver=com.microsoft.sqlserver.jdbc.SQLServerDriver +kylin.source.jdbc.dialect=mssql +kylin.source.jdbc.user=your_username +kylin.source.jdbc.pass=your_password +kylin.source.jdbc.sqoop-home=/usr/hdp/current/sqoop-client/bin +kylin.source.jdbc.filed-delimiter=| +``` + +Amazon Redshift sample: + +``` +kylin.source.default=8 +kylin.source.jdbc.connection-url=jdbc:redshift://hostname:5439/sample +kylin.source.jdbc.driver=com.amazon.redshift.jdbc.Driver +kylin.source.jdbc.dialect=default +kylin.source.jdbc.user=user +kylin.source.jdbc.pass=pass +kylin.source.jdbc.sqoop-home=/usr/hdp/current/sqoop-client/bin +kylin.source.default=8 +kylin.source.jdbc.filed-delimiter=| +``` + +There is another parameter specifing how many splits should be divided. Sqoop would run a mapper for each split. + +``` +kylin.source.jdbc.sqoop-mapper-num=4 +``` + +To make each mapper gets even input, the splitting column is selected by the following rules: + * ShardBy column, if exists; + * Partition date column, if exists; + * High cardinality column, if exists; + * Numeric column, if exists; + * A column at first glance. + +Please note, when configure these parameters in `conf/kylin.properties`, all your projects are using the JDBC as data source. If you need access different types of data source, you need configure these parameters at project level, this is the recommended way (since Kylin v2.4.0). ## Load tables from JDBC data source
