[jira] [Comment Edited] (KYLIN-3044) Support SQL Server as data source
[ https://issues.apache.org/jira/browse/KYLIN-3044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16262514#comment-16262514 ] Kaige Liu edited comment on KYLIN-3044 at 11/22/17 1:32 PM: - Sqoop splits data to a couple of parts and import them parallel. I add a property kylin.source.jdbc.sqoop-mapper-num to specify how many splits should be divided. Sqoop would run a mapper for each split. To make each mapper gets even input, split column is chosen by following some rules: 1. Prefer ClusteredBy column 2. Prefer DistributedBy column 3. Prefer Partition date column 4. Prefer Higher cardinality column 5. Prefer numeric column 6. Pick a column at first glance Patch updated. was (Author: liukaige): Sqoop splits data to a couple of parts and import them parallel. I add a property kylin.source.jdbc.sqoop-mapper-num to specify how many splits should be divided. Sqoop would run a mapper for each split. To make each mapper gets even input, split column is chosen following some rules: 1. Prefer ClusteredBy column 2. Prefer DistributedBy column 3. Prefer Partition date column 4. Prefer Higher cardinality column 5. Prefer numeric column 6. Pick a column at first glance Patch updated. > Support SQL Server as data source > - > > Key: KYLIN-3044 > URL: https://issues.apache.org/jira/browse/KYLIN-3044 > Project: Kylin > Issue Type: Task >Reporter: Kaige Liu >Assignee: Kaige Liu > Attachments: KYLIN-3044-sqlserver-as-datasource.patch, > KYLIN-3044-sqlserver-as-datasource.patch > > > [KYLIN-1351|https://issues.apache.org/jira/browse/KYLIN-1351] has added > Vertica as data source. Base on the work of KYLIN-1351, I'd like to enable > SQL Server as data source of kylin. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (KYLIN-3044) Support SQL Server as data source
[ https://issues.apache.org/jira/browse/KYLIN-3044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16262000#comment-16262000 ] Kaige Liu edited comment on KYLIN-3044 at 11/22/17 6:12 AM: - To use SQL Server as KYLIN data source, following properties should be added to kylin.properties: kylin.source.jdbc.connection-url=jdbc:sqlserver://youdbhost:1433;database=sample kylin.source.jdbc.driver=com.microsoft.sqlserver.jdbc.SQLServerDriver kylin.source.jdbc.dialect=mssql kylin.source.jdbc.user=user kylin.source.jdbc.pass=pass kylin.source.jdbc.sqoop-home=/usr/hdp/current/sqoop-client/bin kylin.source.default=8 kylin.source.jdbc.filed-delimiter=| JDBC driver will not be shipped by KYLIN. Users should add proper driver by themselves. For release package, jdbc driver jar should be added to $KYLIN_HOME/ext For IDE, add jdbc driver jar to class path manually. Add sqoop needs the jdbc driver as well, users should also add jdbc driver to $SQOOP_HOME/lib [~liyang.g...@gmail.com], [~Shaofengshi] please help review my patch. Thanks. was (Author: liukaige): To use SQL Server as KYLIN data source, following properties should be added to kylin.properties: kylin.source.jdbc.connection-url=jdbc:sqlserver://youdbhost:1433;database=sample kylin.source.jdbc.driver=com.microsoft.sqlserver.jdbc.SQLServerDriver kylin.source.jdbc.dialect=mssql kylin.source.jdbc.user=user kylin.source.jdbc.pass=pass kylin.source.jdbc.sqoop-home=/usr/hdp/current/sqoop-client/bin kylin.source.default=8 kylin.source.jdbc.filed-delimiter=| JDBC driver will not be shipped by KYLIN. Users should add proper driver by themselves. For release package, jdbc driver jar should be added to $KYLIN_HOME/ext For IDE, add jdbc driver jar to class path manually. [~liyang.g...@gmail.com], [~Shaofengshi] please help review my patch. Thanks. > Support SQL Server as data source > - > > Key: KYLIN-3044 > URL: https://issues.apache.org/jira/browse/KYLIN-3044 > Project: Kylin > Issue Type: Task >Reporter: Kaige Liu >Assignee: Kaige Liu > Attachments: KYLIN-3304-sqlserver-as-datasource.patch > > > [KYLIN-1351|https://issues.apache.org/jira/browse/KYLIN-1351] has added > Vertica as data source. Base on the work of KYLIN-1351, I'd like to enable > SQL Server as data source of kylin. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (KYLIN-3044) Support SQL Server as data source
[ https://issues.apache.org/jira/browse/KYLIN-3044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16262000#comment-16262000 ] Kaige Liu edited comment on KYLIN-3044 at 11/22/17 5:51 AM: - To use SQL Server as KYLIN data source, following properties should be added to kylin.properties: kylin.source.jdbc.connection-url=jdbc:sqlserver://youdbhost:1433;database=sample kylin.source.jdbc.driver=com.microsoft.sqlserver.jdbc.SQLServerDriver kylin.source.jdbc.dialect=mssql kylin.source.jdbc.user=user kylin.source.jdbc.pass=pass kylin.source.jdbc.sqoop-home=/usr/hdp/current/sqoop-client/bin kylin.source.default=8 kylin.source.jdbc.filed-delimiter=| JDBC driver will not be shipped by KYLIN. Users should add proper driver by themselves. For release package, jdbc driver jar should be added to $KYLIN_HOME/ext For IDE, add jdbc driver jar to class path manually. [~liyang.g...@gmail.com], [~Shaofengshi] please help review my patch. Thanks. was (Author: liukaige): To use SQL Server as KYLIN data source, following properties should be added to kylin.properties: *kylin.source.jdbc.connection-url=jdbc:sqlserver://youdbhost:1433;database=sample kylin.source.jdbc.driver=com.microsoft.sqlserver.jdbc.SQLServerDriver kylin.source.jdbc.dialect=mssql kylin.source.jdbc.user=user kylin.source.jdbc.pass=pass kylin.source.jdbc.sqoop-home=/usr/hdp/current/sqoop-client/bin kylin.source.default=8 kylin.source.jdbc.filed-delimiter=|* JDBC driver will not be shipped by KYLIN. Users should add proper driver by themselves. For release package, jdbc driver jar should be added to $KYLIN_HOME/ext For IDE, add jdbc driver jar to class path manually. [~liyang.g...@gmail.com], [~Shaofengshi] please help review my patch. Thanks. > Support SQL Server as data source > - > > Key: KYLIN-3044 > URL: https://issues.apache.org/jira/browse/KYLIN-3044 > Project: Kylin > Issue Type: Task >Reporter: Kaige Liu >Assignee: Kaige Liu > Attachments: KYLIN-3304-sqlserver-as-datasource.patch > > > [KYLIN-1351|https://issues.apache.org/jira/browse/KYLIN-1351] has added > Vertica as data source. Base on the work of KYLIN-1351, I'd like to enable > SQL Server as data source of kylin. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (KYLIN-3044) Support SQL Server as data source
[ https://issues.apache.org/jira/browse/KYLIN-3044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16262000#comment-16262000 ] Kaige Liu edited comment on KYLIN-3044 at 11/22/17 5:51 AM: - To use SQL Server as KYLIN data source, following properties should be added to kylin.properties: *kylin.source.jdbc.connection-url=jdbc:sqlserver://youdbhost:1433;database=sample kylin.source.jdbc.driver=com.microsoft.sqlserver.jdbc.SQLServerDriver kylin.source.jdbc.dialect=mssql kylin.source.jdbc.user=user kylin.source.jdbc.pass=pass kylin.source.jdbc.sqoop-home=/usr/hdp/current/sqoop-client/bin kylin.source.default=8 kylin.source.jdbc.filed-delimiter=|* JDBC driver will not be shipped by KYLIN. Users should add proper driver by themselves. For release package, jdbc driver jar should be added to $KYLIN_HOME/ext For IDE, add jdbc driver jar to class path manually. [~liyang.g...@gmail.com], [~Shaofengshi] please help review my patch. Thanks. was (Author: liukaige): To use SQL Server as KYLIN data source, following properties should be added to kylin.properties: *kylin.source.jdbc.connection-url=jdbc:sqlserver://youdbhost:1433;database=sample kylin.source.jdbc.driver=com.microsoft.sqlserver.jdbc.SQLServerDriver kylin.source.jdbc.dialect=mssql kylin.source.jdbc.user=user kylin.source.jdbc.pass=pass kylin.source.jdbc.sqoop-home=/usr/hdp/current/sqoop-client/bin kylin.source.default=8 kylin.source.jdbc.filed-delimiter=|* JDBC driver will not be shipped by KYLIN. Users should add proper driver by themselves. For release package, jdbc driver jar should be added to $KYLIN_HOME/ext For IDE, add jdbc driver jar to class path manually. [~liyang.g...@gmail.com] [~Shaofengshi] please help review my patch. Thanks. > Support SQL Server as data source > - > > Key: KYLIN-3044 > URL: https://issues.apache.org/jira/browse/KYLIN-3044 > Project: Kylin > Issue Type: Task >Reporter: Kaige Liu >Assignee: Kaige Liu > Attachments: KYLIN-3304-sqlserver-as-datasource.patch > > > [KYLIN-1351|https://issues.apache.org/jira/browse/KYLIN-1351] has added > Vertica as data source. Base on the work of KYLIN-1351, I'd like to enable > SQL Server as data source of kylin. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
