[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16783830#comment-16783830 ] Daniel Dai commented on HIVE-1555: -- [~Tagar], yes, you can store password in a Hadoop credential store, and point hive.sql.dbcp.password.keystore there. See https://cwiki.apache.org/confluence/display/Hive/JdbcStorageHandler. > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hive > Issue Type: New Feature > Components: JDBC >Reporter: Bob Robertson >Assignee: Gunther Hagleitner >Priority: Major > Labels: TODOC2.2 > Fix For: 2.3.0 > > Attachments: HIVE-1555.7.patch, HIVE-1555.8.patch, HIVE-1555.9.patch, > JDBCStorageHandler Design Doc.pdf > > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16783654#comment-16783654 ] Ruslan Dautkhanov commented on HIVE-1555: - Is this possible to store password in a Hadoop credential store? Metadata is visible to all users.. Alternatively Hive should always return password field redacted for commands like `show create table`. > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hive > Issue Type: New Feature > Components: JDBC >Reporter: Bob Robertson >Assignee: Gunther Hagleitner >Priority: Major > Labels: TODOC2.2 > Fix For: 2.3.0 > > Attachments: HIVE-1555.7.patch, HIVE-1555.8.patch, HIVE-1555.9.patch, > JDBCStorageHandler Design Doc.pdf > > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15889718#comment-15889718 ] Lefty Leverenz commented on HIVE-1555: -- Should this be documented in the wiki? If so, please add a TODOC2.2 label. > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hive > Issue Type: New Feature > Components: JDBC >Reporter: Bob Robertson >Assignee: Gunther Hagleitner > Fix For: 2.2.0 > > Attachments: HIVE-1555.7.patch, HIVE-1555.8.patch, HIVE-1555.9.patch, > JDBCStorageHandler Design Doc.pdf > > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15887005#comment-15887005 ] Jason Dere commented on HIVE-1555: -- +1 > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hive > Issue Type: New Feature > Components: JDBC >Reporter: Bob Robertson >Assignee: Gunther Hagleitner > Attachments: HIVE-1555.7.patch, HIVE-1555.8.patch, HIVE-1555.9.patch, > JDBCStorageHandler Design Doc.pdf > > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15884282#comment-15884282 ] Hive QA commented on HIVE-1555: --- Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12854637/HIVE-1555.9.patch {color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 10284 tests executed *Failed tests:* {noformat} TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) (batchId=236) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=224) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3783/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3783/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3783/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 2 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12854637 - PreCommit-HIVE-Build > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hive > Issue Type: New Feature > Components: JDBC >Reporter: Bob Robertson >Assignee: Gunther Hagleitner > Attachments: HIVE-1555.7.patch, HIVE-1555.8.patch, HIVE-1555.9.patch, > JDBCStorageHandler Design Doc.pdf > > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15880424#comment-15880424 ] Hive QA commented on HIVE-1555: --- Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12854126/HIVE-1555.8.patch {color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 10279 tests executed *Failed tests:* {noformat} TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) (batchId=236) org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[jdbc_handler] (batchId=136) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=224) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23] (batchId=224) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3723/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3723/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3723/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 4 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12854126 - PreCommit-HIVE-Build > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hive > Issue Type: New Feature > Components: JDBC >Reporter: Bob Robertson >Assignee: Gunther Hagleitner > Attachments: HIVE-1555.3.patch, HIVE-1555.4.patch, HIVE-1555.5.patch, > HIVE-1555.6.patch, HIVE-1555.7.patch, HIVE-1555.8.patch, JDBCStorageHandler > Design Doc.pdf > > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15879733#comment-15879733 ] Hive QA commented on HIVE-1555: --- Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12854037/HIVE-1555.7.patch {color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 10274 tests executed *Failed tests:* {noformat} TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) (batchId=236) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[jdbc_handler] (batchId=52) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr] (batchId=140) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=224) org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver.org.apache.hadoop.hive.cli.TestSparkNegativeCliDriver (batchId=231) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3705/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3705/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3705/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 5 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12854037 - PreCommit-HIVE-Build > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hive > Issue Type: New Feature > Components: JDBC >Reporter: Bob Robertson >Assignee: Gunther Hagleitner > Attachments: HIVE-1555.3.patch, HIVE-1555.4.patch, HIVE-1555.5.patch, > HIVE-1555.6.patch, HIVE-1555.7.patch, JDBCStorageHandler Design Doc.pdf > > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15877579#comment-15877579 ] Hive QA commented on HIVE-1555: --- Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12853848/HIVE-1555.6.patch {color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10278 tests executed *Failed tests:* {noformat} TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) (batchId=236) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[jdbc_handler] (batchId=52) org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys] (batchId=159) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=224) org.apache.hive.hcatalog.pig.TestHCatLoaderComplexSchema.testTupleInBagInTupleInBag[1] (batchId=173) org.apache.hive.hcatalog.pig.TestHCatLoaderComplexSchema.testTupleInBagInTupleInBag[2] (batchId=173) org.apache.hive.hcatalog.pig.TestTextFileHCatStorer.testWriteSmallint (batchId=173) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3684/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3684/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3684/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 7 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12853848 - PreCommit-HIVE-Build > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hive > Issue Type: New Feature > Components: JDBC >Reporter: Bob Robertson >Assignee: Gunther Hagleitner > Attachments: HIVE-1555.3.patch, HIVE-1555.4.patch, HIVE-1555.5.patch, > HIVE-1555.6.patch, JDBCStorageHandler Design Doc.pdf > > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15875376#comment-15875376 ] Brock Noland commented on HIVE-1555: I think the resource closing in GenericJdbcDatabaseAccessor.cleanupResources could be a littler tigher. For example if ResultSet.close throws, the statement and connection are not cleaned up. > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hive > Issue Type: New Feature > Components: JDBC >Reporter: Bob Robertson >Assignee: Gunther Hagleitner > Attachments: HIVE-1555.3.patch, HIVE-1555.4.patch, HIVE-1555.5.patch, > JDBCStorageHandler Design Doc.pdf > > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15875371#comment-15875371 ] Gunther Hagleitner commented on HIVE-1555: -- [~thejas] do you have a chance to take a look? > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hive > Issue Type: New Feature > Components: JDBC >Reporter: Bob Robertson >Assignee: Gunther Hagleitner > Attachments: HIVE-1555.3.patch, HIVE-1555.4.patch, HIVE-1555.5.patch, > JDBCStorageHandler Design Doc.pdf > > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15868626#comment-15868626 ] Gunther Hagleitner commented on HIVE-1555: -- Test failures aren't related. [~ashutoshc] do you have some feedback? > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hive > Issue Type: New Feature > Components: JDBC >Reporter: Bob Robertson >Assignee: Gunther Hagleitner > Attachments: HIVE-1555.3.patch, HIVE-1555.4.patch, HIVE-1555.5.patch, > JDBCStorageHandler Design Doc.pdf > > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15862547#comment-15862547 ] Hive QA commented on HIVE-1555: --- Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12852219/HIVE-1555.5.patch {color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 10261 tests executed *Failed tests:* {noformat} TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) (batchId=236) org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys] (batchId=159) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=224) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query23] (batchId=224) org.apache.hive.hcatalog.pig.TestRCFileHCatStorer.testWriteDecimalXY (batchId=173) org.apache.hive.hcatalog.pig.TestTextFileHCatStorer.testWriteDecimal (batchId=173) org.apache.hive.hcatalog.pig.TestTextFileHCatStorer.testWriteDecimalXY (batchId=173) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3507/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3507/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3507/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 7 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12852219 - PreCommit-HIVE-Build > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hive > Issue Type: New Feature > Components: JDBC >Reporter: Bob Robertson >Assignee: Gunther Hagleitner > Attachments: HIVE-1555.3.patch, HIVE-1555.4.patch, HIVE-1555.5.patch, > JDBCStorageHandler Design Doc.pdf > > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15862370#comment-15862370 ] Hive QA commented on HIVE-1555: --- Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12852198/HIVE-1555.4.patch {color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10261 tests executed *Failed tests:* {noformat} TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) (batchId=236) TestHCatNonPartitioned - did not produce a TEST-*.xml file (likely timed out) (batchId=175) TestHCatPartitionPublish - did not produce a TEST-*.xml file (likely timed out) (batchId=175) TestHCatPartitioned - did not produce a TEST-*.xml file (likely timed out) (batchId=175) org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys] (batchId=159) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[jdbc_handler] (batchId=149) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=224) org.apache.hive.hcatalog.cli.TestPermsGrp.testCustomPerms (batchId=175) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3505/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3505/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3505/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 8 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12852198 - PreCommit-HIVE-Build > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hive > Issue Type: New Feature > Components: JDBC >Reporter: Bob Robertson >Assignee: Gunther Hagleitner > Attachments: HIVE-1555.3.patch, HIVE-1555.4.patch, JDBCStorageHandler > Design Doc.pdf > > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15862349#comment-15862349 ] Hive QA commented on HIVE-1555: --- Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12852192/HIVE-1555.3.patch {color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 10272 tests executed *Failed tests:* {noformat} TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) (batchId=236) org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys] (batchId=159) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[jdbc_handler] (batchId=149) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=224) org.apache.hive.jdbc.TestJdbcWithMiniHS2.testConnectionSchemaAPIs (batchId=218) org.apache.hive.jdbc.TestJdbcWithMiniHS2.testHttpHeaderSize (batchId=218) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3504/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3504/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3504/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 6 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12852192 - PreCommit-HIVE-Build > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hive > Issue Type: New Feature > Components: JDBC >Reporter: Bob Robertson >Assignee: Gunther Hagleitner > Attachments: HIVE-1555.1.patch, HIVE-1555.2.patch, HIVE-1555.3.patch, > JDBCStorageHandler Design Doc.pdf > > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15862331#comment-15862331 ] Hive QA commented on HIVE-1555: --- Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12852188/HIVE-1555.2.patch {color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 10272 tests executed *Failed tests:* {noformat} TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) (batchId=236) org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[jdbc_handler] (batchId=52) org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys] (batchId=159) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[jdbc_handler] (batchId=149) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=224) org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[vector_count_distinct] (batchId=106) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3503/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3503/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3503/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 6 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12852188 - PreCommit-HIVE-Build > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hive > Issue Type: New Feature > Components: JDBC >Reporter: Bob Robertson >Assignee: Gunther Hagleitner > Attachments: HIVE-1555.1.patch, HIVE-1555.2.patch, JDBCStorageHandler > Design Doc.pdf > > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15862315#comment-15862315 ] Hive QA commented on HIVE-1555: --- Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12852188/HIVE-1555.2.patch {color:green}SUCCESS:{color} +1 due to 6 test(s) being added or modified. {color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 10257 tests executed *Failed tests:* {noformat} TestDerbyConnector - did not produce a TEST-*.xml file (likely timed out) (batchId=236) TestSparkCliDriver - did not produce a TEST-*.xml file (likely timed out) (batchId=131) [union_remove_15.q,bucket_map_join_tez1.q,scriptfile1.q,groupby7_noskew.q,bucketmapjoin1.q,subquery_multiinsert.q,auto_join8.q,auto_join6.q,groupby2_map_skew.q,lateral_view_explode2.q,join28.q,load_dyn_part1.q,skewjoinopt17.q,union_remove_20.q,bucketmapjoin5.q] org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[jdbc_handler] (batchId=52) org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_join_with_different_encryption_keys] (batchId=159) org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[jdbc_handler] (batchId=149) org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query14] (batchId=224) {noformat} Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/3502/testReport Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/3502/console Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-3502/ Messages: {noformat} Executing org.apache.hive.ptest.execution.TestCheckPhase Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 6 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12852188 - PreCommit-HIVE-Build > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hive > Issue Type: New Feature > Components: JDBC >Reporter: Bob Robertson >Assignee: Gunther Hagleitner > Attachments: HIVE-1555.1.patch, HIVE-1555.2.patch, JDBCStorageHandler > Design Doc.pdf > > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15862266#comment-15862266 ] Gunther Hagleitner commented on HIVE-1555: -- Can I take this over? It's been a few months w/o update again. I've brought some code. Well I took [~charithe]'s recommendation and took https://github.com/QubitProducts/hive-jdbc-storage-handler I haven't tested w/ mysql or any other db yet. But you can write queries against derby in q files now. > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hive > Issue Type: New Feature > Components: JDBC >Reporter: Bob Robertson >Assignee: Gunther Hagleitner > Attachments: JDBCStorageHandler Design Doc.pdf > > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15681496#comment-15681496 ] Dmitry Zagorulkin commented on HIVE-1555: - [~xuefuz] I've implemented simple solution with some hard code by one. It's tested with oracle database. beeline> !connect jdbc:hive2://localhost:1 Connecting to jdbc:hive2://localhost:1 Enter username for jdbc:hive2://localhost:1: Enter password for jdbc:hive2://localhost:1: Connected to: Apache Hive (version 2.2.0-SNAPSHOT) Driver: Hive JDBC (version 2.2.0-SNAPSHOT) Transaction isolation: TRANSACTION_REPEATABLE_READ 0: jdbc:hive2://localhost:1> 0: jdbc:hive2://localhost:1> SET hive.metastore.warehouse.dir=${env:HOME}/Documents/hive-warehouse; No rows affected (0.158 seconds) 0: jdbc:hive2://localhost:1> 0: jdbc:hive2://localhost:1> CREATE EXTERNAL TABLE books3 ( . . . . . . . . . . . . . . . .> book_idINT, . . . . . . . . . . . . . . . .> book_name STRING, . . . . . . . . . . . . . . . .> author_nameSTRING, . . . . . . . . . . . . . . . .> book_isbn STRING . . . . . . . . . . . . . . . .> ) . . . . . . . . . . . . . . . .> STORED BY "org.apache.hive.storagehandler.JDBCStorageHandler" . . . . . . . . . . . . . . . .> TBLPROPERTIES ( . . . . . . . . . . . . . . . .> "mapred.jdbc.driver.class" = "oracle.jdbc.OracleDriver", . . . . . . . . . . . . . . . .> "mapred.jdbc.url" = "jdbc:oracle:thin:@//localhost:49161/XE", . . . . . . . . . . . . . . . .> "mapred.jdbc.username" = "*", . . . . . . . . . . . . . . . .> "mapred.jdbc.password" = "*", . . . . . . . . . . . . . . . .> "hive.jdbc.update.on.duplicate" = "true", . . . . . . . . . . . . . . . .> "mapreduce.jdbc.input.table.name" = "books" . . . . . . . . . . . . . . . .> ); No rows affected (2.297 seconds) 0: jdbc:hive2://localhost:1> 0: jdbc:hive2://localhost:1> 0: jdbc:hive2://localhost:1> select * from books3; +-+---+-+---+ | books3.book_id | books3.book_name | books3.author_name | books3.book_isbn | +-+---+-+---+ | 124123 | name | author | 132321adsaf31 | | 13 | name2 | author2 | asd213fadsf | | 2345236 | name3 | author3 | asdfds1234123 | +-+---+-+---+ 3 rows selected (2.146 seconds) 0: jdbc:hive2://localhost:1> explain select * from books3; ++ | Explain | ++ | STAGE DEPENDENCIES:| | Stage-0 is a root stage | || | STAGE PLANS: | | Stage: Stage-0 | | Fetch Operator | | limit: -1| | Processor Tree: | | TableScan | | alias: books3| | Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE | | Select Operator | | expressions: book_id (type: string), book_name (type: string), author_name (type: string), book_isbn (type: string) | | outputColumnNames: _col0, _col1, _col2, _col3 | | Statistics: Num rows: 1 Data size: 0 Basic stats: PARTIAL Column stats: NONE | | ListSink | || ++ 17 rows selected (0.508 seconds) What do you think about add information about external table inside serde properties section? > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hive > Issue Type: New Feature > Components: JDBC >Reporter: Bob Robertson >Assignee: Dmitry Zagorulkin > Attachments: JDBCStorageHandler Design Doc.pdf > > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15506789#comment-15506789 ] Dmitry Zagorulkin commented on HIVE-1555: - Actually it works fine with mysql and postgres. For oracle it does not work. See here: http://stackoverflow.com/questions/39594861/creating-sparksql-jdbc-federation-with-oracle-db-table-fails-with-strange-error#comment66500144_39594861 > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hive > Issue Type: New Feature > Components: JDBC >Reporter: Bob Robertson >Assignee: Teddy Choi > Attachments: JDBCStorageHandler Design Doc.pdf > > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-1555) JDBC Storage Handler
[ https://issues.apache.org/jira/browse/HIVE-1555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15401447#comment-15401447 ] Yuming Wang commented on HIVE-1555: --- Spark SQL support it. https://community.hortonworks.com/articles/4671/sparksql-jdbc-federation.html {code:sql} CREATE TABLE mysql_federated_sample USING org.apache.spark.sql.jdbc OPTIONS ( driver "com.mysql.jdbc.Driver", url "jdbc:mysql://localhost/hive?user=hive&password=hive", dbtable "TBLS" ); {code} > JDBC Storage Handler > > > Key: HIVE-1555 > URL: https://issues.apache.org/jira/browse/HIVE-1555 > Project: Hive > Issue Type: New Feature > Components: JDBC >Reporter: Bob Robertson >Assignee: Teddy Choi > Attachments: JDBCStorageHandler Design Doc.pdf > > Original Estimate: 24h > Remaining Estimate: 24h > > With the Cassandra and HBase Storage Handlers I thought it would make sense > to include a generic JDBC RDBMS Storage Handler so that you could import a > standard DB table into Hive. Many people must want to perform HiveQL joins, > etc against tables in other systems etc. -- This message was sent by Atlassian JIRA (v6.3.4#6332)