[
https://issues.apache.org/jira/browse/HIVE-20720?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16649834#comment-16649834
]
Hive QA commented on HIVE-20720:
--------------------------------
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12943878/HIVE-20720.6.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 15090 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[external_jdbc_table_partition]
(batchId=91)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[external_jdbc_table_typeconversion]
(batchId=3)
org.apache.hive.storage.jdbc.dao.TestGenericJdbcDatabaseAccessor.testGetRecordIterator
(batchId=236)
org.apache.hive.storage.jdbc.dao.TestGenericJdbcDatabaseAccessor.testGetRecordIterator_emptyResultSet
(batchId=236)
org.apache.hive.storage.jdbc.dao.TestGenericJdbcDatabaseAccessor.testGetRecordIterator_largeOffset
(batchId=236)
org.apache.hive.storage.jdbc.dao.TestGenericJdbcDatabaseAccessor.testGetRecordIterator_offsets
(batchId=236)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/14468/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/14468/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-14468/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12943878 - PreCommit-HIVE-Build
> Add partition column option to JDBC handler
> -------------------------------------------
>
> Key: HIVE-20720
> URL: https://issues.apache.org/jira/browse/HIVE-20720
> Project: Hive
> Issue Type: New Feature
> Components: StorageHandler
> Reporter: Daniel Dai
> Assignee: Daniel Dai
> Priority: Major
> Attachments: HIVE-20720.1.patch, HIVE-20720.2.patch,
> HIVE-20720.3.patch, HIVE-20720.4.patch, HIVE-20720.5.patch, HIVE-20720.6.patch
>
>
> Currently JdbcStorageHandler does not split input in Tez. The reason is
> numSplit of JdbcInputFormat.getSplits can only pass via "mapreduce.job.maps"
> in Tez. And "mapreduce.job.maps" is not a valid param if authorizer(eg.
> SQLStdAuth) is in use. User ends up always use 1 split.
> We need to rely on this new feature if we want to support multi-splits. Here
> is my proposal:
> 1. Specify partitionColumn/numPartitions, and optional lowerBound/upperBound
> in tblproperties if user want to split jdbc data source. In case
> lowerBound/upperBound is not specified, JdbcStorageHandler will run max/min
> query to get this in planner. We can currently limit partitionColumn to only
> numeric/date/timestamp column for simplicity
> 2. If partitionColumn/numPartitions are not specified, don't split input
> 3. Splits are equal intervals without respect to data distribution
> 4. There is also a "hive.sql.query.split" flag vetos the split (can be set
> manually or automatically by calcite)
> 5. If partitionColumn is not defined, but numPartitions is defined, use
> original limit/offset logic (however, don't rely on numSplit).
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
