[
https://issues.apache.org/jira/browse/HIVE-23485?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17118106#comment-17118106
]
Hive QA commented on HIVE-23485:
--------------------------------
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/13004158/HIVE-23485.02.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 29 failed/errored test(s), 17288 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=224)
org.apache.hadoop.hive.cli.TestKuduCliDriver.testCliDriver[kudu_complex_queries]
(batchId=223)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mm_bhif]
(batchId=43)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[acid_vectorization_original_tez]
(batchId=17)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[tez_tag]
(batchId=16)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[vector_join_part_col_char]
(batchId=16)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query14]
(batchId=230)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query23]
(batchId=230)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query33]
(batchId=230)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query41]
(batchId=230)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query45]
(batchId=230)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query54]
(batchId=230)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query56]
(batchId=230)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query58]
(batchId=230)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query60]
(batchId=230)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query6]
(batchId=230)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query83]
(batchId=230)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query8]
(batchId=230)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query23]
(batchId=229)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query33]
(batchId=229)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query41]
(batchId=229)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query45]
(batchId=229)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query54]
(batchId=229)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query56]
(batchId=229)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query58]
(batchId=229)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query60]
(batchId=229)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query6]
(batchId=229)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query83]
(batchId=229)
org.apache.hadoop.hive.cli.TestTezPerfConstraintsCliDriver.testCliDriver[query8]
(batchId=229)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/22653/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/22653/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-22653/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 29 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 13004158 - PreCommit-HIVE-Build
> Bound GroupByOperator stats using largest NDV among columns
> -----------------------------------------------------------
>
> Key: HIVE-23485
> URL: https://issues.apache.org/jira/browse/HIVE-23485
> Project: Hive
> Issue Type: Improvement
> Reporter: Stamatis Zampetakis
> Assignee: Stamatis Zampetakis
> Priority: Major
> Attachments: HIVE-23485.01.patch, HIVE-23485.02.patch
>
>
> Consider the following SQL query:
> {code:sql}
> select id, name from person group by id, name;
> {code}
> and assume that the person table contains the following tuples:
> {code:sql}
> insert into person values (0, 'A') ;
> insert into person values (1, 'A') ;
> insert into person values (2, 'B') ;
> insert into person values (3, 'B') ;
> insert into person values (4, 'B') ;
> insert into person values (5, 'C') ;
> {code}
> If we know the number of distinct values (NDV) for all columns in the group
> by clause then we can infer a lower bound for the total number of rows by
> taking the maximun NDV of the involved columns.
> Currently the query in the scenario above has the following plan:
> {noformat}
> Vertex dependency in root stage
> Reducer 2 <- Map 1 (SIMPLE_EDGE)
> Stage-0
> Fetch Operator
> limit:-1
> Stage-1
> Reducer 2 vectorized
> File Output Operator [FS_11]
> Group By Operator [GBY_10] (rows=3 width=92)
> Output:["_col0","_col1"],keys:KEY._col0, KEY._col1
> <-Map 1 [SIMPLE_EDGE] vectorized
> SHUFFLE [RS_9]
> PartitionCols:_col0, _col1
> Group By Operator [GBY_8] (rows=3 width=92)
> Output:["_col0","_col1"],keys:id, name
> Select Operator [SEL_7] (rows=6 width=92)
> Output:["id","name"]
> TableScan [TS_0] (rows=6 width=92)
>
> default@person,person,Tbl:COMPLETE,Col:COMPLETE,Output:["id","name"]{noformat}
> Observe that the stats for group by report 3 rows but given that the ID
> attribute is part of the aggregation the rows cannot be less than 6.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
