[jira] [Commented] (HIVE-10607) Combination of ReducesinkDedup + TopN optimization yields incorrect result if there are multiple GBY in reducer
[ https://issues.apache.org/jira/browse/HIVE-10607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529903#comment-14529903 ] Hive QA commented on HIVE-10607: {color:red}Overall{color}: -1 at least one tests failed Here are the results of testing the latest attachment: https://issues.apache.org/jira/secure/attachment/12730369/HIVE-10607.patch {color:red}ERROR:{color} -1 due to 25 failed/errored test(s), 8900 tests executed *Failed tests:* {noformat} org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver_authorization_parts org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_insert_partition_dynamic org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_insert_partition_static org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_join_unencrypted_tbl org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_join_with_different_encryption_keys org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_load_data_to_encrypted_tables org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver_encryption_select_read_only_encrypted_tbl org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_disallow_transform org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_droppartition org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_sba_drop_table org.apache.hadoop.hive.cli.TestNegativeCliDriver.testNegativeCliDriver_authorization_uri_alterpart_loc org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver_limit_pushdown org.apache.hadoop.hive.ql.security.TestStorageBasedClientSideAuthorizationProvider.testSimplePrivileges org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationDrops.testDropDatabase org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationDrops.testDropPartition org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationDrops.testDropTable org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationDrops.testDropView org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationProvider.testSimplePrivileges org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationProviderWithACL.testSimplePrivileges org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationReads.testReadDbFailure org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationReads.testReadDbSuccess org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationReads.testReadTableFailure org.apache.hadoop.hive.ql.security.TestStorageBasedMetastoreAuthorizationReads.testReadTableSuccess org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.TestSQLStdHiveAccessControllerHS2.testConfigProcessing org.apache.hadoop.hive.ql.security.authorization.plugin.sqlstd.TestSQLStdHiveAccessControllerHS2.testConfigProcessingCustomSetWhitelistAppend {noformat} Test results: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3746/testReport Console output: http://ec2-174-129-184-35.compute-1.amazonaws.com/jenkins/job/PreCommit-HIVE-TRUNK-Build/3746/console Test logs: http://ec2-174-129-184-35.compute-1.amazonaws.com/logs/PreCommit-HIVE-TRUNK-Build-3746/ Messages: {noformat} Executing org.apache.hive.ptest.execution.PrepPhase Executing org.apache.hive.ptest.execution.ExecutionPhase Executing org.apache.hive.ptest.execution.ReportingPhase Tests exited with: TestsFailedException: 25 tests failed {noformat} This message is automatically generated. ATTACHMENT ID: 12730369 - PreCommit-HIVE-TRUNK-Build > Combination of ReducesinkDedup + TopN optimization yields incorrect result if > there are multiple GBY in reducer > --- > > Key: HIVE-10607 > URL: https://issues.apache.org/jira/browse/HIVE-10607 > Project: Hive > Issue Type: Bug > Components: Logical Optimizer, Tez >Affects Versions: 0.13.0, 0.14.0, 1.0.0, 1.1.0 >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-10607.patch > > > {code:sql} > select ctinyint, count(cdouble) from (select ctinyint, cdouble from > alltypesorc group by ctinyint, cdouble) t1 group by ctinyint order by > ctinyint limit 20; > {code} > This gives different result set depending on which set of optimizations are > on. In particular in .q test environment following two invocations will give > you different result set: > {code} > * mvn test -Phadoop-2 -Dtest.output.overwrite=true > -Dtest=TestMiniTezCliDriver -Dqfile=test.q > -Dhive.optimize.reducededuplication.min.reducer=1 > -Dhive.limit.pushdown.memory.usage=0.3f > * mvn t
[jira] [Commented] (HIVE-10607) Combination of ReducesinkDedup + TopN optimization yields incorrect result if there are multiple GBY in reducer
[ https://issues.apache.org/jira/browse/HIVE-10607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14529191#comment-14529191 ] Sushanth Sowmyan commented on HIVE-10607: - HIVE-10607 is currently marked as an outage in https://cwiki.apache.org/confluence/display/Hive/Hive+1.2+Release+Status, so yes, this is approved. Please go ahead and commit. > Combination of ReducesinkDedup + TopN optimization yields incorrect result if > there are multiple GBY in reducer > --- > > Key: HIVE-10607 > URL: https://issues.apache.org/jira/browse/HIVE-10607 > Project: Hive > Issue Type: Bug > Components: Logical Optimizer, Tez >Affects Versions: 0.13.0, 0.14.0, 1.0.0, 1.1.0 >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-10607.patch > > > {code:sql} > select ctinyint, count(cdouble) from (select ctinyint, cdouble from > alltypesorc group by ctinyint, cdouble) t1 group by ctinyint order by > ctinyint limit 20; > {code} > This gives different result set depending on which set of optimizations are > on. In particular in .q test environment following two invocations will give > you different result set: > {code} > * mvn test -Phadoop-2 -Dtest.output.overwrite=true > -Dtest=TestMiniTezCliDriver -Dqfile=test.q > -Dhive.optimize.reducededuplication.min.reducer=1 > -Dhive.limit.pushdown.memory.usage=0.3f > * mvn test -Phadoop-2 -Dtest.output.overwrite=true > -Dtest=TestMiniTezCliDriver -Dqfile=test.q > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10607) Combination of ReducesinkDedup + TopN optimization yields incorrect result if there are multiple GBY in reducer
[ https://issues.apache.org/jira/browse/HIVE-10607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14528939#comment-14528939 ] Ashutosh Chauhan commented on HIVE-10607: - yeah.. it will be good to have this in 1.2 too. [~sushanth] is that OK ? > Combination of ReducesinkDedup + TopN optimization yields incorrect result if > there are multiple GBY in reducer > --- > > Key: HIVE-10607 > URL: https://issues.apache.org/jira/browse/HIVE-10607 > Project: Hive > Issue Type: Bug > Components: Logical Optimizer, Tez >Affects Versions: 0.13.0, 0.14.0, 1.0.0, 1.1.0 >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-10607.patch > > > {code:sql} > select ctinyint, count(cdouble) from (select ctinyint, cdouble from > alltypesorc group by ctinyint, cdouble) t1 group by ctinyint order by > ctinyint limit 20; > {code} > This gives different result set depending on which set of optimizations are > on. In particular in .q test environment following two invocations will give > you different result set: > {code} > * mvn test -Phadoop-2 -Dtest.output.overwrite=true > -Dtest=TestMiniTezCliDriver -Dqfile=test.q > -Dhive.optimize.reducededuplication.min.reducer=1 > -Dhive.limit.pushdown.memory.usage=0.3f > * mvn test -Phadoop-2 -Dtest.output.overwrite=true > -Dtest=TestMiniTezCliDriver -Dqfile=test.q > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (HIVE-10607) Combination of ReducesinkDedup + TopN optimization yields incorrect result if there are multiple GBY in reducer
[ https://issues.apache.org/jira/browse/HIVE-10607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14528875#comment-14528875 ] Sergey Shelukhin commented on HIVE-10607: - +1. Are you going to commit to 1.2? > Combination of ReducesinkDedup + TopN optimization yields incorrect result if > there are multiple GBY in reducer > --- > > Key: HIVE-10607 > URL: https://issues.apache.org/jira/browse/HIVE-10607 > Project: Hive > Issue Type: Bug > Components: Logical Optimizer, Tez >Affects Versions: 0.13.0, 0.14.0, 1.0.0, 1.1.0 >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Attachments: HIVE-10607.patch > > > {code:sql} > select ctinyint, count(cdouble) from (select ctinyint, cdouble from > alltypesorc group by ctinyint, cdouble) t1 group by ctinyint order by > ctinyint limit 20; > {code} > This gives different result set depending on which set of optimizations are > on. In particular in .q test environment following two invocations will give > you different result set: > {code} > * mvn test -Phadoop-2 -Dtest.output.overwrite=true > -Dtest=TestMiniTezCliDriver -Dqfile=test.q > -Dhive.optimize.reducededuplication.min.reducer=1 > -Dhive.limit.pushdown.memory.usage=0.3f > * mvn test -Phadoop-2 -Dtest.output.overwrite=true > -Dtest=TestMiniTezCliDriver -Dqfile=test.q > {code} -- This message was sent by Atlassian JIRA (v6.3.4#6332)