[jira] [Commented] (HIVE-18713) Optimize: Transform IN clauses to = when there's only one element
[ https://issues.apache.org/jira/browse/HIVE-18713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16373462#comment-16373462 ] Gopal V commented on HIVE-18713: vector_mapjoin3 for LLAP collided with HIVE-18756, will commit q.out addendum. > Optimize: Transform IN clauses to = when there's only one element > - > > Key: HIVE-18713 > URL: https://issues.apache.org/jira/browse/HIVE-18713 > Project: Hive > Issue Type: Bug > Components: Logical Optimizer >Affects Versions: 3.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Fix For: 3.0.0 > > Attachments: HIVE-18713.1.patch, HIVE-18713.2.patch, > HIVE-18713.3.patch, HIVE-18713.4.patch > > > (col1) IN (col2) can be transformed to (col1) = (col2), to avoid the hash-set > implementation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18713) Optimize: Transform IN clauses to = when there's only one element
[
https://issues.apache.org/jira/browse/HIVE-18713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368068#comment-16368068
]
Hive QA commented on HIVE-18713:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12910982/HIVE-18713.4.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 31 failed/errored test(s), 13785 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=240)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=13)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=78)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_mv]
(batchId=248)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=166)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_1]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1]
(batchId=168)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_string_decimal]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_udf_string_to_boolean]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_div0]
(batchId=167)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=121)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=221)
org.apache.hadoop.hive.metastore.client.TestTablesCreateDropAlterTruncate.testAlterTableNullStorageDescriptorInNew[Embedded]
(batchId=205)
org.apache.hadoop.hive.ql.TestAcidOnTez.testGetSplitsLocks (batchId=224)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=187)
org.apache.hive.hcatalog.listener.TestDbNotificationListener.alterIndex
(batchId=242)
org.apache.hive.hcatalog.listener.TestDbNotificationListener.createIndex
(batchId=242)
org.apache.hive.hcatalog.listener.TestDbNotificationListener.dropIndex
(batchId=242)
org.apache.hive.jdbc.TestJdbcWithMiniLlap.testLlapInputFormatEndToEnd
(batchId=235)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=234)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=234)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=234)
org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveAndKill
(batchId=235)
org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveConflictKill
(batchId=235)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9261/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9261/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9261/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 31 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12910982 - PreCommit-HIVE-Build
> Optimize: Transform IN clauses to = when there's only one element
> -
>
> Key: HIVE-18713
> URL: https://issues.apache.org/jira/browse/HIVE-18713
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-18713.1.patch, HIVE-18713.2.patch,
> HIVE-18713.3.patch, HIVE-18713.4.patch
>
>
> (col1) IN (col2) can be transformed to (col1) = (col2), to avoid the hash-set
> implementation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18713) Optimize: Transform IN clauses to = when there's only one element
[
https://issues.apache.org/jira/browse/HIVE-18713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368050#comment-16368050
]
Hive QA commented on HIVE-18713:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
15s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 44s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 273a08e |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9261/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9261/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Optimize: Transform IN clauses to = when there's only one element
> -
>
> Key: HIVE-18713
> URL: https://issues.apache.org/jira/browse/HIVE-18713
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-18713.1.patch, HIVE-18713.2.patch,
> HIVE-18713.3.patch, HIVE-18713.4.patch
>
>
> (col1) IN (col2) can be transformed to (col1) = (col2), to avoid the hash-set
> implementation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18713) Optimize: Transform IN clauses to = when there's only one element
[
https://issues.apache.org/jira/browse/HIVE-18713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368029#comment-16368029
]
Gopal V commented on HIVE-18713:
No.
{code}
The 1st argument of EQUAL is expected to a primitive type, but struct is found
(state=42000,code=10016)
{code}
> Optimize: Transform IN clauses to = when there's only one element
> -
>
> Key: HIVE-18713
> URL: https://issues.apache.org/jira/browse/HIVE-18713
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-18713.1.patch, HIVE-18713.2.patch,
> HIVE-18713.3.patch, HIVE-18713.4.patch
>
>
> (col1) IN (col2) can be transformed to (col1) = (col2), to avoid the hash-set
> implementation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18713) Optimize: Transform IN clauses to = when there's only one element
[ https://issues.apache.org/jira/browse/HIVE-18713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16368002#comment-16368002 ] Vineet Garg commented on HIVE-18713: Just curious about your latest change in the patch - this "optimization" doesn't work for complex types? > Optimize: Transform IN clauses to = when there's only one element > - > > Key: HIVE-18713 > URL: https://issues.apache.org/jira/browse/HIVE-18713 > Project: Hive > Issue Type: Bug > Components: Vectorization >Affects Versions: 3.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Attachments: HIVE-18713.1.patch, HIVE-18713.2.patch, > HIVE-18713.3.patch, HIVE-18713.4.patch > > > (col1) IN (col2) can be transformed to (col1) = (col2), to avoid the hash-set > implementation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18713) Optimize: Transform IN clauses to = when there's only one element
[ https://issues.apache.org/jira/browse/HIVE-18713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365165#comment-16365165 ] Gopal V commented on HIVE-18713: ppd_join5 is unrelated. > Optimize: Transform IN clauses to = when there's only one element > - > > Key: HIVE-18713 > URL: https://issues.apache.org/jira/browse/HIVE-18713 > Project: Hive > Issue Type: Bug > Components: Vectorization >Affects Versions: 3.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Attachments: HIVE-18713.1.patch, HIVE-18713.2.patch, > HIVE-18713.3.patch > > > (col1) IN (col2) can be transformed to (col1) = (col2), to avoid the hash-set > implementation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18713) Optimize: Transform IN clauses to = when there's only one element
[
https://issues.apache.org/jira/browse/HIVE-18713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365149#comment-16365149
]
Hive QA commented on HIVE-18713:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12910645/HIVE-18713.3.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 32 failed/errored test(s), 13103 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=240)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=36)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=78)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=166)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multi_column_in_single]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_1]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=121)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query33]
(batchId=250)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query60]
(batchId=250)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query89]
(batchId=250)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query33]
(batchId=248)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query60]
(batchId=248)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query89]
(batchId=248)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=221)
org.apache.hadoop.hive.metastore.client.TestTablesCreateDropAlterTruncate.testAlterTableNullStorageDescriptorInNew[Embedded]
(batchId=205)
org.apache.hadoop.hive.ql.TestAcidOnTez.testGetSplitsLocks (batchId=224)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=187)
org.apache.hive.hcatalog.common.TestHiveClientCache.testCloseAllClients
(batchId=199)
org.apache.hive.hcatalog.listener.TestDbNotificationListener.alterIndex
(batchId=242)
org.apache.hive.hcatalog.listener.TestDbNotificationListener.createIndex
(batchId=242)
org.apache.hive.hcatalog.listener.TestDbNotificationListener.dropIndex
(batchId=242)
org.apache.hive.jdbc.TestJdbcWithMiniLlap.testLlapInputFormatEndToEnd
(batchId=235)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=234)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=234)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=234)
org.apache.hive.jdbc.TestTriggersMoveWorkloadManager.testTriggerMoveConflictKill
(batchId=235)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9222/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9222/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9222/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 32 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12910645 - PreCommit-HIVE-Build
> Optimize: Transform IN clauses to = when there's only one element
> -
>
> Key: HIVE-18713
> URL: https://issues.apache.org/jira/browse/HIVE-18713
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-18713.1.patch, HIVE-18713.2.patch,
> HIVE-18713.3.patch
>
>
> (col1) IN (col2) can be transformed to (col1) = (col2), to avoid the hash-set
> implementation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18713) Optimize: Transform IN clauses to = when there's only one element
[
https://issues.apache.org/jira/browse/HIVE-18713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16365122#comment-16365122
]
Hive QA commented on HIVE-18713:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 21s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / 7ddac02 |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9222/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9222/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Optimize: Transform IN clauses to = when there's only one element
> -
>
> Key: HIVE-18713
> URL: https://issues.apache.org/jira/browse/HIVE-18713
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-18713.1.patch, HIVE-18713.2.patch,
> HIVE-18713.3.patch
>
>
> (col1) IN (col2) can be transformed to (col1) = (col2), to avoid the hash-set
> implementation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18713) Optimize: Transform IN clauses to = when there's only one element
[ https://issues.apache.org/jira/browse/HIVE-18713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364880#comment-16364880 ] Gopal V commented on HIVE-18713: rewrite triggered correctly in query33 query89 query60 query89 triggered incorrectly in ppd_join5 multi_column_in_single > Optimize: Transform IN clauses to = when there's only one element > - > > Key: HIVE-18713 > URL: https://issues.apache.org/jira/browse/HIVE-18713 > Project: Hive > Issue Type: Bug > Components: Vectorization >Affects Versions: 3.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Attachments: HIVE-18713.1.patch, HIVE-18713.2.patch, > HIVE-18713.3.patch > > > (col1) IN (col2) can be transformed to (col1) = (col2), to avoid the hash-set > implementation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18713) Optimize: Transform IN clauses to = when there's only one element
[
https://issues.apache.org/jira/browse/HIVE-18713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364873#comment-16364873
]
Hive QA commented on HIVE-18713:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12910621/HIVE-18713.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 35 failed/errored test(s), 13099 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestAccumuloCliDriver.testCliDriver[accumulo_queries]
(batchId=240)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_hook]
(batchId=12)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join5] (batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[row__id] (batchId=78)
org.apache.hadoop.hive.cli.TestEncryptedHDFSCliDriver.testCliDriver[encryption_move_tbl]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=166)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multi_column_in_single]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_1]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=160)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_opt_shuffle_serde]
(batchId=179)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[ppd_join5]
(batchId=121)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query1]
(batchId=250)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query33]
(batchId=250)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query60]
(batchId=250)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query89]
(batchId=250)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query33]
(batchId=248)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query60]
(batchId=248)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query89]
(batchId=248)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=221)
org.apache.hadoop.hive.metastore.client.TestTablesCreateDropAlterTruncate.testAlterTableNullStorageDescriptorInNew[Embedded]
(batchId=205)
org.apache.hadoop.hive.metastore.client.TestTablesList.testListTableNamesByFilterNullDatabase[Embedded]
(batchId=205)
org.apache.hadoop.hive.ql.TestAcidOnTez.testGetSplitsLocks (batchId=224)
org.apache.hadoop.hive.ql.TestAutoPurgeTables.testNoAutoPurge (batchId=230)
org.apache.hive.beeline.cli.TestHiveCli.testNoErrorDB (batchId=187)
org.apache.hive.hcatalog.listener.TestDbNotificationListener.alterIndex
(batchId=242)
org.apache.hive.hcatalog.listener.TestDbNotificationListener.createIndex
(batchId=242)
org.apache.hive.hcatalog.listener.TestDbNotificationListener.dropIndex
(batchId=242)
org.apache.hive.jdbc.TestJdbcWithMiniLlap.testLlapInputFormatEndToEnd
(batchId=235)
org.apache.hive.jdbc.TestSSL.testConnectionMismatch (batchId=234)
org.apache.hive.jdbc.TestSSL.testConnectionWrongCertCN (batchId=234)
org.apache.hive.jdbc.TestSSL.testMetastoreConnectionWrongCertCN (batchId=234)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/9217/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/9217/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-9217/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 35 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12910621 - PreCommit-HIVE-Build
> Optimize: Transform IN clauses to = when there's only one element
> -
>
> Key: HIVE-18713
> URL: https://issues.apache.org/jira/browse/HIVE-18713
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-18713.1.patch, HIVE-18713.2.patch
>
>
> (col1) IN (col2) can be transformed to (col1) = (col2), to avoid the
[jira] [Commented] (HIVE-18713) Optimize: Transform IN clauses to = when there's only one element
[
https://issues.apache.org/jira/browse/HIVE-18713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364810#comment-16364810
]
Hive QA commented on HIVE-18713:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
0s{color} | {color:blue} Findbugs executables are not available. {color} |
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:red}-1{color} | {color:red} asflicense {color} | {color:red} 0m
14s{color} | {color:red} The patch generated 49 ASF License warnings. {color} |
| {color:black}{color} | {color:black} {color} | {color:black} 15m 45s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | /data/hiveptest/working/yetus/dev-support/hive-personality.sh |
| git revision | master / a2d22b4 |
| Default Java | 1.8.0_111 |
| asflicense |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9217/yetus/patch-asflicense-problems.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-9217/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Optimize: Transform IN clauses to = when there's only one element
> -
>
> Key: HIVE-18713
> URL: https://issues.apache.org/jira/browse/HIVE-18713
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-18713.1.patch, HIVE-18713.2.patch
>
>
> (col1) IN (col2) can be transformed to (col1) = (col2), to avoid the hash-set
> implementation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18713) Optimize: Transform IN clauses to = when there's only one element
[
https://issues.apache.org/jira/browse/HIVE-18713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364712#comment-16364712
]
Vineet Garg commented on HIVE-18713:
[~gopalv] Just a minor nitpick - cbo_rp_simple_select contains test for
{{RETURN PATH}} feature ({{set hive.cbo.returnpath.hiveop=true)}}. I think
{{cbo_simple_select}} would be better suited for the test.
> Optimize: Transform IN clauses to = when there's only one element
> -
>
> Key: HIVE-18713
> URL: https://issues.apache.org/jira/browse/HIVE-18713
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-18713.1.patch, HIVE-18713.2.patch
>
>
> (col1) IN (col2) can be transformed to (col1) = (col2), to avoid the hash-set
> implementation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18713) Optimize: Transform IN clauses to = when there's only one element
[ https://issues.apache.org/jira/browse/HIVE-18713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364593#comment-16364593 ] Vineet Garg commented on HIVE-18713: Can we add a test? Otherwise looks good to me. +1 pending tests. > Optimize: Transform IN clauses to = when there's only one element > - > > Key: HIVE-18713 > URL: https://issues.apache.org/jira/browse/HIVE-18713 > Project: Hive > Issue Type: Bug > Components: Vectorization >Affects Versions: 3.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Attachments: HIVE-18713.1.patch > > > (col1) IN (col2) can be transformed to (col1) = (col2), to avoid the hash-set > implementation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18713) Optimize: Transform IN clauses to = when there's only one element
[ https://issues.apache.org/jira/browse/HIVE-18713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364495#comment-16364495 ] Xuefu Zhang commented on HIVE-18713: Okay. Make sense. Thanks for the explanation. > Optimize: Transform IN clauses to = when there's only one element > - > > Key: HIVE-18713 > URL: https://issues.apache.org/jira/browse/HIVE-18713 > Project: Hive > Issue Type: Bug > Components: Vectorization >Affects Versions: 3.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Attachments: HIVE-18713.1.patch > > > (col1) IN (col2) can be transformed to (col1) = (col2), to avoid the hash-set > implementation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18713) Optimize: Transform IN clauses to = when there's only one element
[
https://issues.apache.org/jira/browse/HIVE-18713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364443#comment-16364443
]
Gopal V commented on HIVE-18713:
IN clause with >1 elements has to be protected against having a NULL in the
list and can't return a false for a non-match, which makes the expression
expansion a bit wider & less useful.
{{IN returns NULL not only if the expression on the left hand side is NULL, but
also if no match is found in the list and one of the expressions in the list is
NULL.}}
A single NULL element is satisfied by the col=NULL being NULL.
> Optimize: Transform IN clauses to = when there's only one element
> -
>
> Key: HIVE-18713
> URL: https://issues.apache.org/jira/browse/HIVE-18713
> Project: Hive
> Issue Type: Bug
> Components: Vectorization
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-18713.1.patch
>
>
> (col1) IN (col2) can be transformed to (col1) = (col2), to avoid the hash-set
> implementation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18713) Optimize: Transform IN clauses to = when there's only one element
[ https://issues.apache.org/jira/browse/HIVE-18713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16364332#comment-16364332 ] Xuefu Zhang commented on HIVE-18713: I'm wondering if it's a good idea to extend this such that if the number of elements is less than a threshold (say, 3), we convert it to ( (col1) = (a) OR (col1) = (b) OR (col1) = (c) ). > Optimize: Transform IN clauses to = when there's only one element > - > > Key: HIVE-18713 > URL: https://issues.apache.org/jira/browse/HIVE-18713 > Project: Hive > Issue Type: Bug > Components: Vectorization >Affects Versions: 3.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Attachments: HIVE-18713.1.patch > > > (col1) IN (col2) can be transformed to (col1) = (col2), to avoid the hash-set > implementation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
