[jira] [Commented] (HIVE-22805) Vectorization with conditional array or map is not implemented and throws an error

2020-02-05 Thread Marta Kuczora (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030670#comment-17030670
 ] 

Marta Kuczora commented on HIVE-22805:
--

+1

Thanks a lot [~rameshkumar] for the review and [~pvary] for the patch.

> Vectorization with conditional array or map is not implemented and throws an 
> error
> --
>
> Key: HIVE-22805
> URL: https://issues.apache.org/jira/browse/HIVE-22805
> Project: Hive
>  Issue Type: Bug
>  Components: Vectorization
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-22805.2.patch, HIVE-22805.3.patch, HIVE-22805.patch
>
>
> The following queries throw an "Not supported" Exception:
> {code}
> DROP TABLE IF EXISTS cond_vector;
> CREATE TABLE cond_vector(a STRING) STORED AS ORC;
> INSERT OVERWRITE TABLE cond_vector VALUES("a/b");
> set hive.fetch.task.conversion=minimal;
> set hive.execution.mode=container;
> SELECT IF(1=1, MAP("a","b"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, MAP("Mathematics","78"), NULL) 
> FROM cond_vector;
> SELECT IF(1=1, ARRAY("c", "d"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, ARRAY("a", "b"), NULL) FROM 
> cond_vector;
> {code}
> One example of the Exception (maybe not exactly this on 4.0.0 branch, but 
> close enough to check):
> {code}
> Caused by: java.lang.RuntimeException: Not supported
>   at 
> org.apache.hadoop.hive.ql.exec.vector.ListColumnVector.copySelected(ListColumnVector.java:161)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprNull.evaluate(IfExprCondExprNull.java:87)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:95)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprBase.conditionalEvaluate(IfExprCondExprBase.java:68)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:110)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:95)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:146)
>  
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22805) Vectorization with conditional array or map is not implemented and throws an error

2020-02-05 Thread Peter Vary (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030616#comment-17030616
 ] 

Peter Vary commented on HIVE-22805:
---

[~rameshkumar]: Sorry, I have missed your non-binding +1 on RB. Thanks for the 
review!

[~kuczoram]: Could you please review?

Thanks,
Peter

> Vectorization with conditional array or map is not implemented and throws an 
> error
> --
>
> Key: HIVE-22805
> URL: https://issues.apache.org/jira/browse/HIVE-22805
> Project: Hive
>  Issue Type: Bug
>  Components: Vectorization
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-22805.2.patch, HIVE-22805.3.patch, HIVE-22805.patch
>
>
> The following queries throw an "Not supported" Exception:
> {code}
> DROP TABLE IF EXISTS cond_vector;
> CREATE TABLE cond_vector(a STRING) STORED AS ORC;
> INSERT OVERWRITE TABLE cond_vector VALUES("a/b");
> set hive.fetch.task.conversion=minimal;
> set hive.execution.mode=container;
> SELECT IF(1=1, MAP("a","b"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, MAP("Mathematics","78"), NULL) 
> FROM cond_vector;
> SELECT IF(1=1, ARRAY("c", "d"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, ARRAY("a", "b"), NULL) FROM 
> cond_vector;
> {code}
> One example of the Exception (maybe not exactly this on 4.0.0 branch, but 
> close enough to check):
> {code}
> Caused by: java.lang.RuntimeException: Not supported
>   at 
> org.apache.hadoop.hive.ql.exec.vector.ListColumnVector.copySelected(ListColumnVector.java:161)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprNull.evaluate(IfExprCondExprNull.java:87)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:95)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprBase.conditionalEvaluate(IfExprCondExprBase.java:68)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:110)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:95)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:146)
>  
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22805) Vectorization with conditional array or map is not implemented and throws an error

2020-02-05 Thread Peter Vary (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030547#comment-17030547
 ] 

Peter Vary commented on HIVE-22805:
---

[~rameshkumar], [~mustafaiman]: Could you please review?

Thanks,
Peter

> Vectorization with conditional array or map is not implemented and throws an 
> error
> --
>
> Key: HIVE-22805
> URL: https://issues.apache.org/jira/browse/HIVE-22805
> Project: Hive
>  Issue Type: Bug
>  Components: Vectorization
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-22805.2.patch, HIVE-22805.3.patch, HIVE-22805.patch
>
>
> The following queries throw an "Not supported" Exception:
> {code}
> DROP TABLE IF EXISTS cond_vector;
> CREATE TABLE cond_vector(a STRING) STORED AS ORC;
> INSERT OVERWRITE TABLE cond_vector VALUES("a/b");
> set hive.fetch.task.conversion=minimal;
> set hive.execution.mode=container;
> SELECT IF(1=1, MAP("a","b"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, MAP("Mathematics","78"), NULL) 
> FROM cond_vector;
> SELECT IF(1=1, ARRAY("c", "d"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, ARRAY("a", "b"), NULL) FROM 
> cond_vector;
> {code}
> One example of the Exception (maybe not exactly this on 4.0.0 branch, but 
> close enough to check):
> {code}
> Caused by: java.lang.RuntimeException: Not supported
>   at 
> org.apache.hadoop.hive.ql.exec.vector.ListColumnVector.copySelected(ListColumnVector.java:161)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprNull.evaluate(IfExprCondExprNull.java:87)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:95)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprBase.conditionalEvaluate(IfExprCondExprBase.java:68)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:110)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:95)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:146)
>  
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22805) Vectorization with conditional array or map is not implemented and throws an error

2020-02-04 Thread Hive QA (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030321#comment-17030321
 ] 

Hive QA commented on HIVE-22805:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992621/HIVE-22805.3.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:green}SUCCESS:{color} +1 due to 17966 tests passed

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/20467/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20467/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20467/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12992621 - PreCommit-HIVE-Build

> Vectorization with conditional array or map is not implemented and throws an 
> error
> --
>
> Key: HIVE-22805
> URL: https://issues.apache.org/jira/browse/HIVE-22805
> Project: Hive
>  Issue Type: Bug
>  Components: Vectorization
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-22805.2.patch, HIVE-22805.3.patch, HIVE-22805.patch
>
>
> The following queries throw an "Not supported" Exception:
> {code}
> DROP TABLE IF EXISTS cond_vector;
> CREATE TABLE cond_vector(a STRING) STORED AS ORC;
> INSERT OVERWRITE TABLE cond_vector VALUES("a/b");
> set hive.fetch.task.conversion=minimal;
> set hive.execution.mode=container;
> SELECT IF(1=1, MAP("a","b"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, MAP("Mathematics","78"), NULL) 
> FROM cond_vector;
> SELECT IF(1=1, ARRAY("c", "d"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, ARRAY("a", "b"), NULL) FROM 
> cond_vector;
> {code}
> One example of the Exception (maybe not exactly this on 4.0.0 branch, but 
> close enough to check):
> {code}
> Caused by: java.lang.RuntimeException: Not supported
>   at 
> org.apache.hadoop.hive.ql.exec.vector.ListColumnVector.copySelected(ListColumnVector.java:161)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprNull.evaluate(IfExprCondExprNull.java:87)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:95)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprBase.conditionalEvaluate(IfExprCondExprBase.java:68)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:110)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:95)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:146)
>  
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22805) Vectorization with conditional array or map is not implemented and throws an error

2020-02-04 Thread Hive QA (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17030301#comment-17030301
 ] 

Hive QA commented on HIVE-22805:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m 
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  8m 
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
58s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
27s{color} | {color:blue} storage-api in master has 58 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  4m 
32s{color} | {color:blue} ql in master has 1533 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
38s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
 0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  5m 
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
17s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m 20s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20467/dev-support/hive-personality.sh
 |
| git revision | master / 2a66c3f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: storage-api ql U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20467/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Vectorization with conditional array or map is not implemented and throws an 
> error
> --
>
> Key: HIVE-22805
> URL: https://issues.apache.org/jira/browse/HIVE-22805
> Project: Hive
>  Issue Type: Bug
>  Components: Vectorization
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-22805.2.patch, HIVE-22805.3.patch, HIVE-22805.patch
>
>
> The following queries throw an "Not supported" Exception:
> {code}
> DROP TABLE IF EXISTS cond_vector;
> CREATE TABLE cond_vector(a STRING) STORED AS ORC;
> INSERT OVERWRITE TABLE cond_vector VALUES("a/b");
> set hive.fetch.task.conversion=minimal;
> set hive.execution.mode=container;
> SELECT IF(1=1, MAP("a","b"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, MAP("Mathematics","78"), NULL) 
> FROM cond_vector;
> SELECT IF(1=1, ARRAY("c", "d"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, ARRAY("a", "b"), NULL) FROM 
> cond_vector;
> {code}
> One example of the Exception (maybe not exactly this on 4.0.0 branch, but 
> close enough to check):

[jira] [Commented] (HIVE-22805) Vectorization with conditional array or map is not implemented and throws an error

2020-02-04 Thread Peter Vary (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029772#comment-17029772
 ] 

Peter Vary commented on HIVE-22805:
---

Thanks for the review [~rameshkumar]!
Implemented shallowCopy too, and added related tests.

Here is the [RB|https://reviews.apache.org/r/72081/] link, so it is easier to 
review.

Thanks,
Peter

> Vectorization with conditional array or map is not implemented and throws an 
> error
> --
>
> Key: HIVE-22805
> URL: https://issues.apache.org/jira/browse/HIVE-22805
> Project: Hive
>  Issue Type: Bug
>  Components: Vectorization
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-22805.2.patch, HIVE-22805.patch
>
>
> The following queries throw an "Not supported" Exception:
> {code}
> DROP TABLE IF EXISTS cond_vector;
> CREATE TABLE cond_vector(a STRING) STORED AS ORC;
> INSERT OVERWRITE TABLE cond_vector VALUES("a/b");
> set hive.fetch.task.conversion=minimal;
> set hive.execution.mode=container;
> SELECT IF(1=1, MAP("a","b"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, MAP("Mathematics","78"), NULL) 
> FROM cond_vector;
> SELECT IF(1=1, ARRAY("c", "d"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, ARRAY("a", "b"), NULL) FROM 
> cond_vector;
> {code}
> One example of the Exception (maybe not exactly this on 4.0.0 branch, but 
> close enough to check):
> {code}
> Caused by: java.lang.RuntimeException: Not supported
>   at 
> org.apache.hadoop.hive.ql.exec.vector.ListColumnVector.copySelected(ListColumnVector.java:161)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprNull.evaluate(IfExprCondExprNull.java:87)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:95)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprBase.conditionalEvaluate(IfExprCondExprBase.java:68)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:110)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:95)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:146)
>  
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22805) Vectorization with conditional array or map is not implemented and throws an error

2020-02-03 Thread Ramesh Kumar Thangarajan (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17029345#comment-17029345
 ] 

Ramesh Kumar Thangarajan commented on HIVE-22805:
-

Your patch looks good to me. Thank you for fixing this the right way!!

Is it ok to call shallowCopyTo from copySelected? Do you think it makes sense 
to implement shallowCopyTo() for classes inherited from 
MultiValuedColumnVector? Although we do not call shallowCopyTo() directly for 
Map or List, if we have a case with nested list like:
{code:java}
select IF(1=1, ARRAY(ARRAY("a", "b"),ARRAY("c", "d")), NULL)
{code}
, then we may throw some similar error since 
MultiValuedColumnVector::shallowCopyTo is not implemented – since we have the 
new ListColumnVector::copySelected calling the shallowCopyTo for its child and 
similarly for MapColumnVector too.

> Vectorization with conditional array or map is not implemented and throws an 
> error
> --
>
> Key: HIVE-22805
> URL: https://issues.apache.org/jira/browse/HIVE-22805
> Project: Hive
>  Issue Type: Bug
>  Components: Vectorization
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-22805.2.patch, HIVE-22805.patch
>
>
> The following queries throw an "Not supported" Exception:
> {code}
> DROP TABLE IF EXISTS cond_vector;
> CREATE TABLE cond_vector(a STRING) STORED AS ORC;
> INSERT OVERWRITE TABLE cond_vector VALUES("a/b");
> set hive.fetch.task.conversion=minimal;
> set hive.execution.mode=container;
> SELECT IF(1=1, MAP("a","b"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, MAP("Mathematics","78"), NULL) 
> FROM cond_vector;
> SELECT IF(1=1, ARRAY("c", "d"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, ARRAY("a", "b"), NULL) FROM 
> cond_vector;
> {code}
> One example of the Exception (maybe not exactly this on 4.0.0 branch, but 
> close enough to check):
> {code}
> Caused by: java.lang.RuntimeException: Not supported
>   at 
> org.apache.hadoop.hive.ql.exec.vector.ListColumnVector.copySelected(ListColumnVector.java:161)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprNull.evaluate(IfExprCondExprNull.java:87)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:95)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprBase.conditionalEvaluate(IfExprCondExprBase.java:68)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:110)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:95)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:146)
>  
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22805) Vectorization with conditional array or map is not implemented and throws an error

2020-01-31 Thread Hive QA (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17027946#comment-17027946
 ] 

Hive QA commented on HIVE-22805:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992364/HIVE-22805.2.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:green}SUCCESS:{color} +1 due to 17959 tests passed

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/20436/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20436/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20436/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12992364 - PreCommit-HIVE-Build

> Vectorization with conditional array or map is not implemented and throws an 
> error
> --
>
> Key: HIVE-22805
> URL: https://issues.apache.org/jira/browse/HIVE-22805
> Project: Hive
>  Issue Type: Bug
>  Components: Vectorization
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-22805.2.patch, HIVE-22805.patch
>
>
> The following queries throw an "Not supported" Exception:
> {code}
> DROP TABLE IF EXISTS cond_vector;
> CREATE TABLE cond_vector(a STRING) STORED AS ORC;
> INSERT OVERWRITE TABLE cond_vector VALUES("a/b");
> set hive.fetch.task.conversion=minimal;
> set hive.execution.mode=container;
> SELECT IF(1=1, MAP("a","b"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, MAP("Mathematics","78"), NULL) 
> FROM cond_vector;
> SELECT IF(1=1, ARRAY("c", "d"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, ARRAY("a", "b"), NULL) FROM 
> cond_vector;
> {code}
> One example of the Exception (maybe not exactly this on 4.0.0 branch, but 
> close enough to check):
> {code}
> Caused by: java.lang.RuntimeException: Not supported
>   at 
> org.apache.hadoop.hive.ql.exec.vector.ListColumnVector.copySelected(ListColumnVector.java:161)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprNull.evaluate(IfExprCondExprNull.java:87)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:95)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprBase.conditionalEvaluate(IfExprCondExprBase.java:68)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:110)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:95)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:146)
>  
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22805) Vectorization with conditional array or map is not implemented and throws an error

2020-01-31 Thread Hive QA (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17027940#comment-17027940
 ] 

Hive QA commented on HIVE-22805:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  1m 
58s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
52s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
24s{color} | {color:blue} storage-api in master has 58 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  4m 
10s{color} | {color:blue} ql in master has 1537 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
14s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
29s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  5m 
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
21s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
17s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 30m 36s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20436/dev-support/hive-personality.sh
 |
| git revision | master / d4248dc |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: storage-api ql U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20436/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Vectorization with conditional array or map is not implemented and throws an 
> error
> --
>
> Key: HIVE-22805
> URL: https://issues.apache.org/jira/browse/HIVE-22805
> Project: Hive
>  Issue Type: Bug
>  Components: Vectorization
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-22805.2.patch, HIVE-22805.patch
>
>
> The following queries throw an "Not supported" Exception:
> {code}
> DROP TABLE IF EXISTS cond_vector;
> CREATE TABLE cond_vector(a STRING) STORED AS ORC;
> INSERT OVERWRITE TABLE cond_vector VALUES("a/b");
> set hive.fetch.task.conversion=minimal;
> set hive.execution.mode=container;
> SELECT IF(1=1, MAP("a","b"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, MAP("Mathematics","78"), NULL) 
> FROM cond_vector;
> SELECT IF(1=1, ARRAY("c", "d"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, ARRAY("a", "b"), NULL) FROM 
> cond_vector;
> {code}
> One example of the Exception (maybe not exactly this on 4.0.0 branch, but 
> close enough to check):
> {code}
> Caused

[jira] [Commented] (HIVE-22805) Vectorization with conditional array or map is not implemented and throws an error

2020-01-31 Thread Hive QA (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1702#comment-1702
 ] 

Hive QA commented on HIVE-22805:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12992333/HIVE-22805.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 17944 tests 
executed
*Failed tests:*
{noformat}
TestMiniLlapCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=160)

[unionDistinct_1.q,table_nonprintable.q,file_with_header_footer_aggregation.q,vector_offset_limit.q,orc_llap_counters1.q,mm_cttas.q,whroot_external1.q,global_limit.q,rcfile_createas1.q,dynamic_partition_pruning_2.q,intersect_merge.q,results_cache_diff_fs.q,parallel_colstats.q,load_hdfs_file_with_space_in_the_name.q,orc_merge3.q]
org.apache.hive.minikdc.TestJdbcWithMiniKdcCookie.testCookie (batchId=304)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/20431/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/20431/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-20431/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12992333 - PreCommit-HIVE-Build

> Vectorization with conditional array or map is not implemented and throws an 
> error
> --
>
> Key: HIVE-22805
> URL: https://issues.apache.org/jira/browse/HIVE-22805
> Project: Hive
>  Issue Type: Bug
>  Components: Vectorization
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-22805.patch
>
>
> The following queries throw an "Not supported" Exception:
> {code}
> DROP TABLE IF EXISTS cond_vector;
> CREATE TABLE cond_vector(a STRING) STORED AS ORC;
> INSERT OVERWRITE TABLE cond_vector VALUES("a/b");
> set hive.fetch.task.conversion=minimal;
> set hive.execution.mode=container;
> SELECT IF(1=1, MAP("a","b"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, MAP("Mathematics","78"), NULL) 
> FROM cond_vector;
> SELECT IF(1=1, ARRAY("c", "d"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, ARRAY("a", "b"), NULL) FROM 
> cond_vector;
> {code}
> One example of the Exception (maybe not exactly this on 4.0.0 branch, but 
> close enough to check):
> {code}
> Caused by: java.lang.RuntimeException: Not supported
>   at 
> org.apache.hadoop.hive.ql.exec.vector.ListColumnVector.copySelected(ListColumnVector.java:161)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprNull.evaluate(IfExprCondExprNull.java:87)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:95)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprBase.conditionalEvaluate(IfExprCondExprBase.java:68)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:110)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.expressions.IfExprCondExprCondExpr.evaluate(IfExprCondExprCondExpr.java:95)
>   at 
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:146)
>  
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Commented] (HIVE-22805) Vectorization with conditional array or map is not implemented and throws an error

2020-01-31 Thread Hive QA (Jira) 


[ 
https://issues.apache.org/jira/browse/HIVE-22805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17027739#comment-17027739
 ] 

Hive QA commented on HIVE-22805:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  2m  
4s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  8m 
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
54s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
26s{color} | {color:blue} storage-api in master has 58 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  4m 
38s{color} | {color:blue} ql in master has 1537 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
21s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
32s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  5m  
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
18s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
14s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 31m  1s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-20431/dev-support/hive-personality.sh
 |
| git revision | master / d4248dc |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.1 |
| modules | C: storage-api ql U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-20431/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Vectorization with conditional array or map is not implemented and throws an 
> error
> --
>
> Key: HIVE-22805
> URL: https://issues.apache.org/jira/browse/HIVE-22805
> Project: Hive
>  Issue Type: Bug
>  Components: Vectorization
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-22805.patch
>
>
> The following queries throw an "Not supported" Exception:
> {code}
> DROP TABLE IF EXISTS cond_vector;
> CREATE TABLE cond_vector(a STRING) STORED AS ORC;
> INSERT OVERWRITE TABLE cond_vector VALUES("a/b");
> set hive.fetch.task.conversion=minimal;
> set hive.execution.mode=container;
> SELECT IF(1=1, MAP("a","b"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, MAP("Mathematics","78"), NULL) 
> FROM cond_vector;
> SELECT IF(1=1, ARRAY("c", "d"), NULL) FROM cond_vector;
> EXPLAIN VECTORIZATION DETAIL SELECT IF(1=1, ARRAY("a", "b"), NULL) FROM 
> cond_vector;
> {code}
> One example of the Exception (maybe not exactly this on 4.0.0 branch, but 
> close enough to check):
> {code}
> Caused by: