[jira] [Updated] (HIVE-17931) Implement Parquet vectorization reader for Array type
[ https://issues.apache.org/jira/browse/HIVE-17931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Ma updated HIVE-17931: Attachment: HIVE-17931.006.patch Bug fix and add more tests to cover more situations. > Implement Parquet vectorization reader for Array type > - > > Key: HIVE-17931 > URL: https://issues.apache.org/jira/browse/HIVE-17931 > Project: Hive > Issue Type: Sub-task >Reporter: Colin Ma >Assignee: Colin Ma > Attachments: HIVE-17931.001.patch, HIVE-17931.002.patch, > HIVE-17931.003.patch, HIVE-17931.004.patch, HIVE-17931.005.patch, > HIVE-17931.006.patch > > > Parquet vectorized reader can't support array type, it should be supported to > improve the performance when the query with array type. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18054) Make Lineage work with concurrent queries on a Session
[
https://issues.apache.org/jira/browse/HIVE-18054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256579#comment-16256579
]
Hive QA commented on HIVE-18054:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12898089/HIVE-18054.5.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 11384 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=102)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_public_create]
(batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorize_grant_public]
(batchId=92)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7874/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7874/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7874/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12898089 - PreCommit-HIVE-Build
> Make Lineage work with concurrent queries on a Session
> ---
>
> Key: HIVE-18054
> URL: https://issues.apache.org/jira/browse/HIVE-18054
> Project: Hive
> Issue Type: Bug
>Reporter: Andrew Sherman
>Assignee: Andrew Sherman
> Attachments: HIVE-18054.1.patch, HIVE-18054.2.patch,
> HIVE-18054.3.patch, HIVE-18054.4.patch, HIVE-18054.5.patch
>
>
> A Hive Session can contain multiple concurrent sql Operations.
> Lineage is currently tracked in SessionState and is cleared when a query
> completes. This results in Lineage for other running queries being lost.
> To fix this, move LineageState from SessionState to QueryState.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-16828) With CBO enabled, Query on partitioned views throws IndexOutOfBoundException
[
https://issues.apache.org/jira/browse/HIVE-16828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16828:
---
Status: Patch Available (was: Open)
> With CBO enabled, Query on partitioned views throws IndexOutOfBoundException
>
>
> Key: HIVE-16828
> URL: https://issues.apache.org/jira/browse/HIVE-16828
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 2.2.0, 2.1.1, 1.2.0
>Reporter: Adesh Kumar Rao
>Assignee: Vineet Garg
> Attachments: HIVE-16828.2.patch, HIVE-16828.3.patch, HIVE-16828.patch
>
>
> {code:java}
> Caused by: java.lang.AssertionError: Internal error: While invoking method
> 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Filter,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:789)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:534)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:270)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:213)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:374)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(HiveRelFieldTrimmer.java:394)
> ... 98 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
> ... 102 more
> Caused by: java.lang.AssertionError: Internal error: While invoking method
> 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Project,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:789)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:534)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:270)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:213)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:466)
> ... 107 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
> ... 110 more
> Caused by: java.lang.IndexOutOfBoundsException: Index: 94, Size: 94
> at java.util.ArrayList.rangeCheck(ArrayList.java:653)
> at java.util.ArrayList.get(ArrayList.java:429)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(HiveRelFieldTrimmer.java:390)
> {code}
> Reproducible case:
> 1) set "hive.security.authorization.enabled" to "true" in "hive-site.xml".
> 2) {code:java}
> CREATE TABLE table1 (id int) PARTITIONED BY (year int)
> -- create partitioned view
> CREATE VIEW view1 partitioned on (year) as select id, year from table1;
> set hive.cbo.enable=true;
> select year from view1;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-16828) With CBO enabled, Query on partitioned views throws IndexOutOfBoundException
[
https://issues.apache.org/jira/browse/HIVE-16828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256540#comment-16256540
]
Vineet Garg commented on HIVE-16828:
[~ashutoshc] Can you take a look?
> With CBO enabled, Query on partitioned views throws IndexOutOfBoundException
>
>
> Key: HIVE-16828
> URL: https://issues.apache.org/jira/browse/HIVE-16828
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 1.2.0, 2.1.1, 2.2.0
>Reporter: Adesh Kumar Rao
>Assignee: Vineet Garg
> Attachments: HIVE-16828.2.patch, HIVE-16828.3.patch, HIVE-16828.patch
>
>
> {code:java}
> Caused by: java.lang.AssertionError: Internal error: While invoking method
> 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Filter,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:789)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:534)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:270)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:213)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:374)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(HiveRelFieldTrimmer.java:394)
> ... 98 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
> ... 102 more
> Caused by: java.lang.AssertionError: Internal error: While invoking method
> 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Project,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:789)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:534)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:270)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:213)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:466)
> ... 107 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
> ... 110 more
> Caused by: java.lang.IndexOutOfBoundsException: Index: 94, Size: 94
> at java.util.ArrayList.rangeCheck(ArrayList.java:653)
> at java.util.ArrayList.get(ArrayList.java:429)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(HiveRelFieldTrimmer.java:390)
> {code}
> Reproducible case:
> 1) set "hive.security.authorization.enabled" to "true" in "hive-site.xml".
> 2) {code:java}
> CREATE TABLE table1 (id int) PARTITIONED BY (year int)
> -- create partitioned view
> CREATE VIEW view1 partitioned on (year) as select id, year from table1;
> set hive.cbo.enable=true;
> select year from view1;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-16828) With CBO enabled, Query on partitioned views throws IndexOutOfBoundException
[
https://issues.apache.org/jira/browse/HIVE-16828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16828:
---
Attachment: HIVE-16828.3.patch
> With CBO enabled, Query on partitioned views throws IndexOutOfBoundException
>
>
> Key: HIVE-16828
> URL: https://issues.apache.org/jira/browse/HIVE-16828
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 1.2.0, 2.1.1, 2.2.0
>Reporter: Adesh Kumar Rao
>Assignee: Vineet Garg
> Attachments: HIVE-16828.2.patch, HIVE-16828.3.patch, HIVE-16828.patch
>
>
> {code:java}
> Caused by: java.lang.AssertionError: Internal error: While invoking method
> 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Filter,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:789)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:534)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:270)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:213)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:374)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(HiveRelFieldTrimmer.java:394)
> ... 98 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
> ... 102 more
> Caused by: java.lang.AssertionError: Internal error: While invoking method
> 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Project,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:789)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:534)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:270)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:213)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:466)
> ... 107 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
> ... 110 more
> Caused by: java.lang.IndexOutOfBoundsException: Index: 94, Size: 94
> at java.util.ArrayList.rangeCheck(ArrayList.java:653)
> at java.util.ArrayList.get(ArrayList.java:429)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(HiveRelFieldTrimmer.java:390)
> {code}
> Reproducible case:
> 1) set "hive.security.authorization.enabled" to "true" in "hive-site.xml".
> 2) {code:java}
> CREATE TABLE table1 (id int) PARTITIONED BY (year int)
> -- create partitioned view
> CREATE VIEW view1 partitioned on (year) as select id, year from table1;
> set hive.cbo.enable=true;
> select year from view1;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-16828) With CBO enabled, Query on partitioned views throws IndexOutOfBoundException
[
https://issues.apache.org/jira/browse/HIVE-16828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16828:
---
Status: Open (was: Patch Available)
> With CBO enabled, Query on partitioned views throws IndexOutOfBoundException
>
>
> Key: HIVE-16828
> URL: https://issues.apache.org/jira/browse/HIVE-16828
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 2.2.0, 2.1.1, 1.2.0
>Reporter: Adesh Kumar Rao
>Assignee: Vineet Garg
> Attachments: HIVE-16828.2.patch, HIVE-16828.patch
>
>
> {code:java}
> Caused by: java.lang.AssertionError: Internal error: While invoking method
> 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Filter,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:789)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:534)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:270)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:213)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:374)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(HiveRelFieldTrimmer.java:394)
> ... 98 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
> ... 102 more
> Caused by: java.lang.AssertionError: Internal error: While invoking method
> 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Project,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:789)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:534)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:270)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:213)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:466)
> ... 107 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
> ... 110 more
> Caused by: java.lang.IndexOutOfBoundsException: Index: 94, Size: 94
> at java.util.ArrayList.rangeCheck(ArrayList.java:653)
> at java.util.ArrayList.get(ArrayList.java:429)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(HiveRelFieldTrimmer.java:390)
> {code}
> Reproducible case:
> 1) set "hive.security.authorization.enabled" to "true" in "hive-site.xml".
> 2) {code:java}
> CREATE TABLE table1 (id int) PARTITIONED BY (year int)
> -- create partitioned view
> CREATE VIEW view1 partitioned on (year) as select id, year from table1;
> set hive.cbo.enable=true;
> select year from view1;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-16828) With CBO enabled, Query on partitioned views throws IndexOutOfBoundException
[
https://issues.apache.org/jira/browse/HIVE-16828?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256527#comment-16256527
]
Hive QA commented on HIVE-16828:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12898088/HIVE-16828.2.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 11383 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=102)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_public_create]
(batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorize_grant_public]
(batchId=92)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query94]
(batchId=245)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query94]
(batchId=243)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress (batchId=226)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7873/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7873/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7873/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12898088 - PreCommit-HIVE-Build
> With CBO enabled, Query on partitioned views throws IndexOutOfBoundException
>
>
> Key: HIVE-16828
> URL: https://issues.apache.org/jira/browse/HIVE-16828
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 1.2.0, 2.1.1, 2.2.0
>Reporter: Adesh Kumar Rao
>Assignee: Vineet Garg
> Attachments: HIVE-16828.2.patch, HIVE-16828.patch
>
>
> {code:java}
> Caused by: java.lang.AssertionError: Internal error: While invoking method
> 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Filter,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:789)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:534)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:270)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:213)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:374)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(HiveRelFieldTrimmer.java:394)
> ... 98 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
> ... 102 more
> Caused by: java.lang.AssertionError: Internal error: While invoking method
> 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Project,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:789)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:534)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:270)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:213)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:466)
> ... 107 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at
[jira] [Updated] (HIVE-17361) Support LOAD DATA for transactional tables
[ https://issues.apache.org/jira/browse/HIVE-17361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eugene Koifman updated HIVE-17361: -- Attachment: HIVE-17361.14.patch > Support LOAD DATA for transactional tables > -- > > Key: HIVE-17361 > URL: https://issues.apache.org/jira/browse/HIVE-17361 > Project: Hive > Issue Type: New Feature > Components: Transactions >Reporter: Wei Zheng >Assignee: Eugene Koifman >Priority: Critical > Attachments: HIVE-17361.07.patch, HIVE-17361.08.patch, > HIVE-17361.09.patch, HIVE-17361.1.patch, HIVE-17361.10.patch, > HIVE-17361.11.patch, HIVE-17361.12.patch, HIVE-17361.14.patch, > HIVE-17361.2.patch, HIVE-17361.3.patch, HIVE-17361.4.patch > > > LOAD DATA was not supported since ACID was introduced. Need to fill this gap > between ACID table and regular hive table. > Current Documentation is under [DML > Operations|https://cwiki.apache.org/confluence/display/Hive/GettingStarted#GettingStarted-DMLOperations] > and [Loading files into > tables|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DML#LanguageManualDML-Loadingfilesintotables]: > \\ > * Load Data performs very limited validations of the data, in particular it > uses the input file name which may not be in 0_0 which can break some > read logic. (Certainly will for Acid). > * It does not check the schema of the file. This may be a non issue for Acid > which requires ORC which is self describing so Schema Evolution may handle > this seamlessly. (Assuming Schema is not too different). > * It does check that _InputFormat_S are compatible. > * Bucketed (and thus sorted) tables don't support Load Data (but only if > hive.strict.checks.bucketing=true (default)). Will keep this restriction for > Acid. > * Load Data supports OVERWRITE clause > * What happens to file permissions/ownership: rename vs copy differences > \\ > The implementation will follow the same idea as in HIVE-14988 and use a > base_N/ dir for OVERWRITE clause. > \\ > How is minor compaction going to handle delta/base with original files? > Since delta_8_8/_meta_data is created before files are moved, delta_8_8 > becomes visible before it's populated. Is that an issue? -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18080) Performance degradation on VectorizedLogicBench#IfExprLongColumnLongColumnBench when AVX512 is enabled
[
https://issues.apache.org/jira/browse/HIVE-18080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256474#comment-16256474
]
liyunzhang commented on HIVE-18080:
---
[~mmccline]: can you help see the issue? very thanks!
> Performance degradation on
> VectorizedLogicBench#IfExprLongColumnLongColumnBench when AVX512 is enabled
> --
>
> Key: HIVE-18080
> URL: https://issues.apache.org/jira/browse/HIVE-18080
> Project: Hive
> Issue Type: Bug
>Reporter: liyunzhang
>
> Use Xeon(R) Platinum 8180 CPU to test the performance of
> [AVX512|https://en.wikipedia.org/wiki/AVX-512].
> {code}
> #cat /proc/cpuinfo |grep "model name"|head -n 1
> model name: Intel(R) Xeon(R) Platinum 8180 CPU @ 2.50GHz
> {code}
> Before that I have compiled hive with JDK9 as JDK9 enables AVX512
> Use hive microbenchmark(HIVE-10189) to evaluate the performance improvement.
> It seems performance(20%+) in cases in
> {{VectorizedArithmeticBench}},{{VectorizedComparisonBench}},{{VectorizedLikeBench}},{{VectorizedLogicBench}}
> execpt
> {{VectorizedLogicBench#IfExprLongColumnLongColumnBench}},{{VectorizedLogicBench#IfExprRepeatingLongColumnLongColumnBench}}
> and
> {{VectorizedLogicBench#IfExprLongColumnRepeatingLongColumnBench}}.The data is
> like following
> When i use Skylake CPU to evaluate the performance improvement of AVX512.
> I found the performance in VectorizedLogicBench is like following
> || ||AVX2 us/op||AVX512 us/op || (AVX2-AVX512)/AVX2||
> |ColAndColBench|122510| 87014| 28.9%|
> |IfExprLongColumnLongColumnBench | 1325759| 1436073| -8.3% |
> |IfExprLongColumnRepeatingLongColumnBench|1397447|1480450| -5.9%|
> |IfExprRepeatingLongColumnLongColumnBench|1401164|1483062| -5.9% |
> |NotColBench|77042.83|51513.28| 33%|
> There are degradation in
> IfExprLongColumnLongColumnBench,IfExprLongColumnRepeatingLongColumnBench,
> IfExprRepeatingLongColumnLongColumnBench, very confused why there is
> degradation on IfExprLongColumnLongColumnBench cases.
> Here we use {{taskset -cp 1 $pid}} to run the benchmark on single core to
> avoid the impact of dynamic CPU frequency scaling.
> my script
> {code}
> export JAVA_HOME=/home/zly/jdk-9.0.1/
> export PATH=$JAVA_HOME/bin:$PATH
> export LD_LIBRARY_PATH=/home/zly/jdk-9.0.1/mylib
> for i in 0 1 2; do
> java -server -XX:UseAVX=3 -jar benchmarks.jar
> org.apache.hive.benchmark.vectorization.VectorizedLogicBench * -wi 10 -i 20
> -f 1 -bm avgt -tu us >log.logic.avx3.single.$i & export pid=$!
> taskset -cp 1 $pid
> wait $pid
> done
> for i in 0 1 2; do
> java -server -XX:UseAVX=2 -jar benchmarks.jar
> org.apache.hive.benchmark.vectorization.VectorizedLogicBench * -wi 10 -i 20
> -f 1 -bm avgt -tu us >log.logic.avx2.single.$i & export pid=$!
> taskset -cp 1 $pid
> wait $pid
> done
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-14487) Add REBUILD statement for materialized views
[
https://issues.apache.org/jira/browse/HIVE-14487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256473#comment-16256473
]
Hive QA commented on HIVE-14487:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12898081/HIVE-14487.patch
{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 20 failed/errored test(s), 11393 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[materialized_view_create_rewrite]
(batchId=243)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_view] (batchId=39)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[create_view_translate]
(batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[explain_ddl] (batchId=46)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[llap_acid_fast]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_create_rewrite]
(batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[selectDistinctStar]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[union_top_level]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_windowing]
(batchId=167)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_public_create]
(batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorize_grant_public]
(batchId=92)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[union_top_level]
(batchId=130)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query16]
(batchId=245)
org.apache.hadoop.hive.cli.control.TestDanglingQOuts.checkDanglingQOut
(batchId=206)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testViewsReplication
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7872/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7872/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7872/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 20 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12898081 - PreCommit-HIVE-Build
> Add REBUILD statement for materialized views
>
>
> Key: HIVE-14487
> URL: https://issues.apache.org/jira/browse/HIVE-14487
> Project: Hive
> Issue Type: Sub-task
> Components: Materialized views
>Affects Versions: 2.2.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-14487.patch
>
>
> Support for rebuilding existing materialized views. The statement is the
> following:
> {code:sql}
> ALTER MATERIALIZED VIEW [db_name.]materialized_view_name REBUILD;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18077) Vectorization: Add string conversion case for UDFToDouble
[ https://issues.apache.org/jira/browse/HIVE-18077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18077: Attachment: HIVE-18077.07.patch > Vectorization: Add string conversion case for UDFToDouble > - > > Key: HIVE-18077 > URL: https://issues.apache.org/jira/browse/HIVE-18077 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0 > > Attachments: HIVE-18077.01.patch, HIVE-18077.02.patch, > HIVE-18077.04.patch, HIVE-18077.05.patch, HIVE-18077.06.patch, > HIVE-18077.07.patch > > > Add string to float/double vectorization. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18077) Vectorization: Add string conversion case for UDFToDouble
[ https://issues.apache.org/jira/browse/HIVE-18077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18077: Attachment: (was: HIVE-18077.07.patch) > Vectorization: Add string conversion case for UDFToDouble > - > > Key: HIVE-18077 > URL: https://issues.apache.org/jira/browse/HIVE-18077 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0 > > Attachments: HIVE-18077.01.patch, HIVE-18077.02.patch, > HIVE-18077.04.patch, HIVE-18077.05.patch, HIVE-18077.06.patch > > > Add string to float/double vectorization. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18077) Vectorization: Add string conversion case for UDFToDouble
[ https://issues.apache.org/jira/browse/HIVE-18077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18077: Status: Patch Available (was: In Progress) > Vectorization: Add string conversion case for UDFToDouble > - > > Key: HIVE-18077 > URL: https://issues.apache.org/jira/browse/HIVE-18077 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0 > > Attachments: HIVE-18077.01.patch, HIVE-18077.02.patch, > HIVE-18077.04.patch, HIVE-18077.05.patch, HIVE-18077.06.patch, > HIVE-18077.07.patch > > > Add string to float/double vectorization. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18077) Vectorization: Add string conversion case for UDFToDouble
[ https://issues.apache.org/jira/browse/HIVE-18077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18077: Attachment: HIVE-18077.07.patch > Vectorization: Add string conversion case for UDFToDouble > - > > Key: HIVE-18077 > URL: https://issues.apache.org/jira/browse/HIVE-18077 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0 > > Attachments: HIVE-18077.01.patch, HIVE-18077.02.patch, > HIVE-18077.04.patch, HIVE-18077.05.patch, HIVE-18077.06.patch, > HIVE-18077.07.patch > > > Add string to float/double vectorization. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18077) Vectorization: Add string conversion case for UDFToDouble
[ https://issues.apache.org/jira/browse/HIVE-18077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18077: Status: In Progress (was: Patch Available) > Vectorization: Add string conversion case for UDFToDouble > - > > Key: HIVE-18077 > URL: https://issues.apache.org/jira/browse/HIVE-18077 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0 > > Attachments: HIVE-18077.01.patch, HIVE-18077.02.patch, > HIVE-18077.04.patch, HIVE-18077.05.patch, HIVE-18077.06.patch, > HIVE-18077.07.patch > > > Add string to float/double vectorization. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17942) HiveAlterHandler not using conf from HMS Handler
[ https://issues.apache.org/jira/browse/HIVE-17942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar updated HIVE-17942: --- Resolution: Fixed Status: Resolved (was: Patch Available) > HiveAlterHandler not using conf from HMS Handler > > > Key: HIVE-17942 > URL: https://issues.apache.org/jira/browse/HIVE-17942 > Project: Hive > Issue Type: Bug >Affects Versions: 2.1.1 >Reporter: Janaki Lahorani >Assignee: Janaki Lahorani > Fix For: 3.0.0, 2.4.0 > > Attachments: HIVE-17942-branch-2.01.patch, HIVE-17942.1.patch, > HIVE-17942.2.patch, HIVE-17942.3.patch, HIVE-17942.4.patch, HIVE-17942.5.patch > > > When HiveAlterHandler looks for conf, it is not getting the one from thread > local. So, local changes are not visible. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17942) HiveAlterHandler not using conf from HMS Handler
[ https://issues.apache.org/jira/browse/HIVE-17942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256434#comment-16256434 ] Vihang Karajgaonkar commented on HIVE-17942: patch merged to branch-2 as well. Thanks for your contribution [~janulatha] > HiveAlterHandler not using conf from HMS Handler > > > Key: HIVE-17942 > URL: https://issues.apache.org/jira/browse/HIVE-17942 > Project: Hive > Issue Type: Bug >Affects Versions: 2.1.1 >Reporter: Janaki Lahorani >Assignee: Janaki Lahorani > Fix For: 3.0.0, 2.4.0 > > Attachments: HIVE-17942-branch-2.01.patch, HIVE-17942.1.patch, > HIVE-17942.2.patch, HIVE-17942.3.patch, HIVE-17942.4.patch, HIVE-17942.5.patch > > > When HiveAlterHandler looks for conf, it is not getting the one from thread > local. So, local changes are not visible. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17942) HiveAlterHandler not using conf from HMS Handler
[ https://issues.apache.org/jira/browse/HIVE-17942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar updated HIVE-17942: --- Fix Version/s: 2.4.0 > HiveAlterHandler not using conf from HMS Handler > > > Key: HIVE-17942 > URL: https://issues.apache.org/jira/browse/HIVE-17942 > Project: Hive > Issue Type: Bug >Affects Versions: 2.1.1 >Reporter: Janaki Lahorani >Assignee: Janaki Lahorani > Fix For: 3.0.0, 2.4.0 > > Attachments: HIVE-17942-branch-2.01.patch, HIVE-17942.1.patch, > HIVE-17942.2.patch, HIVE-17942.3.patch, HIVE-17942.4.patch, HIVE-17942.5.patch > > > When HiveAlterHandler looks for conf, it is not getting the one from thread > local. So, local changes are not visible. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18077) Vectorization: Add string conversion case for UDFToDouble
[
https://issues.apache.org/jira/browse/HIVE-18077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256428#comment-16256428
]
Hive QA commented on HIVE-18077:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12898076/HIVE-18077.06.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 11388 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=102)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_public_create]
(batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorize_grant_public]
(batchId=92)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7871/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7871/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7871/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12898076 - PreCommit-HIVE-Build
> Vectorization: Add string conversion case for UDFToDouble
> -
>
> Key: HIVE-18077
> URL: https://issues.apache.org/jira/browse/HIVE-18077
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HIVE-18077.01.patch, HIVE-18077.02.patch,
> HIVE-18077.04.patch, HIVE-18077.05.patch, HIVE-18077.06.patch
>
>
> Add string to float/double vectorization.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-16756) Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: / by zero"
[ https://issues.apache.org/jira/browse/HIVE-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vihang Karajgaonkar updated HIVE-16756: --- Attachment: HIVE-16756-branch-2.02.patch patch merged in master branch. Attaching branch-2 patch as well. > Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: > / by zero" > > > Key: HIVE-16756 > URL: https://issues.apache.org/jira/browse/HIVE-16756 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.3.0 >Reporter: Matt McCline >Assignee: Vihang Karajgaonkar >Priority: Critical > Attachments: HIVE-16756-branch-2.02.patch, HIVE-16756.01.patch, > HIVE-16756.02.patch, HIVE-16756.03.patch > > > vectorization_div0.q needs to test the long data type testing. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18026) Hive webhcat principal configuration optimization
[ https://issues.apache.org/jira/browse/HIVE-18026?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256398#comment-16256398 ] Rui Li commented on HIVE-18026: --- The change looks good to me. +1 [~thejas], [~alangates], maybe you also want to take a look. > Hive webhcat principal configuration optimization > - > > Key: HIVE-18026 > URL: https://issues.apache.org/jira/browse/HIVE-18026 > Project: Hive > Issue Type: Bug > Components: WebHCat >Affects Versions: 3.0.0 >Reporter: ZhangBing Lin >Assignee: ZhangBing Lin > Attachments: HIVE-18026.1.patch, HIVE-18026.2.patch > > > Hive webhcat principal configuration optimization，when you configure: > > templeton.kerberos.principal > HTTP/_HOST@ > > The '_HOST' should be replaced by specific host name. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17856) MM tables - IOW is not ACID compliant
[
https://issues.apache.org/jira/browse/HIVE-17856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256387#comment-16256387
]
Steve Yeom commented on HIVE-17856:
---
Yes, Sergey.. Especially one is a multiple IOW statement
and another is a statement with a regular insert and a IOW on the same table.
I had to figure out which is the correct result.
> MM tables - IOW is not ACID compliant
> -
>
> Key: HIVE-17856
> URL: https://issues.apache.org/jira/browse/HIVE-17856
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Sergey Shelukhin
>Assignee: Steve Yeom
> Labels: mm-gap-1
> Attachments: HIVE-17856.1.patch, HIVE-17856.2.patch,
> HIVE-17856.3.patch, HIVE-17856.4.patch, HIVE-17856.5.patch,
> HIVE-17856.6.patch, HIVE-17856.7.patch, HIVE-17856.8.patch, HIVE-17856.9.patch
>
>
> The following tests were removed from mm_all during "integration"... I should
> have never allowed such manner of intergration.
> MM logic should have been kept intact until ACID logic could catch up. Alas,
> here we are.
> {noformat}
> drop table iow0_mm;
> create table iow0_mm(key int) tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> insert overwrite table iow0_mm select key from intermediate;
> insert into table iow0_mm select key + 1 from intermediate;
> select * from iow0_mm order by key;
> insert overwrite table iow0_mm select key + 2 from intermediate;
> select * from iow0_mm order by key;
> drop table iow0_mm;
> drop table iow1_mm;
> create table iow1_mm(key int) partitioned by (key2 int)
> tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> insert overwrite table iow1_mm partition (key2)
> select key as k1, key from intermediate union all select key as k1, key from
> intermediate;
> insert into table iow1_mm partition (key2)
> select key + 1 as k1, key from intermediate union all select key as k1, key
> from intermediate;
> select * from iow1_mm order by key, key2;
> insert overwrite table iow1_mm partition (key2)
> select key + 3 as k1, key from intermediate union all select key + 4 as k1,
> key from intermediate;
> select * from iow1_mm order by key, key2;
> insert overwrite table iow1_mm partition (key2)
> select key + 3 as k1, key + 3 from intermediate union all select key + 2 as
> k1, key + 2 from intermediate;
> select * from iow1_mm order by key, key2;
> drop table iow1_mm;
> {noformat}
> {noformat}
> drop table simple_mm;
> create table simple_mm(key int) stored as orc tblproperties
> ("transactional"="true", "transactional_properties"="insert_only");
> insert into table simple_mm select key from intermediate;
> -insert overwrite table simple_mm select key from intermediate;
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17856) MM tables - IOW is not ACID compliant
[
https://issues.apache.org/jira/browse/HIVE-17856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256376#comment-16256376
]
Sergey Shelukhin commented on HIVE-17856:
-
Which ones were incorrect? Were those the IOW tests? That would make sense
> MM tables - IOW is not ACID compliant
> -
>
> Key: HIVE-17856
> URL: https://issues.apache.org/jira/browse/HIVE-17856
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Sergey Shelukhin
>Assignee: Steve Yeom
> Labels: mm-gap-1
> Attachments: HIVE-17856.1.patch, HIVE-17856.2.patch,
> HIVE-17856.3.patch, HIVE-17856.4.patch, HIVE-17856.5.patch,
> HIVE-17856.6.patch, HIVE-17856.7.patch, HIVE-17856.8.patch, HIVE-17856.9.patch
>
>
> The following tests were removed from mm_all during "integration"... I should
> have never allowed such manner of intergration.
> MM logic should have been kept intact until ACID logic could catch up. Alas,
> here we are.
> {noformat}
> drop table iow0_mm;
> create table iow0_mm(key int) tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> insert overwrite table iow0_mm select key from intermediate;
> insert into table iow0_mm select key + 1 from intermediate;
> select * from iow0_mm order by key;
> insert overwrite table iow0_mm select key + 2 from intermediate;
> select * from iow0_mm order by key;
> drop table iow0_mm;
> drop table iow1_mm;
> create table iow1_mm(key int) partitioned by (key2 int)
> tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> insert overwrite table iow1_mm partition (key2)
> select key as k1, key from intermediate union all select key as k1, key from
> intermediate;
> insert into table iow1_mm partition (key2)
> select key + 1 as k1, key from intermediate union all select key as k1, key
> from intermediate;
> select * from iow1_mm order by key, key2;
> insert overwrite table iow1_mm partition (key2)
> select key + 3 as k1, key from intermediate union all select key + 4 as k1,
> key from intermediate;
> select * from iow1_mm order by key, key2;
> insert overwrite table iow1_mm partition (key2)
> select key + 3 as k1, key + 3 from intermediate union all select key + 2 as
> k1, key + 2 from intermediate;
> select * from iow1_mm order by key, key2;
> drop table iow1_mm;
> {noformat}
> {noformat}
> drop table simple_mm;
> create table simple_mm(key int) stored as orc tblproperties
> ("transactional"="true", "transactional_properties"="insert_only");
> insert into table simple_mm select key from intermediate;
> -insert overwrite table simple_mm select key from intermediate;
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-15911) Create a view based on another view throws an exception “FAILED: NullPointerException null”
[ https://issues.apache.org/jira/browse/HIVE-15911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256370#comment-16256370 ] yangfang commented on HIVE-15911: - [~ashutoshc] I've tested it and this issue has been solved in hive3.0 > Create a view based on another view throws an exception “FAILED: > NullPointerException null” > --- > > Key: HIVE-15911 > URL: https://issues.apache.org/jira/browse/HIVE-15911 > Project: Hive > Issue Type: Bug > Components: Views >Affects Versions: 2.1.0, 2.1.1 > Environment: hive2.1.0 >Reporter: yangfang >Assignee: yangfang > Attachments: 0001-Hive-15911.patch > > > when I create a new view based on another view I an exception “FAILED: > NullPointerException null”: > hive> create view view2(a,b) as select a, b from view1; //view1 is another > view > FAILED: NullPointerException null > hive> > The hive log show error stack： > 2017-02-15T15:40:25,816 ERROR ql.Driver (SessionState.java:printError(1116)) > - FAILED: NullPointerException null > java.lang.NullPointerException > at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:863) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:552) > at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1319) > at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1459) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1239) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1229) > at > org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184) > at > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403) > at > org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821) > at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.apache.hadoop.util.RunJar.run(RunJar.java:221) > at org.apache.hadoop.util.RunJar.main(RunJar.java:136) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17631) upgrade orc to 1.4.0
[
https://issues.apache.org/jira/browse/HIVE-17631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256366#comment-16256366
]
Hive QA commented on HIVE-17631:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12898067/HIVE-17631.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 11388 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[dynamic_semijoin_user_level]
(batchId=147)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_semijoin_reduction]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_semijoin_reduction_3]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mergejoin]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_dynamic_semijoin_reduction2]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_dynamic_semijoin_reduction]
(batchId=149)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=102)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_public_create]
(batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorize_grant_public]
(batchId=92)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgress (batchId=226)
org.apache.hive.beeline.TestBeeLineWithArgs.testQueryProgressParallel
(batchId=226)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7870/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7870/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7870/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 15 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12898067 - PreCommit-HIVE-Build
> upgrade orc to 1.4.0
>
>
> Key: HIVE-17631
> URL: https://issues.apache.org/jira/browse/HIVE-17631
> Project: Hive
> Issue Type: Improvement
>Affects Versions: 3.0.0
>Reporter: Saijin Huang
>Assignee: Sergey Shelukhin
> Attachments: HIVE-17631.patch
>
>
> It seems like orc 1.4.0 has a latest and stable version:
> https://orc.apache.org/docs/releases.html
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17964) HoS: some spark configs doesn't require re-creating a session
[
https://issues.apache.org/jira/browse/HIVE-17964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-17964:
--
Attachment: (was: HIVE-17964.2.patch)
> HoS: some spark configs doesn't require re-creating a session
> -
>
> Key: HIVE-17964
> URL: https://issues.apache.org/jira/browse/HIVE-17964
> Project: Hive
> Issue Type: Improvement
> Components: Spark
>Reporter: Rui Li
>Assignee: Rui Li
>Priority: Minor
> Attachments: HIVE-17964.1.patch, HIVE-17964.2.patch
>
>
> I guess the {{hive.spark.}} configs were initially intended for the RSC.
> Therefore when they're changed, we'll re-create the session for them to take
> effect. There're some configs not related to RSC that also start with
> {{hive.spark.}}. We'd better rename them so that we don't unnecessarily
> re-create sessions, which is usually time consuming.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18025) Push resource plan changes to tez/unmanaged sessions
[ https://issues.apache.org/jira/browse/HIVE-18025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-18025: - Attachment: HIVE-18025.2.patch misc fixes and rebase. > Push resource plan changes to tez/unmanaged sessions > > > Key: HIVE-18025 > URL: https://issues.apache.org/jira/browse/HIVE-18025 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-18025.1.patch, HIVE-18025.2.patch > > > This is to remove MetastoreGlobalTriggersFetcher and make changes so that > DDLTask can push RP changes to tez session pool manager as well for updating > the triggers. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17964) HoS: some spark configs doesn't require re-creating a session
[
https://issues.apache.org/jira/browse/HIVE-17964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-17964:
--
Attachment: (was: HIVE-17964.2.patch)
> HoS: some spark configs doesn't require re-creating a session
> -
>
> Key: HIVE-17964
> URL: https://issues.apache.org/jira/browse/HIVE-17964
> Project: Hive
> Issue Type: Improvement
> Components: Spark
>Reporter: Rui Li
>Assignee: Rui Li
>Priority: Minor
> Attachments: HIVE-17964.1.patch, HIVE-17964.2.patch
>
>
> I guess the {{hive.spark.}} configs were initially intended for the RSC.
> Therefore when they're changed, we'll re-create the session for them to take
> effect. There're some configs not related to RSC that also start with
> {{hive.spark.}}. We'd better rename them so that we don't unnecessarily
> re-create sessions, which is usually time consuming.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17964) HoS: some spark configs doesn't require re-creating a session
[
https://issues.apache.org/jira/browse/HIVE-17964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Li updated HIVE-17964:
--
Attachment: HIVE-17964.2.patch
The output of last test is
[here|https://builds.apache.org/view/H-L/view/Hive/job/PreCommit-HIVE-Build/7842/console].
The error is {{IOException: No space left on device}}. Try again.
> HoS: some spark configs doesn't require re-creating a session
> -
>
> Key: HIVE-17964
> URL: https://issues.apache.org/jira/browse/HIVE-17964
> Project: Hive
> Issue Type: Improvement
> Components: Spark
>Reporter: Rui Li
>Assignee: Rui Li
>Priority: Minor
> Attachments: HIVE-17964.1.patch, HIVE-17964.2.patch,
> HIVE-17964.2.patch, HIVE-17964.2.patch
>
>
> I guess the {{hive.spark.}} configs were initially intended for the RSC.
> Therefore when they're changed, we'll re-create the session for them to take
> effect. There're some configs not related to RSC that also start with
> {{hive.spark.}}. We'd better rename them so that we don't unnecessarily
> re-create sessions, which is usually time consuming.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-17931) Implement Parquet vectorization reader for Array type
[ https://issues.apache.org/jira/browse/HIVE-17931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256318#comment-16256318 ] Ferdinand Xu commented on HIVE-17931: - LGTM +1 > Implement Parquet vectorization reader for Array type > - > > Key: HIVE-17931 > URL: https://issues.apache.org/jira/browse/HIVE-17931 > Project: Hive > Issue Type: Sub-task >Reporter: Colin Ma >Assignee: Colin Ma > Attachments: HIVE-17931.001.patch, HIVE-17931.002.patch, > HIVE-17931.003.patch, HIVE-17931.004.patch, HIVE-17931.005.patch > > > Parquet vectorized reader can't support array type, it should be supported to > improve the performance when the query with array type. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18072) WM - fix various bugs based on cluster testing - part 2
[ https://issues.apache.org/jira/browse/HIVE-18072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18072: Attachment: HIVE-18072.02.patch There were some spurious changes in 01 patch from another patch I was working on in parallel > WM - fix various bugs based on cluster testing - part 2 > --- > > Key: HIVE-18072 > URL: https://issues.apache.org/jira/browse/HIVE-18072 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-18072.01.patch, HIVE-18072.02.patch, > HIVE-18072.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-14560) Support exchange partition between s3 and hdfs tables
[
https://issues.apache.org/jira/browse/HIVE-14560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256270#comment-16256270
]
Ashutosh Chauhan commented on HIVE-14560:
-
+1
> Support exchange partition between s3 and hdfs tables
> -
>
> Key: HIVE-14560
> URL: https://issues.apache.org/jira/browse/HIVE-14560
> Project: Hive
> Issue Type: Bug
>Reporter: Abdullah Yousufi
>Assignee: Abdullah Yousufi
> Fix For: 3.0.0
>
> Attachments: HIVE-14560.02.patch, HIVE-14560.patch
>
>
> {code}
> alter table s3_tbl exchange partition (country='USA', state='CA') with table
> hdfs_tbl;
> {code}
> results in:
> {code}
> Error: Error while processing statement: FAILED: Execution Error, return code
> 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Got
> exception: java.lang.IllegalArgumentException Wrong FS:
> s3a://hive-on-s3/s3_tbl/country=USA/state=CA, expected:
> hdfs://localhost:9000) (state=08S01,code=1)
> {code}
> because the check for whether the s3 destination table path exists occurs on
> the hdfs filesystem.
> Furthermore, exchanging between s3 to hdfs fails because the hdfs rename
> operation is not supported across filesystems. Fix uses copy + deletion in
> the case that the file systems differ.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18056) CachedStore: Have a whitelist/blacklist config to allow selective caching of tables/partitions and allow read while prewarming
[
https://issues.apache.org/jira/browse/HIVE-18056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256268#comment-16256268
]
Hive QA commented on HIVE-18056:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12898065/HIVE-18056.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 16 failed/errored test(s), 11388 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=102)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_public_create]
(batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorize_grant_public]
(batchId=92)
org.apache.hadoop.hive.metastore.cache.TestCachedStore.testAggrStatsRepeatedRead
(batchId=198)
org.apache.hadoop.hive.metastore.cache.TestCachedStore.testDatabaseOps
(batchId=198)
org.apache.hadoop.hive.metastore.cache.TestCachedStore.testPartitionAggrStats
(batchId=198)
org.apache.hadoop.hive.metastore.cache.TestCachedStore.testPartitionAggrStatsBitVector
(batchId=198)
org.apache.hadoop.hive.metastore.cache.TestCachedStore.testPartitionOps
(batchId=198)
org.apache.hadoop.hive.metastore.cache.TestCachedStore.testTableOps
(batchId=198)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7869/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7869/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7869/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 16 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12898065 - PreCommit-HIVE-Build
> CachedStore: Have a whitelist/blacklist config to allow selective caching of
> tables/partitions and allow read while prewarming
> --
>
> Key: HIVE-18056
> URL: https://issues.apache.org/jira/browse/HIVE-18056
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
> Attachments: HIVE-18056.1.patch, HIVE-18056.2.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-14678) Hive-on-MR deprecation warning is not diplayed when engine is set to capital letter 'MR'
[ https://issues.apache.org/jira/browse/HIVE-14678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-14678: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Pushed to master. Thanks, Sergey! > Hive-on-MR deprecation warning is not diplayed when engine is set to capital > letter 'MR' > - > > Key: HIVE-14678 > URL: https://issues.apache.org/jira/browse/HIVE-14678 > Project: Hive > Issue Type: Bug > Components: Beeline, CLI, Configuration, Hive >Affects Versions: 2.1.0, 2.2.0 >Reporter: Raja Mannem >Assignee: Sergey Shelukhin >Priority: Trivial > Fix For: 3.0.0 > > Attachments: HIVE-14678.patch > > > hive> SET hive.execution.engine=mr; > Hive-on-MR is deprecated in Hive 2 and may not be available in the future > versions. Consider using a different execution engine (i.e. tez, spark) or > using Hive 1.X releases. > hive> > hive> SET hive.execution.engine=MR; > hive> > This warning seems to be generated only with small letter 'mr' wand not with > capital letter "MR". > generateMrDeprecationWarning() seem to be displaying this warning and is > being called in different classes. > Ex: > https://github.com/apache/hive/blob/master/cli/src/java/org/apache/hadoop/hive/cli/CliDriver.java#L754 > https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/Driver.java#L1900 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17898) Explain plan output enhancement
[ https://issues.apache.org/jira/browse/HIVE-17898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17898: --- Status: Patch Available (was: Open) > Explain plan output enhancement > --- > > Key: HIVE-17898 > URL: https://issues.apache.org/jira/browse/HIVE-17898 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-17898.1.patch, HIVE-17898.2.patch, > HIVE-17898.3.patch, HIVE-17898.4.patch, HIVE-17898.5.patch, HIVE-17898.6.patch > > > We would like to enhance the explain plan output to display additional > information e.g.: > TableScan operator should have following additional info > * Actual table name (currently only alias name is displayed) > * Database name > * Column names being scanned -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17898) Explain plan output enhancement
[ https://issues.apache.org/jira/browse/HIVE-17898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17898: --- Attachment: HIVE-17898.6.patch > Explain plan output enhancement > --- > > Key: HIVE-17898 > URL: https://issues.apache.org/jira/browse/HIVE-17898 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-17898.1.patch, HIVE-17898.2.patch, > HIVE-17898.3.patch, HIVE-17898.4.patch, HIVE-17898.5.patch, HIVE-17898.6.patch > > > We would like to enhance the explain plan output to display additional > information e.g.: > TableScan operator should have following additional info > * Actual table name (currently only alias name is displayed) > * Database name > * Column names being scanned -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17935) Turn on hive.optimize.sort.dynamic.partition by default
[ https://issues.apache.org/jira/browse/HIVE-17935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-17935: -- Attachment: HIVE-17935.7.patch > Turn on hive.optimize.sort.dynamic.partition by default > --- > > Key: HIVE-17935 > URL: https://issues.apache.org/jira/browse/HIVE-17935 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman > Attachments: HIVE-17935.1.patch, HIVE-17935.2.patch, > HIVE-17935.3.patch, HIVE-17935.4.patch, HIVE-17935.5.patch, > HIVE-17935.6.patch, HIVE-17935.7.patch > > > The config option hive.optimize.sort.dynamic.partition is an optimization for > Hive’s dynamic partitioning feature. It was originally implemented in > [HIVE-6455|https://issues.apache.org/jira/browse/HIVE-6455]. With this > optimization, the dynamic partition columns and bucketing columns (in case of > bucketed tables) are sorted before being fed to the reducers. Since the > partitioning and bucketing columns are sorted, each reducer can keep only one > record writer open at any time thereby reducing the memory pressure on the > reducers. There were some early problems with this optimization and it was > disabled by default in HiveConf in > [HIVE-8151|https://issues.apache.org/jira/browse/HIVE-8151]. Since then > setting hive.optimize.sort.dynamic.partition=true has been used to solve > problems where dynamic partitioning produces with (1) too many small files on > HDFS, which is bad for the cluster and can increase overhead for future Hive > queries over those partitions, and (2) OOM issues in the map tasks because it > trying to simultaneously write to 100 different files. > It now seems that the feature is probably mature enough that it can be > enabled by default. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17898) Explain plan output enhancement
[ https://issues.apache.org/jira/browse/HIVE-17898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17898: --- Status: Open (was: Patch Available) > Explain plan output enhancement > --- > > Key: HIVE-17898 > URL: https://issues.apache.org/jira/browse/HIVE-17898 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-17898.1.patch, HIVE-17898.2.patch, > HIVE-17898.3.patch, HIVE-17898.4.patch, HIVE-17898.5.patch > > > We would like to enhance the explain plan output to display additional > information e.g.: > TableScan operator should have following additional info > * Actual table name (currently only alias name is displayed) > * Database name > * Column names being scanned -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-14731) Use Tez cartesian product edge in Hive (unpartitioned case only)
[ https://issues.apache.org/jira/browse/HIVE-14731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-14731: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) > Use Tez cartesian product edge in Hive (unpartitioned case only) > > > Key: HIVE-14731 > URL: https://issues.apache.org/jira/browse/HIVE-14731 > Project: Hive > Issue Type: Bug >Reporter: Zhiyuan Yang >Assignee: Zhiyuan Yang > Labels: TODOC3.0 > Fix For: 3.0.0 > > Attachments: HIVE-14731.1.patch, HIVE-14731.10.patch, > HIVE-14731.11.patch, HIVE-14731.12.patch, HIVE-14731.13.patch, > HIVE-14731.14.patch, HIVE-14731.15.patch, HIVE-14731.16.patch, > HIVE-14731.17.patch, HIVE-14731.18.patch, HIVE-14731.19.patch, > HIVE-14731.2.patch, HIVE-14731.20.patch, HIVE-14731.21.patch, > HIVE-14731.22.patch, HIVE-14731.23.patch, HIVE-14731.3.patch, > HIVE-14731.4.patch, HIVE-14731.5.patch, HIVE-14731.6.patch, > HIVE-14731.7.patch, HIVE-14731.8.patch, HIVE-14731.9.patch, > HIVE-14731.addendum.patch > > > Given cartesian product edge is available in Tez now (see TEZ-3230), let's > integrate it into Hive on Tez. This allows us to have more than one reducer > in cross product queries. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HIVE-18017) HS2 materialized view registry init loading all tables from metastore
[ https://issues.apache.org/jira/browse/HIVE-18017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez resolved HIVE-18017. Resolution: Fixed Assignee: Jesus Camacho Rodriguez Fix Version/s: 3.0.0 Fixed in HIVE-15436. > HS2 materialized view registry init loading all tables from metastore > - > > Key: HIVE-18017 > URL: https://issues.apache.org/jira/browse/HIVE-18017 > Project: Hive > Issue Type: Bug > Components: HiveServer2, Metastore >Affects Versions: 2.2.0, 2.3.0 > Environment: Hadoop 2.7.3 on EMR >Reporter: Songzhi Liu >Assignee: Jesus Camacho Rodriguez > Fix For: 3.0.0 > > > We're pointing the hive to a remote MySQL metastore with tons of DBs and > tables (10k+ tables). When HS2 starts it tries to init the materialized views > and currently the metastore API tries to do a full load for every database. > It times out after 20 min then HS2 starts without loading MV. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (HIVE-18089) Update golden files for few tests
[ https://issues.apache.org/jira/browse/HIVE-18089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan resolved HIVE-18089. - Resolution: Fixed Fix Version/s: 3.0.0 Pushed to master. > Update golden files for few tests > - > > Key: HIVE-18089 > URL: https://issues.apache.org/jira/browse/HIVE-18089 > Project: Hive > Issue Type: Test > Components: Test >Affects Versions: 3.0.0 >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > Fix For: 3.0.0 > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17856) MM tables - IOW is not ACID compliant
[
https://issues.apache.org/jira/browse/HIVE-17856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256212#comment-16256212
]
Steve Yeom commented on HIVE-17856:
---
mm_all.q tests were taking a bit more time since the correct behavior should be
verified:
in order words, some of the existing expected results are not correct. And I
think we have jiras related to IOW
other than this jira.
> MM tables - IOW is not ACID compliant
> -
>
> Key: HIVE-17856
> URL: https://issues.apache.org/jira/browse/HIVE-17856
> Project: Hive
> Issue Type: Sub-task
> Components: Transactions
>Reporter: Sergey Shelukhin
>Assignee: Steve Yeom
> Labels: mm-gap-1
> Attachments: HIVE-17856.1.patch, HIVE-17856.2.patch,
> HIVE-17856.3.patch, HIVE-17856.4.patch, HIVE-17856.5.patch,
> HIVE-17856.6.patch, HIVE-17856.7.patch, HIVE-17856.8.patch, HIVE-17856.9.patch
>
>
> The following tests were removed from mm_all during "integration"... I should
> have never allowed such manner of intergration.
> MM logic should have been kept intact until ACID logic could catch up. Alas,
> here we are.
> {noformat}
> drop table iow0_mm;
> create table iow0_mm(key int) tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> insert overwrite table iow0_mm select key from intermediate;
> insert into table iow0_mm select key + 1 from intermediate;
> select * from iow0_mm order by key;
> insert overwrite table iow0_mm select key + 2 from intermediate;
> select * from iow0_mm order by key;
> drop table iow0_mm;
> drop table iow1_mm;
> create table iow1_mm(key int) partitioned by (key2 int)
> tblproperties("transactional"="true",
> "transactional_properties"="insert_only");
> insert overwrite table iow1_mm partition (key2)
> select key as k1, key from intermediate union all select key as k1, key from
> intermediate;
> insert into table iow1_mm partition (key2)
> select key + 1 as k1, key from intermediate union all select key as k1, key
> from intermediate;
> select * from iow1_mm order by key, key2;
> insert overwrite table iow1_mm partition (key2)
> select key + 3 as k1, key from intermediate union all select key + 4 as k1,
> key from intermediate;
> select * from iow1_mm order by key, key2;
> insert overwrite table iow1_mm partition (key2)
> select key + 3 as k1, key + 3 from intermediate union all select key + 2 as
> k1, key + 2 from intermediate;
> select * from iow1_mm order by key, key2;
> drop table iow1_mm;
> {noformat}
> {noformat}
> drop table simple_mm;
> create table simple_mm(key int) stored as orc tblproperties
> ("transactional"="true", "transactional_properties"="insert_only");
> insert into table simple_mm select key from intermediate;
> -insert overwrite table simple_mm select key from intermediate;
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HIVE-18089) Update golden files for few tests
[ https://issues.apache.org/jira/browse/HIVE-18089?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan reassigned HIVE-18089: --- > Update golden files for few tests > - > > Key: HIVE-18089 > URL: https://issues.apache.org/jira/browse/HIVE-18089 > Project: Hive > Issue Type: Test > Components: Test >Affects Versions: 3.0.0 >Reporter: Ashutosh Chauhan >Assignee: Ashutosh Chauhan > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18054) Make Lineage work with concurrent queries on a Session
[ https://issues.apache.org/jira/browse/HIVE-18054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18054: -- Attachment: HIVE-18054.5.patch > Make Lineage work with concurrent queries on a Session > --- > > Key: HIVE-18054 > URL: https://issues.apache.org/jira/browse/HIVE-18054 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman > Attachments: HIVE-18054.1.patch, HIVE-18054.2.patch, > HIVE-18054.3.patch, HIVE-18054.4.patch, HIVE-18054.5.patch > > > A Hive Session can contain multiple concurrent sql Operations. > Lineage is currently tracked in SessionState and is cleared when a query > completes. This results in Lineage for other running queries being lost. > To fix this, move LineageState from SessionState to QueryState. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-16828) With CBO enabled, Query on partitioned views throws IndexOutOfBoundException
[
https://issues.apache.org/jira/browse/HIVE-16828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16828:
---
Status: Patch Available (was: Open)
> With CBO enabled, Query on partitioned views throws IndexOutOfBoundException
>
>
> Key: HIVE-16828
> URL: https://issues.apache.org/jira/browse/HIVE-16828
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 2.2.0, 2.1.1, 1.2.0
>Reporter: Adesh Kumar Rao
>Assignee: Vineet Garg
> Attachments: HIVE-16828.2.patch, HIVE-16828.patch
>
>
> {code:java}
> Caused by: java.lang.AssertionError: Internal error: While invoking method
> 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Filter,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:789)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:534)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:270)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:213)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:374)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(HiveRelFieldTrimmer.java:394)
> ... 98 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
> ... 102 more
> Caused by: java.lang.AssertionError: Internal error: While invoking method
> 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Project,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:789)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:534)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:270)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:213)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:466)
> ... 107 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
> ... 110 more
> Caused by: java.lang.IndexOutOfBoundsException: Index: 94, Size: 94
> at java.util.ArrayList.rangeCheck(ArrayList.java:653)
> at java.util.ArrayList.get(ArrayList.java:429)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(HiveRelFieldTrimmer.java:390)
> {code}
> Reproducible case:
> 1) set "hive.security.authorization.enabled" to "true" in "hive-site.xml".
> 2) {code:java}
> CREATE TABLE table1 (id int) PARTITIONED BY (year int)
> -- create partitioned view
> CREATE VIEW view1 partitioned on (year) as select id, year from table1;
> set hive.cbo.enable=true;
> select year from view1;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-16828) With CBO enabled, Query on partitioned views throws IndexOutOfBoundException
[
https://issues.apache.org/jira/browse/HIVE-16828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16828:
---
Status: Open (was: Patch Available)
> With CBO enabled, Query on partitioned views throws IndexOutOfBoundException
>
>
> Key: HIVE-16828
> URL: https://issues.apache.org/jira/browse/HIVE-16828
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 2.2.0, 2.1.1, 1.2.0
>Reporter: Adesh Kumar Rao
>Assignee: Vineet Garg
> Attachments: HIVE-16828.2.patch, HIVE-16828.patch
>
>
> {code:java}
> Caused by: java.lang.AssertionError: Internal error: While invoking method
> 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Filter,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:789)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:534)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:270)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:213)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:374)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(HiveRelFieldTrimmer.java:394)
> ... 98 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
> ... 102 more
> Caused by: java.lang.AssertionError: Internal error: While invoking method
> 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Project,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:789)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:534)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:270)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:213)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:466)
> ... 107 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
> ... 110 more
> Caused by: java.lang.IndexOutOfBoundsException: Index: 94, Size: 94
> at java.util.ArrayList.rangeCheck(ArrayList.java:653)
> at java.util.ArrayList.get(ArrayList.java:429)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(HiveRelFieldTrimmer.java:390)
> {code}
> Reproducible case:
> 1) set "hive.security.authorization.enabled" to "true" in "hive-site.xml".
> 2) {code:java}
> CREATE TABLE table1 (id int) PARTITIONED BY (year int)
> -- create partitioned view
> CREATE VIEW view1 partitioned on (year) as select id, year from table1;
> set hive.cbo.enable=true;
> select year from view1;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-16828) With CBO enabled, Query on partitioned views throws IndexOutOfBoundException
[
https://issues.apache.org/jira/browse/HIVE-16828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16828:
---
Attachment: HIVE-16828.2.patch
> With CBO enabled, Query on partitioned views throws IndexOutOfBoundException
>
>
> Key: HIVE-16828
> URL: https://issues.apache.org/jira/browse/HIVE-16828
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 1.2.0, 2.1.1, 2.2.0
>Reporter: Adesh Kumar Rao
>Assignee: Vineet Garg
> Attachments: HIVE-16828.2.patch, HIVE-16828.patch
>
>
> {code:java}
> Caused by: java.lang.AssertionError: Internal error: While invoking method
> 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Filter,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:789)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:534)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:270)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:213)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:374)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(HiveRelFieldTrimmer.java:394)
> ... 98 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
> ... 102 more
> Caused by: java.lang.AssertionError: Internal error: While invoking method
> 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Project,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:789)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:534)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:270)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:213)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:466)
> ... 107 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
> ... 110 more
> Caused by: java.lang.IndexOutOfBoundsException: Index: 94, Size: 94
> at java.util.ArrayList.rangeCheck(ArrayList.java:653)
> at java.util.ArrayList.get(ArrayList.java:429)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(HiveRelFieldTrimmer.java:390)
> {code}
> Reproducible case:
> 1) set "hive.security.authorization.enabled" to "true" in "hive-site.xml".
> 2) {code:java}
> CREATE TABLE table1 (id int) PARTITIONED BY (year int)
> -- create partitioned view
> CREATE VIEW view1 partitioned on (year) as select id, year from table1;
> set hive.cbo.enable=true;
> select year from view1;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18025) Push resource plan changes to tez/unmanaged sessions
[
https://issues.apache.org/jira/browse/HIVE-18025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256190#comment-16256190
]
Hive QA commented on HIVE-18025:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12898060/HIVE-18025.1.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7868/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7868/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7868/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2017-11-17 00:11:22.019
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-7868/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2017-11-17 00:11:22.022
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 477e9c5 HIVE-18071 : add HS2 jmx information about pools and
current resource plan (Sergey Shelukhin, reviewed by Prasanth Jayachandran)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 477e9c5 HIVE-18071 : add HS2 jmx information about pools and
current resource plan (Sergey Shelukhin, reviewed by Prasanth Jayachandran)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2017-11-17 00:11:22.576
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error:
a/itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestTriggersNoTezSessionPool.java:
No such file or directory
error:
a/itests/hive-unit/src/test/java/org/apache/hive/jdbc/TestTriggersTezSessionPoolManager.java:
No such file or directory
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java: No such file
or directory
error:
a/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolManager.java: No
such file or directory
error:
a/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/TezSessionPoolSession.java: No
such file or directory
error: a/ql/src/java/org/apache/hadoop/hive/ql/exec/tez/WorkloadManager.java:
No such file or directory
error:
a/ql/src/java/org/apache/hadoop/hive/ql/wm/MetastoreGlobalTriggersFetcher.java:
No such file or directory
error:
a/standalone-metastore/src/gen/thrift/gen-javabean/org/apache/hadoop/hive/metastore/api/WMFullResourcePlan.java:
No such file or directory
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12898060 - PreCommit-HIVE-Build
> Push resource plan changes to tez/unmanaged sessions
>
>
> Key: HIVE-18025
> URL: https://issues.apache.org/jira/browse/HIVE-18025
> Project: Hive
> Issue Type: Sub-task
>Affects Versions: 3.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
> Attachments: HIVE-18025.1.patch
>
>
> This is to remove MetastoreGlobalTriggersFetcher and make changes so that
> DDLTask can push RP changes to tez session pool manager as well for updating
> the triggers.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18072) WM - fix various bugs based on cluster testing - part 2
[
https://issues.apache.org/jira/browse/HIVE-18072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256189#comment-16256189
]
Hive QA commented on HIVE-18072:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12898057/HIVE-18072.01.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7867/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7867/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7867/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2017-11-17 00:10:46.425
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-7867/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2017-11-17 00:10:46.428
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
254e216..477e9c5 master -> origin/master
+ git reset --hard HEAD
HEAD is now at 254e216 HIVE-15018: ALTER rewriting flag in materialized view
(Jesus Camacho Rodriguez, reviewed by Ashutosh Chauhan)
+ git clean -f -d
Removing ql/src/java/org/apache/hadoop/hive/ql/parse/SemanticAnalyzer.java.orig
Removing ql/src/test/queries/clientpositive/alter_view_as_select_view.q
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
(use "git pull" to update your local branch)
+ git reset --hard origin/master
HEAD is now at 477e9c5 HIVE-18071 : add HS2 jmx information about pools and
current resource plan (Sergey Shelukhin, reviewed by Prasanth Jayachandran)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2017-11-17 00:10:51.584
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/GuaranteedTasksAllocator.java:167
error:
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/GuaranteedTasksAllocator.java:
patch does not apply
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/exec/tez/WorkloadManager.java:18
error: ql/src/java/org/apache/hadoop/hive/ql/exec/tez/WorkloadManager.java:
patch does not apply
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12898057 - PreCommit-HIVE-Build
> WM - fix various bugs based on cluster testing - part 2
> ---
>
> Key: HIVE-18072
> URL: https://issues.apache.org/jira/browse/HIVE-18072
> Project: Hive
> Issue Type: Sub-task
>Reporter: Sergey Shelukhin
>Assignee: Sergey Shelukhin
> Attachments: HIVE-18072.01.patch, HIVE-18072.patch
>
>
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-15911) Create a view based on another view throws an exception “FAILED: NullPointerException null”
[
https://issues.apache.org/jira/browse/HIVE-15911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256188#comment-16256188
]
Hive QA commented on HIVE-15911:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12853014/0001-Hive-15911.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 18 failed/errored test(s), 11389 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[alter_view_as_select_view]
(batchId=72)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=102)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_public_create]
(batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorize_grant_public]
(batchId=92)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testCancelRenewTokenFlow
(batchId=244)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testConnection
(batchId=244)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testIsValid (batchId=244)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testIsValidNeg
(batchId=244)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testNegativeProxyAuth
(batchId=244)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testNegativeTokenAuth
(batchId=244)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testProxyAuth
(batchId=244)
org.apache.hive.minikdc.TestJdbcWithDBTokenStoreNoDoAs.testTokenAuth
(batchId=244)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7866/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7866/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7866/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 18 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12853014 - PreCommit-HIVE-Build
> Create a view based on another view throws an exception “FAILED:
> NullPointerException null”
> ---
>
> Key: HIVE-15911
> URL: https://issues.apache.org/jira/browse/HIVE-15911
> Project: Hive
> Issue Type: Bug
> Components: Views
>Affects Versions: 2.1.0, 2.1.1
> Environment: hive2.1.0
>Reporter: yangfang
>Assignee: yangfang
> Attachments: 0001-Hive-15911.patch
>
>
> when I create a new view based on another view I an exception “FAILED:
> NullPointerException null”:
> hive> create view view2(a,b) as select a, b from view1; //view1 is another
> view
> FAILED: NullPointerException null
> hive>
> The hive log show error stack：
> 2017-02-15T15:40:25,816 ERROR ql.Driver (SessionState.java:printError(1116))
> - FAILED: NullPointerException null
> java.lang.NullPointerException
> at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:863)
> at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:552)
> at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1319)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1459)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1239)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1229)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
>
[jira] [Commented] (HIVE-17713) disallow adding/modifying columns in metastore for Avro/CSV/etc. SerDes
[ https://issues.apache.org/jira/browse/HIVE-17713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256175#comment-16256175 ] Thejas M Nair commented on HIVE-17713: -- Agree > disallow adding/modifying columns in metastore for Avro/CSV/etc. SerDes > --- > > Key: HIVE-17713 > URL: https://issues.apache.org/jira/browse/HIVE-17713 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin > > This is kind of a followup from HIVE-11985 > When a SerDe uses an external schema (either embedded, or via a file link), > it's not using the information stored in metastore for the columns. So, if > the users modify table schema via add column and other such commands it won't > have effect on the serde, since it's using the external schema to figure out > what the columns are; leading to confusion and bugs. We should not allow such > a modification for SerDes not in hive.serdes.using.metastore.for.schema -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HIVE-18088) Add WM event traces at query level for debugging
[ https://issues.apache.org/jira/browse/HIVE-18088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran reassigned HIVE-18088: > Add WM event traces at query level for debugging > > > Key: HIVE-18088 > URL: https://issues.apache.org/jira/browse/HIVE-18088 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > > For debugging and testing purpose, expose workload manager events via /jmx > endpoint and print summary at the scope of query. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18071) add HS2 jmx information about pools and current resource plan
[ https://issues.apache.org/jira/browse/HIVE-18071?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18071: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Committed to master. Thanks for the review > add HS2 jmx information about pools and current resource plan > - > > Key: HIVE-18071 > URL: https://issues.apache.org/jira/browse/HIVE-18071 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Fix For: 3.0.0 > > Attachments: HIVE-18071.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-14487) Add REBUILD statement for materialized views
[
https://issues.apache.org/jira/browse/HIVE-14487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-14487:
---
Attachment: HIVE-14487.patch
> Add REBUILD statement for materialized views
>
>
> Key: HIVE-14487
> URL: https://issues.apache.org/jira/browse/HIVE-14487
> Project: Hive
> Issue Type: Sub-task
> Components: Materialized views
>Affects Versions: 2.2.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-14487.patch
>
>
> Support for rebuilding existing materialized views. The statement is the
> following:
> {code:sql}
> ALTER MATERIALIZED VIEW [db_name.]materialized_view_name REBUILD;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-14487) Add REBUILD statement for materialized views
[
https://issues.apache.org/jira/browse/HIVE-14487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-14487:
---
Status: Patch Available (was: In Progress)
> Add REBUILD statement for materialized views
>
>
> Key: HIVE-14487
> URL: https://issues.apache.org/jira/browse/HIVE-14487
> Project: Hive
> Issue Type: Sub-task
> Components: Materialized views
>Affects Versions: 2.2.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-14487.patch
>
>
> Support for rebuilding existing materialized views. The statement is the
> following:
> {code:sql}
> ALTER MATERIALIZED VIEW [db_name.]materialized_view_name REBUILD;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18077) Vectorization: Add string conversion case for UDFToDouble
[ https://issues.apache.org/jira/browse/HIVE-18077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18077: Status: Patch Available (was: In Progress) > Vectorization: Add string conversion case for UDFToDouble > - > > Key: HIVE-18077 > URL: https://issues.apache.org/jira/browse/HIVE-18077 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0 > > Attachments: HIVE-18077.01.patch, HIVE-18077.02.patch, > HIVE-18077.04.patch, HIVE-18077.05.patch, HIVE-18077.06.patch > > > Add string to float/double vectorization. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18077) Vectorization: Add string conversion case for UDFToDouble
[ https://issues.apache.org/jira/browse/HIVE-18077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18077: Attachment: HIVE-18077.06.patch > Vectorization: Add string conversion case for UDFToDouble > - > > Key: HIVE-18077 > URL: https://issues.apache.org/jira/browse/HIVE-18077 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0 > > Attachments: HIVE-18077.01.patch, HIVE-18077.02.patch, > HIVE-18077.04.patch, HIVE-18077.05.patch, HIVE-18077.06.patch > > > Add string to float/double vectorization. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18077) Vectorization: Add string conversion case for UDFToDouble
[ https://issues.apache.org/jira/browse/HIVE-18077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt McCline updated HIVE-18077: Status: In Progress (was: Patch Available) > Vectorization: Add string conversion case for UDFToDouble > - > > Key: HIVE-18077 > URL: https://issues.apache.org/jira/browse/HIVE-18077 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0 > > Attachments: HIVE-18077.01.patch, HIVE-18077.02.patch, > HIVE-18077.04.patch, HIVE-18077.05.patch > > > Add string to float/double vectorization. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17898) Explain plan output enhancement
[
https://issues.apache.org/jira/browse/HIVE-17898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256111#comment-16256111
]
Hive QA commented on HIVE-17898:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12898051/HIVE-17898.5.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 75 failed/errored test(s), 11388 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[auto_sortmerge_join_2]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[input4] (batchId=80)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[join0] (batchId=58)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parallel_join0]
(batchId=72)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_ppd_decimal]
(batchId=9)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_outer_join3]
(batchId=32)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_outer_join4]
(batchId=83)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_outer_join6]
(batchId=41)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[explainuser_2]
(batchId=147)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[llap_smb]
(batchId=148)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_smb_mapjoin_14]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_13]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_5]
(batchId=167)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_6]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucket_map_join_tez1]
(batchId=166)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[cluster]
(batchId=155)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[column_access_stats]
(batchId=161)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[correlationoptimizer2]
(batchId=159)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[correlationoptimizer3]
(batchId=166)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_semijoin_reduction_2]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[partition_shared_scan]
(batchId=152)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[smb_mapjoin_17]
(batchId=149)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_in]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_notin]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_select]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_join]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_empty]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_main]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_auto_smb_mapjoin_14]
(batchId=155)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[auto_sortmerge_join_16]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[infer_bucket_sort_map_operators]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[quotedid_smb]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=102)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[tez-tag]
(batchId=101)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_public_create]
(batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorize_grant_public]
(batchId=92)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_join32]
(batchId=141)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[auto_smb_mapjoin_14]
(batchId=129)
[jira] [Commented] (HIVE-16238) LLAP: reset/end has to be invoked for o.a.h.hive.q.io.orc.encoded.EncodedReaderImpl
[ https://issues.apache.org/jira/browse/HIVE-16238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256081#comment-16256081 ] Sergey Shelukhin commented on HIVE-16238: - This will be resolved with ORC upgrade to 1.4.0 that is pending > LLAP: reset/end has to be invoked for > o.a.h.hive.q.io.orc.encoded.EncodedReaderImpl > > > Key: HIVE-16238 > URL: https://issues.apache.org/jira/browse/HIVE-16238 > Project: Hive > Issue Type: Bug > Components: llap >Reporter: Rajesh Balamohan >Assignee: Sergey Shelukhin > > org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl has codec > reference which is never reset/end. This would cause gradual off-heap memory > increase in llap. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17631) upgrade orc to 1.4.0
[ https://issues.apache.org/jira/browse/HIVE-17631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-17631: Status: Patch Available (was: Open) > upgrade orc to 1.4.0 > > > Key: HIVE-17631 > URL: https://issues.apache.org/jira/browse/HIVE-17631 > Project: Hive > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Saijin Huang >Assignee: Sergey Shelukhin > Attachments: HIVE-17631.patch > > > It seems like orc 1.4.0 has a latest and stable version: > https://orc.apache.org/docs/releases.html -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17631) upgrade orc to 1.4.0
[ https://issues.apache.org/jira/browse/HIVE-17631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-17631: Attachment: HIVE-17631.patch [~prasanth_j] [~ashutoshc] can you take a look? > upgrade orc to 1.4.0 > > > Key: HIVE-17631 > URL: https://issues.apache.org/jira/browse/HIVE-17631 > Project: Hive > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Saijin Huang >Assignee: Sergey Shelukhin > Attachments: HIVE-17631.patch > > > It seems like orc 1.4.0 has a latest and stable version: > https://orc.apache.org/docs/releases.html -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18056) CachedStore: Have a whitelist/blacklist config to allow selective caching of tables/partitions and allow read while prewarming
[ https://issues.apache.org/jira/browse/HIVE-18056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-18056: Attachment: HIVE-18056.2.patch > CachedStore: Have a whitelist/blacklist config to allow selective caching of > tables/partitions and allow read while prewarming > -- > > Key: HIVE-18056 > URL: https://issues.apache.org/jira/browse/HIVE-18056 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > Attachments: HIVE-18056.1.patch, HIVE-18056.2.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18056) CachedStore: Have a whitelist/blacklist config to allow selective caching of tables/partitions and allow read while prewarming
[ https://issues.apache.org/jira/browse/HIVE-18056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vaibhav Gumashta updated HIVE-18056: Status: Patch Available (was: Open) > CachedStore: Have a whitelist/blacklist config to allow selective caching of > tables/partitions and allow read while prewarming > -- > > Key: HIVE-18056 > URL: https://issues.apache.org/jira/browse/HIVE-18056 > Project: Hive > Issue Type: Bug > Components: Metastore >Affects Versions: 3.0.0 >Reporter: Vaibhav Gumashta >Assignee: Vaibhav Gumashta > Attachments: HIVE-18056.1.patch, HIVE-18056.2.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18071) add HS2 jmx information about pools and current resource plan
[ https://issues.apache.org/jira/browse/HIVE-18071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256038#comment-16256038 ] Prasanth Jayachandran commented on HIVE-18071: -- +1 > add HS2 jmx information about pools and current resource plan > - > > Key: HIVE-18071 > URL: https://issues.apache.org/jira/browse/HIVE-18071 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-18071.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18025) Push resource plan changes to tez/unmanaged sessions
[ https://issues.apache.org/jira/browse/HIVE-18025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-18025: - Attachment: HIVE-18025.1.patch [~sershe] can you please take a look? > Push resource plan changes to tez/unmanaged sessions > > > Key: HIVE-18025 > URL: https://issues.apache.org/jira/browse/HIVE-18025 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-18025.1.patch > > > This is to remove MetastoreGlobalTriggersFetcher and make changes so that > DDLTask can push RP changes to tez session pool manager as well for updating > the triggers. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18025) Push resource plan changes to tez/unmanaged sessions
[ https://issues.apache.org/jira/browse/HIVE-18025?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prasanth Jayachandran updated HIVE-18025: - Status: Patch Available (was: Open) > Push resource plan changes to tez/unmanaged sessions > > > Key: HIVE-18025 > URL: https://issues.apache.org/jira/browse/HIVE-18025 > Project: Hive > Issue Type: Sub-task >Affects Versions: 3.0.0 >Reporter: Prasanth Jayachandran >Assignee: Prasanth Jayachandran > Attachments: HIVE-18025.1.patch > > > This is to remove MetastoreGlobalTriggersFetcher and make changes so that > DDLTask can push RP changes to tez session pool manager as well for updating > the triggers. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-15631) Optimize for hive client logs , you can filter the log for each session itself.
[
https://issues.apache.org/jira/browse/HIVE-15631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256028#comment-16256028
]
Hive QA commented on HIVE-15631:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12847533/HIVE_15631.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7864/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7864/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7864/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2017-11-16 21:58:16.698
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-7864/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2017-11-16 21:58:16.701
+ cd apache-github-source-source
+ git fetch origin
>From https://github.com/apache/hive
8d06b39..254e216 master -> origin/master
+ git reset --hard HEAD
HEAD is now at 8d06b39 HIVE-15344 : Spelling errors in logging and exceptions
for metastore and service directories (Grant Sohn via Ashutosh Chauhan)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is behind 'origin/master' by 1 commit, and can be fast-forwarded.
(use "git pull" to update your local branch)
+ git reset --hard origin/master
HEAD is now at 254e216 HIVE-15018: ALTER rewriting flag in materialized view
(Jesus Camacho Rodriguez, reviewed by Ashutosh Chauhan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2017-11-16 21:58:22.379
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh
/data/hiveptest/working/scratch/build.patch
error: patch failed:
ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java:51
error: ql/src/java/org/apache/hadoop/hive/ql/session/SessionState.java: patch
does not apply
error: patch failed:
common/src/java/org/apache/hadoop/hive/conf/HiveConf.java:3275
error: common/src/java/org/apache/hadoop/hive/conf/HiveConf.java: patch does
not apply
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12847533 - PreCommit-HIVE-Build
> Optimize for hive client logs , you can filter the log for each session
> itself.
> ---
>
> Key: HIVE-15631
> URL: https://issues.apache.org/jira/browse/HIVE-15631
> Project: Hive
> Issue Type: Improvement
> Components: CLI, Clients, Hive
>Reporter: tartarus
>Assignee: tartarus
> Attachments: HIVE_15631.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> We have several hadoop cluster, about 15 thousand nodes. Every day we use
> hive to submit above 100 thousand jobs.
> So we have a large file of hive logs on every client host every day, but i
> don not know the logs of my session submitted was which line.
> So i hope to print the hive.session.id on every line of logs, and then i
> could use grep to find the logs of my session submitted.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Assigned] (HIVE-17631) upgrade orc to 1.4.0
[ https://issues.apache.org/jira/browse/HIVE-17631?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin reassigned HIVE-17631: --- Assignee: Sergey Shelukhin (was: Saijin Huang) > upgrade orc to 1.4.0 > > > Key: HIVE-17631 > URL: https://issues.apache.org/jira/browse/HIVE-17631 > Project: Hive > Issue Type: Improvement >Affects Versions: 3.0.0 >Reporter: Saijin Huang >Assignee: Sergey Shelukhin > > It seems like orc 1.4.0 has a latest and stable version: > https://orc.apache.org/docs/releases.html -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-15440) Multiple inserts query from the union table failed by java.lang.IllegalArgumentException
[
https://issues.apache.org/jira/browse/HIVE-15440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16256024#comment-16256024
]
Hive QA commented on HIVE-15440:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12844231/HIVE-15440.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 12 failed/errored test(s), 11388 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[explainuser_2]
(batchId=147)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_semijoin_reduction_3]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_union_multiinsert]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=102)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorization_public_create]
(batchId=92)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[authorize_grant_public]
(batchId=92)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7861/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7861/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7861/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 12 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12844231 - PreCommit-HIVE-Build
> Multiple inserts query from the union table failed by
> java.lang.IllegalArgumentException
>
>
> Key: HIVE-15440
> URL: https://issues.apache.org/jira/browse/HIVE-15440
> Project: Hive
> Issue Type: Bug
>Affects Versions: 2.0.0
>Reporter: Aleksey Vovchenko
>Assignee: Aleksey Vovchenko
> Attachments: HIVE-15440.1.patch
>
>
> STEP 1. Configure Hive on Tez
> STEP2. Create test tables
> hive> create table t1(x int);
> hive> create table t2(x int);
> hive> create table t1_tmp(x int);
> hive> create table t2_tmp(x int);
> STEP 3. Run query
> hive> from(select x from t1 union all select x from t2)tbl insert overwrite
> table t1_tmp select x insert overwrite table t2_tmp select x;
> RESULT
> 2016-11-30 13:50:51,610 ERROR [main]: exec.Task (TezTask.java:execute(197)) -
> Failed to execute tez graph.
> java.lang.IllegalArgumentException: VertexGroup must have at least 2 members
> at org.apache.tez.dag.api.VertexGroup.(VertexGroup.java:77)
> at org.apache.tez.dag.api.DAG.createVertexGroup(DAG.java:177)
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.build(TezTask.java:333)
> at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:169)
> at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
> at
> org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85)
> at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1635)
> at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1395)
> at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1208)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1035)
> at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1025)
> at
> org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:201)
> at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:153)
> at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:364)
> at
> org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:712)
> at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:631)
> at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:570)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
[jira] [Updated] (HIVE-18087) Simple select query finds nothing when non-native table has partition columns
[ https://issues.apache.org/jira/browse/HIVE-18087?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Olson updated HIVE-18087: Description: A non-native table can be created with partition columns defined. However, the existence of partition columns for a non-native table is problematic. Nothing disallows the table creation, and the documentation [1] does not mention that non-native tables cannot have partition columns. In fact, it suggests that "PARTITIONED BY" can be specified. With this table definition, when running a query that does not launch a MR job, the query completes successfully and immediately, but no rows are ever returned. I have not yet been able to identify the specific code logic path that prevents it from working as expected, perhaps something in the query plan related to partition pruning. This bug is similar to HIVE-18086. We see this issue with a query like "select \* from my_table;" and the HIVE-18086 NullPointerException with a query like "select count(\*) from my_table;". We encountered this bug using the HBaseStorageHandler. However, enabling debug logging suggests that the query is being short-circuited without actually connecting to HBase at all. [1] https://cwiki.apache.org/confluence/display/Hive/StorageHandlers#StorageHandlers-DDL was: A non-native table can be created with partition columns defined. However, the existence of partition columns for a non-native table is problematic. Nothing disallows the table creation, and the documentation [1] does not mention that non-native tables cannot have partition columns. In fact, it suggests that "PARTITIONED BY" can be specified. With this table definition, when running a query that does not launch a MR job, the query completes successfully and immediately, but no rows are ever returned. I have not yet been able to identify the specific code logic path that prevents it from working as expected, perhaps something in the query plan related to partition pruning. This bug is similar to HIVE-18086. We see this issue with a query like "select * from my_table;" and the HIVE-18086 NullPointerException with a query like "select count(*) from my_table;". We encountered this bug using the HBaseStorageHandler. However, enabling debug logging suggests that the query is being short-circuited without actually connecting to HBase at all. [1] https://cwiki.apache.org/confluence/display/Hive/StorageHandlers#StorageHandlers-DDL > Simple select query finds nothing when non-native table has partition columns > - > > Key: HIVE-18087 > URL: https://issues.apache.org/jira/browse/HIVE-18087 > Project: Hive > Issue Type: Bug > Components: Query Processor > Environment: CDH 5.5.5 on RHEL 6.8 >Reporter: Andrew Olson > > A non-native table can be created with partition columns defined. However, > the existence of partition columns for a non-native table is problematic. > Nothing disallows the table creation, and the documentation [1] does not > mention that non-native tables cannot have partition columns. In fact, it > suggests that "PARTITIONED BY" can be specified. > With this table definition, when running a query that does not launch a MR > job, the query completes successfully and immediately, but no rows are ever > returned. I have not yet been able to identify the specific code logic path > that prevents it from working as expected, perhaps something in the query > plan related to partition pruning. > This bug is similar to HIVE-18086. We see this issue with a query like > "select \* from my_table;" and the HIVE-18086 NullPointerException with a > query like "select count(\*) from my_table;". > We encountered this bug using the HBaseStorageHandler. However, enabling > debug logging suggests that the query is being short-circuited without > actually connecting to HBase at all. > [1] > https://cwiki.apache.org/confluence/display/Hive/StorageHandlers#StorageHandlers-DDL -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18072) WM - fix various bugs based on cluster testing - part 2
[ https://issues.apache.org/jira/browse/HIVE-18072?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Shelukhin updated HIVE-18072: Attachment: HIVE-18072.01.patch Updated > WM - fix various bugs based on cluster testing - part 2 > --- > > Key: HIVE-18072 > URL: https://issues.apache.org/jira/browse/HIVE-18072 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-18072.01.patch, HIVE-18072.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-15911) Create a view based on another view throws an exception “FAILED: NullPointerException null”
[ https://issues.apache.org/jira/browse/HIVE-15911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16255975#comment-16255975 ] Ashutosh Chauhan commented on HIVE-15911: - [~yangfang] I am not able to repro this issue on latest master with test case you have added. > Create a view based on another view throws an exception “FAILED: > NullPointerException null” > --- > > Key: HIVE-15911 > URL: https://issues.apache.org/jira/browse/HIVE-15911 > Project: Hive > Issue Type: Bug > Components: Views >Affects Versions: 2.1.0, 2.1.1 > Environment: hive2.1.0 >Reporter: yangfang >Assignee: yangfang > Attachments: 0001-Hive-15911.patch > > > when I create a new view based on another view I an exception “FAILED: > NullPointerException null”: > hive> create view view2(a,b) as select a, b from view1; //view1 is another > view > FAILED: NullPointerException null > hive> > The hive log show error stack： > 2017-02-15T15:40:25,816 ERROR ql.Driver (SessionState.java:printError(1116)) > - FAILED: NullPointerException null > java.lang.NullPointerException > at org.apache.hadoop.hive.ql.Driver.doAuthorization(Driver.java:863) > at org.apache.hadoop.hive.ql.Driver.compile(Driver.java:552) > at org.apache.hadoop.hive.ql.Driver.compileInternal(Driver.java:1319) > at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1459) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1239) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1229) > at > org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:233) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:184) > at > org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:403) > at > org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:821) > at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:759) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at org.apache.hadoop.util.RunJar.run(RunJar.java:221) > at org.apache.hadoop.util.RunJar.main(RunJar.java:136) -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-15018) ALTER rewriting flag in materialized view
[
https://issues.apache.org/jira/browse/HIVE-15018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jesus Camacho Rodriguez updated HIVE-15018:
---
Resolution: Fixed
Fix Version/s: 3.0.0
Status: Resolved (was: Patch Available)
Pushed to master, thanks for reviewing [~ashutoshc]!
> ALTER rewriting flag in materialized view
> --
>
> Key: HIVE-15018
> URL: https://issues.apache.org/jira/browse/HIVE-15018
> Project: Hive
> Issue Type: Sub-task
> Components: Materialized views
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Fix For: 3.0.0
>
> Attachments: HIVE-15018.01.patch, HIVE-15018.patch
>
>
> We should extend the ALTER statement in case we want to change the rewriting
> behavior of the materialized view after we have created it.
> {code:sql}
> ALTER MATERIALIZED VIEW [db_name.]materialized_view_name DISABLE REWRITE;
> {code}
> {code:sql}
> ALTER MATERIALIZED VIEW [db_name.]materialized_view_name ENABLE REWRITE;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-15631) Optimize for hive client logs , you can filter the log for each session itself.
[
https://issues.apache.org/jira/browse/HIVE-15631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16255957#comment-16255957
]
Prasanth Jayachandran commented on HIVE-15631:
--
I would actually recommend setting sessionId in MDC context (and cleanup when
session is closed). With MDC, sessionId can be included in the pattern layout
use {X} option or the MDC key. Also adding it to MDC has benefit of using
routing appender to redirect log lines corresponding to a specific session to
its own files. MDC is used by operation logging and LLAP to redirect
query/session specific log lines to separate files. We could use that for
hive.log as well.
> Optimize for hive client logs , you can filter the log for each session
> itself.
> ---
>
> Key: HIVE-15631
> URL: https://issues.apache.org/jira/browse/HIVE-15631
> Project: Hive
> Issue Type: Improvement
> Components: CLI, Clients, Hive
>Reporter: tartarus
>Assignee: tartarus
> Attachments: HIVE_15631.patch
>
> Original Estimate: 24h
> Remaining Estimate: 24h
>
> We have several hadoop cluster, about 15 thousand nodes. Every day we use
> hive to submit above 100 thousand jobs.
> So we have a large file of hive logs on every client host every day, but i
> don not know the logs of my session submitted was which line.
> So i hope to print the hive.session.id on every line of logs, and then i
> could use grep to find the logs of my session submitted.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18054) Make Lineage work with concurrent queries on a Session
[
https://issues.apache.org/jira/browse/HIVE-18054?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16255950#comment-16255950
]
Hive QA commented on HIVE-18054:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12898017/HIVE-18054.4.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 27 failed/errored test(s), 11389 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[ctas_blobstore_to_blobstore]
(batchId=246)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[ctas_hdfs_to_blobstore]
(batchId=246)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_blobstore_to_blobstore]
(batchId=246)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_empty_into_blobstore]
(batchId=246)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_into_table]
(batchId=246)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_directory]
(batchId=246)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_move]
(batchId=246)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_merge_only]
(batchId=246)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_dynamic_partitions_move_only]
(batchId=246)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[insert_overwrite_table]
(batchId=246)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[orc_format_nonpart]
(batchId=246)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[orc_format_part]
(batchId=246)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[orc_nonstd_partitions_loc]
(batchId=246)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[rcfile_format_nonpart]
(batchId=246)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[rcfile_format_part]
(batchId=246)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[rcfile_nonstd_partitions_loc]
(batchId=246)
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[zero_rows_blobstore]
(batchId=246)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[index_auto_update]
(batchId=72)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[insert_with_move_files_from_source_dir]
(batchId=83)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestSparkPerfCliDriver.testCliDriver[query39]
(batchId=245)
org.apache.hadoop.hive.ql.optimizer.TestGenMapRedUtilsCreateConditionalTask.testConditionalMoveTaskIsOptimized
(batchId=274)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7859/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7859/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7859/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 27 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12898017 - PreCommit-HIVE-Build
> Make Lineage work with concurrent queries on a Session
> ---
>
> Key: HIVE-18054
> URL: https://issues.apache.org/jira/browse/HIVE-18054
> Project: Hive
> Issue Type: Bug
>Reporter: Andrew Sherman
>Assignee: Andrew Sherman
> Attachments: HIVE-18054.1.patch, HIVE-18054.2.patch,
> HIVE-18054.3.patch, HIVE-18054.4.patch
>
>
> A Hive Session can contain multiple concurrent sql Operations.
> Lineage is currently tracked in SessionState and is cleared when a query
> completes. This results in Lineage for other running queries being lost.
> To fix this, move LineageState from SessionState to QueryState.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-17898) Explain plan output enhancement
[ https://issues.apache.org/jira/browse/HIVE-17898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17898: --- Status: Patch Available (was: Open) > Explain plan output enhancement > --- > > Key: HIVE-17898 > URL: https://issues.apache.org/jira/browse/HIVE-17898 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-17898.1.patch, HIVE-17898.2.patch, > HIVE-17898.3.patch, HIVE-17898.4.patch, HIVE-17898.5.patch > > > We would like to enhance the explain plan output to display additional > information e.g.: > TableScan operator should have following additional info > * Actual table name (currently only alias name is displayed) > * Database name > * Column names being scanned -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17898) Explain plan output enhancement
[ https://issues.apache.org/jira/browse/HIVE-17898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17898: --- Attachment: HIVE-17898.5.patch > Explain plan output enhancement > --- > > Key: HIVE-17898 > URL: https://issues.apache.org/jira/browse/HIVE-17898 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-17898.1.patch, HIVE-17898.2.patch, > HIVE-17898.3.patch, HIVE-17898.4.patch, HIVE-17898.5.patch > > > We would like to enhance the explain plan output to display additional > information e.g.: > TableScan operator should have following additional info > * Actual table name (currently only alias name is displayed) > * Database name > * Column names being scanned -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17898) Explain plan output enhancement
[ https://issues.apache.org/jira/browse/HIVE-17898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17898: --- Status: Open (was: Patch Available) > Explain plan output enhancement > --- > > Key: HIVE-17898 > URL: https://issues.apache.org/jira/browse/HIVE-17898 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-17898.1.patch, HIVE-17898.2.patch, > HIVE-17898.3.patch, HIVE-17898.4.patch > > > We would like to enhance the explain plan output to display additional > information e.g.: > TableScan operator should have following additional info > * Actual table name (currently only alias name is displayed) > * Database name > * Column names being scanned -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-18086) NullPointerException initializing query job when non-native table has partition columns
[
https://issues.apache.org/jira/browse/HIVE-18086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Olson updated HIVE-18086:
Description:
A non-native table can be created with partition columns defined. However, the
existence of partition columns for a non-native table is problematic. Nothing
disallows the table creation, and the documentation [1] does not mention that
non-native tables cannot have partition columns. In fact, it suggests that
"PARTITIONED BY" can be specified.
When launching the MR job for any query a NullPointerException is thrown. A
null path value originates in
{{org.apache.hadoop.hive.ql.exec.Utilities#createDummyFileForEmptyTable(...)}}
because "if this isn't a hive table we can't create an empty file for it" and
is added to the list of input paths that the
{{org.apache.hadoop.hive.ql.exec.mr.ExecDriver}} provides to the
FileInputFormat, ultimately causing the NPE in the Path constructor because the
child path is null.
{noformat}
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
java.lang.NullPointerException
at org.apache.hadoop.fs.Path.(Path.java:115)
at
org.apache.hadoop.mapred.FileInputFormat.setInputPaths(FileInputFormat.java:438)
at
org.apache.hadoop.hive.ql.exec.Utilities.setInputPaths(Utilities.java:3550)
at
org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:371)
at
org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:137)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1690)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1449)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1235)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1053)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1043)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:209)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:161)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:372)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:759)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:617)
{noformat}
We encountered this bug using the HBaseStorageHandler.
[1]
https://cwiki.apache.org/confluence/display/Hive/StorageHandlers#StorageHandlers-DDL
was:
A non-native table can be created with partition columns defined. However, the
existence of partition columns for a non-native table is problematic. Nothing
disallows the table creation, and the documentation [1] does not mention that
non-native tables cannot have partition columns. In fact, it suggests that
PARTITIONED BY can be specified.
When launching the MR job for any query a NullPointerException is thrown. A
null path value originates in
{{org.apache.hadoop.hive.ql.exec.Utilities#createDummyFileForEmptyTable(...)}}
because "if this isn't a hive table we can't create an empty file for it" and
is added to the list of input paths that the
org.apache.hadoop.hive.ql.exec.mr.ExecDriver provides to the FileInputFormat,
ultimately causing the NPE in the Path constructor because the child path is
null.
{noformat}
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
java.lang.NullPointerException
at org.apache.hadoop.fs.Path.(Path.java:115)
at
org.apache.hadoop.mapred.FileInputFormat.setInputPaths(FileInputFormat.java:438)
at
org.apache.hadoop.hive.ql.exec.Utilities.setInputPaths(Utilities.java:3550)
at
org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:371)
at
org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:137)
at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160)
at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1690)
at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1449)
at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1235)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1053)
at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1043)
at
org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:209)
at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:161)
at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:372)
at
org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:759)
at
[jira] [Commented] (HIVE-15631) Optimize for hive client logs , you can filter the log for each session itself.
[ https://issues.apache.org/jira/browse/HIVE-15631?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16255921#comment-16255921 ] Ashutosh Chauhan commented on HIVE-15631: - [~prasanth_j] [~thejas] this seems pretty useful for debugging. > Optimize for hive client logs , you can filter the log for each session > itself. > --- > > Key: HIVE-15631 > URL: https://issues.apache.org/jira/browse/HIVE-15631 > Project: Hive > Issue Type: Improvement > Components: CLI, Clients, Hive >Reporter: tartarus >Assignee: tartarus > Attachments: HIVE_15631.patch > > Original Estimate: 24h > Remaining Estimate: 24h > > We have several hadoop cluster, about 15 thousand nodes. Every day we use > hive to submit above 100 thousand jobs. > So we have a large file of hive logs on every client host every day, but i > don not know the logs of my session submitted was which line. > So i hope to print the hive.session.id on every line of logs, and then i > could use grep to find the logs of my session submitted. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-15018) ALTER rewriting flag in materialized view
[
https://issues.apache.org/jira/browse/HIVE-15018?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16255910#comment-16255910
]
Ashutosh Chauhan commented on HIVE-15018:
-
+1
> ALTER rewriting flag in materialized view
> --
>
> Key: HIVE-15018
> URL: https://issues.apache.org/jira/browse/HIVE-15018
> Project: Hive
> Issue Type: Sub-task
> Components: Materialized views
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-15018.01.patch, HIVE-15018.patch
>
>
> We should extend the ALTER statement in case we want to change the rewriting
> behavior of the materialized view after we have created it.
> {code:sql}
> ALTER MATERIALIZED VIEW [db_name.]materialized_view_name DISABLE REWRITE;
> {code}
> {code:sql}
> ALTER MATERIALIZED VIEW [db_name.]materialized_view_name ENABLE REWRITE;
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-18072) WM - fix various bugs based on cluster testing - part 2
[ https://issues.apache.org/jira/browse/HIVE-18072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16255892#comment-16255892 ] Sergey Shelukhin commented on HIVE-18072: - Hmm, I cannot repro most of these.. looks like all cluster tests OOMed in various ways and lots of minor tests timed out. I'll see what I can repro and fix that, then do another run. > WM - fix various bugs based on cluster testing - part 2 > --- > > Key: HIVE-18072 > URL: https://issues.apache.org/jira/browse/HIVE-18072 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-18072.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-15344) Spelling errors in logging and exceptions for metastore and service directories
[ https://issues.apache.org/jira/browse/HIVE-15344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-15344: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Pushed to master. Thanks, Grant! > Spelling errors in logging and exceptions for metastore and service > directories > --- > > Key: HIVE-15344 > URL: https://issues.apache.org/jira/browse/HIVE-15344 > Project: Hive > Issue Type: Bug > Components: CLI, Metastore, Server Infrastructure >Reporter: Grant Sohn >Assignee: Grant Sohn >Priority: Trivial > Fix For: 3.0.0 > > Attachments: HIVE-15344.1.patch > > > More spelling errors in logging and exception messages. > verififcation -> verification > indexxes -> indexes > resouce -> resource -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18072) WM - fix various bugs based on cluster testing - part 2
[ https://issues.apache.org/jira/browse/HIVE-18072?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16255860#comment-16255860 ] Prasanth Jayachandran commented on HIVE-18072: -- Left minor comments in RB > WM - fix various bugs based on cluster testing - part 2 > --- > > Key: HIVE-18072 > URL: https://issues.apache.org/jira/browse/HIVE-18072 > Project: Hive > Issue Type: Sub-task >Reporter: Sergey Shelukhin >Assignee: Sergey Shelukhin > Attachments: HIVE-18072.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-15344) Spelling errors in logging and exceptions for metastore and service directories
[ https://issues.apache.org/jira/browse/HIVE-15344?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16255859#comment-16255859 ] Ashutosh Chauhan commented on HIVE-15344: - +1 > Spelling errors in logging and exceptions for metastore and service > directories > --- > > Key: HIVE-15344 > URL: https://issues.apache.org/jira/browse/HIVE-15344 > Project: Hive > Issue Type: Bug > Components: CLI, Metastore, Server Infrastructure >Reporter: Grant Sohn >Assignee: Grant Sohn >Priority: Trivial > Attachments: HIVE-15344.1.patch > > > More spelling errors in logging and exception messages. > verififcation -> verification > indexxes -> indexes > resouce -> resource -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17714) move custom SerDe schema considerations into metastore from QL
[ https://issues.apache.org/jira/browse/HIVE-17714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16255855#comment-16255855 ] Sergey Shelukhin commented on HIVE-17714: - Initial motivation for HIVE-11985 was the field length, which is not a problem anymore (at least some fields were converted to text). However, as per the above another reason not to store is the potential inconsistency. There are 2 further concerns: * [~owen.omalley] appears to have -1d the store-recreated-schema approach above; and [~alangates] also doesn't like it. * In order to recreate the schema, you need the SerDe :) Otherwise how would you know the schema from the SerDe? Unless you are willing to rely on user who creates the table to re-create it for you and send you the correct schema (which you won't be able to validate either, without the SerDe). Which is even more brittle than just re-creating it. > move custom SerDe schema considerations into metastore from QL > -- > > Key: HIVE-17714 > URL: https://issues.apache.org/jira/browse/HIVE-17714 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Alan Gates > > Columns in metastore for tables that use external schema don't have the type > information (since HIVE-11985) and may be entirely inconsistent (since > forever, due to issues like HIVE-17713; or for SerDes that allow an URL for > the schema, due to a change in the underlying file). > Currently, if you trace the usage of ConfVars.SERDESUSINGMETASTOREFORSCHEMA, > and to MetaStoreUtils.getFieldsFromDeserializer, you'd see that the code in > QL handles this in Hive. So, for the most part metastore just returns > whatever is stored for columns in the database. > One exception appears to be get_fields_with_environment_context, which is > interesting... so getTable will return incorrect columns (potentially), but > get_fields/get_schema will return correct ones from SerDe as far as I can > tell. > As part of separating the metastore, we should make sure all the APIs return > the correct schema for the columns; it's not a good idea to have everyone > reimplement getFieldsFromDeserializer. > Note: this should also remove a flag introduced in HIVE-17731 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18085) Run checkstyle on storage-api module with proper configuration
[
https://issues.apache.org/jira/browse/HIVE-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16255851#comment-16255851
]
Hive QA commented on HIVE-18085:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12898009/HIVE-18085.0.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 7 failed/errored test(s), 11388 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=102)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7858/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7858/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7858/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 7 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12898009 - PreCommit-HIVE-Build
> Run checkstyle on storage-api module with proper configuration
> --
>
> Key: HIVE-18085
> URL: https://issues.apache.org/jira/browse/HIVE-18085
> Project: Hive
> Issue Type: Sub-task
>Reporter: Adam Szita
>Assignee: Adam Szita
> Attachments: HIVE-18085.0.patch
>
>
> Module storage-api is disconnected from the root Hive pom, so we need to add
> the proper plugin configuration.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-16756) Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: / by zero"
[ https://issues.apache.org/jira/browse/HIVE-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16255812#comment-16255812 ] Vihang Karajgaonkar commented on HIVE-16756: +1 I would merging this change towards the EOD unless there are more review comments. > Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: > / by zero" > > > Key: HIVE-16756 > URL: https://issues.apache.org/jira/browse/HIVE-16756 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.3.0 >Reporter: Matt McCline >Assignee: Vihang Karajgaonkar >Priority: Critical > Attachments: HIVE-16756.01.patch, HIVE-16756.02.patch, > HIVE-16756.03.patch > > > vectorization_div0.q needs to test the long data type testing. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17942) HiveAlterHandler not using conf from HMS Handler
[ https://issues.apache.org/jira/browse/HIVE-17942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Janaki Lahorani updated HIVE-17942: --- Attachment: HIVE-17942-branch-2.01.patch > HiveAlterHandler not using conf from HMS Handler > > > Key: HIVE-17942 > URL: https://issues.apache.org/jira/browse/HIVE-17942 > Project: Hive > Issue Type: Bug >Affects Versions: 2.1.1 >Reporter: Janaki Lahorani >Assignee: Janaki Lahorani > Fix For: 3.0.0 > > Attachments: HIVE-17942-branch-2.01.patch, HIVE-17942.1.patch, > HIVE-17942.2.patch, HIVE-17942.3.patch, HIVE-17942.4.patch, HIVE-17942.5.patch > > > When HiveAlterHandler looks for conf, it is not getting the one from thread > local. So, local changes are not visible. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18077) Vectorization: Add string conversion case for UDFToDouble
[
https://issues.apache.org/jira/browse/HIVE-18077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16255752#comment-16255752
]
Hive QA commented on HIVE-18077:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12898007/HIVE-18077.05.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 11388 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBlobstoreCliDriver.testCliDriver[buckets]
(batchId=246)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[unionDistinct_1]
(batchId=146)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_vectorized_dynamic_partition_pruning]
(batchId=173)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=102)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.testReadDataFromEncryptedHiveTableByHCatMR[1]
(batchId=187)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7857/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7857/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7857/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12898007 - PreCommit-HIVE-Build
> Vectorization: Add string conversion case for UDFToDouble
> -
>
> Key: HIVE-18077
> URL: https://issues.apache.org/jira/browse/HIVE-18077
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Fix For: 3.0.0
>
> Attachments: HIVE-18077.01.patch, HIVE-18077.02.patch,
> HIVE-18077.04.patch, HIVE-18077.05.patch
>
>
> Add string to float/double vectorization.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (HIVE-15440) Multiple inserts query from the union table failed by java.lang.IllegalArgumentException
[ https://issues.apache.org/jira/browse/HIVE-15440?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16255740#comment-16255740 ] Ashutosh Chauhan commented on HIVE-15440: - [~AleKsey Vovchenko] I can't repro this error any more on latest master. Are you still able to repro this on latest master? > Multiple inserts query from the union table failed by > java.lang.IllegalArgumentException > > > Key: HIVE-15440 > URL: https://issues.apache.org/jira/browse/HIVE-15440 > Project: Hive > Issue Type: Bug >Affects Versions: 2.0.0 >Reporter: Aleksey Vovchenko >Assignee: Aleksey Vovchenko > Attachments: HIVE-15440.1.patch > > > STEP 1. Configure Hive on Tez > STEP2. Create test tables > hive> create table t1(x int); > hive> create table t2(x int); > hive> create table t1_tmp(x int); > hive> create table t2_tmp(x int); > STEP 3. Run query > hive> from(select x from t1 union all select x from t2)tbl insert overwrite > table t1_tmp select x insert overwrite table t2_tmp select x; > RESULT > 2016-11-30 13:50:51,610 ERROR [main]: exec.Task (TezTask.java:execute(197)) - > Failed to execute tez graph. > java.lang.IllegalArgumentException: VertexGroup must have at least 2 members > at org.apache.tez.dag.api.VertexGroup.(VertexGroup.java:77) > at org.apache.tez.dag.api.DAG.createVertexGroup(DAG.java:177) > at org.apache.hadoop.hive.ql.exec.tez.TezTask.build(TezTask.java:333) > at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:169) > at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:160) > at > org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:85) > at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1635) > at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1395) > at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1208) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1035) > at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1025) > at > org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:201) > at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:153) > at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:364) > at > org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:712) > at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:631) > at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:570) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:483) > at org.apache.hadoop.util.RunJar.main(RunJar.java:212) > This issue doesn't happen on MR mode. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-15343) Spelling errors in logging and exceptions for beeline, common, hbase-handler, hcatalog, llap-server, orc, serde and shims
[ https://issues.apache.org/jira/browse/HIVE-15343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ashutosh Chauhan updated HIVE-15343: Resolution: Fixed Fix Version/s: 3.0.0 Status: Resolved (was: Patch Available) Pushed to master. Thanks, Grant! > Spelling errors in logging and exceptions for beeline, common, hbase-handler, > hcatalog, llap-server, orc, serde and shims > - > > Key: HIVE-15343 > URL: https://issues.apache.org/jira/browse/HIVE-15343 > Project: Hive > Issue Type: Bug > Components: Beeline, HBase Handler, HCatalog, llap, ORC, > Serializers/Deserializers, Shims >Reporter: Grant Sohn >Assignee: Grant Sohn >Priority: Trivial > Fix For: 3.0.0 > > Attachments: HIVE-15343.1.patch > > > There are a set of misspelled words in the logs and exceptions. > Wtaited -> Waited > prioroty -> priority -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-15343) Spelling errors in logging and exceptions for beeline, common, hbase-handler, hcatalog, llap-server, orc, serde and shims
[ https://issues.apache.org/jira/browse/HIVE-15343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16255674#comment-16255674 ] Ashutosh Chauhan commented on HIVE-15343: - +1 > Spelling errors in logging and exceptions for beeline, common, hbase-handler, > hcatalog, llap-server, orc, serde and shims > - > > Key: HIVE-15343 > URL: https://issues.apache.org/jira/browse/HIVE-15343 > Project: Hive > Issue Type: Bug > Components: Beeline, HBase Handler, HCatalog, llap, ORC, > Serializers/Deserializers, Shims >Reporter: Grant Sohn >Assignee: Grant Sohn >Priority: Trivial > Attachments: HIVE-15343.1.patch > > > There are a set of misspelled words in the logs and exceptions. > Wtaited -> Waited > prioroty -> priority -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18084) Upgrade checkstyle version to support lambdas
[
https://issues.apache.org/jira/browse/HIVE-18084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16255634#comment-16255634
]
Hive QA commented on HIVE-18084:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12898005/HIVE-18084.0.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 11360 tests
executed
*Failed tests:*
{noformat}
TestMiniLlapCliDriver - did not produce a TEST-*.xml file (likely timed out)
(batchId=146)
[intersect_all.q,unionDistinct_1.q,orc_ppd_schema_evol_3a.q,table_nonprintable.q,tez_union_dynamic_partition.q,tez_union_dynamic_partition_2.q,temp_table_external.q,global_limit.q,llap_udf.q,schemeAuthority.q,cte_2.q,rcfile_createas1.q,dynamic_partition_pruning_2.q,intersect_merge.q,parallel_colstats.q]
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[dbtxnmgr_showlocks]
(batchId=77)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[insert_values_orig_table_use_metadata]
(batchId=162)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[ppd_union_view]
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=156)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_2]
(batchId=102)
org.apache.hadoop.hive.ql.TestTxnCommands.testMergeOnTezEdges (batchId=288)
org.apache.hadoop.hive.ql.parse.TestReplicationScenarios.testConstraints
(batchId=223)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc
(batchId=244)
{noformat}
Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/7856/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/7856/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-7856/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12898005 - PreCommit-HIVE-Build
> Upgrade checkstyle version to support lambdas
> -
>
> Key: HIVE-18084
> URL: https://issues.apache.org/jira/browse/HIVE-18084
> Project: Hive
> Issue Type: Sub-task
>Reporter: Adam Szita
>Assignee: Adam Szita
> Attachments: HIVE-18084.0.patch, diff.txt
>
>
> Current version does not support lambdas in source files so it skips them. We
> need to upgrade checkstyle version to fix this.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (HIVE-18054) Make Lineage work with concurrent queries on a Session
[ https://issues.apache.org/jira/browse/HIVE-18054?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Sherman updated HIVE-18054: -- Attachment: HIVE-18054.4.patch > Make Lineage work with concurrent queries on a Session > --- > > Key: HIVE-18054 > URL: https://issues.apache.org/jira/browse/HIVE-18054 > Project: Hive > Issue Type: Bug >Reporter: Andrew Sherman >Assignee: Andrew Sherman > Attachments: HIVE-18054.1.patch, HIVE-18054.2.patch, > HIVE-18054.3.patch, HIVE-18054.4.patch > > > A Hive Session can contain multiple concurrent sql Operations. > Lineage is currently tracked in SessionState and is cleared when a query > completes. This results in Lineage for other running queries being lost. > To fix this, move LineageState from SessionState to QueryState. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-17714) move custom SerDe schema considerations into metastore from QL
[ https://issues.apache.org/jira/browse/HIVE-17714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16255584#comment-16255584 ] Vihang Karajgaonkar commented on HIVE-17714: [~sershe] Do you have any further thoughts or comments? If my understanding stated above is correct, is it reasonable to revert HIVE-11985 (fully or partially) instead of adding Deserializer usage on all the HMS APIs a better approach? > move custom SerDe schema considerations into metastore from QL > -- > > Key: HIVE-17714 > URL: https://issues.apache.org/jira/browse/HIVE-17714 > Project: Hive > Issue Type: Bug >Reporter: Sergey Shelukhin >Assignee: Alan Gates > > Columns in metastore for tables that use external schema don't have the type > information (since HIVE-11985) and may be entirely inconsistent (since > forever, due to issues like HIVE-17713; or for SerDes that allow an URL for > the schema, due to a change in the underlying file). > Currently, if you trace the usage of ConfVars.SERDESUSINGMETASTOREFORSCHEMA, > and to MetaStoreUtils.getFieldsFromDeserializer, you'd see that the code in > QL handles this in Hive. So, for the most part metastore just returns > whatever is stored for columns in the database. > One exception appears to be get_fields_with_environment_context, which is > interesting... so getTable will return incorrect columns (potentially), but > get_fields/get_schema will return correct ones from SerDe as far as I can > tell. > As part of separating the metastore, we should make sure all the APIs return > the correct schema for the columns; it's not a good idea to have everyone > reimplement getFieldsFromDeserializer. > Note: this should also remove a flag introduced in HIVE-17731 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-16756) Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: / by zero"
[ https://issues.apache.org/jira/browse/HIVE-16756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16255563#comment-16255563 ] Vihang Karajgaonkar commented on HIVE-16756: Test failures are known flaky tests. > Vectorization: LongColModuloLongColumn throws "java.lang.ArithmeticException: > / by zero" > > > Key: HIVE-16756 > URL: https://issues.apache.org/jira/browse/HIVE-16756 > Project: Hive > Issue Type: Bug > Components: Hive >Affects Versions: 2.3.0 >Reporter: Matt McCline >Assignee: Vihang Karajgaonkar >Priority: Critical > Attachments: HIVE-16756.01.patch, HIVE-16756.02.patch, > HIVE-16756.03.patch > > > vectorization_div0.q needs to test the long data type testing. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HIVE-14745) Remove jira user/password from profiles by using another command to submit results to jira
[ https://issues.apache.org/jira/browse/HIVE-14745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Szita reassigned HIVE-14745: - Assignee: Adam Szita (was: Barna Zsombor Klara) > Remove jira user/password from profiles by using another command to submit > results to jira > -- > > Key: HIVE-14745 > URL: https://issues.apache.org/jira/browse/HIVE-14745 > Project: Hive > Issue Type: Sub-task > Components: Hive, Testing Infrastructure >Reporter: Sergio Peña >Assignee: Adam Szita > > Hive ptest uses some properties files per branch that contain information > about how to execute the tests. > This profile includes the user & password to submit the results to JIRA. We > should get rid of this sensitive information from the profile by moving the > jira submission task to another command or script executed directly by > Jenkins. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (HIVE-18077) Vectorization: Add string conversion case for UDFToDouble
[ https://issues.apache.org/jira/browse/HIVE-18077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1629#comment-1629 ] Teddy Choi commented on HIVE-18077: --- +1 pending tests. Looks good to me. :) > Vectorization: Add string conversion case for UDFToDouble > - > > Key: HIVE-18077 > URL: https://issues.apache.org/jira/browse/HIVE-18077 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Fix For: 3.0.0 > > Attachments: HIVE-18077.01.patch, HIVE-18077.02.patch, > HIVE-18077.04.patch, HIVE-18077.05.patch > > > Add string to float/double vectorization. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
