[jira] [Commented] (HIVE-19353) Vectorization: ConstantVectorExpression --> RuntimeException: Unexpected column vector type LIST
[
https://issues.apache.org/jira/browse/HIVE-19353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16458133#comment-16458133
]
Vineet Garg commented on HIVE-19353:
Can we add a negative test? Also if it not necessary can you move
{{vector_constant}} from {{minillap.shared.query.files}} to
{{minillap.query.files}}?
> Vectorization: ConstantVectorExpression --> RuntimeException: Unexpected
> column vector type LIST
> -
>
> Key: HIVE-19353
> URL: https://issues.apache.org/jira/browse/HIVE-19353
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-19353.01.patch
>
>
> Found by enabling vectorization for
> org.apache.hive.jdbc.TestJdbcDriver2.testResultSetMetaData
> {noformat}
> Caused by: java.lang.RuntimeException: Unexpected column vector type LIST
> at
> org.apache.hadoop.hive.ql.exec.vector.expressions.ConstantVectorExpression.evaluate(ConstantVectorExpression.java:237)
> ~[hive-exec-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorSelectOperator.process(VectorSelectOperator.java:146)
> ~[hive-exec-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.Operator.vectorForward(Operator.java:955)
> ~[hive-exec-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:928)
> ~[hive-exec-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.TableScanOperator.process(TableScanOperator.java:125)
> ~[hive-exec-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.vector.VectorMapOperator.closeOp(VectorMapOperator.java:984)
> ~[hive-exec-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:722)
> ~[hive-exec-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]
> at
> org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:193)
> ~[hive-exec-3.1.0-SNAPSHOT.jar:3.1.0-SNAPSHOT]{noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-19346) TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_create_rewrite_5] failling
[
https://issues.apache.org/jira/browse/HIVE-19346?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg reassigned HIVE-19346:
--
> TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_create_rewrite_5]
> failling
> -
>
> Key: HIVE-19346
> URL: https://issues.apache.org/jira/browse/HIVE-19346
> Project: Hive
> Issue Type: Sub-task
>Reporter: Vineet Garg
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> {noformat}
> Error Message
> Client Execution succeeded but contained differences (error code = 1) after
> executing materialized_view_create_rewrite_5.q
> 402c402
> < totalSize1053
> ---
> > totalSize1055
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19346) TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_create_rewrite_5] failling
[
https://issues.apache.org/jira/browse/HIVE-19346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16457796#comment-16457796
]
Vineet Garg commented on HIVE-19346:
[~jcamachorodriguez] Would you mind taking a quick look at it? Not sure why it
start failing recently. Probably just need golden file output
> TestMiniLlapLocalCliDriver.testCliDriver[materialized_view_create_rewrite_5]
> failling
> -
>
> Key: HIVE-19346
> URL: https://issues.apache.org/jira/browse/HIVE-19346
> Project: Hive
> Issue Type: Sub-task
>Reporter: Vineet Garg
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
>
> {noformat}
> Error Message
> Client Execution succeeded but contained differences (error code = 1) after
> executing materialized_view_create_rewrite_5.q
> 402c402
> < totalSize1053
> ---
> > totalSize1055
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19159) TestMTQueries.testMTQueries1 failure
[ https://issues.apache.org/jira/browse/HIVE-19159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16457797#comment-16457797 ] Vineet Garg commented on HIVE-19159: [~abstractdog] This is still failing in branch-3. Can you take a look please? > TestMTQueries.testMTQueries1 failure > > > Key: HIVE-19159 > URL: https://issues.apache.org/jira/browse/HIVE-19159 > Project: Hive > Issue Type: Sub-task > Components: Test >Reporter: Vineet Garg >Assignee: Laszlo Bodor >Priority: Critical > Attachments: HIVE-19159.01.patch > > > I have confirmed that HIVE-18051 caused this failure -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19347) TestTriggersWorkloadManager tests are failing consistently
[
https://issues.apache.org/jira/browse/HIVE-19347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16457806#comment-16457806
]
Vineet Garg commented on HIVE-19347:
[~prasanth_j] [~sershe] Can you take a look? Not sure what caused the failure
but these tests are failing on branch-3 and master.
> TestTriggersWorkloadManager tests are failing consistently
> --
>
> Key: HIVE-19347
> URL: https://issues.apache.org/jira/browse/HIVE-19347
> Project: Hive
> Issue Type: Sub-task
>Reporter: Vineet Garg
>Priority: Major
>
> {noformat}
> Error Message
> Expected query to succeed expected null, but was: Error while processing statement: FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 3,
> vertexId=vertex_1524884047358_0001_21_01, diagnostics=[Task failed,
> taskId=task_1524884047358_0001_21_01_00, diagnostics=[TaskAttempt 0
> failed, info=[Error: Error while running task ( failure ) :
> attempt_1524884047358_0001_21_01_00_0:java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException:
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.io.IOException: java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:80)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:419)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> ... 15 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19347) TestTriggersWorkloadManager tests are failing consistently
[
https://issues.apache.org/jira/browse/HIVE-19347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-19347:
---
Description:
I see the following tests failing on branch-3
* org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesWrite
10 sec 14
* org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomReadOps
7.7 sec 14
*
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedFiles
15 sec 14
* org.apache.hive.jdbc.TestTriggersWorkloadManager.testMultipleTriggers2
17 sec 14
*
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryExecutionTime
1.5 sec 14
*
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryElapsedTime
{noformat}
Error Message
Expected query to succeed expected null, but was: TestTriggersWorkloadManager tests are failing consistently
> --
>
> Key: HIVE-19347
> URL: https://issues.apache.org/jira/browse/HIVE-19347
> Project: Hive
> Issue Type: Sub-task
>Reporter: Vineet Garg
>Priority: Major
>
> I see the following tests failing on branch-3
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesWrite
> 10 sec 14
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomReadOps
> 7.7 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedFiles
> 15 sec 14
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testMultipleTriggers2
> 17 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryExecutionTime
> 1.5 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryElapsedTime
> {noformat}
> Error Message
> Expected query to succeed expected null, but was: Error while processing statement: FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 3,
> vertexId=vertex_1524884047358_0001_21_01, diagnostics=[Task failed,
> taskId=task_1524884047358_0001_21_01_00, diagnostics=[TaskAttempt 0
> failed, info=[Error: Error while running task ( failure ) :
> attempt_1524884047358_0001_21_01_00_0:java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException:
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.io.IOException: java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:80)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:419)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> ... 15 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19347) TestTriggersWorkloadManager tests are failing consistently
[
https://issues.apache.org/jira/browse/HIVE-19347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-19347:
---
Description:
I see the following tests failing on branch-3
* org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesWrite
10 sec 14
* org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomReadOps
7.7 sec 14
*
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedFiles
15 sec 14
* org.apache.hive.jdbc.TestTriggersWorkloadManager.testMultipleTriggers2
17 sec 14
*
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryExecutionTime
1.5 sec 14
*
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryElapsedTime
*
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerVertexRawInputSplitsNoKill
20 sec 18
* org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighShuffleBytes
1.4 sec 18
* org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomNonExistent
2.6 sec 18
* org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesRead
{noformat}
Error Message
Expected query to succeed expected null, but was: TestTriggersWorkloadManager tests are failing consistently
> --
>
> Key: HIVE-19347
> URL: https://issues.apache.org/jira/browse/HIVE-19347
> Project: Hive
> Issue Type: Sub-task
>Reporter: Vineet Garg
>Priority: Major
>
> I see the following tests failing on branch-3
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesWrite
> 10 sec 14
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomReadOps
> 7.7 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedFiles
> 15 sec 14
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testMultipleTriggers2
> 17 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryExecutionTime
> 1.5 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryElapsedTime
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerVertexRawInputSplitsNoKill
> 20 sec 18
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighShuffleBytes
> 1.4 sec 18
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomNonExistent
> 2.6 sec 18
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesRead
> {noformat}
> Error Message
> Expected query to succeed expected null, but was: Error while processing statement: FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 3,
> vertexId=vertex_1524884047358_0001_21_01, diagnostics=[Task failed,
> taskId=task_1524884047358_0001_21_01_00, diagnostics=[TaskAttempt 0
> failed, info=[Error: Error while running task ( failure ) :
> attempt_1524884047358_0001_21_01_00_0:java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException:
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.io.IOException: java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:80)
> at
>
[jira] [Commented] (HIVE-19348) org.apache.hadoop.hive.ql.plan.mapping.TestOperatorCmp are failing
[
https://issues.apache.org/jira/browse/HIVE-19348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16457819#comment-16457819
]
Vineet Garg commented on HIVE-19348:
[~kgyrtkirk] These tests are failing both in branch-3 and master. Can you take
a look please?
> org.apache.hadoop.hive.ql.plan.mapping.TestOperatorCmp are failing
> ---
>
> Key: HIVE-19348
> URL: https://issues.apache.org/jira/browse/HIVE-19348
> Project: Hive
> Issue Type: Sub-task
>Reporter: Vineet Garg
>Assignee: Zoltan Haindrich
>Priority: Major
>
> {noformat}
> Error Message
> expected:<1> but was:<2>
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19348) org.apache.hadoop.hive.ql.plan.mapping.TestOperatorCmp are failing
[
https://issues.apache.org/jira/browse/HIVE-19348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-19348:
---
Description:
*
org.apache.hadoop.hive.ql.plan.mapping.TestOperatorCmp.testDifferentFiltersAreNotMatched
2.7 sec 7
*
org.apache.hadoop.hive.ql.plan.mapping.TestOperatorCmp.testUnrelatedFiltersAreNotMatched0
1.4 sec 7
*
org.apache.hadoop.hive.ql.plan.mapping.TestOperatorCmp.testUnrelatedFiltersAreNotMatched1
1.8 sec 7
*
org.apache.hadoop.hive.ql.plan.mapping.TestOperatorCmp.testSameFiltersMatched
1.8 sec 7
*
org.apache.hadoop.hive.ql.plan.mapping.TestReOptimization.testNotReExecutedIfAssertionError
{noformat}
Error Message
expected:<1> but was:<2>
{noformat}
was:
{noformat}
Error Message
expected:<1> but was:<2>
{noformat}
> org.apache.hadoop.hive.ql.plan.mapping.TestOperatorCmp are failing
> ---
>
> Key: HIVE-19348
> URL: https://issues.apache.org/jira/browse/HIVE-19348
> Project: Hive
> Issue Type: Sub-task
>Reporter: Vineet Garg
>Assignee: Zoltan Haindrich
>Priority: Major
>
> *
> org.apache.hadoop.hive.ql.plan.mapping.TestOperatorCmp.testDifferentFiltersAreNotMatched
>2.7 sec 7
> *
> org.apache.hadoop.hive.ql.plan.mapping.TestOperatorCmp.testUnrelatedFiltersAreNotMatched0
> 1.4 sec 7
> *
> org.apache.hadoop.hive.ql.plan.mapping.TestOperatorCmp.testUnrelatedFiltersAreNotMatched1
> 1.8 sec 7
> *
> org.apache.hadoop.hive.ql.plan.mapping.TestOperatorCmp.testSameFiltersMatched
> 1.8 sec 7
> *
> org.apache.hadoop.hive.ql.plan.mapping.TestReOptimization.testNotReExecutedIfAssertionError
> {noformat}
> Error Message
> expected:<1> but was:<2>
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-19348) org.apache.hadoop.hive.ql.plan.mapping.TestOperatorCmp are failing
[
https://issues.apache.org/jira/browse/HIVE-19348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg reassigned HIVE-19348:
--
> org.apache.hadoop.hive.ql.plan.mapping.TestOperatorCmp are failing
> ---
>
> Key: HIVE-19348
> URL: https://issues.apache.org/jira/browse/HIVE-19348
> Project: Hive
> Issue Type: Sub-task
>Reporter: Vineet Garg
>Assignee: Zoltan Haindrich
>Priority: Major
>
> {noformat}
> Error Message
> expected:<1> but was:<2>
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-19347) TestTriggersWorkloadManager tests are failing consistently
[
https://issues.apache.org/jira/browse/HIVE-19347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg reassigned HIVE-19347:
--
Assignee: Vineet Garg
> TestTriggersWorkloadManager tests are failing consistently
> --
>
> Key: HIVE-19347
> URL: https://issues.apache.org/jira/browse/HIVE-19347
> Project: Hive
> Issue Type: Sub-task
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
>
> I see the following tests failing on branch-3
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesWrite
> 10 sec 14
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomReadOps
> 7.7 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedFiles
> 15 sec 14
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testMultipleTriggers2
> 17 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryExecutionTime
> 1.5 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryElapsedTime
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerVertexRawInputSplitsNoKill
> 20 sec 18
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighShuffleBytes
> 1.4 sec 18
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomNonExistent
> 2.6 sec 18
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesRead
> {noformat}
> Error Message
> Expected query to succeed expected null, but was: Error while processing statement: FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 3,
> vertexId=vertex_1524884047358_0001_21_01, diagnostics=[Task failed,
> taskId=task_1524884047358_0001_21_01_00, diagnostics=[TaskAttempt 0
> failed, info=[Error: Error while running task ( failure ) :
> attempt_1524884047358_0001_21_01_00_0:java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException:
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.io.IOException: java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:80)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:419)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> ... 15 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17055) Flaky test: TestMiniLlapCliDriver.testCliDriver[llap_smb]
[ https://issues.apache.org/jira/browse/HIVE-17055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16457862#comment-16457862 ] Vineet Garg commented on HIVE-17055: tez_smb_1 is failing with plan difference on branch-3 (https://builds.apache.org/job/PreCommit-HIVE-Build/10541/testReport/org.apache.hadoop.hive.cli/TestMiniLlapLocalCliDriver/testCliDriver_tez_smb_1_/). I don't see it failing on master though > Flaky test: TestMiniLlapCliDriver.testCliDriver[llap_smb] > - > > Key: HIVE-17055 > URL: https://issues.apache.org/jira/browse/HIVE-17055 > Project: Hive > Issue Type: Sub-task >Reporter: Janaki Lahorani >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-17055.1.patch > > > Following tests are also failing with same symptoms: > * TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1] > * TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_main] > Client Execution succeeded but contained differences (error code = 1) after > executing llap_smb.q > 324,325c324,325 > < 2000 9 52 > < 2001 0 139630 > --- > > 2001 4 139630 > > 2001 6 52 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-19350) Vectorization: Turn off vectorization for explainuser_1.q / spark_explainuser_1
[ https://issues.apache.org/jira/browse/HIVE-19350?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16457868#comment-16457868 ] Vineet Garg commented on HIVE-19350: +1 pending tests > Vectorization: Turn off vectorization for explainuser_1.q / > spark_explainuser_1 > --- > > Key: HIVE-19350 > URL: https://issues.apache.org/jira/browse/HIVE-19350 > Project: Hive > Issue Type: Bug > Components: Hive >Reporter: Matt McCline >Assignee: Matt McCline >Priority: Critical > Attachments: HIVE-19350.01.patch > > > Seem like the operator number instability issue to me that Pengcheng Xiong > worked on that could occur with vectorization. > For now, turning off vectorization for: > TestMiniLlapLocalCliDriver.testCliDriver[explainuser_1] > TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1] > Follow up Jira is HIVE-19351. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-19347) TestTriggersWorkloadManager tests are failing consistently
[
https://issues.apache.org/jira/browse/HIVE-19347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-19347:
---
Description:
Caused by the patch which turned on vectorization. Following tests are failing
due to the patch:
*
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1]
*
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[explainuser_1]
* org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesWrite
10 sec 14
* org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomReadOps
7.7 sec 14
*
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedFiles
15 sec 14
* org.apache.hive.jdbc.TestTriggersWorkloadManager.testMultipleTriggers2
17 sec 14
*
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryExecutionTime
1.5 sec 14
*
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryElapsedTime
*
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerVertexRawInputSplitsNoKill
20 sec 18
* org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighShuffleBytes
1.4 sec 18
* org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomNonExistent
2.6 sec 18
* org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesRead
{noformat}
Error Message
Expected query to succeed expected null, but was: TestTriggersWorkloadManager tests are failing consistently
> --
>
> Key: HIVE-19347
> URL: https://issues.apache.org/jira/browse/HIVE-19347
> Project: Hive
> Issue Type: Sub-task
>Reporter: Vineet Garg
>Assignee: Matt McCline
>Priority: Blocker
>
> Caused by the patch which turned on vectorization. Following tests are
> failing due to the patch:
> *
> org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1]
> *
> org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[explainuser_1]
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesWrite
> 10 sec 14
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomReadOps
> 7.7 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedFiles
> 15 sec 14
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testMultipleTriggers2
> 17 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryExecutionTime
> 1.5 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryElapsedTime
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerVertexRawInputSplitsNoKill
> 20 sec 18
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighShuffleBytes
> 1.4 sec 18
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomNonExistent
> 2.6 sec 18
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesRead
> {noformat}
> Error Message
> Expected query to succeed expected null, but was: Error while processing statement: FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 3,
> vertexId=vertex_1524884047358_0001_21_01, diagnostics=[Task failed,
> taskId=task_1524884047358_0001_21_01_00, diagnostics=[TaskAttempt 0
> failed, info=[Error: Error while running task ( failure ) :
> attempt_1524884047358_0001_21_01_00_0:java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException:
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
>
[jira] [Commented] (HIVE-19347) TestTriggersWorkloadManager tests are failing consistently
[
https://issues.apache.org/jira/browse/HIVE-19347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16457858#comment-16457858
]
Vineet Garg commented on HIVE-19347:
Following tests are also caused by the vectorization being turned on:
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[explainuser_1]
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1]
> TestTriggersWorkloadManager tests are failing consistently
> --
>
> Key: HIVE-19347
> URL: https://issues.apache.org/jira/browse/HIVE-19347
> Project: Hive
> Issue Type: Sub-task
>Reporter: Vineet Garg
>Assignee: Matt McCline
>Priority: Blocker
>
> I see the following tests failing on branch-3
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesWrite
> 10 sec 14
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomReadOps
> 7.7 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedFiles
> 15 sec 14
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testMultipleTriggers2
> 17 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryExecutionTime
> 1.5 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryElapsedTime
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerVertexRawInputSplitsNoKill
> 20 sec 18
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighShuffleBytes
> 1.4 sec 18
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomNonExistent
> 2.6 sec 18
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesRead
> {noformat}
> Error Message
> Expected query to succeed expected null, but was: Error while processing statement: FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 3,
> vertexId=vertex_1524884047358_0001_21_01, diagnostics=[Task failed,
> taskId=task_1524884047358_0001_21_01_00, diagnostics=[TaskAttempt 0
> failed, info=[Error: Error while running task ( failure ) :
> attempt_1524884047358_0001_21_01_00_0:java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException:
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.io.IOException: java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:80)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:419)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> ... 15 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19347) TestTriggersWorkloadManager tests are failing consistently
[
https://issues.apache.org/jira/browse/HIVE-19347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-19347:
---
Description:
Caused by the patch which turned on vectorization. Following tests are failing
due to the patch:
*
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1]
*
* org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesWrite
10 sec 14
* org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomReadOps
7.7 sec 14
*
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedFiles
15 sec 14
* org.apache.hive.jdbc.TestTriggersWorkloadManager.testMultipleTriggers2
17 sec 14
*
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryExecutionTime
1.5 sec 14
*
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryElapsedTime
*
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerVertexRawInputSplitsNoKill
20 sec 18
* org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighShuffleBytes
1.4 sec 18
* org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomNonExistent
2.6 sec 18
* org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesRead
{noformat}
Error Message
Expected query to succeed expected null, but was: TestTriggersWorkloadManager tests are failing consistently
> --
>
> Key: HIVE-19347
> URL: https://issues.apache.org/jira/browse/HIVE-19347
> Project: Hive
> Issue Type: Sub-task
>Reporter: Vineet Garg
>Assignee: Matt McCline
>Priority: Blocker
>
> Caused by the patch which turned on vectorization. Following tests are
> failing due to the patch:
> *
> org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1]
> *
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesWrite
> 10 sec 14
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomReadOps
> 7.7 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedFiles
> 15 sec 14
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testMultipleTriggers2
> 17 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryExecutionTime
> 1.5 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryElapsedTime
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerVertexRawInputSplitsNoKill
> 20 sec 18
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighShuffleBytes
> 1.4 sec 18
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomNonExistent
> 2.6 sec 18
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesRead
> {noformat}
> Error Message
> Expected query to succeed expected null, but was: Error while processing statement: FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 3,
> vertexId=vertex_1524884047358_0001_21_01, diagnostics=[Task failed,
> taskId=task_1524884047358_0001_21_01_00, diagnostics=[TaskAttempt 0
> failed, info=[Error: Error while running task ( failure ) :
> attempt_1524884047358_0001_21_01_00_0:java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException:
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at
[jira] [Comment Edited] (HIVE-17055) Flaky test: TestMiniLlapCliDriver.testCliDriver[llap_smb]
[ https://issues.apache.org/jira/browse/HIVE-17055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16457829#comment-16457829 ] Vineet Garg edited comment on HIVE-17055 at 4/28/18 11:01 PM: -- tez_smb_main is consistently failing (both on master and branch-3). Probably need golden file output. [~djaiswal] Would you mind taking a look? was (Author: vgarg): tez_smb_main is consistently failing. Probably need golden file output. [~djaiswal] Would you mind taking a look? > Flaky test: TestMiniLlapCliDriver.testCliDriver[llap_smb] > - > > Key: HIVE-17055 > URL: https://issues.apache.org/jira/browse/HIVE-17055 > Project: Hive > Issue Type: Sub-task >Reporter: Janaki Lahorani >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-17055.1.patch > > > Following tests are also failing with same symptoms: > * TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1] > * TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_main] > Client Execution succeeded but contained differences (error code = 1) after > executing llap_smb.q > 324,325c324,325 > < 2000 9 52 > < 2001 0 139630 > --- > > 2001 4 139630 > > 2001 6 52 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-17055) Flaky test: TestMiniLlapCliDriver.testCliDriver[llap_smb]
[ https://issues.apache.org/jira/browse/HIVE-17055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16457829#comment-16457829 ] Vineet Garg commented on HIVE-17055: tez_smb_main is consistently failing. Probably need golden file output. [~djaiswal] Would you mind taking a look? > Flaky test: TestMiniLlapCliDriver.testCliDriver[llap_smb] > - > > Key: HIVE-17055 > URL: https://issues.apache.org/jira/browse/HIVE-17055 > Project: Hive > Issue Type: Sub-task >Reporter: Janaki Lahorani >Assignee: Deepak Jaiswal >Priority: Major > Attachments: HIVE-17055.1.patch > > > Following tests are also failing with same symptoms: > * TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_1] > * TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_main] > Client Execution succeeded but contained differences (error code = 1) after > executing llap_smb.q > 324,325c324,325 > < 2000 9 52 > < 2001 0 139630 > --- > > 2001 4 139630 > > 2001 6 52 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-19347) TestTriggersWorkloadManager tests are failing consistently
[
https://issues.apache.org/jira/browse/HIVE-19347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg reassigned HIVE-19347:
--
Assignee: Matt McCline (was: Vineet Garg)
> TestTriggersWorkloadManager tests are failing consistently
> --
>
> Key: HIVE-19347
> URL: https://issues.apache.org/jira/browse/HIVE-19347
> Project: Hive
> Issue Type: Sub-task
>Reporter: Vineet Garg
>Assignee: Matt McCline
>Priority: Major
>
> I see the following tests failing on branch-3
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesWrite
> 10 sec 14
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomReadOps
> 7.7 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedFiles
> 15 sec 14
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testMultipleTriggers2
> 17 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryExecutionTime
> 1.5 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryElapsedTime
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerVertexRawInputSplitsNoKill
> 20 sec 18
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighShuffleBytes
> 1.4 sec 18
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomNonExistent
> 2.6 sec 18
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesRead
> {noformat}
> Error Message
> Expected query to succeed expected null, but was: Error while processing statement: FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 3,
> vertexId=vertex_1524884047358_0001_21_01, diagnostics=[Task failed,
> taskId=task_1524884047358_0001_21_01_00, diagnostics=[TaskAttempt 0
> failed, info=[Error: Error while running task ( failure ) :
> attempt_1524884047358_0001_21_01_00_0:java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException:
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.io.IOException: java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:80)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:419)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> ... 15 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19347) TestTriggersWorkloadManager tests are failing consistently
[
https://issues.apache.org/jira/browse/HIVE-19347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-19347:
---
Priority: Blocker (was: Major)
> TestTriggersWorkloadManager tests are failing consistently
> --
>
> Key: HIVE-19347
> URL: https://issues.apache.org/jira/browse/HIVE-19347
> Project: Hive
> Issue Type: Sub-task
>Reporter: Vineet Garg
>Assignee: Matt McCline
>Priority: Blocker
>
> I see the following tests failing on branch-3
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesWrite
> 10 sec 14
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomReadOps
> 7.7 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedFiles
> 15 sec 14
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testMultipleTriggers2
> 17 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryExecutionTime
> 1.5 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryElapsedTime
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerVertexRawInputSplitsNoKill
> 20 sec 18
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighShuffleBytes
> 1.4 sec 18
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomNonExistent
> 2.6 sec 18
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesRead
> {noformat}
> Error Message
> Expected query to succeed expected null, but was: Error while processing statement: FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 3,
> vertexId=vertex_1524884047358_0001_21_01, diagnostics=[Task failed,
> taskId=task_1524884047358_0001_21_01_00, diagnostics=[TaskAttempt 0
> failed, info=[Error: Error while running task ( failure ) :
> attempt_1524884047358_0001_21_01_00_0:java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException:
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.io.IOException: java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:80)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:419)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> ... 15 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19347) TestTriggersWorkloadManager tests are failing consistently
[
https://issues.apache.org/jira/browse/HIVE-19347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16457857#comment-16457857
]
Vineet Garg commented on HIVE-19347:
[~mmccline] These failures are caused by HIVE-19269. Can you take a look please?
> TestTriggersWorkloadManager tests are failing consistently
> --
>
> Key: HIVE-19347
> URL: https://issues.apache.org/jira/browse/HIVE-19347
> Project: Hive
> Issue Type: Sub-task
>Reporter: Vineet Garg
>Assignee: Matt McCline
>Priority: Major
>
> I see the following tests failing on branch-3
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesWrite
> 10 sec 14
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomReadOps
> 7.7 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedFiles
> 15 sec 14
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testMultipleTriggers2
> 17 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryExecutionTime
> 1.5 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryElapsedTime
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerVertexRawInputSplitsNoKill
> 20 sec 18
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighShuffleBytes
> 1.4 sec 18
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomNonExistent
> 2.6 sec 18
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesRead
> {noformat}
> Error Message
> Expected query to succeed expected null, but was: Error while processing statement: FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 3,
> vertexId=vertex_1524884047358_0001_21_01, diagnostics=[Task failed,
> taskId=task_1524884047358_0001_21_01_00, diagnostics=[TaskAttempt 0
> failed, info=[Error: Error while running task ( failure ) :
> attempt_1524884047358_0001_21_01_00_0:java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException:
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.io.IOException: java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:80)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:419)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> ... 15 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-19347) TestTriggersWorkloadManager tests are failing consistently
[
https://issues.apache.org/jira/browse/HIVE-19347?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16457859#comment-16457859
]
Vineet Garg commented on HIVE-19347:
org.apache.hive.jdbc.TestJdbcDriver2.testResultSetMetaData also failing due to
vectorization
> TestTriggersWorkloadManager tests are failing consistently
> --
>
> Key: HIVE-19347
> URL: https://issues.apache.org/jira/browse/HIVE-19347
> Project: Hive
> Issue Type: Sub-task
>Reporter: Vineet Garg
>Assignee: Matt McCline
>Priority: Blocker
>
> Caused by the patch which turned on vectorization. Following tests are
> failing due to the patch:
> *
> org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1]
> *
> org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[explainuser_1]
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesWrite
> 10 sec 14
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomReadOps
> 7.7 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedFiles
> 15 sec 14
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testMultipleTriggers2
> 17 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryExecutionTime
> 1.5 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryElapsedTime
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerVertexRawInputSplitsNoKill
> 20 sec 18
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighShuffleBytes
> 1.4 sec 18
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomNonExistent
> 2.6 sec 18
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesRead
> {noformat}
> Error Message
> Expected query to succeed expected null, but was: Error while processing statement: FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 3,
> vertexId=vertex_1524884047358_0001_21_01, diagnostics=[Task failed,
> taskId=task_1524884047358_0001_21_01_00, diagnostics=[TaskAttempt 0
> failed, info=[Error: Error while running task ( failure ) :
> attempt_1524884047358_0001_21_01_00_0:java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException:
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
> org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
> java.io.IOException: java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:80)
> at
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:419)
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:267)
> ... 15 more
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-19347) TestTriggersWorkloadManager tests are failing consistently
[
https://issues.apache.org/jira/browse/HIVE-19347?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-19347:
---
Description:
Caused by the patch which turned on vectorization. Following tests are failing
due to the patch:
* org.apache.hive.jdbc.TestJdbcDriver2.testResultSetMetaData
*
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1]
*
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[explainuser_1]
* org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesWrite
10 sec 14
* org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomReadOps
7.7 sec 14
*
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedFiles
15 sec 14
* org.apache.hive.jdbc.TestTriggersWorkloadManager.testMultipleTriggers2
17 sec 14
*
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryExecutionTime
1.5 sec 14
*
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryElapsedTime
*
org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerVertexRawInputSplitsNoKill
20 sec 18
* org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighShuffleBytes
1.4 sec 18
* org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomNonExistent
2.6 sec 18
* org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesRead
{noformat}
Error Message
Expected query to succeed expected null, but was: TestTriggersWorkloadManager tests are failing consistently
> --
>
> Key: HIVE-19347
> URL: https://issues.apache.org/jira/browse/HIVE-19347
> Project: Hive
> Issue Type: Sub-task
>Reporter: Vineet Garg
>Assignee: Matt McCline
>Priority: Blocker
>
> Caused by the patch which turned on vectorization. Following tests are
> failing due to the patch:
> * org.apache.hive.jdbc.TestJdbcDriver2.testResultSetMetaData
> *
> org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_explainuser_1]
> *
> org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[explainuser_1]
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesWrite
> 10 sec 14
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomReadOps
> 7.7 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomCreatedFiles
> 15 sec 14
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testMultipleTriggers2
> 17 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryExecutionTime
> 1.5 sec 14
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerSlowQueryElapsedTime
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerVertexRawInputSplitsNoKill
> 20 sec 18
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighShuffleBytes
> 1.4 sec 18
> *
> org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerCustomNonExistent
> 2.6 sec 18
> * org.apache.hive.jdbc.TestTriggersWorkloadManager.testTriggerHighBytesRead
> {noformat}
> Error Message
> Expected query to succeed expected null, but was: Error while processing statement: FAILED: Execution Error, return code 2 from
> org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed, vertexName=Map 3,
> vertexId=vertex_1524884047358_0001_21_01, diagnostics=[Task failed,
> taskId=task_1524884047358_0001_21_01_00, diagnostics=[TaskAttempt 0
> failed, info=[Error: Error while running task ( failure ) :
> attempt_1524884047358_0001_21_01_00_0:java.lang.RuntimeException:
> org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException:
> java.io.IOException: java.lang.NullPointerException
> at
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:296)
> at org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:250)
> at
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:374)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1962)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at
>
[jira] [Commented] (HIVE-17914) Provide support for complex expressions in ON clauses for SEMIjoins
[ https://issues.apache.org/jira/browse/HIVE-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16221239#comment-16221239 ] Vineet Garg commented on HIVE-17914: [~jcamachorodriguez] FYI we already have jira (HIVE-17766) tracking this. > Provide support for complex expressions in ON clauses for SEMIjoins > --- > > Key: HIVE-17914 > URL: https://issues.apache.org/jira/browse/HIVE-17914 > Project: Hive > Issue Type: Bug > Components: Operators, Parser >Affects Versions: 3.0.0 >Reporter: Jesus Camacho Rodriguez >Assignee: Jesus Camacho Rodriguez > Attachments: HIVE-17914.patch > > > Similar to HIVE-15251, HIVE-15211. > Currently, we have some restrictions on the predicates that we can use in ON > clauses for semi joins. It is important for subquery decorrelation to support > complex conditions. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17767) Rewrite correlated EXISTS/IN subqueries into LEFT SEMI JOIN
[ https://issues.apache.org/jira/browse/HIVE-17767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17767: --- Status: Patch Available (was: Open) > Rewrite correlated EXISTS/IN subqueries into LEFT SEMI JOIN > --- > > Key: HIVE-17767 > URL: https://issues.apache.org/jira/browse/HIVE-17767 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-17767.1.patch > > > Currently such queries are written into group by + inner join with value > generator and is inefficient. Value generator consists of join with outer > query to fetch all correlated values. This value generator could be > completely eliminated if such queries are instead rewritten into LEFT SEMI > JOIN. > Note that to do this first hive need to support LEFT SEMI JOIN with non-equi > condition (HIVE-17766). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17767) Rewrite correlated EXISTS/IN subqueries into LEFT SEMI JOIN
[ https://issues.apache.org/jira/browse/HIVE-17767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17767: --- Attachment: HIVE-17767.1.patch First patch removes earlier commit (which reverted another commit) and adds a rewrite of correlated EXISTS/IN into LEFT SEMI JOIN. This patch isn't yet ready to be committed since HIVE-17766 isn't in yet. > Rewrite correlated EXISTS/IN subqueries into LEFT SEMI JOIN > --- > > Key: HIVE-17767 > URL: https://issues.apache.org/jira/browse/HIVE-17767 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-17767.1.patch > > > Currently such queries are written into group by + inner join with value > generator and is inefficient. Value generator consists of join with outer > query to fetch all correlated values. This value generator could be > completely eliminated if such queries are instead rewritten into LEFT SEMI > JOIN. > Note that to do this first hive need to support LEFT SEMI JOIN with non-equi > condition (HIVE-17766). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17898) Explain plan output enhancement
[ https://issues.apache.org/jira/browse/HIVE-17898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17898: --- Status: Patch Available (was: Open) > Explain plan output enhancement > --- > > Key: HIVE-17898 > URL: https://issues.apache.org/jira/browse/HIVE-17898 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-17898.1.patch > > > We would like to enhance the explain plan output to display additional > information e.g.: > TableScan operator should have following additional info > * Actual table name (currently only alias name is displayed) > * Database name > * Column names being scanned -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (HIVE-17898) Explain plan output enhancement
[ https://issues.apache.org/jira/browse/HIVE-17898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-17898: --- Status: Open (was: Patch Available) > Explain plan output enhancement > --- > > Key: HIVE-17898 > URL: https://issues.apache.org/jira/browse/HIVE-17898 > Project: Hive > Issue Type: Improvement > Components: Query Planning >Reporter: Vineet Garg >Assignee: Vineet Garg > Attachments: HIVE-17898.1.patch > > > We would like to enhance the explain plan output to display additional > information e.g.: > TableScan operator should have following additional info > * Actual table name (currently only alias name is displayed) > * Database name > * Column names being scanned -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (HIVE-18462) Explain formatted for queries with map join has columnExprMap with unformatted column name
[
https://issues.apache.org/jira/browse/HIVE-18462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg reassigned HIVE-18462:
--
> Explain formatted for queries with map join has columnExprMap with
> unformatted column name
> --
>
> Key: HIVE-18462
> URL: https://issues.apache.org/jira/browse/HIVE-18462
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
>
> e.g.
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:Column[_col0]",
> "_col1":"0:Column[_col1]",
> "_col2":"1:Column[_col0]",
> "_col3":"1:Column[_col1]"
>}
> {code}
> It is better formatted as:
> "columnExprMap:":{
> "_col0":"0:_col0",
> "_col1":"0:_col1",
> "_col2":"1:_col0",
> "_col3":"1:_col1"
>}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18462) Explain formatted for queries with map join has columnExprMap with unformatted column name
[
https://issues.apache.org/jira/browse/HIVE-18462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18462:
---
Description:
e.g.
{code:sql}
"columnExprMap:":{
"_col0":"0:Column[_col0]",
"_col1":"0:Column[_col1]",
"_col2":"1:Column[_col0]",
"_col3":"1:Column[_col1]"
}
{code}
It is better formatted as:
{code:sql}
"columnExprMap:":{
"_col0":"0:_col0",
"_col1":"0:_col1",
"_col2":"1:_col0",
"_col3":"1:_col1"
}
{code}
was:
e.g.
{code:sql}
"columnExprMap:":{
"_col0":"0:Column[_col0]",
"_col1":"0:Column[_col1]",
"_col2":"1:Column[_col0]",
"_col3":"1:Column[_col1]"
}
{code}
It is better formatted as:
"columnExprMap:":{
"_col0":"0:_col0",
"_col1":"0:_col1",
"_col2":"1:_col0",
"_col3":"1:_col1"
}
> Explain formatted for queries with map join has columnExprMap with
> unformatted column name
> --
>
> Key: HIVE-18462
> URL: https://issues.apache.org/jira/browse/HIVE-18462
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
>
> e.g.
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:Column[_col0]",
> "_col1":"0:Column[_col1]",
> "_col2":"1:Column[_col0]",
> "_col3":"1:Column[_col1]"
> }
> {code}
> It is better formatted as:
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:_col0",
> "_col1":"0:_col1",
> "_col2":"1:_col0",
> "_col3":"1:_col1"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18462) Explain formatted for queries with map join has columnExprMap with unformatted column name
[
https://issues.apache.org/jira/browse/HIVE-18462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18462:
---
Status: Patch Available (was: Open)
> Explain formatted for queries with map join has columnExprMap with
> unformatted column name
> --
>
> Key: HIVE-18462
> URL: https://issues.apache.org/jira/browse/HIVE-18462
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-18462.1.patch
>
>
> e.g.
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:Column[_col0]",
> "_col1":"0:Column[_col1]",
> "_col2":"1:Column[_col0]",
> "_col3":"1:Column[_col1]"
> }
> {code}
> It is better formatted as:
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:_col0",
> "_col1":"0:_col1",
> "_col2":"1:_col0",
> "_col3":"1:_col1"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18462) Explain formatted for queries with map join has columnExprMap with unformatted column name
[
https://issues.apache.org/jira/browse/HIVE-18462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18462:
---
Attachment: HIVE-18462.1.patch
> Explain formatted for queries with map join has columnExprMap with
> unformatted column name
> --
>
> Key: HIVE-18462
> URL: https://issues.apache.org/jira/browse/HIVE-18462
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-18462.1.patch
>
>
> e.g.
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:Column[_col0]",
> "_col1":"0:Column[_col1]",
> "_col2":"1:Column[_col0]",
> "_col3":"1:Column[_col1]"
> }
> {code}
> It is better formatted as:
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:_col0",
> "_col1":"0:_col1",
> "_col2":"1:_col0",
> "_col3":"1:_col1"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-18490) Query with EXISTS and NOT EXISTS with non-equi predicate can produce wrong result
[
https://issues.apache.org/jira/browse/HIVE-18490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg reassigned HIVE-18490:
--
> Query with EXISTS and NOT EXISTS with non-equi predicate can produce wrong
> result
> -
>
> Key: HIVE-18490
> URL: https://issues.apache.org/jira/browse/HIVE-18490
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Critical
> Attachments: HIVE-18490.1.patch
>
>
> Queries such as following can produce wrong result
> {code:sql}
> select
>count(ws_order_number)
> from
>web_sales ws1
> where
> and exists (select *
> from web_sales ws2
> where ws1.ws_order_number = ws2.ws_order_number
> and ws1.ws_warehouse_sk <> ws2.ws_warehouse_sk)
> and not exists(select *
>from web_returns wr1
>where ws1.ws_order_number = wr1.wr_order_number);
> {code}
> This query is simplified version of tpcds query 94. Such queries are
> rewritten into LEFT SEMI JOIN and LEFT OUTER JOIN with residual
> predicate/filter (non-equi join key). Problem is that these joins are being
> merged, we shouldn't be merging since semi join has non-equi join filter.
> Basically the underlying issue is that if a query has multiple join with LEFT
> SEMI JOIN with non-equi join key it is being merged with other joins. Merge
> logic should check such cases and avoid merging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-16605) Enforce NOT NULL constraints
[
https://issues.apache.org/jira/browse/HIVE-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg reassigned HIVE-16605:
--
Assignee: Vineet Garg
> Enforce NOT NULL constraints
>
>
> Key: HIVE-16605
> URL: https://issues.apache.org/jira/browse/HIVE-16605
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Carter Shanklin
>Assignee: Vineet Garg
>Priority: Major
>
> Since NOT NULL is so common it would be great to have tables start to enforce
> that.
> [~ekoifman] described a possible approach in HIVE-16575:
> {quote}
> One way to enforce not null constraint is to have the optimizer add
> enforce_not_null UDF which throws if it sees a NULL, otherwise it's pass
> through.
> So if 'b' has not null constraint,
> Insert into T select a,b,c... would become
> Insert into T select a, enforce_not_null(b), c.
> This would work for any table type.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16605) Enforce NOT NULL constraints
[
https://issues.apache.org/jira/browse/HIVE-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16605:
---
Attachment: HIVE-16605.1.patch
> Enforce NOT NULL constraints
>
>
> Key: HIVE-16605
> URL: https://issues.apache.org/jira/browse/HIVE-16605
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Carter Shanklin
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-16605.1.patch
>
>
> Since NOT NULL is so common it would be great to have tables start to enforce
> that.
> [~ekoifman] described a possible approach in HIVE-16575:
> {quote}
> One way to enforce not null constraint is to have the optimizer add
> enforce_not_null UDF which throws if it sees a NULL, otherwise it's pass
> through.
> So if 'b' has not null constraint,
> Insert into T select a,b,c... would become
> Insert into T select a, enforce_not_null(b), c.
> This would work for any table type.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16605) Enforce NOT NULL constraints
[
https://issues.apache.org/jira/browse/HIVE-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16326763#comment-16326763
]
Vineet Garg commented on HIVE-16605:
Attached is first patch which supports NOT NULL constraint enforcement on
INSERT statements. Note that this patch is not yet fully tested therefore is
not yet ready to be committed.
> Enforce NOT NULL constraints
>
>
> Key: HIVE-16605
> URL: https://issues.apache.org/jira/browse/HIVE-16605
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Carter Shanklin
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-16605.1.patch
>
>
> Since NOT NULL is so common it would be great to have tables start to enforce
> that.
> [~ekoifman] described a possible approach in HIVE-16575:
> {quote}
> One way to enforce not null constraint is to have the optimizer add
> enforce_not_null UDF which throws if it sees a NULL, otherwise it's pass
> through.
> So if 'b' has not null constraint,
> Insert into T select a,b,c... would become
> Insert into T select a, enforce_not_null(b), c.
> This would work for any table type.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16605) Enforce NOT NULL constraints
[
https://issues.apache.org/jira/browse/HIVE-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16605:
---
Status: Patch Available (was: Open)
> Enforce NOT NULL constraints
>
>
> Key: HIVE-16605
> URL: https://issues.apache.org/jira/browse/HIVE-16605
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Carter Shanklin
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-16605.1.patch
>
>
> Since NOT NULL is so common it would be great to have tables start to enforce
> that.
> [~ekoifman] described a possible approach in HIVE-16575:
> {quote}
> One way to enforce not null constraint is to have the optimizer add
> enforce_not_null UDF which throws if it sees a NULL, otherwise it's pass
> through.
> So if 'b' has not null constraint,
> Insert into T select a,b,c... would become
> Insert into T select a, enforce_not_null(b), c.
> This would work for any table type.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18490) Query with EXISTS and NOT EXISTS with non-equi predicate can produce wrong result
[
https://issues.apache.org/jira/browse/HIVE-18490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18490:
---
Status: Patch Available (was: Open)
> Query with EXISTS and NOT EXISTS with non-equi predicate can produce wrong
> result
> -
>
> Key: HIVE-18490
> URL: https://issues.apache.org/jira/browse/HIVE-18490
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Critical
> Attachments: HIVE-18490.1.patch, HIVE-18490.2.patch
>
>
> Queries such as following can produce wrong result
> {code:sql}
> select
>count(ws_order_number)
> from
>web_sales ws1
> where
> and exists (select *
> from web_sales ws2
> where ws1.ws_order_number = ws2.ws_order_number
> and ws1.ws_warehouse_sk <> ws2.ws_warehouse_sk)
> and not exists(select *
>from web_returns wr1
>where ws1.ws_order_number = wr1.wr_order_number);
> {code}
> This query is simplified version of tpcds query 94. Such queries are
> rewritten into LEFT SEMI JOIN and LEFT OUTER JOIN with residual
> predicate/filter (non-equi join key). Problem is that these joins are being
> merged, we shouldn't be merging since semi join has non-equi join filter.
> Basically the underlying issue is that if a query has multiple join with LEFT
> SEMI JOIN with non-equi join key it is being merged with other joins. Merge
> logic should check such cases and avoid merging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18490) Query with EXISTS and NOT EXISTS with non-equi predicate can produce wrong result
[
https://issues.apache.org/jira/browse/HIVE-18490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16332935#comment-16332935
]
Vineet Garg commented on HIVE-18490:
[~jcamachorodriguez] Can you take a look? This patch disables merging of joins
if there is a left semi join with residual predicate.
> Query with EXISTS and NOT EXISTS with non-equi predicate can produce wrong
> result
> -
>
> Key: HIVE-18490
> URL: https://issues.apache.org/jira/browse/HIVE-18490
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Critical
> Attachments: HIVE-18490.1.patch, HIVE-18490.2.patch
>
>
> Queries such as following can produce wrong result
> {code:sql}
> select
>count(ws_order_number)
> from
>web_sales ws1
> where
> and exists (select *
> from web_sales ws2
> where ws1.ws_order_number = ws2.ws_order_number
> and ws1.ws_warehouse_sk <> ws2.ws_warehouse_sk)
> and not exists(select *
>from web_returns wr1
>where ws1.ws_order_number = wr1.wr_order_number);
> {code}
> This query is simplified version of tpcds query 94. Such queries are
> rewritten into LEFT SEMI JOIN and LEFT OUTER JOIN with residual
> predicate/filter (non-equi join key). Problem is that these joins are being
> merged, we shouldn't be merging since semi join has non-equi join filter.
> Basically the underlying issue is that if a query has multiple join with LEFT
> SEMI JOIN with non-equi join key it is being merged with other joins. Merge
> logic should check such cases and avoid merging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18490) Query with EXISTS and NOT EXISTS with non-equi predicate can produce wrong result
[
https://issues.apache.org/jira/browse/HIVE-18490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18490:
---
Status: Open (was: Patch Available)
> Query with EXISTS and NOT EXISTS with non-equi predicate can produce wrong
> result
> -
>
> Key: HIVE-18490
> URL: https://issues.apache.org/jira/browse/HIVE-18490
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Critical
> Attachments: HIVE-18490.1.patch, HIVE-18490.2.patch
>
>
> Queries such as following can produce wrong result
> {code:sql}
> select
>count(ws_order_number)
> from
>web_sales ws1
> where
> and exists (select *
> from web_sales ws2
> where ws1.ws_order_number = ws2.ws_order_number
> and ws1.ws_warehouse_sk <> ws2.ws_warehouse_sk)
> and not exists(select *
>from web_returns wr1
>where ws1.ws_order_number = wr1.wr_order_number);
> {code}
> This query is simplified version of tpcds query 94. Such queries are
> rewritten into LEFT SEMI JOIN and LEFT OUTER JOIN with residual
> predicate/filter (non-equi join key). Problem is that these joins are being
> merged, we shouldn't be merging since semi join has non-equi join filter.
> Basically the underlying issue is that if a query has multiple join with LEFT
> SEMI JOIN with non-equi join key it is being merged with other joins. Merge
> logic should check such cases and avoid merging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18490) Query with EXISTS and NOT EXISTS with non-equi predicate can produce wrong result
[
https://issues.apache.org/jira/browse/HIVE-18490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18490:
---
Attachment: HIVE-18490.2.patch
> Query with EXISTS and NOT EXISTS with non-equi predicate can produce wrong
> result
> -
>
> Key: HIVE-18490
> URL: https://issues.apache.org/jira/browse/HIVE-18490
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Critical
> Attachments: HIVE-18490.1.patch, HIVE-18490.2.patch
>
>
> Queries such as following can produce wrong result
> {code:sql}
> select
>count(ws_order_number)
> from
>web_sales ws1
> where
> and exists (select *
> from web_sales ws2
> where ws1.ws_order_number = ws2.ws_order_number
> and ws1.ws_warehouse_sk <> ws2.ws_warehouse_sk)
> and not exists(select *
>from web_returns wr1
>where ws1.ws_order_number = wr1.wr_order_number);
> {code}
> This query is simplified version of tpcds query 94. Such queries are
> rewritten into LEFT SEMI JOIN and LEFT OUTER JOIN with residual
> predicate/filter (non-equi join key). Problem is that these joins are being
> merged, we shouldn't be merging since semi join has non-equi join filter.
> Basically the underlying issue is that if a query has multiple join with LEFT
> SEMI JOIN with non-equi join key it is being merged with other joins. Merge
> logic should check such cases and avoid merging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18462) Explain formatted for queries with map join has columnExprMap with unformatted column name
[
https://issues.apache.org/jira/browse/HIVE-18462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333883#comment-16333883
]
Vineet Garg commented on HIVE-18462:
Thanks for pointing it out [~anishek]. I have reverted and re-commited with
appropriate JIRA number.
> Explain formatted for queries with map join has columnExprMap with
> unformatted column name
> --
>
> Key: HIVE-18462
> URL: https://issues.apache.org/jira/browse/HIVE-18462
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-18462.1.patch, HIVE-18462.2.patch
>
>
> e.g.
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:Column[_col0]",
> "_col1":"0:Column[_col1]",
> "_col2":"1:Column[_col0]",
> "_col3":"1:Column[_col1]"
> }
> {code}
> It is better formatted as:
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:_col0",
> "_col1":"0:_col1",
> "_col2":"1:_col0",
> "_col3":"1:_col1"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18490) Query with EXISTS and NOT EXISTS with non-equi predicate can produce wrong result
[
https://issues.apache.org/jira/browse/HIVE-18490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18490:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Query with EXISTS and NOT EXISTS with non-equi predicate can produce wrong
> result
> -
>
> Key: HIVE-18490
> URL: https://issues.apache.org/jira/browse/HIVE-18490
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Critical
> Attachments: HIVE-18490.1.patch, HIVE-18490.2.patch
>
>
> Queries such as following can produce wrong result
> {code:sql}
> select
>count(ws_order_number)
> from
>web_sales ws1
> where
> and exists (select *
> from web_sales ws2
> where ws1.ws_order_number = ws2.ws_order_number
> and ws1.ws_warehouse_sk <> ws2.ws_warehouse_sk)
> and not exists(select *
>from web_returns wr1
>where ws1.ws_order_number = wr1.wr_order_number);
> {code}
> This query is simplified version of tpcds query 94. Such queries are
> rewritten into LEFT SEMI JOIN and LEFT OUTER JOIN with residual
> predicate/filter (non-equi join key). Problem is that these joins are being
> merged, we shouldn't be merging since semi join has non-equi join filter.
> Basically the underlying issue is that if a query has multiple join with LEFT
> SEMI JOIN with non-equi join key it is being merged with other joins. Merge
> logic should check such cases and avoid merging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18490) Query with EXISTS and NOT EXISTS with non-equi predicate can produce wrong result
[
https://issues.apache.org/jira/browse/HIVE-18490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16333885#comment-16333885
]
Vineet Garg commented on HIVE-18490:
Thanks [~jcamachorodriguez]. Pushed this to mater.
> Query with EXISTS and NOT EXISTS with non-equi predicate can produce wrong
> result
> -
>
> Key: HIVE-18490
> URL: https://issues.apache.org/jira/browse/HIVE-18490
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Critical
> Attachments: HIVE-18490.1.patch, HIVE-18490.2.patch
>
>
> Queries such as following can produce wrong result
> {code:sql}
> select
>count(ws_order_number)
> from
>web_sales ws1
> where
> and exists (select *
> from web_sales ws2
> where ws1.ws_order_number = ws2.ws_order_number
> and ws1.ws_warehouse_sk <> ws2.ws_warehouse_sk)
> and not exists(select *
>from web_returns wr1
>where ws1.ws_order_number = wr1.wr_order_number);
> {code}
> This query is simplified version of tpcds query 94. Such queries are
> rewritten into LEFT SEMI JOIN and LEFT OUTER JOIN with residual
> predicate/filter (non-equi join key). Problem is that these joins are being
> merged, we shouldn't be merging since semi join has non-equi join filter.
> Basically the underlying issue is that if a query has multiple join with LEFT
> SEMI JOIN with non-equi join key it is being merged with other joins. Merge
> logic should check such cases and avoid merging.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16828) With CBO enabled, Query on partitioned views throws IndexOutOfBoundException
[
https://issues.apache.org/jira/browse/HIVE-16828?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16828:
---
Fix Version/s: 3.0.0
> With CBO enabled, Query on partitioned views throws IndexOutOfBoundException
>
>
> Key: HIVE-16828
> URL: https://issues.apache.org/jira/browse/HIVE-16828
> Project: Hive
> Issue Type: Bug
> Components: CBO
>Affects Versions: 1.2.0, 2.1.1, 2.2.0
>Reporter: Adesh Kumar Rao
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-16828.2.patch, HIVE-16828.3.patch, HIVE-16828.patch
>
>
> {code:java}
> Caused by: java.lang.AssertionError: Internal error: While invoking method
> 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Filter,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:789)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:534)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:270)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:213)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:374)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(HiveRelFieldTrimmer.java:394)
> ... 98 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
> ... 102 more
> Caused by: java.lang.AssertionError: Internal error: While invoking method
> 'public org.apache.calcite.sql2rel.RelFieldTrimmer$TrimResult
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(org.apache.calcite.rel.core.Project,org.apache.calcite.util.ImmutableBitSet,java.util.Set)'
> at org.apache.calcite.util.Util.newInternal(Util.java:789)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:534)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.dispatchTrimFields(RelFieldTrimmer.java:270)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimChild(RelFieldTrimmer.java:213)
> at
> org.apache.calcite.sql2rel.RelFieldTrimmer.trimFields(RelFieldTrimmer.java:466)
> ... 107 more
> Caused by: java.lang.reflect.InvocationTargetException
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:483)
> at org.apache.calcite.util.ReflectUtil$2.invoke(ReflectUtil.java:531)
> ... 110 more
> Caused by: java.lang.IndexOutOfBoundsException: Index: 94, Size: 94
> at java.util.ArrayList.rangeCheck(ArrayList.java:653)
> at java.util.ArrayList.get(ArrayList.java:429)
> at
> org.apache.hadoop.hive.ql.optimizer.calcite.rules.HiveRelFieldTrimmer.trimFields(HiveRelFieldTrimmer.java:390)
> {code}
> Reproducible case:
> 1) set "hive.security.authorization.enabled" to "true" in "hive-site.xml".
> 2) {code:java}
> CREATE TABLE table1 (id int) PARTITIONED BY (year int)
> -- create partitioned view
> CREATE VIEW view1 partitioned on (year) as select id, year from table1;
> set hive.cbo.enable=true;
> select year from view1;
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16605) Enforce NOT NULL constraints
[
https://issues.apache.org/jira/browse/HIVE-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16605:
---
Attachment: HIVE-16605.2.patch
> Enforce NOT NULL constraints
>
>
> Key: HIVE-16605
> URL: https://issues.apache.org/jira/browse/HIVE-16605
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Carter Shanklin
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-16605.1.patch, HIVE-16605.2.patch
>
>
> Since NOT NULL is so common it would be great to have tables start to enforce
> that.
> [~ekoifman] described a possible approach in HIVE-16575:
> {quote}
> One way to enforce not null constraint is to have the optimizer add
> enforce_not_null UDF which throws if it sees a NULL, otherwise it's pass
> through.
> So if 'b' has not null constraint,
> Insert into T select a,b,c... would become
> Insert into T select a, enforce_not_null(b), c.
> This would work for any table type.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16605) Enforce NOT NULL constraints
[
https://issues.apache.org/jira/browse/HIVE-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16329343#comment-16329343
]
Vineet Garg commented on HIVE-16605:
There was a typo in testconfiguration in earlier patch which probably caused
all of these failures. Retrying the patch.
> Enforce NOT NULL constraints
>
>
> Key: HIVE-16605
> URL: https://issues.apache.org/jira/browse/HIVE-16605
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Carter Shanklin
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-16605.1.patch, HIVE-16605.2.patch
>
>
> Since NOT NULL is so common it would be great to have tables start to enforce
> that.
> [~ekoifman] described a possible approach in HIVE-16575:
> {quote}
> One way to enforce not null constraint is to have the optimizer add
> enforce_not_null UDF which throws if it sees a NULL, otherwise it's pass
> through.
> So if 'b' has not null constraint,
> Insert into T select a,b,c... would become
> Insert into T select a, enforce_not_null(b), c.
> This would work for any table type.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16605) Enforce NOT NULL constraints
[
https://issues.apache.org/jira/browse/HIVE-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16605:
---
Status: Patch Available (was: Open)
> Enforce NOT NULL constraints
>
>
> Key: HIVE-16605
> URL: https://issues.apache.org/jira/browse/HIVE-16605
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Carter Shanklin
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-16605.1.patch, HIVE-16605.2.patch
>
>
> Since NOT NULL is so common it would be great to have tables start to enforce
> that.
> [~ekoifman] described a possible approach in HIVE-16575:
> {quote}
> One way to enforce not null constraint is to have the optimizer add
> enforce_not_null UDF which throws if it sees a NULL, otherwise it's pass
> through.
> So if 'b' has not null constraint,
> Insert into T select a,b,c... would become
> Insert into T select a, enforce_not_null(b), c.
> This would work for any table type.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16605) Enforce NOT NULL constraints
[
https://issues.apache.org/jira/browse/HIVE-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16605:
---
Status: Open (was: Patch Available)
> Enforce NOT NULL constraints
>
>
> Key: HIVE-16605
> URL: https://issues.apache.org/jira/browse/HIVE-16605
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Carter Shanklin
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-16605.1.patch, HIVE-16605.2.patch
>
>
> Since NOT NULL is so common it would be great to have tables start to enforce
> that.
> [~ekoifman] described a possible approach in HIVE-16575:
> {quote}
> One way to enforce not null constraint is to have the optimizer add
> enforce_not_null UDF which throws if it sees a NULL, otherwise it's pass
> through.
> So if 'b' has not null constraint,
> Insert into T select a,b,c... would become
> Insert into T select a, enforce_not_null(b), c.
> This would work for any table type.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18462) Explain formatted for queries with map join has columnExprMap with unformatted column name
[
https://issues.apache.org/jira/browse/HIVE-18462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18462:
---
Status: Patch Available (was: Open)
> Explain formatted for queries with map join has columnExprMap with
> unformatted column name
> --
>
> Key: HIVE-18462
> URL: https://issues.apache.org/jira/browse/HIVE-18462
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-18462.1.patch
>
>
> e.g.
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:Column[_col0]",
> "_col1":"0:Column[_col1]",
> "_col2":"1:Column[_col0]",
> "_col3":"1:Column[_col1]"
> }
> {code}
> It is better formatted as:
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:_col0",
> "_col1":"0:_col1",
> "_col2":"1:_col0",
> "_col3":"1:_col1"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16605) Enforce NOT NULL constraints
[
https://issues.apache.org/jira/browse/HIVE-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16605:
---
Attachment: (was: HIVE-16605.1.patch)
> Enforce NOT NULL constraints
>
>
> Key: HIVE-16605
> URL: https://issues.apache.org/jira/browse/HIVE-16605
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Carter Shanklin
>Assignee: Vineet Garg
>Priority: Major
>
> Since NOT NULL is so common it would be great to have tables start to enforce
> that.
> [~ekoifman] described a possible approach in HIVE-16575:
> {quote}
> One way to enforce not null constraint is to have the optimizer add
> enforce_not_null UDF which throws if it sees a NULL, otherwise it's pass
> through.
> So if 'b' has not null constraint,
> Insert into T select a,b,c... would become
> Insert into T select a, enforce_not_null(b), c.
> This would work for any table type.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18462) Explain formatted for queries with map join has columnExprMap with unformatted column name
[
https://issues.apache.org/jira/browse/HIVE-18462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18462:
---
Attachment: HIVE-18462.1.patch
> Explain formatted for queries with map join has columnExprMap with
> unformatted column name
> --
>
> Key: HIVE-18462
> URL: https://issues.apache.org/jira/browse/HIVE-18462
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-18462.1.patch
>
>
> e.g.
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:Column[_col0]",
> "_col1":"0:Column[_col1]",
> "_col2":"1:Column[_col0]",
> "_col3":"1:Column[_col1]"
> }
> {code}
> It is better formatted as:
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:_col0",
> "_col1":"0:_col1",
> "_col2":"1:_col0",
> "_col3":"1:_col1"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18462) Explain formatted for queries with map join has columnExprMap with unformatted column name
[
https://issues.apache.org/jira/browse/HIVE-18462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18462:
---
Status: Open (was: Patch Available)
> Explain formatted for queries with map join has columnExprMap with
> unformatted column name
> --
>
> Key: HIVE-18462
> URL: https://issues.apache.org/jira/browse/HIVE-18462
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
>
> e.g.
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:Column[_col0]",
> "_col1":"0:Column[_col1]",
> "_col2":"1:Column[_col0]",
> "_col3":"1:Column[_col1]"
> }
> {code}
> It is better formatted as:
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:_col0",
> "_col1":"0:_col1",
> "_col2":"1:_col0",
> "_col3":"1:_col1"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18462) Explain formatted for queries with map join has columnExprMap with unformatted column name
[
https://issues.apache.org/jira/browse/HIVE-18462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18462:
---
Attachment: (was: HIVE-18462.1.patch)
> Explain formatted for queries with map join has columnExprMap with
> unformatted column name
> --
>
> Key: HIVE-18462
> URL: https://issues.apache.org/jira/browse/HIVE-18462
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
>
> e.g.
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:Column[_col0]",
> "_col1":"0:Column[_col1]",
> "_col2":"1:Column[_col0]",
> "_col3":"1:Column[_col1]"
> }
> {code}
> It is better formatted as:
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:_col0",
> "_col1":"0:_col1",
> "_col2":"1:_col0",
> "_col3":"1:_col1"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16605) Enforce NOT NULL constraints
[
https://issues.apache.org/jira/browse/HIVE-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16605:
---
Status: Open (was: Patch Available)
> Enforce NOT NULL constraints
>
>
> Key: HIVE-16605
> URL: https://issues.apache.org/jira/browse/HIVE-16605
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Carter Shanklin
>Assignee: Vineet Garg
>Priority: Major
>
> Since NOT NULL is so common it would be great to have tables start to enforce
> that.
> [~ekoifman] described a possible approach in HIVE-16575:
> {quote}
> One way to enforce not null constraint is to have the optimizer add
> enforce_not_null UDF which throws if it sees a NULL, otherwise it's pass
> through.
> So if 'b' has not null constraint,
> Insert into T select a,b,c... would become
> Insert into T select a, enforce_not_null(b), c.
> This would work for any table type.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16605) Enforce NOT NULL constraints
[
https://issues.apache.org/jira/browse/HIVE-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16605:
---
Status: Patch Available (was: Open)
> Enforce NOT NULL constraints
>
>
> Key: HIVE-16605
> URL: https://issues.apache.org/jira/browse/HIVE-16605
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Carter Shanklin
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-16605.1.patch
>
>
> Since NOT NULL is so common it would be great to have tables start to enforce
> that.
> [~ekoifman] described a possible approach in HIVE-16575:
> {quote}
> One way to enforce not null constraint is to have the optimizer add
> enforce_not_null UDF which throws if it sees a NULL, otherwise it's pass
> through.
> So if 'b' has not null constraint,
> Insert into T select a,b,c... would become
> Insert into T select a, enforce_not_null(b), c.
> This would work for any table type.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16605) Enforce NOT NULL constraints
[
https://issues.apache.org/jira/browse/HIVE-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16605:
---
Attachment: HIVE-16605.1.patch
> Enforce NOT NULL constraints
>
>
> Key: HIVE-16605
> URL: https://issues.apache.org/jira/browse/HIVE-16605
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Carter Shanklin
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-16605.1.patch
>
>
> Since NOT NULL is so common it would be great to have tables start to enforce
> that.
> [~ekoifman] described a possible approach in HIVE-16575:
> {quote}
> One way to enforce not null constraint is to have the optimizer add
> enforce_not_null UDF which throws if it sees a NULL, otherwise it's pass
> through.
> So if 'b' has not null constraint,
> Insert into T select a,b,c... would become
> Insert into T select a, enforce_not_null(b), c.
> This would work for any table type.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18462) Explain formatted for queries with map join has columnExprMap with unformatted column name
[
https://issues.apache.org/jira/browse/HIVE-18462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18462:
---
Status: Patch Available (was: Open)
> Explain formatted for queries with map join has columnExprMap with
> unformatted column name
> --
>
> Key: HIVE-18462
> URL: https://issues.apache.org/jira/browse/HIVE-18462
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-18462.1.patch, HIVE-18462.2.patch
>
>
> e.g.
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:Column[_col0]",
> "_col1":"0:Column[_col1]",
> "_col2":"1:Column[_col0]",
> "_col3":"1:Column[_col1]"
> }
> {code}
> It is better formatted as:
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:_col0",
> "_col1":"0:_col1",
> "_col2":"1:_col0",
> "_col3":"1:_col1"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18462) Explain formatted for queries with map join has columnExprMap with unformatted column name
[
https://issues.apache.org/jira/browse/HIVE-18462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18462:
---
Attachment: (was: HIVE-18462.2.patch)
> Explain formatted for queries with map join has columnExprMap with
> unformatted column name
> --
>
> Key: HIVE-18462
> URL: https://issues.apache.org/jira/browse/HIVE-18462
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-18462.1.patch, HIVE-18462.2.patch
>
>
> e.g.
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:Column[_col0]",
> "_col1":"0:Column[_col1]",
> "_col2":"1:Column[_col0]",
> "_col3":"1:Column[_col1]"
> }
> {code}
> It is better formatted as:
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:_col0",
> "_col1":"0:_col1",
> "_col2":"1:_col0",
> "_col3":"1:_col1"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18462) Explain formatted for queries with map join has columnExprMap with unformatted column name
[
https://issues.apache.org/jira/browse/HIVE-18462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18462:
---
Attachment: HIVE-18462.2.patch
> Explain formatted for queries with map join has columnExprMap with
> unformatted column name
> --
>
> Key: HIVE-18462
> URL: https://issues.apache.org/jira/browse/HIVE-18462
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-18462.1.patch, HIVE-18462.2.patch
>
>
> e.g.
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:Column[_col0]",
> "_col1":"0:Column[_col1]",
> "_col2":"1:Column[_col0]",
> "_col3":"1:Column[_col1]"
> }
> {code}
> It is better formatted as:
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:_col0",
> "_col1":"0:_col1",
> "_col2":"1:_col0",
> "_col3":"1:_col1"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18462) Explain formatted for queries with map join has columnExprMap with unformatted column name
[
https://issues.apache.org/jira/browse/HIVE-18462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16328504#comment-16328504
]
Vineet Garg commented on HIVE-18462:
[~ashutoshc] Can you take a look?
> Explain formatted for queries with map join has columnExprMap with
> unformatted column name
> --
>
> Key: HIVE-18462
> URL: https://issues.apache.org/jira/browse/HIVE-18462
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-18462.1.patch, HIVE-18462.2.patch
>
>
> e.g.
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:Column[_col0]",
> "_col1":"0:Column[_col1]",
> "_col2":"1:Column[_col0]",
> "_col3":"1:Column[_col1]"
> }
> {code}
> It is better formatted as:
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:_col0",
> "_col1":"0:_col1",
> "_col2":"1:_col0",
> "_col3":"1:_col1"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18462) Explain formatted for queries with map join has columnExprMap with unformatted column name
[
https://issues.apache.org/jira/browse/HIVE-18462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18462:
---
Attachment: HIVE-18462.2.patch
> Explain formatted for queries with map join has columnExprMap with
> unformatted column name
> --
>
> Key: HIVE-18462
> URL: https://issues.apache.org/jira/browse/HIVE-18462
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-18462.1.patch, HIVE-18462.2.patch
>
>
> e.g.
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:Column[_col0]",
> "_col1":"0:Column[_col1]",
> "_col2":"1:Column[_col0]",
> "_col3":"1:Column[_col1]"
> }
> {code}
> It is better formatted as:
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:_col0",
> "_col1":"0:_col1",
> "_col2":"1:_col0",
> "_col3":"1:_col1"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18462) Explain formatted for queries with map join has columnExprMap with unformatted column name
[
https://issues.apache.org/jira/browse/HIVE-18462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18462:
---
Status: Patch Available (was: Open)
> Explain formatted for queries with map join has columnExprMap with
> unformatted column name
> --
>
> Key: HIVE-18462
> URL: https://issues.apache.org/jira/browse/HIVE-18462
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-18462.1.patch, HIVE-18462.2.patch
>
>
> e.g.
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:Column[_col0]",
> "_col1":"0:Column[_col1]",
> "_col2":"1:Column[_col0]",
> "_col3":"1:Column[_col1]"
> }
> {code}
> It is better formatted as:
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:_col0",
> "_col1":"0:_col1",
> "_col2":"1:_col0",
> "_col3":"1:_col1"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18462) Explain formatted for queries with map join has columnExprMap with unformatted column name
[
https://issues.apache.org/jira/browse/HIVE-18462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18462:
---
Status: Open (was: Patch Available)
> Explain formatted for queries with map join has columnExprMap with
> unformatted column name
> --
>
> Key: HIVE-18462
> URL: https://issues.apache.org/jira/browse/HIVE-18462
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-18462.1.patch, HIVE-18462.2.patch
>
>
> e.g.
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:Column[_col0]",
> "_col1":"0:Column[_col1]",
> "_col2":"1:Column[_col0]",
> "_col3":"1:Column[_col1]"
> }
> {code}
> It is better formatted as:
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:_col0",
> "_col1":"0:_col1",
> "_col2":"1:_col0",
> "_col3":"1:_col1"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18462) Explain formatted for queries with map join has columnExprMap with unformatted column name
[
https://issues.apache.org/jira/browse/HIVE-18462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18462:
---
Status: Open (was: Patch Available)
> Explain formatted for queries with map join has columnExprMap with
> unformatted column name
> --
>
> Key: HIVE-18462
> URL: https://issues.apache.org/jira/browse/HIVE-18462
> Project: Hive
> Issue Type: Improvement
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-18462.1.patch, HIVE-18462.2.patch
>
>
> e.g.
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:Column[_col0]",
> "_col1":"0:Column[_col1]",
> "_col2":"1:Column[_col0]",
> "_col3":"1:Column[_col1]"
> }
> {code}
> It is better formatted as:
> {code:sql}
> "columnExprMap:":{
> "_col0":"0:_col0",
> "_col1":"0:_col1",
> "_col2":"1:_col0",
> "_col3":"1:_col1"
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18548) Fix log4j import
[
https://issues.apache.org/jira/browse/HIVE-18548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16340233#comment-16340233
]
Vineet Garg commented on HIVE-18548:
cc [~prasanth_j]
> Fix log4j import
>
>
> Key: HIVE-18548
> URL: https://issues.apache.org/jira/browse/HIVE-18548
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
>
> LLAP server code could run into invalid reference to log4j.MDC due to code
> importing {{org.apache.log4j.MDC}} instead of {{org.slf4j.MDC }}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-18548) Fix log4j import
[
https://issues.apache.org/jira/browse/HIVE-18548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg reassigned HIVE-18548:
--
> Fix log4j import
>
>
> Key: HIVE-18548
> URL: https://issues.apache.org/jira/browse/HIVE-18548
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
>
> LLAP server code could run into invalid reference to log4j.MDC due to code
> importing {{org.apache.log4j.MDC}} instead of {{org.slf4j.MDC }}.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18548) Fix log4j import
[
https://issues.apache.org/jira/browse/HIVE-18548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18548:
---
Description: LLAP server code could run into invalid reference to log4j.MDC
due to code importing {{org.apache.log4j.MDC}} instead of {{org.slf4j.MDC}}
(was: LLAP server code could run into invalid reference to log4j.MDC due to
code importing {{org.apache.log4j.MDC}} instead of {org.slf4j.MDC }.)
> Fix log4j import
>
>
> Key: HIVE-18548
> URL: https://issues.apache.org/jira/browse/HIVE-18548
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
>
> LLAP server code could run into invalid reference to log4j.MDC due to code
> importing {{org.apache.log4j.MDC}} instead of {{org.slf4j.MDC}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18548) Fix log4j import
[
https://issues.apache.org/jira/browse/HIVE-18548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18548:
---
Description: LLAP server code could run into invalid reference to log4j.MDC
due to code importing {{org.apache.log4j.MDC}} instead of {org.slf4j.MDC }.
(was: LLAP server code could run into invalid reference to log4j.MDC due to
code importing {{org.apache.log4j.MDC}} instead of {{org.slf4j.MDC }}.)
> Fix log4j import
>
>
> Key: HIVE-18548
> URL: https://issues.apache.org/jira/browse/HIVE-18548
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
>
> LLAP server code could run into invalid reference to log4j.MDC due to code
> importing {{org.apache.log4j.MDC}} instead of {org.slf4j.MDC }.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18548) Fix log4j import
[
https://issues.apache.org/jira/browse/HIVE-18548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18548:
---
Status: Patch Available (was: Open)
> Fix log4j import
>
>
> Key: HIVE-18548
> URL: https://issues.apache.org/jira/browse/HIVE-18548
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18548.1.patch
>
>
> LLAP server code could run into invalid reference to log4j.MDC due to code
> importing {{org.apache.log4j.MDC}} instead of {{org.slf4j.MDC}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18548) Fix log4j import
[
https://issues.apache.org/jira/browse/HIVE-18548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18548:
---
Attachment: HIVE-18548.1.patch
> Fix log4j import
>
>
> Key: HIVE-18548
> URL: https://issues.apache.org/jira/browse/HIVE-18548
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18548.1.patch
>
>
> LLAP server code could run into invalid reference to log4j.MDC due to code
> importing {{org.apache.log4j.MDC}} instead of {{org.slf4j.MDC}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18548) Fix log4j import
[
https://issues.apache.org/jira/browse/HIVE-18548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16342405#comment-16342405
]
Vineet Garg commented on HIVE-18548:
Failures are unrelated
> Fix log4j import
>
>
> Key: HIVE-18548
> URL: https://issues.apache.org/jira/browse/HIVE-18548
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18548.1.patch
>
>
> LLAP server code could run into invalid reference to log4j.MDC due to code
> importing {{org.apache.log4j.MDC}} instead of {{org.slf4j.MDC}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18548) Fix log4j import
[
https://issues.apache.org/jira/browse/HIVE-18548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18548:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
Pushed to master.
> Fix log4j import
>
>
> Key: HIVE-18548
> URL: https://issues.apache.org/jira/browse/HIVE-18548
> Project: Hive
> Issue Type: Bug
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 3.0.0
>
> Attachments: HIVE-18548.1.patch
>
>
> LLAP server code could run into invalid reference to log4j.MDC due to code
> importing {{org.apache.log4j.MDC}} instead of {{org.slf4j.MDC}}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18598) Disallow constraints with EXTERNAL table
[ https://issues.apache.org/jira/browse/HIVE-18598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-18598: --- Status: Patch Available (was: Open) > Disallow constraints with EXTERNAL table > > > Key: HIVE-18598 > URL: https://issues.apache.org/jira/browse/HIVE-18598 > Project: Hive > Issue Type: Task > Components: SQL >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-18598.1.patch > > > Since Hive do not manage the data for external tables and can not enforce > constraints it doesn't make sense to allow constraints like PRIMARY KEY, NOT > NULL etc for such tables. > User can still specify RELY to signal optimizer for constraint related > optimizations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-18598) Disallow constraints with EXTERNAL table
[ https://issues.apache.org/jira/browse/HIVE-18598?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16348054#comment-16348054 ] Vineet Garg commented on HIVE-18598: First patch only patches CREATE TABLE statement. ALTER TABLE is yet to be patched up. > Disallow constraints with EXTERNAL table > > > Key: HIVE-18598 > URL: https://issues.apache.org/jira/browse/HIVE-18598 > Project: Hive > Issue Type: Task > Components: SQL >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-18598.1.patch > > > Since Hive do not manage the data for external tables and can not enforce > constraints it doesn't make sense to allow constraints like PRIMARY KEY, NOT > NULL etc for such tables. > User can still specify RELY to signal optimizer for constraint related > optimizations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18598) Disallow constraints with EXTERNAL table
[ https://issues.apache.org/jira/browse/HIVE-18598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-18598: --- Attachment: HIVE-18598.1.patch > Disallow constraints with EXTERNAL table > > > Key: HIVE-18598 > URL: https://issues.apache.org/jira/browse/HIVE-18598 > Project: Hive > Issue Type: Task > Components: SQL >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-18598.1.patch > > > Since Hive do not manage the data for external tables and can not enforce > constraints it doesn't make sense to allow constraints like PRIMARY KEY, NOT > NULL etc for such tables. > User can still specify RELY to signal optimizer for constraint related > optimizations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-18598) Disallow constraints with EXTERNAL table
[ https://issues.apache.org/jira/browse/HIVE-18598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg reassigned HIVE-18598: -- > Disallow constraints with EXTERNAL table > > > Key: HIVE-18598 > URL: https://issues.apache.org/jira/browse/HIVE-18598 > Project: Hive > Issue Type: Task > Components: SQL >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-18598.1.patch > > > Since Hive do not manage the data for external tables and can not enforce > constraints it doesn't make sense to allow constraints like PRIMARY KEY, NOT > NULL etc for such tables. > User can still specify RELY to signal optimizer for constraint related > optimizations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-16605) Enforce NOT NULL constraints
[
https://issues.apache.org/jira/browse/HIVE-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16605:
---
Status: Open (was: Patch Available)
> Enforce NOT NULL constraints
>
>
> Key: HIVE-16605
> URL: https://issues.apache.org/jira/browse/HIVE-16605
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Carter Shanklin
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-16605.1.patch, HIVE-16605.2.patch,
> HIVE-16605.3.patch
>
>
> Since NOT NULL is so common it would be great to have tables start to enforce
> that.
> [~ekoifman] described a possible approach in HIVE-16575:
> {quote}
> One way to enforce not null constraint is to have the optimizer add
> enforce_not_null UDF which throws if it sees a NULL, otherwise it's pass
> through.
> So if 'b' has not null constraint,
> Insert into T select a,b,c... would become
> Insert into T select a, enforce_not_null(b), c.
> This would work for any table type.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16605) Enforce NOT NULL constraints
[
https://issues.apache.org/jira/browse/HIVE-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16605:
---
Attachment: HIVE-16605.3.patch
> Enforce NOT NULL constraints
>
>
> Key: HIVE-16605
> URL: https://issues.apache.org/jira/browse/HIVE-16605
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Carter Shanklin
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-16605.1.patch, HIVE-16605.2.patch,
> HIVE-16605.3.patch
>
>
> Since NOT NULL is so common it would be great to have tables start to enforce
> that.
> [~ekoifman] described a possible approach in HIVE-16575:
> {quote}
> One way to enforce not null constraint is to have the optimizer add
> enforce_not_null UDF which throws if it sees a NULL, otherwise it's pass
> through.
> So if 'b' has not null constraint,
> Insert into T select a,b,c... would become
> Insert into T select a, enforce_not_null(b), c.
> This would work for any table type.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16605) Enforce NOT NULL constraints
[
https://issues.apache.org/jira/browse/HIVE-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16605:
---
Status: Patch Available (was: Open)
> Enforce NOT NULL constraints
>
>
> Key: HIVE-16605
> URL: https://issues.apache.org/jira/browse/HIVE-16605
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Carter Shanklin
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-16605.1.patch, HIVE-16605.2.patch,
> HIVE-16605.3.patch
>
>
> Since NOT NULL is so common it would be great to have tables start to enforce
> that.
> [~ekoifman] described a possible approach in HIVE-16575:
> {quote}
> One way to enforce not null constraint is to have the optimizer add
> enforce_not_null UDF which throws if it sees a NULL, otherwise it's pass
> through.
> So if 'b' has not null constraint,
> Insert into T select a,b,c... would become
> Insert into T select a, enforce_not_null(b), c.
> This would work for any table type.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16605) Enforce NOT NULL constraints
[
https://issues.apache.org/jira/browse/HIVE-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16605:
---
Status: Patch Available (was: Open)
> Enforce NOT NULL constraints
>
>
> Key: HIVE-16605
> URL: https://issues.apache.org/jira/browse/HIVE-16605
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Carter Shanklin
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-16605.1.patch, HIVE-16605.2.patch,
> HIVE-16605.3.patch, HIVE-16605.4.patch, HIVE-16605.5.patch
>
>
> Since NOT NULL is so common it would be great to have tables start to enforce
> that.
> [~ekoifman] described a possible approach in HIVE-16575:
> {quote}
> One way to enforce not null constraint is to have the optimizer add
> enforce_not_null UDF which throws if it sees a NULL, otherwise it's pass
> through.
> So if 'b' has not null constraint,
> Insert into T select a,b,c... would become
> Insert into T select a, enforce_not_null(b), c.
> This would work for any table type.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16605) Enforce NOT NULL constraints
[
https://issues.apache.org/jira/browse/HIVE-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16605:
---
Attachment: HIVE-16605.5.patch
> Enforce NOT NULL constraints
>
>
> Key: HIVE-16605
> URL: https://issues.apache.org/jira/browse/HIVE-16605
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Carter Shanklin
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-16605.1.patch, HIVE-16605.2.patch,
> HIVE-16605.3.patch, HIVE-16605.4.patch, HIVE-16605.5.patch
>
>
> Since NOT NULL is so common it would be great to have tables start to enforce
> that.
> [~ekoifman] described a possible approach in HIVE-16575:
> {quote}
> One way to enforce not null constraint is to have the optimizer add
> enforce_not_null UDF which throws if it sees a NULL, otherwise it's pass
> through.
> So if 'b' has not null constraint,
> Insert into T select a,b,c... would become
> Insert into T select a, enforce_not_null(b), c.
> This would work for any table type.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16605) Enforce NOT NULL constraints
[
https://issues.apache.org/jira/browse/HIVE-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16605:
---
Status: Open (was: Patch Available)
> Enforce NOT NULL constraints
>
>
> Key: HIVE-16605
> URL: https://issues.apache.org/jira/browse/HIVE-16605
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Carter Shanklin
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-16605.1.patch, HIVE-16605.2.patch,
> HIVE-16605.3.patch, HIVE-16605.4.patch, HIVE-16605.5.patch
>
>
> Since NOT NULL is so common it would be great to have tables start to enforce
> that.
> [~ekoifman] described a possible approach in HIVE-16575:
> {quote}
> One way to enforce not null constraint is to have the optimizer add
> enforce_not_null UDF which throws if it sees a NULL, otherwise it's pass
> through.
> So if 'b' has not null constraint,
> Insert into T select a,b,c... would become
> Insert into T select a, enforce_not_null(b), c.
> This would work for any table type.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18647) Cannot create table: Unknown column 'CREATION_METADATA_MV_CREATION_METADATA_ID_OID'
[
https://issues.apache.org/jira/browse/HIVE-18647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355949#comment-16355949
]
Vineet Garg commented on HIVE-18647:
[~jcamachorodriguez] committed HIVE-18641 which should fix this
> Cannot create table: Unknown column
> 'CREATION_METADATA_MV_CREATION_METADATA_ID_OID'
> ---
>
> Key: HIVE-18647
> URL: https://issues.apache.org/jira/browse/HIVE-18647
> Project: Hive
> Issue Type: Bug
>Reporter: Rui Li
>Priority: Major
>
> I'm using latest master branch code and mysql as metastore.
> Creating table hits this error:
> {noformat}
> 2018-02-07T22:04:55,438 ERROR [41f91bf4-bc49-4a73-baee-e2a1d79b8a4e main]
> metastore.RetryingHMSHandler: Retrying HMSHandler after 2000 ms (attempt 1 of
> 10) with error: javax.jdo.JDODataStoreException: Insert of object
> "org.apache.hadoop.hive.metastore.model.MTable@28d16af8" using statement
> "INSERT INTO `TBLS`
> (`TBL_ID`,`CREATE_TIME`,`CREATION_METADATA_MV_CREATION_METADATA_ID_OID`,`DB_ID`,`LAST_ACCESS_TIME`,`OWNER`,`RETENTION`,`IS_REWRITE_ENABLED`,`SD_ID`,`TBL_NAME`,`TBL_TYPE`,`VIEW_EXPANDED_TEXT`,`VIEW_ORIGINAL_TEXT`)
> VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?)" failed : Unknown column
> 'CREATION_METADATA_MV_CREATION_METADATA_ID_OID' in 'field list'
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:729)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:749)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.createTable(ObjectStore.java:1125)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97)
> at com.sun.proxy.$Proxy36.createTable(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1506)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1412)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:1614)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16605) Enforce NOT NULL constraints
[
https://issues.apache.org/jira/browse/HIVE-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16605:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
Failures are unrelated. Pushed to master
> Enforce NOT NULL constraints
>
>
> Key: HIVE-16605
> URL: https://issues.apache.org/jira/browse/HIVE-16605
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Carter Shanklin
>Assignee: Vineet Garg
>Priority: Major
> Labels: TODOC3.0
> Attachments: HIVE-16605.1.patch, HIVE-16605.2.patch,
> HIVE-16605.3.patch, HIVE-16605.4.patch, HIVE-16605.5.patch
>
>
> Since NOT NULL is so common it would be great to have tables start to enforce
> that.
> [~ekoifman] described a possible approach in HIVE-16575:
> {quote}
> One way to enforce not null constraint is to have the optimizer add
> enforce_not_null UDF which throws if it sees a NULL, otherwise it's pass
> through.
> So if 'b' has not null constraint,
> Insert into T select a,b,c... would become
> Insert into T select a, enforce_not_null(b), c.
> This would work for any table type.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16605) Enforce NOT NULL constraints
[
https://issues.apache.org/jira/browse/HIVE-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16605:
---
Labels: TODOC3.0 (was: )
> Enforce NOT NULL constraints
>
>
> Key: HIVE-16605
> URL: https://issues.apache.org/jira/browse/HIVE-16605
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Carter Shanklin
>Assignee: Vineet Garg
>Priority: Major
> Labels: TODOC3.0
> Attachments: HIVE-16605.1.patch, HIVE-16605.2.patch,
> HIVE-16605.3.patch, HIVE-16605.4.patch, HIVE-16605.5.patch
>
>
> Since NOT NULL is so common it would be great to have tables start to enforce
> that.
> [~ekoifman] described a possible approach in HIVE-16575:
> {quote}
> One way to enforce not null constraint is to have the optimizer add
> enforce_not_null UDF which throws if it sees a NULL, otherwise it's pass
> through.
> So if 'b' has not null constraint,
> Insert into T select a,b,c... would become
> Insert into T select a, enforce_not_null(b), c.
> This would work for any table type.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-16605) Enforce NOT NULL constraints
[
https://issues.apache.org/jira/browse/HIVE-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16355910#comment-16355910
]
Vineet Garg commented on HIVE-16605:
Need to document the following:
* Hive Conf {{hive.constraint.notnull.enforce}}
* NOT NULL constraint is enforced for INSERT/MERGE/UPDATE statements.
> Enforce NOT NULL constraints
>
>
> Key: HIVE-16605
> URL: https://issues.apache.org/jira/browse/HIVE-16605
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Carter Shanklin
>Assignee: Vineet Garg
>Priority: Major
> Labels: TODOC3.0
> Attachments: HIVE-16605.1.patch, HIVE-16605.2.patch,
> HIVE-16605.3.patch, HIVE-16605.4.patch, HIVE-16605.5.patch
>
>
> Since NOT NULL is so common it would be great to have tables start to enforce
> that.
> [~ekoifman] described a possible approach in HIVE-16575:
> {quote}
> One way to enforce not null constraint is to have the optimizer add
> enforce_not_null UDF which throws if it sees a NULL, otherwise it's pass
> through.
> So if 'b' has not null constraint,
> Insert into T select a,b,c... would become
> Insert into T select a, enforce_not_null(b), c.
> This would work for any table type.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Reopened] (HIVE-18647) Cannot create table: Unknown column 'CREATION_METADATA_MV_CREATION_METADATA_ID_OID'
[
https://issues.apache.org/jira/browse/HIVE-18647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg reopened HIVE-18647:
> Cannot create table: Unknown column
> 'CREATION_METADATA_MV_CREATION_METADATA_ID_OID'
> ---
>
> Key: HIVE-18647
> URL: https://issues.apache.org/jira/browse/HIVE-18647
> Project: Hive
> Issue Type: Bug
>Reporter: Rui Li
>Priority: Major
> Fix For: 3.0.0
>
>
> I'm using latest master branch code and mysql as metastore.
> Creating table hits this error:
> {noformat}
> 2018-02-07T22:04:55,438 ERROR [41f91bf4-bc49-4a73-baee-e2a1d79b8a4e main]
> metastore.RetryingHMSHandler: Retrying HMSHandler after 2000 ms (attempt 1 of
> 10) with error: javax.jdo.JDODataStoreException: Insert of object
> "org.apache.hadoop.hive.metastore.model.MTable@28d16af8" using statement
> "INSERT INTO `TBLS`
> (`TBL_ID`,`CREATE_TIME`,`CREATION_METADATA_MV_CREATION_METADATA_ID_OID`,`DB_ID`,`LAST_ACCESS_TIME`,`OWNER`,`RETENTION`,`IS_REWRITE_ENABLED`,`SD_ID`,`TBL_NAME`,`TBL_TYPE`,`VIEW_EXPANDED_TEXT`,`VIEW_ORIGINAL_TEXT`)
> VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?)" failed : Unknown column
> 'CREATION_METADATA_MV_CREATION_METADATA_ID_OID' in 'field list'
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:729)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:749)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.createTable(ObjectStore.java:1125)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97)
> at com.sun.proxy.$Proxy36.createTable(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1506)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1412)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:1614)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18647) Cannot create table: Unknown column 'CREATION_METADATA_MV_CREATION_METADATA_ID_OID'
[
https://issues.apache.org/jira/browse/HIVE-18647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16357400#comment-16357400
]
Vineet Garg commented on HIVE-18647:
I am also running into same issue with Derby
> Cannot create table: Unknown column
> 'CREATION_METADATA_MV_CREATION_METADATA_ID_OID'
> ---
>
> Key: HIVE-18647
> URL: https://issues.apache.org/jira/browse/HIVE-18647
> Project: Hive
> Issue Type: Bug
>Reporter: Rui Li
>Priority: Major
> Fix For: 3.0.0
>
>
> I'm using latest master branch code and mysql as metastore.
> Creating table hits this error:
> {noformat}
> 2018-02-07T22:04:55,438 ERROR [41f91bf4-bc49-4a73-baee-e2a1d79b8a4e main]
> metastore.RetryingHMSHandler: Retrying HMSHandler after 2000 ms (attempt 1 of
> 10) with error: javax.jdo.JDODataStoreException: Insert of object
> "org.apache.hadoop.hive.metastore.model.MTable@28d16af8" using statement
> "INSERT INTO `TBLS`
> (`TBL_ID`,`CREATE_TIME`,`CREATION_METADATA_MV_CREATION_METADATA_ID_OID`,`DB_ID`,`LAST_ACCESS_TIME`,`OWNER`,`RETENTION`,`IS_REWRITE_ENABLED`,`SD_ID`,`TBL_NAME`,`TBL_TYPE`,`VIEW_EXPANDED_TEXT`,`VIEW_ORIGINAL_TEXT`)
> VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?)" failed : Unknown column
> 'CREATION_METADATA_MV_CREATION_METADATA_ID_OID' in 'field list'
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:729)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:749)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.createTable(ObjectStore.java:1125)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97)
> at com.sun.proxy.$Proxy36.createTable(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1506)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1412)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:1614)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18647) Cannot create table: "message:Exception thrown when executing query : SELECT DISTINCT.."
[
https://issues.apache.org/jira/browse/HIVE-18647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-18647:
---
Summary: Cannot create table: "message:Exception thrown when executing
query : SELECT DISTINCT.." (was: Cannot create table: Unknown column
'CREATION_METADATA_MV_CREATION_METADATA_ID_OID')
> Cannot create table: "message:Exception thrown when executing query : SELECT
> DISTINCT.."
>
>
> Key: HIVE-18647
> URL: https://issues.apache.org/jira/browse/HIVE-18647
> Project: Hive
> Issue Type: Bug
>Reporter: Rui Li
>Priority: Major
> Fix For: 3.0.0
>
>
> I'm using latest master branch code and mysql as metastore.
> Creating table hits this error:
> {noformat}
> 2018-02-07T22:04:55,438 ERROR [41f91bf4-bc49-4a73-baee-e2a1d79b8a4e main]
> metastore.RetryingHMSHandler: Retrying HMSHandler after 2000 ms (attempt 1 of
> 10) with error: javax.jdo.JDODataStoreException: Insert of object
> "org.apache.hadoop.hive.metastore.model.MTable@28d16af8" using statement
> "INSERT INTO `TBLS`
> (`TBL_ID`,`CREATE_TIME`,`CREATION_METADATA_MV_CREATION_METADATA_ID_OID`,`DB_ID`,`LAST_ACCESS_TIME`,`OWNER`,`RETENTION`,`IS_REWRITE_ENABLED`,`SD_ID`,`TBL_NAME`,`TBL_TYPE`,`VIEW_EXPANDED_TEXT`,`VIEW_ORIGINAL_TEXT`)
> VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?)" failed : Unknown column
> 'CREATION_METADATA_MV_CREATION_METADATA_ID_OID' in 'field list'
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:729)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:749)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.createTable(ObjectStore.java:1125)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97)
> at com.sun.proxy.$Proxy36.createTable(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1506)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1412)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:1614)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18647) Cannot create table: "message:Exception thrown when executing query : SELECT DISTINCT.."
[
https://issues.apache.org/jira/browse/HIVE-18647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16357406#comment-16357406
]
Vineet Garg commented on HIVE-18647:
Modified the Jira title to reflect new error/exception message. I get the
following exception:
{code:java}
FAILED: Execution Error, return code 1 from
org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:Exception thrown
when executing query : SELECT DISTINCT
'org.apache.hadoop.hive.metastore.model.MTable' AS
NUCLEUS_TYPE,A0.BUCKETING_VERSION,A0.CREATE_TIME,A0.LAST_ACCESS_TIME,A0.LOAD_IN_BUCKETED_TABLE,A0.OWNER,A0.RETENTION,A0.IS_REWRITE_ENABLED,A0.TBL_NAME,A0.TBL_TYPE,A0.TBL_ID
FROM TBLS A0 LEFT OUTER JOIN DBS B0 ON A0.DB_ID = B0.DB_ID WHERE A0.TBL_NAME =
? AND B0."NAME" = ?){code}
> Cannot create table: "message:Exception thrown when executing query : SELECT
> DISTINCT.."
>
>
> Key: HIVE-18647
> URL: https://issues.apache.org/jira/browse/HIVE-18647
> Project: Hive
> Issue Type: Bug
>Reporter: Rui Li
>Priority: Major
> Fix For: 3.0.0
>
>
> I'm using latest master branch code and mysql as metastore.
> Creating table hits this error:
> {noformat}
> 2018-02-07T22:04:55,438 ERROR [41f91bf4-bc49-4a73-baee-e2a1d79b8a4e main]
> metastore.RetryingHMSHandler: Retrying HMSHandler after 2000 ms (attempt 1 of
> 10) with error: javax.jdo.JDODataStoreException: Insert of object
> "org.apache.hadoop.hive.metastore.model.MTable@28d16af8" using statement
> "INSERT INTO `TBLS`
> (`TBL_ID`,`CREATE_TIME`,`CREATION_METADATA_MV_CREATION_METADATA_ID_OID`,`DB_ID`,`LAST_ACCESS_TIME`,`OWNER`,`RETENTION`,`IS_REWRITE_ENABLED`,`SD_ID`,`TBL_NAME`,`TBL_TYPE`,`VIEW_EXPANDED_TEXT`,`VIEW_ORIGINAL_TEXT`)
> VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?)" failed : Unknown column
> 'CREATION_METADATA_MV_CREATION_METADATA_ID_OID' in 'field list'
> at
> org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:543)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:729)
> at
> org.datanucleus.api.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:749)
> at
> org.apache.hadoop.hive.metastore.ObjectStore.createTable(ObjectStore.java:1125)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at
> org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:97)
> at com.sun.proxy.$Proxy36.createTable(Unknown Source)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1506)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_core(HiveMetaStore.java:1412)
> at
> org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.create_table_with_environment_context(HiveMetaStore.java:1614)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> {noformat}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Assigned] (HIVE-18657) Fix checkstyle violations for Semantic Analyzer
[ https://issues.apache.org/jira/browse/HIVE-18657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg reassigned HIVE-18657: -- > Fix checkstyle violations for Semantic Analyzer > --- > > Key: HIVE-18657 > URL: https://issues.apache.org/jira/browse/HIVE-18657 > Project: Hive > Issue Type: Task >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > > SemanticAnalyzer.java has quite a few checkstyle violations which should be > fixed. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18598) Disallow NOT NULL constraints to be ENABLED/ENFORCED with EXTERNAL table
[ https://issues.apache.org/jira/browse/HIVE-18598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-18598: --- Status: Open (was: Patch Available) > Disallow NOT NULL constraints to be ENABLED/ENFORCED with EXTERNAL table > > > Key: HIVE-18598 > URL: https://issues.apache.org/jira/browse/HIVE-18598 > Project: Hive > Issue Type: Task > Components: SQL >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-18598.1.patch, HIVE-18598.2.patch, > HIVE-18598.3.patch > > > HIVE-16605 is enabling/enforcing NOT NULL constraint. But since Hive do not > manage the data for external tables and can not enforce constraints it > doesn't make sense to allow NOT NULL constraints to be enabled/enforced on > external table. > User can still specify RELY to signal optimizer for constraint related > optimizations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18598) Disallow NOT NULL constraints to be ENABLED/ENFORCED with EXTERNAL table
[ https://issues.apache.org/jira/browse/HIVE-18598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-18598: --- Status: Patch Available (was: Open) > Disallow NOT NULL constraints to be ENABLED/ENFORCED with EXTERNAL table > > > Key: HIVE-18598 > URL: https://issues.apache.org/jira/browse/HIVE-18598 > Project: Hive > Issue Type: Task > Components: SQL >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-18598.1.patch, HIVE-18598.2.patch, > HIVE-18598.3.patch > > > HIVE-16605 is enabling/enforcing NOT NULL constraint. But since Hive do not > manage the data for external tables and can not enforce constraints it > doesn't make sense to allow NOT NULL constraints to be enabled/enforced on > external table. > User can still specify RELY to signal optimizer for constraint related > optimizations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18598) Disallow NOT NULL constraints to be ENABLED/ENFORCED with EXTERNAL table
[ https://issues.apache.org/jira/browse/HIVE-18598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-18598: --- Attachment: HIVE-18598.3.patch > Disallow NOT NULL constraints to be ENABLED/ENFORCED with EXTERNAL table > > > Key: HIVE-18598 > URL: https://issues.apache.org/jira/browse/HIVE-18598 > Project: Hive > Issue Type: Task > Components: SQL >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-18598.1.patch, HIVE-18598.2.patch, > HIVE-18598.3.patch > > > HIVE-16605 is enabling/enforcing NOT NULL constraint. But since Hive do not > manage the data for external tables and can not enforce constraints it > doesn't make sense to allow NOT NULL constraints to be enabled/enforced on > external table. > User can still specify RELY to signal optimizer for constraint related > optimizations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-18598) Disallow NOT NULL constraints to be ENABLED/ENFORCED with EXTERNAL table
[ https://issues.apache.org/jira/browse/HIVE-18598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-18598: --- Attachment: HIVE-18598.2.patch > Disallow NOT NULL constraints to be ENABLED/ENFORCED with EXTERNAL table > > > Key: HIVE-18598 > URL: https://issues.apache.org/jira/browse/HIVE-18598 > Project: Hive > Issue Type: Task > Components: SQL >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-18598.1.patch, HIVE-18598.2.patch > > > HIVE-16605 is enabling/enforcing NOT NULL constraint. But since Hive do not > manage the data for external tables and can not enforce constraints it > doesn't make sense to allow NOT NULL constraints to be enabled/enforced on > external table. > User can still specify RELY to signal optimizer for constraint related > optimizations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-16605) Enforce NOT NULL constraints
[
https://issues.apache.org/jira/browse/HIVE-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16605:
---
Attachment: HIVE-16605.4.patch
> Enforce NOT NULL constraints
>
>
> Key: HIVE-16605
> URL: https://issues.apache.org/jira/browse/HIVE-16605
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Carter Shanklin
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-16605.1.patch, HIVE-16605.2.patch,
> HIVE-16605.3.patch, HIVE-16605.4.patch
>
>
> Since NOT NULL is so common it would be great to have tables start to enforce
> that.
> [~ekoifman] described a possible approach in HIVE-16575:
> {quote}
> One way to enforce not null constraint is to have the optimizer add
> enforce_not_null UDF which throws if it sees a NULL, otherwise it's pass
> through.
> So if 'b' has not null constraint,
> Insert into T select a,b,c... would become
> Insert into T select a, enforce_not_null(b), c.
> This would work for any table type.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-16605) Enforce NOT NULL constraints
[
https://issues.apache.org/jira/browse/HIVE-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16605:
---
Status: Patch Available (was: Open)
> Enforce NOT NULL constraints
>
>
> Key: HIVE-16605
> URL: https://issues.apache.org/jira/browse/HIVE-16605
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Carter Shanklin
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-16605.1.patch, HIVE-16605.2.patch,
> HIVE-16605.3.patch, HIVE-16605.4.patch
>
>
> Since NOT NULL is so common it would be great to have tables start to enforce
> that.
> [~ekoifman] described a possible approach in HIVE-16575:
> {quote}
> One way to enforce not null constraint is to have the optimizer add
> enforce_not_null UDF which throws if it sees a NULL, otherwise it's pass
> through.
> So if 'b' has not null constraint,
> Insert into T select a,b,c... would become
> Insert into T select a, enforce_not_null(b), c.
> This would work for any table type.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18598) Disallow NOT NULL constraints to be ENABLED/ENFORCED with EXTERNAL table
[ https://issues.apache.org/jira/browse/HIVE-18598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Vineet Garg updated HIVE-18598: --- Status: Patch Available (was: Open) > Disallow NOT NULL constraints to be ENABLED/ENFORCED with EXTERNAL table > > > Key: HIVE-18598 > URL: https://issues.apache.org/jira/browse/HIVE-18598 > Project: Hive > Issue Type: Task > Components: SQL >Reporter: Vineet Garg >Assignee: Vineet Garg >Priority: Major > Attachments: HIVE-18598.1.patch, HIVE-18598.2.patch > > > HIVE-16605 is enabling/enforcing NOT NULL constraint. But since Hive do not > manage the data for external tables and can not enforce constraints it > doesn't make sense to allow NOT NULL constraints to be enabled/enforced on > external table. > User can still specify RELY to signal optimizer for constraint related > optimizations. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-16605) Enforce NOT NULL constraints
[
https://issues.apache.org/jira/browse/HIVE-16605?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-16605:
---
Status: Open (was: Patch Available)
> Enforce NOT NULL constraints
>
>
> Key: HIVE-16605
> URL: https://issues.apache.org/jira/browse/HIVE-16605
> Project: Hive
> Issue Type: New Feature
>Affects Versions: 3.0.0
>Reporter: Carter Shanklin
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-16605.1.patch, HIVE-16605.2.patch,
> HIVE-16605.3.patch, HIVE-16605.4.patch
>
>
> Since NOT NULL is so common it would be great to have tables start to enforce
> that.
> [~ekoifman] described a possible approach in HIVE-16575:
> {quote}
> One way to enforce not null constraint is to have the optimizer add
> enforce_not_null UDF which throws if it sees a NULL, otherwise it's pass
> through.
> So if 'b' has not null constraint,
> Insert into T select a,b,c... would become
> Insert into T select a, enforce_not_null(b), c.
> This would work for any table type.
> {quote}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
