[jira] [Commented] (HIVE-16646) Alias in transform ... as clause shouldn't be case sensitive

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007665#comment-16007665
 ] 

Hive QA commented on HIVE-16646:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12867633/HIVE-16646.2.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 10688 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[mapjoin2] 
(batchId=236)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
 (batchId=144)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_join30]
 (batchId=149)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_3] 
(batchId=97)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] 
(batchId=97)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5212/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5212/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5212/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12867633 - PreCommit-HIVE-Build

> Alias in transform ... as clause shouldn't be case sensitive
> 
>
> Key: HIVE-16646
> URL: https://issues.apache.org/jira/browse/HIVE-16646
> Project: Hive
>  Issue Type: Bug
>  Components: hpl/sql
>Reporter: Yibing Shi
>Assignee: Yibing Shi
> Attachments: HIVE-16646.1.patch, HIVE-16646.2.patch
>
>
> Create a table like below:
> {code:sql}
> CREATE TABLE hive_bug(col1 string);
> {code}
> Run below query in Hive:
> {code}
> from hive_bug select transform(col1) using '/bin/cat' as ( string);
> {code}
> The result would be:
> {noformat}
> 0: jdbc:hive2://localhost:1> from hive_bug select transform(col1) using 
> '/bin/cat' as ( string);
> ..
> INFO  : OK
> +---+--+
> |   |
> +---+--+
> +---+--+
> {noformat}
> The output column name is ** instead of the lowercase .



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-13583) E061-14: Search Conditions

[Lefty Leverenz (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-13583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007654#comment-16007654
 ] 

Lefty Leverenz commented on HIVE-13583:
---

Does this need any wiki documentation?

> E061-14: Search Conditions
> --
>
> Key: HIVE-13583
> URL: https://issues.apache.org/jira/browse/HIVE-13583
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Carter Shanklin
>Assignee: Zoltan Haindrich
> Fix For: 3.0.0
>
> Attachments: HIVE-13583.1.patch, HIVE-13583.2.patch
>
>
> This is a part of the SQL:2011 Analytics Complete Umbrella JIRA HIVE-13554. 
> Support for various forms of search conditions are mandatory in the SQL 
> standard. For example, " is not true;" Hive should support those 
> forms mandated by the standard.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16555) Add a new thrift API call for get_metastore_uuid

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007640#comment-16007640
 ] 

Hive QA commented on HIVE-16555:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12867643/HIVE-16555.08.patch

{color:green}SUCCESS:{color} +1 due to 5 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 10697 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[schema_evol_orc_acid_part_update]
 (batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
 (batchId=144)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_3] 
(batchId=97)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] 
(batchId=97)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5211/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5211/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5211/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12867643 - PreCommit-HIVE-Build

> Add a new thrift API call for get_metastore_uuid
> 
>
> Key: HIVE-16555
> URL: https://issues.apache.org/jira/browse/HIVE-16555
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
> Attachments: HIVE-16555.01.patch, HIVE-16555.02.patch, 
> HIVE-16555.03.patch, HIVE-16555.04.patch, HIVE-16555.05.patch, 
> HIVE-16555.06.patch, HIVE-16555.07.patch, HIVE-16555.08.patch
>
>
> Sub-task of the main JIRA to add the new thrift API



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16659) Query plan should reflect hive.spark.use.groupby.shuffle

[Rui Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rui Li updated HIVE-16659:
--
Component/s: Spark

> Query plan should reflect hive.spark.use.groupby.shuffle
> 
>
> Key: HIVE-16659
> URL: https://issues.apache.org/jira/browse/HIVE-16659
> Project: Hive
>  Issue Type: Bug
>  Components: Spark
>Reporter: Rui Li
>Assignee: Rui Li
>
> It's useful to show the shuffle type used in the query plan. Currently it 
> shows "GROUP" no matter what we set for hive.spark.use.groupby.shuffle.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (HIVE-16659) Query plan should reflect hive.spark.use.groupby.shuffle

[Rui Li (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Rui Li reassigned HIVE-16659:
-


> Query plan should reflect hive.spark.use.groupby.shuffle
> 
>
> Key: HIVE-16659
> URL: https://issues.apache.org/jira/browse/HIVE-16659
> Project: Hive
>  Issue Type: Bug
>Reporter: Rui Li
>Assignee: Rui Li
>
> It's useful to show the shuffle type used in the query plan. Currently it 
> shows "GROUP" no matter what we set for hive.spark.use.groupby.shuffle.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16658) TestTimestampTZ.java has missed the ASF header

[Rui Li (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007607#comment-16007607
 ] 

Rui Li commented on HIVE-16658:
---

+1. Thanks for the fix, [~txhsj]

> TestTimestampTZ.java has missed the ASF header
> --
>
> Key: HIVE-16658
> URL: https://issues.apache.org/jira/browse/HIVE-16658
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Saijin Huang
>Assignee: Saijin Huang
>Priority: Minor
> Attachments: HIVE-16658.1.patch
>
>
> when i test the issue HIVE-14412, TestTimestampTZ has missed the ASF header



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16634) LLAP Use a pool of connections to a single AM from a daemon

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007605#comment-16007605
 ] 

Hive QA commented on HIVE-16634:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12867626/HIVE-16634.02.patch

{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 20 failed/errored test(s), 10688 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
 (batchId=144)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_3] 
(batchId=97)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] 
(batchId=97)
org.apache.hadoop.hive.llap.daemon.impl.TestTaskExecutorService.testDoKillMultiple
 (batchId=287)
org.apache.hadoop.hive.llap.daemon.impl.TestTaskExecutorService.testDontKillMultiple
 (batchId=287)
org.apache.hadoop.hive.llap.daemon.impl.TestTaskExecutorService.testFinishablePreeptsNonFinishable
 (batchId=287)
org.apache.hadoop.hive.llap.daemon.impl.TestTaskExecutorService.testPreemptionQueueComparator
 (batchId=287)
org.apache.hadoop.hive.llap.daemon.impl.TestTaskExecutorService.testPreemptionStateOnTaskMoveToFinishableState
 (batchId=287)
org.apache.hadoop.hive.llap.daemon.impl.TestTaskExecutorService.testPreemptionStateOnTaskMoveToNonFinishableState
 (batchId=287)
org.apache.hadoop.hive.llap.daemon.impl.TestTaskExecutorService.testWaitQueueAcceptAfterAMTaskReport
 (batchId=287)
org.apache.hadoop.hive.llap.daemon.impl.TestTaskExecutorService.testWaitQueuePreemption
 (batchId=287)
org.apache.hadoop.hive.llap.daemon.impl.comparator.TestFirstInFirstOutComparator.testWaitQueueComparator
 (batchId=287)
org.apache.hadoop.hive.llap.daemon.impl.comparator.TestFirstInFirstOutComparator.testWaitQueueComparatorParallelism
 (batchId=287)
org.apache.hadoop.hive.llap.daemon.impl.comparator.TestFirstInFirstOutComparator.testWaitQueueComparatorWithinDagPriority
 (batchId=287)
org.apache.hadoop.hive.llap.daemon.impl.comparator.TestFirstInFirstOutComparator.testWaitQueueComparatorWithinSameDagPriority
 (batchId=287)
org.apache.hadoop.hive.llap.daemon.impl.comparator.TestShortestJobFirstComparator.testWaitQueueComparator
 (batchId=287)
org.apache.hadoop.hive.llap.daemon.impl.comparator.TestShortestJobFirstComparator.testWaitQueueComparatorAging
 (batchId=287)
org.apache.hadoop.hive.llap.daemon.impl.comparator.TestShortestJobFirstComparator.testWaitQueueComparatorParallelism
 (batchId=287)
org.apache.hadoop.hive.llap.daemon.impl.comparator.TestShortestJobFirstComparator.testWaitQueueComparatorWithinDagPriority
 (batchId=287)
org.apache.hadoop.hive.llap.daemon.impl.comparator.TestShortestJobFirstComparator.testWaitQueueComparatorWithinSameDagPriority
 (batchId=287)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5210/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5210/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5210/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 20 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12867626 - PreCommit-HIVE-Build

> LLAP Use a pool of connections to a single AM from a daemon
> ---
>
> Key: HIVE-16634
> URL: https://issues.apache.org/jira/browse/HIVE-16634
> Project: Hive
>  Issue Type: Bug
>  Components: llap
>Reporter: Rajesh Balamohan
>Assignee: Siddharth Seth
> Attachments: HIVE-16634.01.patch, HIVE-16634.02.patch, 
> locked-threads-ipc.png
>
>
> !locked-threads-ipc.png!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-14731) Use Tez cartesian product edge in Hive (unpartitioned case only)

[Zhiyuan Yang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-14731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007602#comment-16007602
 ] 

Zhiyuan Yang commented on HIVE-14731:
-

Thanks [~hagleitn] for review! New patch was uploaded to address your comment.

> Use Tez cartesian product edge in Hive (unpartitioned case only)
> 
>
> Key: HIVE-14731
> URL: https://issues.apache.org/jira/browse/HIVE-14731
> Project: Hive
>  Issue Type: Bug
>Reporter: Zhiyuan Yang
>Assignee: Zhiyuan Yang
> Attachments: HIVE-14731.10.patch, HIVE-14731.11.patch, 
> HIVE-14731.12.patch, HIVE-14731.13.patch, HIVE-14731.14.patch, 
> HIVE-14731.15.patch, HIVE-14731.16.patch, HIVE-14731.17.patch, 
> HIVE-14731.1.patch, HIVE-14731.2.patch, HIVE-14731.3.patch, 
> HIVE-14731.4.patch, HIVE-14731.5.patch, HIVE-14731.6.patch, 
> HIVE-14731.7.patch, HIVE-14731.8.patch, HIVE-14731.9.patch
>
>
> Given cartesian product edge is available in Tez now (see TEZ-3230), let's 
> integrate it into Hive on Tez. This allows us to have more than one reducer 
> in cross product queries.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-14731) Use Tez cartesian product edge in Hive (unpartitioned case only)

[Zhiyuan Yang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-14731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhiyuan Yang updated HIVE-14731:

Attachment: HIVE-14731.17.patch

> Use Tez cartesian product edge in Hive (unpartitioned case only)
> 
>
> Key: HIVE-14731
> URL: https://issues.apache.org/jira/browse/HIVE-14731
> Project: Hive
>  Issue Type: Bug
>Reporter: Zhiyuan Yang
>Assignee: Zhiyuan Yang
> Attachments: HIVE-14731.10.patch, HIVE-14731.11.patch, 
> HIVE-14731.12.patch, HIVE-14731.13.patch, HIVE-14731.14.patch, 
> HIVE-14731.15.patch, HIVE-14731.16.patch, HIVE-14731.17.patch, 
> HIVE-14731.1.patch, HIVE-14731.2.patch, HIVE-14731.3.patch, 
> HIVE-14731.4.patch, HIVE-14731.5.patch, HIVE-14731.6.patch, 
> HIVE-14731.7.patch, HIVE-14731.8.patch, HIVE-14731.9.patch
>
>
> Given cartesian product edge is available in Tez now (see TEZ-3230), let's 
> integrate it into Hive on Tez. This allows us to have more than one reducer 
> in cross product queries.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-14731) Use Tez cartesian product edge in Hive (unpartitioned case only)

[Zhiyuan Yang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-14731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Zhiyuan Yang updated HIVE-14731:

Status: Patch Available  (was: Open)

> Use Tez cartesian product edge in Hive (unpartitioned case only)
> 
>
> Key: HIVE-14731
> URL: https://issues.apache.org/jira/browse/HIVE-14731
> Project: Hive
>  Issue Type: Bug
>Reporter: Zhiyuan Yang
>Assignee: Zhiyuan Yang
> Attachments: HIVE-14731.10.patch, HIVE-14731.11.patch, 
> HIVE-14731.12.patch, HIVE-14731.13.patch, HIVE-14731.14.patch, 
> HIVE-14731.15.patch, HIVE-14731.16.patch, HIVE-14731.17.patch, 
> HIVE-14731.1.patch, HIVE-14731.2.patch, HIVE-14731.3.patch, 
> HIVE-14731.4.patch, HIVE-14731.5.patch, HIVE-14731.6.patch, 
> HIVE-14731.7.patch, HIVE-14731.8.patch, HIVE-14731.9.patch
>
>
> Given cartesian product edge is available in Tez now (see TEZ-3230), let's 
> integrate it into Hive on Tez. This allows us to have more than one reducer 
> in cross product queries.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16658) TestTimestampTZ.java has missed the ASF header

[Saijin Huang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Saijin Huang updated HIVE-16658:

Description: when i test the issue HIVE-14412, TestTimestampTZ has missed 
the ASF header

> TestTimestampTZ.java has missed the ASF header
> --
>
> Key: HIVE-16658
> URL: https://issues.apache.org/jira/browse/HIVE-16658
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Saijin Huang
>Assignee: Saijin Huang
>Priority: Minor
> Attachments: HIVE-16658.1.patch
>
>
> when i test the issue HIVE-14412, TestTimestampTZ has missed the ASF header



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16658) TestTimestampTZ.java has missed the ASF header

[Saijin Huang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Saijin Huang updated HIVE-16658:

Status: Patch Available  (was: Open)

> TestTimestampTZ.java has missed the ASF header
> --
>
> Key: HIVE-16658
> URL: https://issues.apache.org/jira/browse/HIVE-16658
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Saijin Huang
>Assignee: Saijin Huang
>Priority: Minor
> Attachments: HIVE-16658.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (HIVE-16658) TestTimestampTZ.java has missed the ASF header

[Saijin Huang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Saijin Huang reassigned HIVE-16658:
---


> TestTimestampTZ.java has missed the ASF header
> --
>
> Key: HIVE-16658
> URL: https://issues.apache.org/jira/browse/HIVE-16658
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Saijin Huang
>Assignee: Saijin Huang
>Priority: Minor
> Attachments: HIVE-16658.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16658) TestTimestampTZ.java has missed the ASF header

[Saijin Huang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16658?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Saijin Huang updated HIVE-16658:

Attachment: HIVE-16658.1.patch

> TestTimestampTZ.java has missed the ASF header
> --
>
> Key: HIVE-16658
> URL: https://issues.apache.org/jira/browse/HIVE-16658
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Saijin Huang
>Assignee: Saijin Huang
>Priority: Minor
> Attachments: HIVE-16658.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-14389) Beeline should not output query and prompt to stdout

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-14389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007567#comment-16007567
 ] 

Hive QA commented on HIVE-14389:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12867625/HIVE-14389.04.patch

{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 10687 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
 (batchId=144)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_3] 
(batchId=97)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] 
(batchId=97)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5209/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5209/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5209/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12867625 - PreCommit-HIVE-Build

> Beeline should not output query and prompt to stdout
> 
>
> Key: HIVE-14389
> URL: https://issues.apache.org/jira/browse/HIVE-14389
> Project: Hive
>  Issue Type: Improvement
>  Components: Beeline
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
> Attachments: HIVE-14389.01.patch, HIVE-14389.02.patch, 
> HIVE-14389.03.patch, HIVE-14389.04.patch
>
>
> It seems that the Beeline prints the query along with the results in the 
> stdout when a script file is passed. The output file in the example below 
> needs to only have the results and not the query.
> {noformat}
> .vihang-MBP:bin vihang$ ./beeline --showheader=false --outformat=tsv2 -u 
> "jdbc:hive2://localhost:1" -f /tmp/query.sql > /tmp/query.out 2> 
> /tmp/query.err
> OK
> $ cat /tmp/query.out
> 1: jdbc:hive2://localhost:1/default> select * from likes limit 4;
> +---+--+--+
> | 1 | chocolate|
> | 1 | car  |
> | 1 | games|
> | 1 | chess|
> +---+--+--+
> 1: jdbc:hive2://localhost:1/default>
> 1: jdbc:hive2://localhost:1/default>
> $
> {noformat}
> A lot of people use HiveCLI and in order to transition from HiveCLI scripts 
> to Beeline, this needs to be taken care of. The output files generated by 
> beeline should contain only the results and nothing else.
> Similarly, when not in silent mode, query are being printed out on stdout, 
> which is adding garbage along with results, as just like HIVE CLI does, users 
> would like to have only the results on stdout, not errors/debugging info/etc, 
> like the full query. 
> Query could be printed out, no problem, as long as it is not on stdout (with 
> results), instead, it must be printed out along with the debugging info.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16143) Improve msck repair batching

[Lefty Leverenz (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007566#comment-16007566
 ] 

Lefty Leverenz commented on HIVE-16143:
---

I left some minor doc comments on the review board.

> Improve msck repair batching
> 
>
> Key: HIVE-16143
> URL: https://issues.apache.org/jira/browse/HIVE-16143
> Project: Hive
>  Issue Type: Improvement
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
> Attachments: HIVE-16143.01.patch, HIVE-16143.02.patch, 
> HIVE-16143.03.patch, HIVE-16143.04.patch, HIVE-16143.05.patch, 
> HIVE-16143.06.patch, HIVE-16143.07.patch, HIVE-16143.08.patch
>
>
> Currently, the {{msck repair table}} command batches the number of partitions 
> created in the metastore using the config {{HIVE_MSCK_REPAIR_BATCH_SIZE}}. 
> Following snippet shows the batching logic. There can be couple of 
> improvements to this batching logic:
> {noformat} 
> int batch_size = conf.getIntVar(ConfVars.HIVE_MSCK_REPAIR_BATCH_SIZE);
>   if (batch_size > 0 && partsNotInMs.size() > batch_size) {
> int counter = 0;
> for (CheckResult.PartitionResult part : partsNotInMs) {
>   counter++;
>   
> apd.addPartition(Warehouse.makeSpecFromName(part.getPartitionName()), null);
>   repairOutput.add("Repair: Added partition to metastore " + 
> msckDesc.getTableName()
>   + ':' + part.getPartitionName());
>   if (counter % batch_size == 0 || counter == 
> partsNotInMs.size()) {
> db.createPartitions(apd);
> apd = new AddPartitionDesc(table.getDbName(), 
> table.getTableName(), false);
>   }
> }
>   } else {
> for (CheckResult.PartitionResult part : partsNotInMs) {
>   
> apd.addPartition(Warehouse.makeSpecFromName(part.getPartitionName()), null);
>   repairOutput.add("Repair: Added partition to metastore " + 
> msckDesc.getTableName()
>   + ':' + part.getPartitionName());
> }
> db.createPartitions(apd);
>   }
> } catch (Exception e) {
>   LOG.info("Could not bulk-add partitions to metastore; trying one by 
> one", e);
>   repairOutput.clear();
>   msckAddPartitionsOneByOne(db, table, partsNotInMs, repairOutput);
> }
> {noformat}
> 1. If the batch size is too aggressive the code falls back to adding 
> partitions one by one which is almost always very slow. It is easily possible 
> that users increase the batch size to higher value to make the command run 
> faster but end up with a worse performance because code falls back to adding 
> one by one. Users are then expected to determine the tuned value of batch 
> size which works well for their environment. I think the code could handle 
> this situation better by exponentially decaying the batch size instead of 
> falling back to one by one.
> 2. The other issue with this implementation is if lets say first batch 
> succeeds and the second one fails, the code tries to add all the partitions 
> one by one irrespective of whether some of the were successfully added or 
> not. If we need to fall back to one by one we should atleast remove the ones 
> which we know for sure are already added successfully.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16233) llap: Query failed with AllocatorOutOfMemoryException

[Sergey Shelukhin (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16233?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sergey Shelukhin updated HIVE-16233:

Attachment: HIVE-16233.05.patch

Updated patch. Work for me in UT and in cluster, but needs more perf testing, 
probably most of the correctness checks added need to be moved to asserts

> llap: Query failed with AllocatorOutOfMemoryException
> -
>
> Key: HIVE-16233
> URL: https://issues.apache.org/jira/browse/HIVE-16233
> Project: Hive
>  Issue Type: Bug
>  Components: llap
>Reporter: Siddharth Seth
>Assignee: Sergey Shelukhin
> Attachments: HIVE-16233.01.patch, HIVE-16233.02.patch, 
> HIVE-16233.03.patch, HIVE-16233.04.patch, HIVE-16233.05.patch
>
>
> {code}
> TaskAttempt 5 failed, info=[Error: Error while running task ( failure ) : 
> attempt_1488231257387_2288_25_05_56_5:java.lang.RuntimeException: 
> org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: 
> java.io.IOException: 
> org.apache.hadoop.hive.common.io.Allocator$AllocatorOutOfMemoryException: 
> Failed to allocate 262144; at 0 out of 1
> at 
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:211)
> at 
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.run(TezProcessor.java:168)
> at 
> org.apache.tez.runtime.LogicalIOProcessorRuntimeTask.run(LogicalIOProcessorRuntimeTask.java:370)
> at 
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:73)
> at 
> org.apache.tez.runtime.task.TaskRunner2Callable$1.run(TaskRunner2Callable.java:61)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1724)
> at 
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:61)
> at 
> org.apache.tez.runtime.task.TaskRunner2Callable.callInternal(TaskRunner2Callable.java:37)
> at org.apache.tez.common.CallableWithNdc.call(CallableWithNdc.java:36)
> at 
> org.apache.hadoop.hive.llap.daemon.impl.StatsRecordingThreadPool$WrappedCallable.call(StatsRecordingThreadPool.java:110)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: 
> java.io.IOException: java.io.IOException: 
> org.apache.hadoop.hive.common.io.Allocator$AllocatorOutOfMemoryException: 
> Failed to allocate 262144; at 0 out of 1
> at 
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:74)
> at 
> org.apache.hadoop.hive.ql.exec.tez.MapRecordProcessor.run(MapRecordProcessor.java:419)
> at 
> org.apache.hadoop.hive.ql.exec.tez.TezProcessor.initializeAndRunProcessor(TezProcessor.java:185)
> ... 15 more
> Caused by: java.io.IOException: java.io.IOException: 
> org.apache.hadoop.hive.common.io.Allocator$AllocatorOutOfMemoryException: 
> Failed to allocate 262144; at 0 out of 1
> at 
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerChain.handleRecordReaderNextException(HiveIOExceptionHandlerChain.java:121)
> at 
> org.apache.hadoop.hive.io.HiveIOExceptionHandlerUtil.handleRecordReaderNextException(HiveIOExceptionHandlerUtil.java:77)
> at 
> org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.doNext(HiveContextAwareRecordReader.java:365)
> at 
> org.apache.hadoop.hive.ql.io.HiveRecordReader.doNext(HiveRecordReader.java:79)
> at 
> org.apache.hadoop.hive.ql.io.HiveRecordReader.doNext(HiveRecordReader.java:33)
> at 
> org.apache.hadoop.hive.ql.io.HiveContextAwareRecordReader.next(HiveContextAwareRecordReader.java:116)
> at 
> org.apache.hadoop.mapred.split.TezGroupedSplitsInputFormat$TezGroupedSplitsRecordReader.next(TezGroupedSplitsInputFormat.java:151)
> at 
> org.apache.tez.mapreduce.lib.MRReaderMapred.next(MRReaderMapred.java:116)
> at 
> org.apache.hadoop.hive.ql.exec.tez.MapRecordSource.pushRecord(MapRecordSource.java:62)
> ... 17 more
> Caused by: java.io.IOException: 
> org.apache.hadoop.hive.common.io.Allocator$AllocatorOutOfMemoryException: 
> Failed to allocate 262144; at 0 out of 1
> at 
> org.apache.hadoop.hive.ql.io.orc.encoded.EncodedReaderImpl.readEncodedColumns(EncodedReaderImpl.java:425)
> at 
> org.apache.hadoop.hive.llap.io.encoded.OrcEncodedDataReader.performDataRead(OrcEncodedDataReader.java:413)
> 

[jira] [Commented] (HIVE-16645) Commands.java has missed the catch statement and has some code format errors

[Saijin Huang (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007539#comment-16007539
 ] 

Saijin Huang commented on HIVE-16645:
-

hi,[~aihuaxu],can you plz take a look?

> Commands.java has missed the catch statement and has some code format errors
> 
>
> Key: HIVE-16645
> URL: https://issues.apache.org/jira/browse/HIVE-16645
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Saijin Huang
>Assignee: Saijin Huang
>Priority: Minor
> Attachments: HIVE-16645.1.patch
>
>
> In commands.java, the catch statement is missing and the Resultset statement 
> is not closed.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16645) Commands.java has missed the catch statement and has some code format errors

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007533#comment-16007533
 ] 

ASF GitHub Bot commented on HIVE-16645:
---

GitHub user txhsj opened a pull request:

https://github.com/apache/hive/pull/183

HIVE-16645 Commands.java has missed the catch statement and has some …

In commands.java, the catch statement is missing and the Resultset 
statement is not closed.

You can merge this pull request into a Git repository by running:

$ git pull https://github.com/txhsj/hive master

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/hive/pull/183.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #183


commit 706766af52c256999c08f5367d1ec0e02f73600e
Author: 黄赛金10202350 
Date:   2017-05-12T02:15:23Z

HIVE-16645 Commands.java has missed the catch statement and has some code 
format errors




> Commands.java has missed the catch statement and has some code format errors
> 
>
> Key: HIVE-16645
> URL: https://issues.apache.org/jira/browse/HIVE-16645
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Saijin Huang
>Assignee: Saijin Huang
>Priority: Minor
> Attachments: HIVE-16645.1.patch
>
>
> In commands.java, the catch statement is missing and the Resultset statement 
> is not closed.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16555) Add a new thrift API call for get_metastore_uuid

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007526#comment-16007526
 ] 

Hive QA commented on HIVE-16555:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12867643/HIVE-16555.08.patch

{color:green}SUCCESS:{color} +1 due to 5 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 10697 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_3] 
(batchId=97)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] 
(batchId=97)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5208/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5208/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5208/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12867643 - PreCommit-HIVE-Build

> Add a new thrift API call for get_metastore_uuid
> 
>
> Key: HIVE-16555
> URL: https://issues.apache.org/jira/browse/HIVE-16555
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
> Attachments: HIVE-16555.01.patch, HIVE-16555.02.patch, 
> HIVE-16555.03.patch, HIVE-16555.04.patch, HIVE-16555.05.patch, 
> HIVE-16555.06.patch, HIVE-16555.07.patch, HIVE-16555.08.patch
>
>
> Sub-task of the main JIRA to add the new thrift API



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16646) Alias in transform ... as clause shouldn't be case sensitive

[Yongzhi Chen (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007511#comment-16007511
 ] 

Yongzhi Chen commented on HIVE-16646:
-

The patch LGTM +1

> Alias in transform ... as clause shouldn't be case sensitive
> 
>
> Key: HIVE-16646
> URL: https://issues.apache.org/jira/browse/HIVE-16646
> Project: Hive
>  Issue Type: Bug
>  Components: hpl/sql
>Reporter: Yibing Shi
>Assignee: Yibing Shi
> Attachments: HIVE-16646.1.patch, HIVE-16646.2.patch
>
>
> Create a table like below:
> {code:sql}
> CREATE TABLE hive_bug(col1 string);
> {code}
> Run below query in Hive:
> {code}
> from hive_bug select transform(col1) using '/bin/cat' as ( string);
> {code}
> The result would be:
> {noformat}
> 0: jdbc:hive2://localhost:1> from hive_bug select transform(col1) using 
> '/bin/cat' as ( string);
> ..
> INFO  : OK
> +---+--+
> |   |
> +---+--+
> +---+--+
> {noformat}
> The output column name is ** instead of the lowercase .



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16646) Alias in transform ... as clause shouldn't be case sensitive

[Yongzhi Chen (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007510#comment-16007510
 ] 

Yongzhi Chen commented on HIVE-16646:
-

The failures are not related. 
Following two tests passed in my local machine:
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge_incompat2] 
(batchId=79)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[columnstats_part_coltype]
 (batchId=156)
The rest failures aged more than 2.




> Alias in transform ... as clause shouldn't be case sensitive
> 
>
> Key: HIVE-16646
> URL: https://issues.apache.org/jira/browse/HIVE-16646
> Project: Hive
>  Issue Type: Bug
>  Components: hpl/sql
>Reporter: Yibing Shi
>Assignee: Yibing Shi
> Attachments: HIVE-16646.1.patch, HIVE-16646.2.patch
>
>
> Create a table like below:
> {code:sql}
> CREATE TABLE hive_bug(col1 string);
> {code}
> Run below query in Hive:
> {code}
> from hive_bug select transform(col1) using '/bin/cat' as ( string);
> {code}
> The result would be:
> {noformat}
> 0: jdbc:hive2://localhost:1> from hive_bug select transform(col1) using 
> '/bin/cat' as ( string);
> ..
> INFO  : OK
> +---+--+
> |   |
> +---+--+
> +---+--+
> {noformat}
> The output column name is ** instead of the lowercase .



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16655) LLAP: Avoid preempting fragments before they enter the running state

[Sergey Shelukhin (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007503#comment-16007503
 ] 

Sergey Shelukhin commented on HIVE-16655:
-

+1 pending tests... cluster tests might be helpful too

> LLAP: Avoid preempting fragments before they enter the running state
> 
>
> Key: HIVE-16655
> URL: https://issues.apache.org/jira/browse/HIVE-16655
> Project: Hive
>  Issue Type: Bug
>Reporter: Siddharth Seth
>Assignee: Siddharth Seth
> Attachments: HIVE-16655.01.patch
>
>
> Currently in the AM, fragments may be preempted as soon as they are 
> allocated, without knowing whether they will move into the RUNNING state or 
> not. Leads to a lot of unnecessary kills.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-14389) Beeline should not output query and prompt to stdout

[Thejas M Nair (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-14389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007476#comment-16007476
 ] 

Thejas M Nair commented on HIVE-14389:
--

+1


> Beeline should not output query and prompt to stdout
> 
>
> Key: HIVE-14389
> URL: https://issues.apache.org/jira/browse/HIVE-14389
> Project: Hive
>  Issue Type: Improvement
>  Components: Beeline
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
> Attachments: HIVE-14389.01.patch, HIVE-14389.02.patch, 
> HIVE-14389.03.patch, HIVE-14389.04.patch
>
>
> It seems that the Beeline prints the query along with the results in the 
> stdout when a script file is passed. The output file in the example below 
> needs to only have the results and not the query.
> {noformat}
> .vihang-MBP:bin vihang$ ./beeline --showheader=false --outformat=tsv2 -u 
> "jdbc:hive2://localhost:1" -f /tmp/query.sql > /tmp/query.out 2> 
> /tmp/query.err
> OK
> $ cat /tmp/query.out
> 1: jdbc:hive2://localhost:1/default> select * from likes limit 4;
> +---+--+--+
> | 1 | chocolate|
> | 1 | car  |
> | 1 | games|
> | 1 | chess|
> +---+--+--+
> 1: jdbc:hive2://localhost:1/default>
> 1: jdbc:hive2://localhost:1/default>
> $
> {noformat}
> A lot of people use HiveCLI and in order to transition from HiveCLI scripts 
> to Beeline, this needs to be taken care of. The output files generated by 
> beeline should contain only the results and nothing else.
> Similarly, when not in silent mode, query are being printed out on stdout, 
> which is adding garbage along with results, as just like HIVE CLI does, users 
> would like to have only the results on stdout, not errors/debugging info/etc, 
> like the full query. 
> Query could be printed out, no problem, as long as it is not on stdout (with 
> results), instead, it must be printed out along with the debugging info.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16646) Alias in transform ... as clause shouldn't be case sensitive

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007464#comment-16007464
 ] 

Hive QA commented on HIVE-16646:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12867633/HIVE-16646.2.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 10688 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge_incompat2] 
(batchId=79)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[columnstats_part_coltype]
 (batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_join30]
 (batchId=149)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_3] 
(batchId=97)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] 
(batchId=97)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5206/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5206/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5206/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12867633 - PreCommit-HIVE-Build

> Alias in transform ... as clause shouldn't be case sensitive
> 
>
> Key: HIVE-16646
> URL: https://issues.apache.org/jira/browse/HIVE-16646
> Project: Hive
>  Issue Type: Bug
>  Components: hpl/sql
>Reporter: Yibing Shi
>Assignee: Yibing Shi
> Attachments: HIVE-16646.1.patch, HIVE-16646.2.patch
>
>
> Create a table like below:
> {code:sql}
> CREATE TABLE hive_bug(col1 string);
> {code}
> Run below query in Hive:
> {code}
> from hive_bug select transform(col1) using '/bin/cat' as ( string);
> {code}
> The result would be:
> {noformat}
> 0: jdbc:hive2://localhost:1> from hive_bug select transform(col1) using 
> '/bin/cat' as ( string);
> ..
> INFO  : OK
> +---+--+
> |   |
> +---+--+
> +---+--+
> {noformat}
> The output column name is ** instead of the lowercase .



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16655) LLAP: Avoid preempting fragments before they enter the running state

[Siddharth Seth (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Siddharth Seth updated HIVE-16655:
--
Attachment: HIVE-16655.01.patch

The patch.
Relies on the start notifications from Tez, to detemine the state of a 
fragment, and preempt accordingly.

cc [~prasanth_j], [~sershe] for review.

Cannot submit to jenkins since the Tez patch introduces new APIs.

> LLAP: Avoid preempting fragments before they enter the running state
> 
>
> Key: HIVE-16655
> URL: https://issues.apache.org/jira/browse/HIVE-16655
> Project: Hive
>  Issue Type: Bug
>Reporter: Siddharth Seth
>Assignee: Siddharth Seth
> Attachments: HIVE-16655.01.patch
>
>
> Currently in the AM, fragments may be preempted as soon as they are 
> allocated, without knowing whether they will move into the RUNNING state or 
> not. Leads to a lot of unnecessary kills.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16602) Implement shared scans with Tez

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007413#comment-16007413
 ] 

Hive QA commented on HIVE-16602:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12867559/HIVE-16602.03.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 10687 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[explainuser_2] 
(batchId=141)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mapjoin46] 
(batchId=154)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[multiMapJoin2]
 (batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_scalar]
 (batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_select]
 (batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_smb_main]
 (batchId=148)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
 (batchId=144)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_join30]
 (batchId=149)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_3] 
(batchId=97)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] 
(batchId=97)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query1] (batchId=231)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query30] 
(batchId=231)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query6] (batchId=231)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query81] 
(batchId=231)
org.apache.hadoop.hive.cli.TestPerfCliDriver.testCliDriver[query9] (batchId=231)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5205/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5205/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5205/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 15 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12867559 - PreCommit-HIVE-Build

> Implement shared scans with Tez
> ---
>
> Key: HIVE-16602
> URL: https://issues.apache.org/jira/browse/HIVE-16602
> Project: Hive
>  Issue Type: New Feature
>  Components: Physical Optimizer
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
> Attachments: HIVE-16602.01.patch, HIVE-16602.02.patch, 
> HIVE-16602.03.patch, HIVE-16602.patch
>
>
> Given a query plan, the goal is to identify scans on input tables that can be 
> merged so the data is read only once. Optimization will be carried out at the 
> physical level.
> In the longer term, identification of equivalent expressions and 
> reutilization of intermediary results should be done at the logical layer via 
> Spool operator.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16324) Truncate table should not work when EXTERNAL property of table is true

[Vihang Karajgaonkar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vihang Karajgaonkar updated HIVE-16324:
---
Status: Patch Available  (was: Open)

> Truncate table should not work when EXTERNAL property of table is true
> --
>
> Key: HIVE-16324
> URL: https://issues.apache.org/jira/browse/HIVE-16324
> Project: Hive
>  Issue Type: Bug
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
> Attachments: HIVE-16324.01.patch
>
>
> Currently if you create an external table using the command {{CREATE EXTERNAL 
> TABLE table_name}} the {{TRUNCATE table table_name}} command fails as 
> expected because only managed tables should be allowed to be truncated.
> But if you set the external property of a previously managed table using 
> {{ALTER TABLE table_name SET TBLPROPERTIES('EXTERNAL'='true')}}, truncate 
> table command does not object and deletes all the data from the external 
> table.
> Eg: This works but it should not ..
> {noformat}
> 0: jdbc:hive2://localhost:1/default> create table test_ext2 (col1 string);
> No rows affected (0.424 seconds)
> 0: jdbc:hive2://localhost:1/default> alter table test_ext2 set 
> tblproperties ('EXTERNAL'='true');
> No rows affected (0.149 seconds)
> 0: jdbc:hive2://localhost:1/default> insert into table test_ext2 values 
> ("test");
> WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the 
> future versions. Consider using a different execution engine (i.e. spark, 
> tez) or using Hive 1.X releases.
> No rows affected (3.447 seconds)
> 0: jdbc:hive2://localhost:1/default>
> 0: jdbc:hive2://localhost:1/default>
> 0: jdbc:hive2://localhost:1/default> select * from test_ext2;
> +-+
> | test_ext2.col1  |
> +-+
> | test|
> +-+
> 1 row selected (0.147 seconds)
> 0: jdbc:hive2://localhost:1/default> truncate table test_ext2;
> No rows affected (0.138 seconds)
> 0: jdbc:hive2://localhost:1/default> select * from test_ext2;
> +-+
> | test_ext2.col1  |
> +-+
> +-+
> No rows selected (0.134 seconds)
> 0: jdbc:hive2://localhost:1/default>
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16324) Truncate table should not work when EXTERNAL property of table is true

[Vihang Karajgaonkar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vihang Karajgaonkar updated HIVE-16324:
---
Attachment: HIVE-16324.01.patch

> Truncate table should not work when EXTERNAL property of table is true
> --
>
> Key: HIVE-16324
> URL: https://issues.apache.org/jira/browse/HIVE-16324
> Project: Hive
>  Issue Type: Bug
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
> Attachments: HIVE-16324.01.patch
>
>
> Currently if you create an external table using the command {{CREATE EXTERNAL 
> TABLE table_name}} the {{TRUNCATE table table_name}} command fails as 
> expected because only managed tables should be allowed to be truncated.
> But if you set the external property of a previously managed table using 
> {{ALTER TABLE table_name SET TBLPROPERTIES('EXTERNAL'='true')}}, truncate 
> table command does not object and deletes all the data from the external 
> table.
> Eg: This works but it should not ..
> {noformat}
> 0: jdbc:hive2://localhost:1/default> create table test_ext2 (col1 string);
> No rows affected (0.424 seconds)
> 0: jdbc:hive2://localhost:1/default> alter table test_ext2 set 
> tblproperties ('EXTERNAL'='true');
> No rows affected (0.149 seconds)
> 0: jdbc:hive2://localhost:1/default> insert into table test_ext2 values 
> ("test");
> WARNING: Hive-on-MR is deprecated in Hive 2 and may not be available in the 
> future versions. Consider using a different execution engine (i.e. spark, 
> tez) or using Hive 1.X releases.
> No rows affected (3.447 seconds)
> 0: jdbc:hive2://localhost:1/default>
> 0: jdbc:hive2://localhost:1/default>
> 0: jdbc:hive2://localhost:1/default> select * from test_ext2;
> +-+
> | test_ext2.col1  |
> +-+
> | test|
> +-+
> 1 row selected (0.147 seconds)
> 0: jdbc:hive2://localhost:1/default> truncate table test_ext2;
> No rows affected (0.138 seconds)
> 0: jdbc:hive2://localhost:1/default> select * from test_ext2;
> +-+
> | test_ext2.col1  |
> +-+
> +-+
> No rows selected (0.134 seconds)
> 0: jdbc:hive2://localhost:1/default>
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16555) Add a new thrift API call for get_metastore_uuid

[Alan Gates (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007391#comment-16007391
 ] 

Alan Gates commented on HIVE-16555:
---

+1

> Add a new thrift API call for get_metastore_uuid
> 
>
> Key: HIVE-16555
> URL: https://issues.apache.org/jira/browse/HIVE-16555
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
> Attachments: HIVE-16555.01.patch, HIVE-16555.02.patch, 
> HIVE-16555.03.patch, HIVE-16555.04.patch, HIVE-16555.05.patch, 
> HIVE-16555.06.patch, HIVE-16555.07.patch, HIVE-16555.08.patch
>
>
> Sub-task of the main JIRA to add the new thrift API



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15571) Support Insert into for druid storage handler

[Nishant Bangarwa (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007385#comment-16007385
 ] 

Nishant Bangarwa commented on HIVE-15571:
-

[~bslim][~jcamachorodriguez] attached new patch which supports multiple insert 
into in parallel and does not need locking. Also updated the review request. 
Please review again. 

> Support Insert into for druid storage handler
> -
>
> Key: HIVE-15571
> URL: https://issues.apache.org/jira/browse/HIVE-15571
> Project: Hive
>  Issue Type: New Feature
>  Components: Druid integration
>Reporter: slim bouguerra
>Assignee: Nishant Bangarwa
> Attachments: HIVE-15571.01.patch, HIVE-15571.02.patch
>
>
> Add support of inset into operator for druid storage handler.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-15571) Support Insert into for druid storage handler

[Nishant Bangarwa (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-15571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nishant Bangarwa updated HIVE-15571:

Attachment: HIVE-15571.02.patch

> Support Insert into for druid storage handler
> -
>
> Key: HIVE-15571
> URL: https://issues.apache.org/jira/browse/HIVE-15571
> Project: Hive
>  Issue Type: New Feature
>  Components: Druid integration
>Reporter: slim bouguerra
>Assignee: Nishant Bangarwa
> Attachments: HIVE-15571.01.patch, HIVE-15571.02.patch
>
>
> Add support of inset into operator for druid storage handler.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16555) Add a new thrift API call for get_metastore_uuid

[Vihang Karajgaonkar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007381#comment-16007381
 ] 

Vihang Karajgaonkar commented on HIVE-16555:


The metastore install and upgrade script changes are already committed as part 
of another sub-task of this JIRA in HIVE-16556

> Add a new thrift API call for get_metastore_uuid
> 
>
> Key: HIVE-16555
> URL: https://issues.apache.org/jira/browse/HIVE-16555
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
> Attachments: HIVE-16555.01.patch, HIVE-16555.02.patch, 
> HIVE-16555.03.patch, HIVE-16555.04.patch, HIVE-16555.05.patch, 
> HIVE-16555.06.patch, HIVE-16555.07.patch, HIVE-16555.08.patch
>
>
> Sub-task of the main JIRA to add the new thrift API



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-1010) Implement INFORMATION_SCHEMA in Hive

[Gunther Hagleitner (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Gunther Hagleitner updated HIVE-1010:
-
Attachment: HIVE-1010.15.patch

> Implement INFORMATION_SCHEMA in Hive
> 
>
> Key: HIVE-1010
> URL: https://issues.apache.org/jira/browse/HIVE-1010
> Project: Hive
>  Issue Type: New Feature
>  Components: Metastore, Query Processor, Server Infrastructure
>Reporter: Jeff Hammerbacher
>Assignee: Gunther Hagleitner
> Attachments: HIVE-1010.10.patch, HIVE-1010.11.patch, 
> HIVE-1010.12.patch, HIVE-1010.13.patch, HIVE-1010.14.patch, 
> HIVE-1010.15.patch, HIVE-1010.7.patch, HIVE-1010.8.patch, HIVE-1010.9.patch
>
>
> INFORMATION_SCHEMA is part of the SQL92 standard and would be useful to 
> implement using our metastore.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16656) CachedStore: make prewarm and background cache update multithreaded

[Vaibhav Gumashta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vaibhav Gumashta updated HIVE-16656:

Issue Type: Sub-task  (was: Improvement)
Parent: HIVE-16520

> CachedStore: make prewarm and background cache update multithreaded
> ---
>
> Key: HIVE-16656
> URL: https://issues.apache.org/jira/browse/HIVE-16656
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16579) CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

[Vaibhav Gumashta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vaibhav Gumashta updated HIVE-16579:

Issue Type: Sub-task  (was: Improvement)
Parent: HIVE-16520

> CachedStore: improvements to partition col stats caching and cache column 
> stats for unpartitioned table
> ---
>
> Key: HIVE-16579
> URL: https://issues.apache.org/jira/browse/HIVE-16579
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
> Attachments: HIVE-16579.1.patch, HIVE-16579.2.patch, 
> HIVE-16579.3.patch
>
>
> 1. Update stats cache when partitions/table is dropped.
> 2. Update cached partition col stats in the background cache update thread. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16586) Fix Unit test failures when CachedStore is enabled

[Vaibhav Gumashta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vaibhav Gumashta updated HIVE-16586:

Issue Type: Sub-task  (was: Bug)
Parent: HIVE-16520

> Fix Unit test failures when CachedStore is enabled
> --
>
> Key: HIVE-16586
> URL: https://issues.apache.org/jira/browse/HIVE-16586
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Reporter: Daniel Dai
>Assignee: Daniel Dai
> Attachments: enable_cachedstore.patch, HIVE-16586.1.patch, 
> HIVE-16586.2.patch
>
>
> Though we don't plan to turn on CachedStore by default, we want to make sure 
> unit tests pass with CachedStore. I turn on CachedStore in the patch in order 
> to run unit tests with it, but I will turn off CachedStore when committing.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16555) Add a new thrift API call for get_metastore_uuid

[Alan Gates (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007360#comment-16007360
 ] 

Alan Gates commented on HIVE-16555:
---

I missed there was a unique key constraint on the params.  So this looks good.

Were you planning on making the metastore install and upgrade script changes 
separately?

> Add a new thrift API call for get_metastore_uuid
> 
>
> Key: HIVE-16555
> URL: https://issues.apache.org/jira/browse/HIVE-16555
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
> Attachments: HIVE-16555.01.patch, HIVE-16555.02.patch, 
> HIVE-16555.03.patch, HIVE-16555.04.patch, HIVE-16555.05.patch, 
> HIVE-16555.06.patch, HIVE-16555.07.patch, HIVE-16555.08.patch
>
>
> Sub-task of the main JIRA to add the new thrift API



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16645) Commands.java has missed the catch statement and has some code format errors

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007358#comment-16007358
 ] 

Hive QA commented on HIVE-16645:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12867552/HIVE-16645.1.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 10687 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[bucketsortoptimize_insert_1]
 (batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[orc_merge_diff_fs] 
(batchId=1)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
 (batchId=144)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_3] 
(batchId=97)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] 
(batchId=97)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5204/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5204/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5204/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12867552 - PreCommit-HIVE-Build

> Commands.java has missed the catch statement and has some code format errors
> 
>
> Key: HIVE-16645
> URL: https://issues.apache.org/jira/browse/HIVE-16645
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.0.0
>Reporter: Saijin Huang
>Assignee: Saijin Huang
>Priority: Minor
> Attachments: HIVE-16645.1.patch
>
>
> In commands.java, the catch statement is missing and the Resultset statement 
> is not closed.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (HIVE-16656) CachedStore: make prewarm and background cache update multithreaded

[Vaibhav Gumashta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vaibhav Gumashta reassigned HIVE-16656:
---


> CachedStore: make prewarm and background cache update multithreaded
> ---
>
> Key: HIVE-16656
> URL: https://issues.apache.org/jira/browse/HIVE-16656
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-1010) Implement INFORMATION_SCHEMA in Hive

[Gunther Hagleitner (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-1010?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007355#comment-16007355
 ] 

Gunther Hagleitner commented on HIVE-1010:
--

[~thejas] I believe I have addressed all comments. Can you take another look 
please?

> Implement INFORMATION_SCHEMA in Hive
> 
>
> Key: HIVE-1010
> URL: https://issues.apache.org/jira/browse/HIVE-1010
> Project: Hive
>  Issue Type: New Feature
>  Components: Metastore, Query Processor, Server Infrastructure
>Reporter: Jeff Hammerbacher
>Assignee: Gunther Hagleitner
> Attachments: HIVE-1010.10.patch, HIVE-1010.11.patch, 
> HIVE-1010.12.patch, HIVE-1010.13.patch, HIVE-1010.14.patch, 
> HIVE-1010.7.patch, HIVE-1010.8.patch, HIVE-1010.9.patch
>
>
> INFORMATION_SCHEMA is part of the SQL92 standard and would be useful to 
> implement using our metastore.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Issue Comment Deleted] (HIVE-12790) Metastore connection leaks in HiveServer2

[Wei Zheng (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Wei Zheng updated HIVE-12790:
-
Comment: was deleted

(was: Once you get new logs/configs from customer, please remove the outdated 
attachments to avoid confusion)

> Metastore connection leaks in HiveServer2
> -
>
> Key: HIVE-12790
> URL: https://issues.apache.org/jira/browse/HIVE-12790
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 1.1.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
> Fix For: 2.0.0
>
> Attachments: HIVE-12790.2.patch, HIVE-12790.3.patch, 
> HIVE-12790.patch, snippedLog.txt
>
>
> HiveServer2 keeps opening new connections to HMS each time it launches a 
> task. These connections do not appear to be closed when the task completes 
> thus causing a HMS connection leak. "lsof" for the HS2 process shows 
> connections to port 9083.
> {code}
> 2015-12-03 04:20:56,352 INFO  [HiveServer2-Background-Pool: Thread-424756()]: 
> ql.Driver (SessionState.java:printInfo(558)) - Launching Job 11 out of 41
> 2015-12-03 04:20:56,354 INFO  [Thread-405728()]: hive.metastore 
> (HiveMetaStoreClient.java:open(311)) - Trying to connect to metastore with 
> URI thrift://:9083
> 2015-12-03 04:20:56,360 INFO  [Thread-405728()]: hive.metastore 
> (HiveMetaStoreClient.java:open(351)) - Opened a connection to metastore, 
> current connections: 14824
> 2015-12-03 04:20:56,360 INFO  [Thread-405728()]: hive.metastore 
> (HiveMetaStoreClient.java:open(400)) - Connected to metastore.
> 
> 2015-12-03 04:21:06,355 INFO  [HiveServer2-Background-Pool: Thread-424756()]: 
> ql.Driver (SessionState.java:printInfo(558)) - Launching Job 12 out of 41
> 2015-12-03 04:21:06,357 INFO  [Thread-405756()]: hive.metastore 
> (HiveMetaStoreClient.java:open(311)) - Trying to connect to metastore with 
> URI thrift://:9083
> 2015-12-03 04:21:06,362 INFO  [Thread-405756()]: hive.metastore 
> (HiveMetaStoreClient.java:open(351)) - Opened a connection to metastore, 
> current connections: 14825
> 2015-12-03 04:21:06,362 INFO  [Thread-405756()]: hive.metastore 
> (HiveMetaStoreClient.java:open(400)) - Connected to metastore.
> ...
> 2015-12-03 04:21:08,357 INFO  [HiveServer2-Background-Pool: Thread-424756()]: 
> ql.Driver (SessionState.java:printInfo(558)) - Launching Job 13 out of 41
> 2015-12-03 04:21:08,360 INFO  [Thread-405782()]: hive.metastore 
> (HiveMetaStoreClient.java:open(311)) - Trying to connect to metastore with 
> URI thrift://:9083
> 2015-12-03 04:21:08,364 INFO  [Thread-405782()]: hive.metastore 
> (HiveMetaStoreClient.java:open(351)) - Opened a connection to metastore, 
> current connections: 14826
> 2015-12-03 04:21:08,365 INFO  [Thread-405782()]: hive.metastore 
> (HiveMetaStoreClient.java:open(400)) - Connected to metastore.
> ... 
> {code}
> The TaskRunner thread starts a new SessionState each time, which creates a 
> new connection to the HMS (via Hive.get(conf).getMSC()) that is never closed.
> Even SessionState.close(), currently not being called by the TaskRunner 
> thread, does not close this connection.
> Attaching a anonymized log snippet where the number of HMS connections 
> reaches north of 25000+ connections.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-12790) Metastore connection leaks in HiveServer2

[Wei Zheng (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-12790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007341#comment-16007341
 ] 

Wei Zheng commented on HIVE-12790:
--

Once you get new logs/configs from customer, please remove the outdated 
attachments to avoid confusion

> Metastore connection leaks in HiveServer2
> -
>
> Key: HIVE-12790
> URL: https://issues.apache.org/jira/browse/HIVE-12790
> Project: Hive
>  Issue Type: Bug
>  Components: HiveServer2
>Affects Versions: 1.1.0
>Reporter: Naveen Gangam
>Assignee: Naveen Gangam
> Fix For: 2.0.0
>
> Attachments: HIVE-12790.2.patch, HIVE-12790.3.patch, 
> HIVE-12790.patch, snippedLog.txt
>
>
> HiveServer2 keeps opening new connections to HMS each time it launches a 
> task. These connections do not appear to be closed when the task completes 
> thus causing a HMS connection leak. "lsof" for the HS2 process shows 
> connections to port 9083.
> {code}
> 2015-12-03 04:20:56,352 INFO  [HiveServer2-Background-Pool: Thread-424756()]: 
> ql.Driver (SessionState.java:printInfo(558)) - Launching Job 11 out of 41
> 2015-12-03 04:20:56,354 INFO  [Thread-405728()]: hive.metastore 
> (HiveMetaStoreClient.java:open(311)) - Trying to connect to metastore with 
> URI thrift://:9083
> 2015-12-03 04:20:56,360 INFO  [Thread-405728()]: hive.metastore 
> (HiveMetaStoreClient.java:open(351)) - Opened a connection to metastore, 
> current connections: 14824
> 2015-12-03 04:20:56,360 INFO  [Thread-405728()]: hive.metastore 
> (HiveMetaStoreClient.java:open(400)) - Connected to metastore.
> 
> 2015-12-03 04:21:06,355 INFO  [HiveServer2-Background-Pool: Thread-424756()]: 
> ql.Driver (SessionState.java:printInfo(558)) - Launching Job 12 out of 41
> 2015-12-03 04:21:06,357 INFO  [Thread-405756()]: hive.metastore 
> (HiveMetaStoreClient.java:open(311)) - Trying to connect to metastore with 
> URI thrift://:9083
> 2015-12-03 04:21:06,362 INFO  [Thread-405756()]: hive.metastore 
> (HiveMetaStoreClient.java:open(351)) - Opened a connection to metastore, 
> current connections: 14825
> 2015-12-03 04:21:06,362 INFO  [Thread-405756()]: hive.metastore 
> (HiveMetaStoreClient.java:open(400)) - Connected to metastore.
> ...
> 2015-12-03 04:21:08,357 INFO  [HiveServer2-Background-Pool: Thread-424756()]: 
> ql.Driver (SessionState.java:printInfo(558)) - Launching Job 13 out of 41
> 2015-12-03 04:21:08,360 INFO  [Thread-405782()]: hive.metastore 
> (HiveMetaStoreClient.java:open(311)) - Trying to connect to metastore with 
> URI thrift://:9083
> 2015-12-03 04:21:08,364 INFO  [Thread-405782()]: hive.metastore 
> (HiveMetaStoreClient.java:open(351)) - Opened a connection to metastore, 
> current connections: 14826
> 2015-12-03 04:21:08,365 INFO  [Thread-405782()]: hive.metastore 
> (HiveMetaStoreClient.java:open(400)) - Connected to metastore.
> ... 
> {code}
> The TaskRunner thread starts a new SessionState each time, which creates a 
> new connection to the HMS (via Hive.get(conf).getMSC()) that is never closed.
> Even SessionState.close(), currently not being called by the TaskRunner 
> thread, does not close this connection.
> Attaching a anonymized log snippet where the number of HMS connections 
> reaches north of 25000+ connections.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16646) Alias in transform ... as clause shouldn't be case sensitive

[Yibing Shi (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007339#comment-16007339
 ] 

Yibing Shi commented on HIVE-16646:
---

Thank you, [~ychena]!

> Alias in transform ... as clause shouldn't be case sensitive
> 
>
> Key: HIVE-16646
> URL: https://issues.apache.org/jira/browse/HIVE-16646
> Project: Hive
>  Issue Type: Bug
>  Components: hpl/sql
>Reporter: Yibing Shi
>Assignee: Yibing Shi
> Attachments: HIVE-16646.1.patch, HIVE-16646.2.patch
>
>
> Create a table like below:
> {code:sql}
> CREATE TABLE hive_bug(col1 string);
> {code}
> Run below query in Hive:
> {code}
> from hive_bug select transform(col1) using '/bin/cat' as ( string);
> {code}
> The result would be:
> {noformat}
> 0: jdbc:hive2://localhost:1> from hive_bug select transform(col1) using 
> '/bin/cat' as ( string);
> ..
> INFO  : OK
> +---+--+
> |   |
> +---+--+
> +---+--+
> {noformat}
> The output column name is ** instead of the lowercase .



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (HIVE-16655) LLAP: Avoid preempting fragments before they enter the running state

[Siddharth Seth (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Siddharth Seth reassigned HIVE-16655:
-


> LLAP: Avoid preempting fragments before they enter the running state
> 
>
> Key: HIVE-16655
> URL: https://issues.apache.org/jira/browse/HIVE-16655
> Project: Hive
>  Issue Type: Bug
>Reporter: Siddharth Seth
>Assignee: Siddharth Seth
>
> Currently in the AM, fragments may be preempted as soon as they are 
> allocated, without knowing whether they will move into the RUNNING state or 
> not. Leads to a lot of unnecessary kills.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16559) Parquet schema evolution for partitioned tables may break if table and partition serdes differ

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007296#comment-16007296
 ] 

Hive QA commented on HIVE-16559:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12867546/HIVE-16559.02.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 10688 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_join30]
 (batchId=149)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_3] 
(batchId=97)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] 
(batchId=97)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5203/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5203/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5203/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12867546 - PreCommit-HIVE-Build

> Parquet schema evolution for partitioned tables may break if table and 
> partition serdes differ
> --
>
> Key: HIVE-16559
> URL: https://issues.apache.org/jira/browse/HIVE-16559
> Project: Hive
>  Issue Type: Bug
>  Components: Serializers/Deserializers
>Reporter: Barna Zsombor Klara
>Assignee: Barna Zsombor Klara
> Fix For: 3.0.0
>
> Attachments: HIVE-16559.01.patch, HIVE-16559.02.patch
>
>
> Parquet schema evolution should make it possible to have partitions/tables 
>  backed by files with different schemas. Hive should match the table columns 
> with file columns based on the column name if possible.
> However if the serde for a table is missing columns from the serde of a 
> partition Hive fails to match the columns together.
> Steps to reproduce:
> {code}
> CREATE TABLE myparquettable_parted
> (
>   name string,
>   favnumber int,
>   favcolor string,
>   age int,
>   favpet string
> )
> PARTITIONED BY (day string)
> STORED AS PARQUET;
> INSERT OVERWRITE TABLE myparquettable_parted
> PARTITION(day='2017-04-04')
> SELECT
>'mary' as name,
>5 AS favnumber,
>'blue' AS favcolor,
>35 AS age,
>'dog' AS favpet;
> alter table myparquettable_parted
> REPLACE COLUMNS
> (
> favnumber int,
> age int
> );   

[jira] [Updated] (HIVE-16143) Improve msck repair batching

[Vihang Karajgaonkar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16143?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vihang Karajgaonkar updated HIVE-16143:
---
Attachment: (was: HIVE-16143.08.patch)

> Improve msck repair batching
> 
>
> Key: HIVE-16143
> URL: https://issues.apache.org/jira/browse/HIVE-16143
> Project: Hive
>  Issue Type: Improvement
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
> Attachments: HIVE-16143.01.patch, HIVE-16143.02.patch, 
> HIVE-16143.03.patch, HIVE-16143.04.patch, HIVE-16143.05.patch, 
> HIVE-16143.06.patch, HIVE-16143.07.patch, HIVE-16143.08.patch
>
>
> Currently, the {{msck repair table}} command batches the number of partitions 
> created in the metastore using the config {{HIVE_MSCK_REPAIR_BATCH_SIZE}}. 
> Following snippet shows the batching logic. There can be couple of 
> improvements to this batching logic:
> {noformat} 
> int batch_size = conf.getIntVar(ConfVars.HIVE_MSCK_REPAIR_BATCH_SIZE);
>   if (batch_size > 0 && partsNotInMs.size() > batch_size) {
> int counter = 0;
> for (CheckResult.PartitionResult part : partsNotInMs) {
>   counter++;
>   
> apd.addPartition(Warehouse.makeSpecFromName(part.getPartitionName()), null);
>   repairOutput.add("Repair: Added partition to metastore " + 
> msckDesc.getTableName()
>   + ':' + part.getPartitionName());
>   if (counter % batch_size == 0 || counter == 
> partsNotInMs.size()) {
> db.createPartitions(apd);
> apd = new AddPartitionDesc(table.getDbName(), 
> table.getTableName(), false);
>   }
> }
>   } else {
> for (CheckResult.PartitionResult part : partsNotInMs) {
>   
> apd.addPartition(Warehouse.makeSpecFromName(part.getPartitionName()), null);
>   repairOutput.add("Repair: Added partition to metastore " + 
> msckDesc.getTableName()
>   + ':' + part.getPartitionName());
> }
> db.createPartitions(apd);
>   }
> } catch (Exception e) {
>   LOG.info("Could not bulk-add partitions to metastore; trying one by 
> one", e);
>   repairOutput.clear();
>   msckAddPartitionsOneByOne(db, table, partsNotInMs, repairOutput);
> }
> {noformat}
> 1. If the batch size is too aggressive the code falls back to adding 
> partitions one by one which is almost always very slow. It is easily possible 
> that users increase the batch size to higher value to make the command run 
> faster but end up with a worse performance because code falls back to adding 
> one by one. Users are then expected to determine the tuned value of batch 
> size which works well for their environment. I think the code could handle 
> this situation better by exponentially decaying the batch size instead of 
> falling back to one by one.
> 2. The other issue with this implementation is if lets say first batch 
> succeeds and the second one fails, the code tries to add all the partitions 
> one by one irrespective of whether some of the were successfully added or 
> not. If we need to fall back to one by one we should atleast remove the ones 
> which we know for sure are already added successfully.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16639) LLAP: Derive shuffle thread counts and keep-alive connections from instance count

[Siddharth Seth (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Siddharth Seth updated HIVE-16639:
--
Status: Patch Available  (was: Open)

> LLAP: Derive shuffle thread counts and keep-alive connections from instance 
> count
> -
>
> Key: HIVE-16639
> URL: https://issues.apache.org/jira/browse/HIVE-16639
> Project: Hive
>  Issue Type: Improvement
>Reporter: Gopal V
>Assignee: Siddharth Seth
> Attachments: HIVE-16639.01.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16639) LLAP: Derive shuffle thread counts and keep-alive connections from instance count

[Siddharth Seth (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Siddharth Seth updated HIVE-16639:
--
Attachment: HIVE-16639.01.patch

Enables keep-alive by default.
Thread count to 3x cpu count.
-Dhttp.maxconnections set to numInstances + 1

[~gopalv], [~rajesh.balamohan] - could you please take a  look.

> LLAP: Derive shuffle thread counts and keep-alive connections from instance 
> count
> -
>
> Key: HIVE-16639
> URL: https://issues.apache.org/jira/browse/HIVE-16639
> Project: Hive
>  Issue Type: Improvement
>Reporter: Gopal V
>Assignee: Siddharth Seth
> Attachments: HIVE-16639.01.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (HIVE-16639) LLAP: Derive shuffle thread counts and keep-alive connections from instance count

[Siddharth Seth (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16639?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Siddharth Seth reassigned HIVE-16639:
-

Assignee: Siddharth Seth

> LLAP: Derive shuffle thread counts and keep-alive connections from instance 
> count
> -
>
> Key: HIVE-16639
> URL: https://issues.apache.org/jira/browse/HIVE-16639
> Project: Hive
>  Issue Type: Improvement
>Reporter: Gopal V
>Assignee: Siddharth Seth
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16647) Improve the validation output to make the output to stderr and stdout more consistent

[Aihua Xu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aihua Xu updated HIVE-16647:

Attachment: (was: HIVE-16647.1.patch)

> Improve the validation output to make the output to stderr and stdout more 
> consistent
> -
>
> Key: HIVE-16647
> URL: https://issues.apache.org/jira/browse/HIVE-16647
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Affects Versions: 2.2.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>Priority: Minor
> Attachments: HIVE-16647.1.patch
>
>
> Some output are printed to stderr or stdout inconsistently. Here are some of 
> them. Update to make them more consistent.
> *  Version table validation
>   When the version table is missing, the err msg goes to stderr
>   When the version table is not valid, the err msg goes to stdout with a 
> message like "Failed in schema version validation: 
> *  Metastore/schema table validation
> ** When the version table contains the wrong version or there are no rows in 
> the version table, err msg goes to stderr
> ** When there diffs between the schema and metastore tables, the err msg goes 
> to stdout



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16647) Improve the validation output to make the output to stderr and stdout more consistent

[Aihua Xu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aihua Xu updated HIVE-16647:

Attachment: HIVE-16647.1.patch

> Improve the validation output to make the output to stderr and stdout more 
> consistent
> -
>
> Key: HIVE-16647
> URL: https://issues.apache.org/jira/browse/HIVE-16647
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Affects Versions: 2.2.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>Priority: Minor
> Attachments: HIVE-16647.1.patch
>
>
> Some output are printed to stderr or stdout inconsistently. Here are some of 
> them. Update to make them more consistent.
> *  Version table validation
>   When the version table is missing, the err msg goes to stderr
>   When the version table is not valid, the err msg goes to stdout with a 
> message like "Failed in schema version validation: 
> *  Metastore/schema table validation
> ** When the version table contains the wrong version or there are no rows in 
> the version table, err msg goes to stderr
> ** When there diffs between the schema and metastore tables, the err msg goes 
> to stdout



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16625) Extra '\0' characters in the output, when SeparatedValuesOutputFormat is used and the quoting is disabled

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007217#comment-16007217
 ] 

Hive QA commented on HIVE-16625:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12867534/HIVE-16625.02.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5202/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5202/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5202/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit 
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2017-05-11 21:22:50.136
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-5202/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2017-05-11 21:22:50.139
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 6bfa249 HIVE-16557: Vectorization: Specialize ReduceSink empty 
key case (Matt McCline, reviewed by Gopal Vijayaraghavan)
+ git clean -f -d
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 6bfa249 HIVE-16557: Vectorization: Specialize ReduceSink empty 
key case (Matt McCline, reviewed by Gopal Vijayaraghavan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2017-05-11 21:22:50.657
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh 
/data/hiveptest/working/scratch/build.patch
error: patch failed: itests/src/test/resources/testconfiguration.properties:753
error: itests/src/test/resources/testconfiguration.properties: patch does not 
apply
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12867534 - PreCommit-HIVE-Build

> Extra '\0' characters in the output, when SeparatedValuesOutputFormat is used 
> and the quoting is disabled
> -
>
> Key: HIVE-16625
> URL: https://issues.apache.org/jira/browse/HIVE-16625
> Project: Hive
>  Issue Type: Bug
>  Components: Beeline, Testing Infrastructure
>Affects Versions: 3.0.0
>Reporter: Peter Vary
>Assignee: Peter Vary
> Attachments: HIVE-16625.02.patch, HIVE-16625.patch
>
>
> If the output format is using {{SeparatedValuesOutputFormat}}, and the 
> quoting is disabled (by default is disabled), and the value of the cell 
> contains the separator character, then the the output is "quoted" with '\0' 
> characters.
> To reproduce:
> {code}
> create table quotes(s string);
> insert into quotes values('a\ta');
> !set outputFormat tsv2
> select * from quotes;
> {code}
> The result is:
> {code}
> quotes.s
> ^@a   a^@
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16272) support for drop function in incremental replication

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007216#comment-16007216
 ] 

Hive QA commented on HIVE-16272:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12867527/HIVE-16272.1.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5201/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5201/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5201/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Tests exited with: NonZeroExitCodeException
Command 'bash /data/hiveptest/working/scratch/source-prep.sh' failed with exit 
status 1 and output '+ date '+%Y-%m-%d %T.%3N'
2017-05-11 21:21:36.754
+ [[ -n /usr/lib/jvm/java-8-openjdk-amd64 ]]
+ export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64
+ export 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ 
PATH=/usr/lib/jvm/java-8-openjdk-amd64/bin/:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
+ export 'ANT_OPTS=-Xmx1g -XX:MaxPermSize=256m '
+ ANT_OPTS='-Xmx1g -XX:MaxPermSize=256m '
+ export 'MAVEN_OPTS=-Xmx1g '
+ MAVEN_OPTS='-Xmx1g '
+ cd /data/hiveptest/working/
+ tee /data/hiveptest/logs/PreCommit-HIVE-Build-5201/source-prep.txt
+ [[ false == \t\r\u\e ]]
+ mkdir -p maven ivy
+ [[ git = \s\v\n ]]
+ [[ git = \g\i\t ]]
+ [[ -z master ]]
+ [[ -d apache-github-source-source ]]
+ [[ ! -d apache-github-source-source/.git ]]
+ [[ ! -d apache-github-source-source ]]
+ date '+%Y-%m-%d %T.%3N'
2017-05-11 21:21:36.756
+ cd apache-github-source-source
+ git fetch origin
+ git reset --hard HEAD
HEAD is now at 6bfa249 HIVE-16557: Vectorization: Specialize ReduceSink empty 
key case (Matt McCline, reviewed by Gopal Vijayaraghavan)
+ git clean -f -d
Removing 
ql/src/test/results/clientpositive/beeline/create_merge_compressed.q.out
+ git checkout master
Already on 'master'
Your branch is up-to-date with 'origin/master'.
+ git reset --hard origin/master
HEAD is now at 6bfa249 HIVE-16557: Vectorization: Specialize ReduceSink empty 
key case (Matt McCline, reviewed by Gopal Vijayaraghavan)
+ git merge --ff-only origin/master
Already up-to-date.
+ date '+%Y-%m-%d %T.%3N'
2017-05-11 21:21:37.651
+ patchCommandPath=/data/hiveptest/working/scratch/smart-apply-patch.sh
+ patchFilePath=/data/hiveptest/working/scratch/build.patch
+ [[ -f /data/hiveptest/working/scratch/build.patch ]]
+ chmod +x /data/hiveptest/working/scratch/smart-apply-patch.sh
+ /data/hiveptest/working/scratch/smart-apply-patch.sh 
/data/hiveptest/working/scratch/build.patch
error: 
a/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/TestReplicationScenariosAcrossInstances.java:
 No such file or directory
error: 
a/itests/hive-unit/src/test/java/org/apache/hadoop/hive/ql/parse/WareHouse.java:
 No such file or directory
error: a/ql/src/java/org/apache/hadoop/hive/ql/parse/repl/DumpType.java: No 
such file or directory
error: 
a/ql/src/java/org/apache/hadoop/hive/ql/parse/repl/dump/events/EventHandlerFactory.java:
 No such file or directory
error: 
a/ql/src/java/org/apache/hadoop/hive/ql/parse/repl/load/message/DropTableHandler.java:
 No such file or directory
The patch does not appear to apply with p0, p1, or p2
+ exit 1
'
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12867527 - PreCommit-HIVE-Build

> support for drop function in incremental replication 
> -
>
> Key: HIVE-16272
> URL: https://issues.apache.org/jira/browse/HIVE-16272
> Project: Hive
>  Issue Type: Sub-task
>  Components: HiveServer2
>Affects Versions: 2.2.0
>Reporter: anishek
>Assignee: anishek
> Fix For: 3.0.0
>
> Attachments: HIVE-16272.1.patch
>
>
> drop function should work in incremental dump and incremental load



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16643) BeeLine tests output should keep the PREHOOK/POSTHOOK Input/Output orderdering

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007213#comment-16007213
 ] 

Hive QA commented on HIVE-16643:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12867525/HIVE-16643.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 10688 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_2] 
(batchId=236)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_3] 
(batchId=97)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] 
(batchId=97)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5200/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5200/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5200/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12867525 - PreCommit-HIVE-Build

> BeeLine tests output should keep the PREHOOK/POSTHOOK Input/Output orderdering
> --
>
> Key: HIVE-16643
> URL: https://issues.apache.org/jira/browse/HIVE-16643
> Project: Hive
>  Issue Type: New Feature
>  Components: Testing Infrastructure
>Reporter: Peter Vary
>Assignee: Peter Vary
> Attachments: HIVE-16643.patch
>
>
> The {{PreExecutePrinter}} and the {{PostExecutePrinter}} prints the query 
> input and the output list in alphabetical order in {{printEntities}} method.
> Our goal is to have the same output from the BeeLine query tests, and the Cli 
> query tests. Since the BeeLine tests are using test specific databases to run 
> the tests, and only converting the results in the end to remove this specific 
> database names from the output, we have to reorder the lists after this 
> conversion.
> Raw BeeLine output:
> {code}
> [..]
> INFO  : PREHOOK: Output: create_merge_compressed@src_rc_merge_test
> INFO  : PREHOOK: Output: database:create_merge_compressed
> [..]
> {code}
> Before patch BeeLine output:
> {code}
> [..]
> PREHOOK: Output: default@src_rc_merge_test
> PREHOOK: Output: database:default
> [..]
> {code}
> Expected output:
> {code}
> [..]
> PREHOOK: Output: database:default
> PREHOOK: Output: default@src_rc_merge_test
> [..]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16579) CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

[Vaibhav Gumashta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vaibhav Gumashta updated HIVE-16579:

Status: Patch Available  (was: Open)

> CachedStore: improvements to partition col stats caching and cache column 
> stats for unpartitioned table
> ---
>
> Key: HIVE-16579
> URL: https://issues.apache.org/jira/browse/HIVE-16579
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
> Attachments: HIVE-16579.1.patch, HIVE-16579.2.patch, 
> HIVE-16579.3.patch
>
>
> 1. Update stats cache when partitions/table is dropped.
> 2. Update cached partition col stats in the background cache update thread. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16636) TestPerfCli driver is missing query24

[Ashutosh Chauhan (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007200#comment-16007200
 ] 

Ashutosh Chauhan commented on HIVE-16636:
-

+1

> TestPerfCli driver is missing query24
> -
>
> Key: HIVE-16636
> URL: https://issues.apache.org/jira/browse/HIVE-16636
> Project: Hive
>  Issue Type: Test
>Reporter: Vineet Garg
>Assignee: Vineet Garg
> Attachments: HIVE-16636.1.patch
>
>
> TPC-DS query 24 is missing



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16579) CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

[Vaibhav Gumashta (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vaibhav Gumashta updated HIVE-16579:

Attachment: HIVE-16579.3.patch

Removing formatting. 

> CachedStore: improvements to partition col stats caching and cache column 
> stats for unpartitioned table
> ---
>
> Key: HIVE-16579
> URL: https://issues.apache.org/jira/browse/HIVE-16579
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
> Attachments: HIVE-16579.1.patch, HIVE-16579.2.patch, 
> HIVE-16579.3.patch
>
>
> 1. Update stats cache when partitions/table is dropped.
> 2. Update cached partition col stats in the background cache update thread. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16653) Mergejoin should give itself a correct tag

[Pengcheng Xiong (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007192#comment-16007192
 ] 

Pengcheng Xiong commented on HIVE-16653:


[~vgarg], thanks for your comments. This patch is just moving some code above. 
The original code assumes that it is not empty. Thus i think it is safe...

> Mergejoin should give itself a correct tag
> --
>
> Key: HIVE-16653
> URL: https://issues.apache.org/jira/browse/HIVE-16653
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-16653.01.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16653) Mergejoin should give itself a correct tag

[Vineet Garg (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007182#comment-16007182
 ] 

Vineet Garg commented on HIVE-16653:


{code} if (stack.size() < 2 || !(stack.get(stack.size() - 2) instanceof 
DummyStoreOperator)) {code} If stack happens to be empty (not sure if it could 
be) second condition will throw an error. May be it is a good idea to 
check/assert if stack is not empty.

> Mergejoin should give itself a correct tag
> --
>
> Key: HIVE-16653
> URL: https://issues.apache.org/jira/browse/HIVE-16653
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-16653.01.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16636) TestPerfCli driver is missing query24

[Vineet Garg (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007179#comment-16007179
 ] 

Vineet Garg commented on HIVE-16636:


Tests are unrelated. [~ashutoshc] can you take a look?

> TestPerfCli driver is missing query24
> -
>
> Key: HIVE-16636
> URL: https://issues.apache.org/jira/browse/HIVE-16636
> Project: Hive
>  Issue Type: Test
>Reporter: Vineet Garg
>Assignee: Vineet Garg
> Attachments: HIVE-16636.1.patch
>
>
> TPC-DS query 24 is missing



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (HIVE-16636) TestPerfCli driver is missing query24

[Vineet Garg (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007179#comment-16007179
 ] 

Vineet Garg edited comment on HIVE-16636 at 5/11/17 8:58 PM:
-

Test failures are unrelated. [~ashutoshc] can you take a look?


was (Author: vgarg):
Tests are unrelated. [~ashutoshc] can you take a look?

> TestPerfCli driver is missing query24
> -
>
> Key: HIVE-16636
> URL: https://issues.apache.org/jira/browse/HIVE-16636
> Project: Hive
>  Issue Type: Test
>Reporter: Vineet Garg
>Assignee: Vineet Garg
> Attachments: HIVE-16636.1.patch
>
>
> TPC-DS query 24 is missing



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16636) TestPerfCli driver is missing query24

[Vineet Garg (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16636?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007177#comment-16007177
 ] 

Vineet Garg commented on HIVE-16636:


{code} if (stack.size() < 2 || !(stack.get(stack.size() - 2) instanceof 
DummyStoreOperator)) {code} If stack happens to be empty (not sure if it could 
be) second condition will throw an error. May be it is a good idea to 
check/assert if stack is not empty.

> TestPerfCli driver is missing query24
> -
>
> Key: HIVE-16636
> URL: https://issues.apache.org/jira/browse/HIVE-16636
> Project: Hive
>  Issue Type: Test
>Reporter: Vineet Garg
>Assignee: Vineet Garg
> Attachments: HIVE-16636.1.patch
>
>
> TPC-DS query 24 is missing



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Issue Comment Deleted] (HIVE-16636) TestPerfCli driver is missing query24

[Vineet Garg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vineet Garg updated HIVE-16636:
---
Comment: was deleted

(was: {code} if (stack.size() < 2 || !(stack.get(stack.size() - 2) instanceof 
DummyStoreOperator)) {code} If stack happens to be empty (not sure if it could 
be) second condition will throw an error. May be it is a good idea to 
check/assert if stack is not empty.)

> TestPerfCli driver is missing query24
> -
>
> Key: HIVE-16636
> URL: https://issues.apache.org/jira/browse/HIVE-16636
> Project: Hive
>  Issue Type: Test
>Reporter: Vineet Garg
>Assignee: Vineet Garg
> Attachments: HIVE-16636.1.patch
>
>
> TPC-DS query 24 is missing



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16647) Improve the validation output to make the output to stderr and stdout more consistent

[Aihua Xu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aihua Xu updated HIVE-16647:

Attachment: HIVE-16647.1.patch

> Improve the validation output to make the output to stderr and stdout more 
> consistent
> -
>
> Key: HIVE-16647
> URL: https://issues.apache.org/jira/browse/HIVE-16647
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Affects Versions: 2.2.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>Priority: Minor
> Attachments: HIVE-16647.1.patch
>
>
> Some output are printed to stderr or stdout inconsistently. Here are some of 
> them. Update to make them more consistent.
> *  Version table validation
>   When the version table is missing, the err msg goes to stderr
>   When the version table is not valid, the err msg goes to stdout with a 
> message like "Failed in schema version validation: 
> *  Metastore/schema table validation
> ** When the version table contains the wrong version or there are no rows in 
> the version table, err msg goes to stderr
> ** When there diffs between the schema and metastore tables, the err msg goes 
> to stdout



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16647) Improve the validation output to make the output to stderr and stdout more consistent

[Aihua Xu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aihua Xu updated HIVE-16647:

Status: In Progress  (was: Patch Available)

> Improve the validation output to make the output to stderr and stdout more 
> consistent
> -
>
> Key: HIVE-16647
> URL: https://issues.apache.org/jira/browse/HIVE-16647
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Affects Versions: 2.2.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>Priority: Minor
> Attachments: HIVE-16647.1.patch
>
>
> Some output are printed to stderr or stdout inconsistently. Here are some of 
> them. Update to make them more consistent.
> *  Version table validation
>   When the version table is missing, the err msg goes to stderr
>   When the version table is not valid, the err msg goes to stdout with a 
> message like "Failed in schema version validation: 
> *  Metastore/schema table validation
> ** When the version table contains the wrong version or there are no rows in 
> the version table, err msg goes to stderr
> ** When there diffs between the schema and metastore tables, the err msg goes 
> to stdout



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16647) Improve the validation output to make the output to stderr and stdout more consistent

[Aihua Xu (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Aihua Xu updated HIVE-16647:

Attachment: (was: HIVE-16647.1.patch)

> Improve the validation output to make the output to stderr and stdout more 
> consistent
> -
>
> Key: HIVE-16647
> URL: https://issues.apache.org/jira/browse/HIVE-16647
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Affects Versions: 2.2.0
>Reporter: Aihua Xu
>Assignee: Aihua Xu
>Priority: Minor
> Attachments: HIVE-16647.1.patch
>
>
> Some output are printed to stderr or stdout inconsistently. Here are some of 
> them. Update to make them more consistent.
> *  Version table validation
>   When the version table is missing, the err msg goes to stderr
>   When the version table is not valid, the err msg goes to stdout with a 
> message like "Failed in schema version validation: 
> *  Metastore/schema table validation
> ** When the version table contains the wrong version or there are no rows in 
> the version table, err msg goes to stderr
> ** When there diffs between the schema and metastore tables, the err msg goes 
> to stdout



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (HIVE-16654) Optimize a combination of avg(), sum(), count(distinct) etc

[Pengcheng Xiong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pengcheng Xiong reassigned HIVE-16654:
--


> Optimize a combination of avg(), sum(), count(distinct) etc
> ---
>
> Key: HIVE-16654
> URL: https://issues.apache.org/jira/browse/HIVE-16654
> Project: Hive
>  Issue Type: Bug
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
>
> an example rewrite for q28 of tpcds is 
> {code}
> (select LP as B1_LP ,CNT  as B1_CNT,CNTD as B1_CNTD
>   from (select sum(xc0) / sum(xc1) as LP, sum(xc1) as CNT, count(1) as 
> CNTD from (select sum(ss_list_price) as xc0, count(ss_list_price) as xc1 from 
> store_sales  where 
> ss_list_price is not null and ss_quantity between 0 and 5
> and (ss_list_price between 11 and 11+10 
>  or ss_coupon_amt between 460 and 460+1000
>  or ss_wholesale_cost between 14 and 14+20)
>  group by ss_list_price) ss0) ss1) B1
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16654) Optimize a combination of avg(), sum(), count(distinct) etc

[Pengcheng Xiong (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007145#comment-16007145
 ] 

Pengcheng Xiong commented on HIVE-16654:


ccing [~gopalv] and [~ashutoshc]

> Optimize a combination of avg(), sum(), count(distinct) etc
> ---
>
> Key: HIVE-16654
> URL: https://issues.apache.org/jira/browse/HIVE-16654
> Project: Hive
>  Issue Type: Bug
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
>
> an example rewrite for q28 of tpcds is 
> {code}
> (select LP as B1_LP ,CNT  as B1_CNT,CNTD as B1_CNTD
>   from (select sum(xc0) / sum(xc1) as LP, sum(xc1) as CNT, count(1) as 
> CNTD from (select sum(ss_list_price) as xc0, count(ss_list_price) as xc1 from 
> store_sales  where 
> ss_list_price is not null and ss_quantity between 0 and 5
> and (ss_list_price between 11 and 11+10 
>  or ss_coupon_amt between 460 and 460+1000
>  or ss_wholesale_cost between 14 and 14+20)
>  group by ss_list_price) ss0) ss1) B1
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16628) Fix query25 when it uses a mix of MergeJoin and MapJoin

[Pengcheng Xiong (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007142#comment-16007142
 ] 

Pengcheng Xiong commented on HIVE-16628:


pushed to master. thanks [~ashutoshc] for the review.

> Fix query25 when it uses a mix of MergeJoin and MapJoin
> ---
>
> Key: HIVE-16628
> URL: https://issues.apache.org/jira/browse/HIVE-16628
> Project: Hive
>  Issue Type: Sub-task
>Affects Versions: 2.0.0, 2.1.0
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Fix For: 3.0.0
>
> Attachments: HIVE-16628.01.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16628) Fix query25 when it uses a mix of MergeJoin and MapJoin

[Pengcheng Xiong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pengcheng Xiong updated HIVE-16628:
---
Affects Version/s: 2.0.0
   2.1.0

> Fix query25 when it uses a mix of MergeJoin and MapJoin
> ---
>
> Key: HIVE-16628
> URL: https://issues.apache.org/jira/browse/HIVE-16628
> Project: Hive
>  Issue Type: Sub-task
>Affects Versions: 2.0.0, 2.1.0
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Fix For: 3.0.0
>
> Attachments: HIVE-16628.01.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16628) Fix query25 when it uses a mix of MergeJoin and MapJoin

[Pengcheng Xiong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pengcheng Xiong updated HIVE-16628:
---
Fix Version/s: 3.0.0

> Fix query25 when it uses a mix of MergeJoin and MapJoin
> ---
>
> Key: HIVE-16628
> URL: https://issues.apache.org/jira/browse/HIVE-16628
> Project: Hive
>  Issue Type: Sub-task
>Affects Versions: 2.0.0, 2.1.0
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Fix For: 3.0.0
>
> Attachments: HIVE-16628.01.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16653) Mergejoin should give itself a correct tag

[Pengcheng Xiong (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007141#comment-16007141
 ] 

Pengcheng Xiong commented on HIVE-16653:


[~hagleitn] and [~ashutoshc], could u take a look? Thanks.

> Mergejoin should give itself a correct tag
> --
>
> Key: HIVE-16653
> URL: https://issues.apache.org/jira/browse/HIVE-16653
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-16653.01.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16653) Mergejoin should give itself a correct tag

[Pengcheng Xiong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pengcheng Xiong updated HIVE-16653:
---
Attachment: HIVE-16653.01.patch

> Mergejoin should give itself a correct tag
> --
>
> Key: HIVE-16653
> URL: https://issues.apache.org/jira/browse/HIVE-16653
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-16653.01.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16653) Mergejoin should give itself a correct tag

[Pengcheng Xiong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pengcheng Xiong updated HIVE-16653:
---
Status: Patch Available  (was: Open)

> Mergejoin should give itself a correct tag
> --
>
> Key: HIVE-16653
> URL: https://issues.apache.org/jira/browse/HIVE-16653
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
> Attachments: HIVE-16653.01.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (HIVE-16653) Mergejoin should give itself a correct tag

[Pengcheng Xiong (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Pengcheng Xiong reassigned HIVE-16653:
--


> Mergejoin should give itself a correct tag
> --
>
> Key: HIVE-16653
> URL: https://issues.apache.org/jira/browse/HIVE-16653
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Pengcheng Xiong
>Assignee: Pengcheng Xiong
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16642) New Events created as part of replv2 potentially break replv1

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007112#comment-16007112
 ] 

Hive QA commented on HIVE-16642:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12867520/HIVE-16642.1.patch

{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 10690 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestHBaseCliDriver.testCliDriver[hbase_ppd_key_range]
 (batchId=92)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
 (batchId=144)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_join30]
 (batchId=149)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_3] 
(batchId=97)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] 
(batchId=97)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5199/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5199/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5199/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12867520 - PreCommit-HIVE-Build

> New Events created as part of replv2 potentially break replv1
> -
>
> Key: HIVE-16642
> URL: https://issues.apache.org/jira/browse/HIVE-16642
> Project: Hive
>  Issue Type: Sub-task
>  Components: repl
>Reporter: Sushanth Sowmyan
>Assignee: Sushanth Sowmyan
> Attachments: HIVE-16642.1.patch
>
>
> We have a couple of new events introduced, such as 
> \{CREATE,DROP\}\{INDEX,FUNCTION\} since the introduction of replv1, but those 
> which do not have a replv1 ReplicationTask associated with them.
> Thus, for users like Falcon, we potentially wind up throwing a 
> IllegalStateException if replv1 based HiveDR is running on a cluster with 
> these updated events.
> Thus, we should be more graceful when encountering them, returning a 
> NoopReplicationTask equivalent that they can make use of, or ignore, for such 
> newer events.
> In addition, we should add additional test cases so that we track whether or 
> not the creation of these events leads to any backward incompatibility we 
> introduce. To this end, if any of the events should change so that we 
> introduce a backward incompatibility, we should have these tests fail, and 
> alert us to that possibility.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Comment Edited] (HIVE-16555) Add a new thrift API call for get_metastore_uuid

[Vihang Karajgaonkar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007108#comment-16007108
 ] 

Vihang Karajgaonkar edited comment on HIVE-16555 at 5/11/17 8:08 PM:
-

Removed MetastoreDBProperty from the hive_metastore.thrift file. Added java doc 
for the newly added MetastoreClient and Rawstore interface methods. Updated the 
patch on review board as well.


was (Author: vihangk1):
Removed

> Add a new thrift API call for get_metastore_uuid
> 
>
> Key: HIVE-16555
> URL: https://issues.apache.org/jira/browse/HIVE-16555
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
> Attachments: HIVE-16555.01.patch, HIVE-16555.02.patch, 
> HIVE-16555.03.patch, HIVE-16555.04.patch, HIVE-16555.05.patch, 
> HIVE-16555.06.patch, HIVE-16555.07.patch, HIVE-16555.08.patch
>
>
> Sub-task of the main JIRA to add the new thrift API



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16555) Add a new thrift API call for get_metastore_uuid

[Vihang Karajgaonkar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vihang Karajgaonkar updated HIVE-16555:
---
Attachment: HIVE-16555.08.patch

Removed

> Add a new thrift API call for get_metastore_uuid
> 
>
> Key: HIVE-16555
> URL: https://issues.apache.org/jira/browse/HIVE-16555
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
> Attachments: HIVE-16555.01.patch, HIVE-16555.02.patch, 
> HIVE-16555.03.patch, HIVE-16555.04.patch, HIVE-16555.05.patch, 
> HIVE-16555.06.patch, HIVE-16555.07.patch, HIVE-16555.08.patch
>
>
> Sub-task of the main JIRA to add the new thrift API



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16555) Add a new thrift API call for get_metastore_uuid

[Vihang Karajgaonkar (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007075#comment-16007075
 ] 

Vihang Karajgaonkar commented on HIVE-16555:


Thanks [~alangates] for the review. The {{createDbGuidAndPersist}} method also 
is in a transaction block and there is a unique key constraint on column 
{{PROPERTY_KEY}} in the backing table of this API. In the above case if there 
was another call to createDbGuidAndPersist from a 2nd server between the the 
get and create of the 1st server the createDbGuidAndPersist call from the 1st 
server will throw a unique key constraint violation exception which is caught 
and logged as a warning. It then retrieves the UUID set by the 2nd server and 
returns to the client. Do you still see a race condition happening given that 
database will prevent adding the same key twice? I added a test case in the 
patch which calls this API from multiple clients in parallel and makes sure 
that all the clients retrieve the same UUID.

regarding {{MetastoreDBProperty}} let me update the patch after removing the 
structure. Thanks for catching that!

> Add a new thrift API call for get_metastore_uuid
> 
>
> Key: HIVE-16555
> URL: https://issues.apache.org/jira/browse/HIVE-16555
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
> Attachments: HIVE-16555.01.patch, HIVE-16555.02.patch, 
> HIVE-16555.03.patch, HIVE-16555.04.patch, HIVE-16555.05.patch, 
> HIVE-16555.06.patch, HIVE-16555.07.patch
>
>
> Sub-task of the main JIRA to add the new thrift API



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16555) Add a new thrift API call for get_metastore_uuid

[Vihang Karajgaonkar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vihang Karajgaonkar updated HIVE-16555:
---
Attachment: HIVE-16555.07.patch

> Add a new thrift API call for get_metastore_uuid
> 
>
> Key: HIVE-16555
> URL: https://issues.apache.org/jira/browse/HIVE-16555
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
> Attachments: HIVE-16555.01.patch, HIVE-16555.02.patch, 
> HIVE-16555.03.patch, HIVE-16555.04.patch, HIVE-16555.05.patch, 
> HIVE-16555.06.patch, HIVE-16555.07.patch
>
>
> Sub-task of the main JIRA to add the new thrift API



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16555) Add a new thrift API call for get_metastore_uuid

[Vihang Karajgaonkar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vihang Karajgaonkar updated HIVE-16555:
---
Attachment: (was: HIVE-16555.07.patch)

> Add a new thrift API call for get_metastore_uuid
> 
>
> Key: HIVE-16555
> URL: https://issues.apache.org/jira/browse/HIVE-16555
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
> Attachments: HIVE-16555.01.patch, HIVE-16555.02.patch, 
> HIVE-16555.03.patch, HIVE-16555.04.patch, HIVE-16555.05.patch, 
> HIVE-16555.06.patch, HIVE-16555.07.patch
>
>
> Sub-task of the main JIRA to add the new thrift API



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16555) Add a new thrift API call for get_metastore_uuid

[Alan Gates (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007063#comment-16007063
 ] 

Alan Gates commented on HIVE-16555:
---

I think there's a race condition in HiveMetaStore between when you check the 
UUID and when you set it.  getMetastoreDBUuid calls getGuidFromDB and if that 
returns null, then it calls createDbGuidAndPersist which blindly writes a new 
UUID.  But these are called in separate transactions, so another server could 
set the UUID in between and the code wouldn't detect it.

Also, in hive_metastore.thrift you create a new structure MetastoreDBProperty 
but you don't seem to use it anywhere in the thrift interface.

> Add a new thrift API call for get_metastore_uuid
> 
>
> Key: HIVE-16555
> URL: https://issues.apache.org/jira/browse/HIVE-16555
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
> Attachments: HIVE-16555.01.patch, HIVE-16555.02.patch, 
> HIVE-16555.03.patch, HIVE-16555.04.patch, HIVE-16555.05.patch, 
> HIVE-16555.06.patch, HIVE-16555.07.patch
>
>
> Sub-task of the main JIRA to add the new thrift API



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16207) Add support for Complex Types in Fast SerDe

[Teddy Choi (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Teddy Choi updated HIVE-16207:
--
Attachment: HIVE-16207.4.patch

Fixed failing integration bugs.

> Add support for Complex Types in Fast SerDe
> ---
>
> Key: HIVE-16207
> URL: https://issues.apache.org/jira/browse/HIVE-16207
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Matt McCline
>Assignee: Teddy Choi
>Priority: Critical
> Attachments: HIVE-16207.1.patch, HIVE-16207.1.patch.zip, 
> HIVE-16207.2.patch, HIVE-16207.3.patch, HIVE-16207.4.patch, partial.patch
>
>
> Add complex type support to Fast SerDe classes.  This is needed for fully 
> supporting complex types in Vectorization



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16579) CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

[Daniel Dai (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007034#comment-16007034
 ] 

Daniel Dai commented on HIVE-16579:
---

[~vgumashta], are you able to make a patch without the format change?

> CachedStore: improvements to partition col stats caching and cache column 
> stats for unpartitioned table
> ---
>
> Key: HIVE-16579
> URL: https://issues.apache.org/jira/browse/HIVE-16579
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
> Attachments: HIVE-16579.1.patch, HIVE-16579.2.patch
>
>
> 1. Update stats cache when partitions/table is dropped.
> 2. Update cached partition col stats in the background cache update thread. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16600) Refactor SetSparkReducerParallelism#needSetParallelism to enable parallel order by in multi_insert cases

[Hive QA (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16600?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007027#comment-16007027
 ] 

Hive QA commented on HIVE-16600:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12867515/HIVE-16600.3.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 15 failed/errored test(s), 10686 tests 
executed
*Failed tests:*
{noformat}
TestHiveServer2SessionTimeout - did not produce a TEST-*.xml file (likely timed 
out) (batchId=226)
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[mapjoin2] 
(batchId=236)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[columnstats_part_coltype]
 (batchId=156)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_if_expr]
 (batchId=144)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainanalyze_3] 
(batchId=97)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[explainuser_3] 
(batchId=97)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testAddJarDataNucleusUnCaching 
(batchId=225)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testEnableThriftSerializeInTasks 
(batchId=225)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testConnection (batchId=237)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testIsValid (batchId=237)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testIsValidNeg (batchId=237)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testNegativeProxyAuth 
(batchId=237)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testNegativeTokenAuth 
(batchId=237)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testProxyAuth (batchId=237)
org.apache.hive.minikdc.TestJdbcWithDBTokenStore.testTokenAuth (batchId=237)
{noformat}

Test results: https://builds.apache.org/job/PreCommit-HIVE-Build/5198/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/5198/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-5198/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 15 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12867515 - PreCommit-HIVE-Build

> Refactor SetSparkReducerParallelism#needSetParallelism to enable parallel 
> order by in multi_insert cases
> 
>
> Key: HIVE-16600
> URL: https://issues.apache.org/jira/browse/HIVE-16600
> Project: Hive
>  Issue Type: Sub-task
>Reporter: liyunzhang_intel
>Assignee: liyunzhang_intel
> Attachments: HIVE-16600.1.patch, HIVE-16600.2.patch, 
> HIVE-16600.3.patch, mr.explain.log.HIVE-16600
>
>
> multi_insert_gby.case.q
> {code}
> set hive.exec.reducers.bytes.per.reducer=256;
> set hive.optimize.sampling.orderby=true;
> drop table if exists e1;
> drop table if exists e2;
> create table e1 (key string, value string);
> create table e2 (key string);
> FROM (select key, cast(key as double) as keyD, value from src order by key) a
> INSERT OVERWRITE TABLE e1
> SELECT key, value
> INSERT OVERWRITE TABLE e2
> SELECT key;
> select * from e1;
> select * from e2;
> {code} 
> the parallelism of Sort is 1 even we enable parallel order 
> by("hive.optimize.sampling.orderby" is set as "true").  This is not 
> reasonable because the parallelism  should be calcuated by  
> [Utilities.estimateReducers|https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/SetSparkReducerParallelism.java#L170]
> this is because SetSparkReducerParallelism#needSetParallelism returns false 
> when [children size of 
> RS|https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/optimizer/spark/SetSparkReducerParallelism.java#L207]
>  is greater than 1.
> in this case, the children size of {{RS[2]}} is two.
> the logical plan of the case
> {code}
>TS[0]-SEL[1]-RS[2]-SEL[3]-SEL[4]-FS[5]
> -SEL[6]-FS[7]
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-14389) Beeline should not output query and prompt to stdout

[Yongzhi Chen (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-14389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007023#comment-16007023
 ] 

Yongzhi Chen commented on HIVE-14389:
-

PATCH 4 looks good.  +1

> Beeline should not output query and prompt to stdout
> 
>
> Key: HIVE-14389
> URL: https://issues.apache.org/jira/browse/HIVE-14389
> Project: Hive
>  Issue Type: Improvement
>  Components: Beeline
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
> Attachments: HIVE-14389.01.patch, HIVE-14389.02.patch, 
> HIVE-14389.03.patch, HIVE-14389.04.patch
>
>
> It seems that the Beeline prints the query along with the results in the 
> stdout when a script file is passed. The output file in the example below 
> needs to only have the results and not the query.
> {noformat}
> .vihang-MBP:bin vihang$ ./beeline --showheader=false --outformat=tsv2 -u 
> "jdbc:hive2://localhost:1" -f /tmp/query.sql > /tmp/query.out 2> 
> /tmp/query.err
> OK
> $ cat /tmp/query.out
> 1: jdbc:hive2://localhost:1/default> select * from likes limit 4;
> +---+--+--+
> | 1 | chocolate|
> | 1 | car  |
> | 1 | games|
> | 1 | chess|
> +---+--+--+
> 1: jdbc:hive2://localhost:1/default>
> 1: jdbc:hive2://localhost:1/default>
> $
> {noformat}
> A lot of people use HiveCLI and in order to transition from HiveCLI scripts 
> to Beeline, this needs to be taken care of. The output files generated by 
> beeline should contain only the results and nothing else.
> Similarly, when not in silent mode, query are being printed out on stdout, 
> which is adding garbage along with results, as just like HIVE CLI does, users 
> would like to have only the results on stdout, not errors/debugging info/etc, 
> like the full query. 
> Query could be printed out, no problem, as long as it is not on stdout (with 
> results), instead, it must be printed out along with the debugging info.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (HIVE-16646) Alias in transform ... as clause shouldn't be case sensitive

[Yongzhi Chen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yongzhi Chen reassigned HIVE-16646:
---

Assignee: Yibing Shi  (was: Yongzhi Chen)

> Alias in transform ... as clause shouldn't be case sensitive
> 
>
> Key: HIVE-16646
> URL: https://issues.apache.org/jira/browse/HIVE-16646
> Project: Hive
>  Issue Type: Bug
>  Components: hpl/sql
>Reporter: Yibing Shi
>Assignee: Yibing Shi
> Attachments: HIVE-16646.1.patch, HIVE-16646.2.patch
>
>
> Create a table like below:
> {code:sql}
> CREATE TABLE hive_bug(col1 string);
> {code}
> Run below query in Hive:
> {code}
> from hive_bug select transform(col1) using '/bin/cat' as ( string);
> {code}
> The result would be:
> {noformat}
> 0: jdbc:hive2://localhost:1> from hive_bug select transform(col1) using 
> '/bin/cat' as ( string);
> ..
> INFO  : OK
> +---+--+
> |   |
> +---+--+
> +---+--+
> {noformat}
> The output column name is ** instead of the lowercase .



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16646) Alias in transform ... as clause shouldn't be case sensitive

[Yongzhi Chen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yongzhi Chen updated HIVE-16646:

Attachment: HIVE-16646.2.patch

> Alias in transform ... as clause shouldn't be case sensitive
> 
>
> Key: HIVE-16646
> URL: https://issues.apache.org/jira/browse/HIVE-16646
> Project: Hive
>  Issue Type: Bug
>  Components: hpl/sql
>Reporter: Yibing Shi
>Assignee: Yongzhi Chen
> Attachments: HIVE-16646.1.patch, HIVE-16646.2.patch
>
>
> Create a table like below:
> {code:sql}
> CREATE TABLE hive_bug(col1 string);
> {code}
> Run below query in Hive:
> {code}
> from hive_bug select transform(col1) using '/bin/cat' as ( string);
> {code}
> The result would be:
> {noformat}
> 0: jdbc:hive2://localhost:1> from hive_bug select transform(col1) using 
> '/bin/cat' as ( string);
> ..
> INFO  : OK
> +---+--+
> |   |
> +---+--+
> +---+--+
> {noformat}
> The output column name is ** instead of the lowercase .



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16646) Alias in transform ... as clause shouldn't be case sensitive

[Yongzhi Chen (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007014#comment-16007014
 ] 

Yongzhi Chen commented on HIVE-16646:
-

[~Yibing], I generate another patch to include the output and also change a 
little on the test to make it more relevant to the fix. 
In order to attach it, I have to temporary change the assignee of the bug, then 
change back to you.  

> Alias in transform ... as clause shouldn't be case sensitive
> 
>
> Key: HIVE-16646
> URL: https://issues.apache.org/jira/browse/HIVE-16646
> Project: Hive
>  Issue Type: Bug
>  Components: hpl/sql
>Reporter: Yibing Shi
>Assignee: Yibing Shi
> Attachments: HIVE-16646.1.patch
>
>
> Create a table like below:
> {code:sql}
> CREATE TABLE hive_bug(col1 string);
> {code}
> Run below query in Hive:
> {code}
> from hive_bug select transform(col1) using '/bin/cat' as ( string);
> {code}
> The result would be:
> {noformat}
> 0: jdbc:hive2://localhost:1> from hive_bug select transform(col1) using 
> '/bin/cat' as ( string);
> ..
> INFO  : OK
> +---+--+
> |   |
> +---+--+
> +---+--+
> {noformat}
> The output column name is ** instead of the lowercase .



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (HIVE-16646) Alias in transform ... as clause shouldn't be case sensitive

[Yongzhi Chen (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yongzhi Chen reassigned HIVE-16646:
---

Assignee: Yongzhi Chen  (was: Yibing Shi)

> Alias in transform ... as clause shouldn't be case sensitive
> 
>
> Key: HIVE-16646
> URL: https://issues.apache.org/jira/browse/HIVE-16646
> Project: Hive
>  Issue Type: Bug
>  Components: hpl/sql
>Reporter: Yibing Shi
>Assignee: Yongzhi Chen
> Attachments: HIVE-16646.1.patch
>
>
> Create a table like below:
> {code:sql}
> CREATE TABLE hive_bug(col1 string);
> {code}
> Run below query in Hive:
> {code}
> from hive_bug select transform(col1) using '/bin/cat' as ( string);
> {code}
> The result would be:
> {noformat}
> 0: jdbc:hive2://localhost:1> from hive_bug select transform(col1) using 
> '/bin/cat' as ( string);
> ..
> INFO  : OK
> +---+--+
> |   |
> +---+--+
> +---+--+
> {noformat}
> The output column name is ** instead of the lowercase .



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-14731) Use Tez cartesian product edge in Hive (unpartitioned case only)

[Gunther Hagleitner (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-14731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007003#comment-16007003
 ] 

Gunther Hagleitner commented on HIVE-14731:
---

[~aplusplus] the condition should be (cross prod enabled && !outerjoin && 
!iscrossproduct) right? I think you're missing the first condition. Otherwise 
I'm +1 assuming all tests pass.

> Use Tez cartesian product edge in Hive (unpartitioned case only)
> 
>
> Key: HIVE-14731
> URL: https://issues.apache.org/jira/browse/HIVE-14731
> Project: Hive
>  Issue Type: Bug
>Reporter: Zhiyuan Yang
>Assignee: Zhiyuan Yang
> Attachments: HIVE-14731.10.patch, HIVE-14731.11.patch, 
> HIVE-14731.12.patch, HIVE-14731.13.patch, HIVE-14731.14.patch, 
> HIVE-14731.15.patch, HIVE-14731.16.patch, HIVE-14731.1.patch, 
> HIVE-14731.2.patch, HIVE-14731.3.patch, HIVE-14731.4.patch, 
> HIVE-14731.5.patch, HIVE-14731.6.patch, HIVE-14731.7.patch, 
> HIVE-14731.8.patch, HIVE-14731.9.patch
>
>
> Given cartesian product edge is available in Tez now (see TEZ-3230), let's 
> integrate it into Hive on Tez. This allows us to have more than one reducer 
> in cross product queries.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-15051) Test framework integration with findbugs, rat checks etc.

[Thejas M Nair (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-15051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16007002#comment-16007002
 ] 

Thejas M Nair commented on HIVE-15051:
--

[~pvary]
Where would the details of the findbugs result (or other steps) show up ? Would 
it be accessible via jenkins job ?
What are remaining steps to get this as part of our precommit testing ?
Thanks again for looking into this!


> Test framework integration with findbugs, rat checks etc.
> -
>
> Key: HIVE-15051
> URL: https://issues.apache.org/jira/browse/HIVE-15051
> Project: Hive
>  Issue Type: Sub-task
>  Components: Testing Infrastructure
>Reporter: Peter Vary
>Assignee: Peter Vary
> Attachments: beeline.out, ql.out
>
>
> Find a way to integrate code analysis tools like findbugs, rat checks to 
> PreCommit tests, thus removing the burden from reviewers to check the code 
> style and other checks which could be done by code. 
> Might worth to take a look on Yetus, but keep in mind the Hive has a specific 
> parallel test framework.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16555) Add a new thrift API call for get_metastore_uuid

[Vihang Karajgaonkar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vihang Karajgaonkar updated HIVE-16555:
---
Attachment: HIVE-16555.07.patch

Added Sahil's suggestion from review board

> Add a new thrift API call for get_metastore_uuid
> 
>
> Key: HIVE-16555
> URL: https://issues.apache.org/jira/browse/HIVE-16555
> Project: Hive
>  Issue Type: Sub-task
>  Components: Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
> Attachments: HIVE-16555.01.patch, HIVE-16555.02.patch, 
> HIVE-16555.03.patch, HIVE-16555.04.patch, HIVE-16555.05.patch, 
> HIVE-16555.06.patch, HIVE-16555.07.patch
>
>
> Sub-task of the main JIRA to add the new thrift API



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16579) CachedStore: improvements to partition col stats caching and cache column stats for unpartitioned table

[Thejas M Nair (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006980#comment-16006980
 ] 

Thejas M Nair commented on HIVE-16579:
--

[~vgumashta] Can you please include a review board link or pull request ?


> CachedStore: improvements to partition col stats caching and cache column 
> stats for unpartitioned table
> ---
>
> Key: HIVE-16579
> URL: https://issues.apache.org/jira/browse/HIVE-16579
> Project: Hive
>  Issue Type: Improvement
>  Components: Metastore
>Affects Versions: 3.0.0
>Reporter: Vaibhav Gumashta
>Assignee: Vaibhav Gumashta
> Attachments: HIVE-16579.1.patch, HIVE-16579.2.patch
>
>
> 1. Update stats cache when partitions/table is dropped.
> 2. Update cached partition col stats in the background cache update thread. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (HIVE-16652) LlapInputFormat: Seeing "output error" WARN message

[Jason Dere (JIRA)]

[ 
https://issues.apache.org/jira/browse/HIVE-16652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16006967#comment-16006967
 ] 

Jason Dere commented on HIVE-16652:
---

I suspect we may be closing the LlapTaskUmbilicalExternalClient and the 
umbilical connection to LLAP daemon before the LlapTaskUmbilicalExternalClient 
has the chance to send the heartbeat response back to the LLAP daemon.


> LlapInputFormat: Seeing "output error" WARN message
> ---
>
> Key: HIVE-16652
> URL: https://issues.apache.org/jira/browse/HIVE-16652
> Project: Hive
>  Issue Type: Sub-task
>  Components: llap
>Reporter: Jason Dere
>Assignee: Jason Dere
>
> Another warning message I'm seeing in the logs for TestJdbcWithMiniLlap after 
> adding the line to close the RecordReader in the test:
> {noformat}
> 2017-05-11T11:08:34,511  WARN [IPC Server handler 0 on 54847] ipc.Server: IPC 
> Server handler 0 on 54847, call Call#341 Retry#0 heartbeat({  
> containerId=container_6830411502416918223_0003_00_00, requestId=2, 
> startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, 
> taskAttemptId=attempt_6830411502416918223_0003_0_00_00_0, eventCount=2 
> }), rpc version=2, client version=1, methodsFingerPrint=996603002 from 
> 10.22.8.180:54849: output error
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16634) LLAP Use a pool of connections to a single AM from a daemon

[Siddharth Seth (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Siddharth Seth updated HIVE-16634:
--
Attachment: HIVE-16634.02.patch

Updated to add a cap. One downside is that it will always created the 
configured instances. Don't think seeding with a single instance helps a lot.

> LLAP Use a pool of connections to a single AM from a daemon
> ---
>
> Key: HIVE-16634
> URL: https://issues.apache.org/jira/browse/HIVE-16634
> Project: Hive
>  Issue Type: Bug
>  Components: llap
>Reporter: Rajesh Balamohan
>Assignee: Siddharth Seth
> Attachments: HIVE-16634.01.patch, HIVE-16634.02.patch, 
> locked-threads-ipc.png
>
>
> !locked-threads-ipc.png!



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Assigned] (HIVE-16652) LlapInputFormat: Seeing "output error" WARN message

[Jason Dere (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16652?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Dere reassigned HIVE-16652:
-


> LlapInputFormat: Seeing "output error" WARN message
> ---
>
> Key: HIVE-16652
> URL: https://issues.apache.org/jira/browse/HIVE-16652
> Project: Hive
>  Issue Type: Sub-task
>  Components: llap
>Reporter: Jason Dere
>Assignee: Jason Dere
>
> Another warning message I'm seeing in the logs for TestJdbcWithMiniLlap after 
> adding the line to close the RecordReader in the test:
> {noformat}
> 2017-05-11T11:08:34,511  WARN [IPC Server handler 0 on 54847] ipc.Server: IPC 
> Server handler 0 on 54847, call Call#341 Retry#0 heartbeat({  
> containerId=container_6830411502416918223_0003_00_00, requestId=2, 
> startIndex=0, preRoutedStartIndex=0, maxEventsToGet=500, 
> taskAttemptId=attempt_6830411502416918223_0003_0_00_00_0, eventCount=2 
> }), rpc version=2, client version=1, methodsFingerPrint=996603002 from 
> 10.22.8.180:54849: output error
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-14389) Beeline should not output query and prompt to stdout

[Vihang Karajgaonkar (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-14389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Vihang Karajgaonkar updated HIVE-14389:
---
Attachment: HIVE-14389.04.patch

Fixed the test failure. Also added some negative tests as per Peter's suggestion

> Beeline should not output query and prompt to stdout
> 
>
> Key: HIVE-14389
> URL: https://issues.apache.org/jira/browse/HIVE-14389
> Project: Hive
>  Issue Type: Improvement
>  Components: Beeline
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Minor
> Attachments: HIVE-14389.01.patch, HIVE-14389.02.patch, 
> HIVE-14389.03.patch, HIVE-14389.04.patch
>
>
> It seems that the Beeline prints the query along with the results in the 
> stdout when a script file is passed. The output file in the example below 
> needs to only have the results and not the query.
> {noformat}
> .vihang-MBP:bin vihang$ ./beeline --showheader=false --outformat=tsv2 -u 
> "jdbc:hive2://localhost:1" -f /tmp/query.sql > /tmp/query.out 2> 
> /tmp/query.err
> OK
> $ cat /tmp/query.out
> 1: jdbc:hive2://localhost:1/default> select * from likes limit 4;
> +---+--+--+
> | 1 | chocolate|
> | 1 | car  |
> | 1 | games|
> | 1 | chess|
> +---+--+--+
> 1: jdbc:hive2://localhost:1/default>
> 1: jdbc:hive2://localhost:1/default>
> $
> {noformat}
> A lot of people use HiveCLI and in order to transition from HiveCLI scripts 
> to Beeline, this needs to be taken care of. The output files generated by 
> beeline should contain only the results and nothing else.
> Similarly, when not in silent mode, query are being printed out on stdout, 
> which is adding garbage along with results, as just like HIVE CLI does, users 
> would like to have only the results on stdout, not errors/debugging info/etc, 
> like the full query. 
> Query could be printed out, no problem, as long as it is not on stdout (with 
> results), instead, it must be printed out along with the debugging info.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (HIVE-16651) LlapProtocolClientProxy stack trace when using llap input format

[Jason Dere (JIRA)]

 [ 
https://issues.apache.org/jira/browse/HIVE-16651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Dere updated HIVE-16651:
--
Attachment: HIVE-16651.1.patch

Looks like this has to do with cancelling the requestManagerFuture in 
LlapProtocolClientProxy. Patch to ignore CancellationException in the 
FuturesCallback. I suppose another fix for this might be to avoid cancelling 
the future in the first place.

[~sseth], can you review?

> LlapProtocolClientProxy stack trace when using llap input format
> 
>
> Key: HIVE-16651
> URL: https://issues.apache.org/jira/browse/HIVE-16651
> Project: Hive
>  Issue Type: Sub-task
>  Components: llap
>Reporter: Jason Dere
>Assignee: Jason Dere
> Attachments: HIVE-16651.1.patch
>
>
> Seeing this after LlapBaseRecordReader.close():
> {noformat}
> 16/06/28 22:05:32 WARN LlapProtocolClientProxy: RequestManager shutdown with 
> error
> java.util.concurrent.CancellationException
>   at java.util.concurrent.FutureTask.report(FutureTask.java:121)
>   at java.util.concurrent.FutureTask.get(FutureTask.java:192)
>   at 
> com.google.common.util.concurrent.Uninterruptibles.getUninterruptibly(Uninterruptibles.java:135)
>   at com.google.common.util.concurrent.Futures$4.run(Futures.java:1170)
>   at 
> com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:297)
>   at 
> com.google.common.util.concurrent.ExecutionList.executeListener(ExecutionList.java:156)
>   at 
> com.google.common.util.concurrent.ExecutionList.execute(ExecutionList.java:145)
>   at 
> com.google.common.util.concurrent.ListenableFutureTask.done(ListenableFutureTask.java:91)
>   at java.util.concurrent.FutureTask.finishCompletion(FutureTask.java:384)
>   at java.util.concurrent.FutureTask.cancel(FutureTask.java:180)
>   at 
> org.apache.hadoop.hive.llap.tez.LlapProtocolClientProxy.serviceStop(LlapProtocolClientProxy.java:131)
>   at 
> org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
>   at 
> org.apache.hadoop.hive.llap.ext.LlapTaskUmbilicalExternalClient.serviceStop(LlapTaskUmbilicalExternalClient.java:135)
>   at 
> org.apache.hadoop.service.AbstractService.stop(AbstractService.java:221)
>   at 
> org.apache.hadoop.service.AbstractService.close(AbstractService.java:250)
>   at 
> org.apache.hadoop.hive.llap.LlapBaseRecordReader.close(LlapBaseRecordReader.java:84)
>   at 
> org.apache.hadoop.hive.llap.LlapRowRecordReader.close(LlapRowRecordReader.java:80)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)