[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load

2018-06-24 Thread mahesh kumar behera (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521875#comment-16521875
 ] 

mahesh kumar behera commented on HIVE-19924:


https://github.com/apache/hive/pull/376

> Tag distcp jobs run by Repl Load
> 
>
> Key: HIVE-19924
> URL: https://issues.apache.org/jira/browse/HIVE-19924
> Project: Hive
>  Issue Type: Task
>  Components: repl
>Affects Versions: 3.1.0, 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19924.01.patch
>
>
> Add tags in jobconf for distcp related jobs started by replication. This will 
> allow hive to kill these jobs in case beacon retries, or hs2 dies and beacon 
> issues a kill command.
>  * one of the tags should definitely be the query_id that starts the job : 
> With this flow beacon before retrying the bootstrap load, will issue a kill 
> command to hs2 with the query id of the previous issued command. hs2 will 
> then kill an running jobs on yarn tagged with the Query_id.
>  * To get around the additional failure point as mentioned above. The jobs 
> can be tagged with an additional unique tag_id provided by Beacon in the WITH 
> clause in repl load command to be used to tag distcp jobs ). Enhance the kill 
> api to take the tag as input and kill jobs associated with that tag. Problem 
> here is how do we validate the association of the tag with a hive query id to 
> make sure this api is not used to kill jobs run by other components, however 
> we can provide this capability to only admins and should be ok in that case.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19938) Upgrade scripts for information schema

2018-06-24 Thread Daniel Dai (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521873#comment-16521873
 ] 

Daniel Dai commented on HIVE-19938:
---

Upload HIVE-19938.3.patch to fix ptest failure.

> Upgrade scripts for information schema
> --
>
> Key: HIVE-19938
> URL: https://issues.apache.org/jira/browse/HIVE-19938
> Project: Hive
>  Issue Type: Bug
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-19938.1.patch, HIVE-19938.2.patch, 
> HIVE-19938.3.patch
>
>
> To make schematool -upgradeSchema work for information schema.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-19938) Upgrade scripts for information schema

2018-06-24 Thread Daniel Dai (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Daniel Dai updated HIVE-19938:
--
Attachment: HIVE-19938.3.patch

> Upgrade scripts for information schema
> --
>
> Key: HIVE-19938
> URL: https://issues.apache.org/jira/browse/HIVE-19938
> Project: Hive
>  Issue Type: Bug
>Reporter: Daniel Dai
>Assignee: Daniel Dai
>Priority: Major
> Attachments: HIVE-19938.1.patch, HIVE-19938.2.patch, 
> HIVE-19938.3.patch
>
>
> To make schematool -upgradeSchema work for information schema.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19924) Tag distcp jobs run by Repl Load

2018-06-24 Thread mahesh kumar behera (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521871#comment-16521871
 ] 

mahesh kumar behera commented on HIVE-19924:


[~sankarh]

pease review the patch

> Tag distcp jobs run by Repl Load
> 
>
> Key: HIVE-19924
> URL: https://issues.apache.org/jira/browse/HIVE-19924
> Project: Hive
>  Issue Type: Task
>  Components: repl
>Affects Versions: 3.1.0, 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19924.01.patch
>
>
> Add tags in jobconf for distcp related jobs started by replication. This will 
> allow hive to kill these jobs in case beacon retries, or hs2 dies and beacon 
> issues a kill command.
>  * one of the tags should definitely be the query_id that starts the job : 
> With this flow beacon before retrying the bootstrap load, will issue a kill 
> command to hs2 with the query id of the previous issued command. hs2 will 
> then kill an running jobs on yarn tagged with the Query_id.
>  * To get around the additional failure point as mentioned above. The jobs 
> can be tagged with an additional unique tag_id provided by Beacon in the WITH 
> clause in repl load command to be used to tag distcp jobs ). Enhance the kill 
> api to take the tag as input and kill jobs associated with that tag. Problem 
> here is how do we validate the association of the tag with a hive query id to 
> make sure this api is not used to kill jobs run by other components, however 
> we can provide this capability to only admins and should be ok in that case.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-19924) Tag distcp jobs run by Repl Load

2018-06-24 Thread mahesh kumar behera (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

mahesh kumar behera updated HIVE-19924:
---
Status: Patch Available  (was: Open)

> Tag distcp jobs run by Repl Load
> 
>
> Key: HIVE-19924
> URL: https://issues.apache.org/jira/browse/HIVE-19924
> Project: Hive
>  Issue Type: Task
>  Components: repl
>Affects Versions: 3.1.0, 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19924.01.patch
>
>
> Add tags in jobconf for distcp related jobs started by replication. This will 
> allow hive to kill these jobs in case beacon retries, or hs2 dies and beacon 
> issues a kill command.
>  * one of the tags should definitely be the query_id that starts the job : 
> With this flow beacon before retrying the bootstrap load, will issue a kill 
> command to hs2 with the query id of the previous issued command. hs2 will 
> then kill an running jobs on yarn tagged with the Query_id.
>  * To get around the additional failure point as mentioned above. The jobs 
> can be tagged with an additional unique tag_id provided by Beacon in the WITH 
> clause in repl load command to be used to tag distcp jobs ). Enhance the kill 
> api to take the tag as input and kill jobs associated with that tag. Problem 
> here is how do we validate the association of the tag with a hive query id to 
> make sure this api is not used to kill jobs run by other components, however 
> we can provide this capability to only admins and should be ok in that case.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-18705) Improve HiveMetaStoreClient.dropDatabase

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-18705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521864#comment-16521864
 ] 

Hive QA commented on HIVE-18705:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
36s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
24s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
30s{color} | {color:blue} hbase-handler in master has 15 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 
58s{color} | {color:blue} ql in master has 2280 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
38s{color} | {color:blue} service in master has 48 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 
11s{color} | {color:blue} standalone-metastore in master has 228 extant 
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  2m 
21s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:red}-1{color} | {color:red} mvninstall {color} | {color:red}  0m 
20s{color} | {color:red} service in the patch failed. {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  2m 
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
11s{color} | {color:green} The patch hbase-handler passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
37s{color} | {color:green} ql: The patch generated 0 new + 0 unchanged - 12 
fixed = 0 total (was 12) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
13s{color} | {color:green} The patch service passed checkstyle {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
22s{color} | {color:green} The patch standalone-metastore passed checkstyle 
{color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  8m 
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  2m 
23s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
13s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 42m  3s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12072/dev-support/hive-personality.sh
 |
| git revision | master / 37bcf7b |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| mvninstall | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-12072/yetus/patch-mvninstall-service.txt
 |
| modules | C: hbase-handler ql service standalone-metastore U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-12072/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Improve HiveMetaStoreClient.dropDatabase
> 
>
> Key: HIVE-18705
> URL: https://issues.apache.org/jira/browse/HIVE-18705
> Project: Hive
>

[jira] [Resolved] (HIVE-19965) Make HiveEndPoint use IMetaStoreClient.add_partition

2018-06-24 Thread Eugene Koifman (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eugene Koifman resolved HIVE-19965.
---
   Resolution: Fixed
Fix Version/s: 3.1.0

> Make HiveEndPoint use IMetaStoreClient.add_partition
> 
>
> Key: HIVE-19965
> URL: https://issues.apache.org/jira/browse/HIVE-19965
> Project: Hive
>  Issue Type: Bug
>  Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Blocker
> Fix For: 3.1.0
>
> Attachments: HIVE-19965.01-branch-3.patch, HIVE-19965.01.patch, 
> HIVE-19965.02-branch-3.patch
>
>
> it currently uses "alter table add partition if exists..."
> which since HIVE-18814 requires X lock on the table which blocks other 
> streaming writers from making progress.  
> HIVE-19961 to investigate if X lock can be relaxed, etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-18729) Druid Time column type

2018-06-24 Thread Ashutosh Chauhan (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-18729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521853#comment-16521853
 ] 

Ashutosh Chauhan commented on HIVE-18729:
-

+1

> Druid Time column type
> --
>
> Key: HIVE-18729
> URL: https://issues.apache.org/jira/browse/HIVE-18729
> Project: Hive
>  Issue Type: Task
>  Components: Druid integration
>Reporter: slim bouguerra
>Assignee: Jesus Camacho Rodriguez
>Priority: Blocker
> Attachments: HIVE-18729.branch-3.patch, HIVE-18729.patch
>
>
> I have talked Offline with [~jcamachorodriguez] about this and agreed that 
> the best way to go is to support both cases where Druid time column can be 
> Timestamp or Timestamp with local time zone. 
> In fact, for the Hive-Druid internal table, this makes perfect sense since we 
> have Hive metadata about the time column during the CTAS statement then we 
> can handle both cases as we do for another type of storage eg ORC.
> For the Druid external tables, we can have a default type and allow the user 
> to override that via table properties. 
> CC [~ashutoshc] and [~nishantbangarwa]. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-12192) Hive should carry out timestamp computations in UTC

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-12192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521852#comment-16521852
 ] 

Hive QA commented on HIVE-12192:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928922/HIVE-12192.25.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/12071/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12071/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12071/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL 
https://issues.apache.org/jira/secure/attachment/12928922/HIVE-12192.25.patch 
was found in seen patch url's cache and a test was probably run already on it. 
Aborting...
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12928922 - PreCommit-HIVE-Build

> Hive should carry out timestamp computations in UTC
> ---
>
> Key: HIVE-12192
> URL: https://issues.apache.org/jira/browse/HIVE-12192
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Ryan Blue
>Assignee: Jesus Camacho Rodriguez
>Priority: Blocker
>  Labels: timestamp
> Fix For: 3.1.0
>
> Attachments: HIVE-12192.01.patch, HIVE-12192.02.patch, 
> HIVE-12192.03.patch, HIVE-12192.04.patch, HIVE-12192.05.patch, 
> HIVE-12192.06.patch, HIVE-12192.07.patch, HIVE-12192.08.patch, 
> HIVE-12192.09.patch, HIVE-12192.10.patch, HIVE-12192.11.patch, 
> HIVE-12192.12.patch, HIVE-12192.13.patch, HIVE-12192.14.patch, 
> HIVE-12192.15.patch, HIVE-12192.16.patch, HIVE-12192.17.patch, 
> HIVE-12192.18.patch, HIVE-12192.19.patch, HIVE-12192.20.patch, 
> HIVE-12192.21.patch, HIVE-12192.22.patch, HIVE-12192.23.patch, 
> HIVE-12192.24.patch, HIVE-12192.25.patch, HIVE-12192.patch
>
>
> Hive currently uses the "local" time of a java.sql.Timestamp to represent the 
> SQL data type TIMESTAMP WITHOUT TIME ZONE. The purpose is to be able to use 
> {{Timestamp#getYear()}} and similar methods to implement SQL functions like 
> {{year}}.
> When the SQL session's time zone is a DST zone, such as America/Los_Angeles 
> that alternates between PST and PDT, there are times that cannot be 
> represented because the effective zone skips them.
> {code}
> hive> select TIMESTAMP '2015-03-08 02:10:00.101';
> 2015-03-08 03:10:00.101
> {code}
> Using UTC instead of the SQL session time zone as the underlying zone for a 
> java.sql.Timestamp avoids this bug, while still returning correct values for 
> {{getYear}} etc. Using UTC as the convenience representation (timestamp 
> without time zone has no real zone) would make timestamp calculations more 
> consistent and avoid similar problems in the future.
> Notably, this would break the {{unix_timestamp}} UDF that specifies the 
> result is with respect to ["the default timezone and default 
> locale"|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-DateFunctions].
>  That function would need to be updated to use the 
> {{System.getProperty("user.timezone")}} zone.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-12192) Hive should carry out timestamp computations in UTC

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-12192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521851#comment-16521851
 ] 

Hive QA commented on HIVE-12192:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928922/HIVE-12192.25.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/12069/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12069/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12069/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL 
https://issues.apache.org/jira/secure/attachment/12928922/HIVE-12192.25.patch 
was found in seen patch url's cache and a test was probably run already on it. 
Aborting...
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12928922 - PreCommit-HIVE-Build

> Hive should carry out timestamp computations in UTC
> ---
>
> Key: HIVE-12192
> URL: https://issues.apache.org/jira/browse/HIVE-12192
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Ryan Blue
>Assignee: Jesus Camacho Rodriguez
>Priority: Blocker
>  Labels: timestamp
> Fix For: 3.1.0
>
> Attachments: HIVE-12192.01.patch, HIVE-12192.02.patch, 
> HIVE-12192.03.patch, HIVE-12192.04.patch, HIVE-12192.05.patch, 
> HIVE-12192.06.patch, HIVE-12192.07.patch, HIVE-12192.08.patch, 
> HIVE-12192.09.patch, HIVE-12192.10.patch, HIVE-12192.11.patch, 
> HIVE-12192.12.patch, HIVE-12192.13.patch, HIVE-12192.14.patch, 
> HIVE-12192.15.patch, HIVE-12192.16.patch, HIVE-12192.17.patch, 
> HIVE-12192.18.patch, HIVE-12192.19.patch, HIVE-12192.20.patch, 
> HIVE-12192.21.patch, HIVE-12192.22.patch, HIVE-12192.23.patch, 
> HIVE-12192.24.patch, HIVE-12192.25.patch, HIVE-12192.patch
>
>
> Hive currently uses the "local" time of a java.sql.Timestamp to represent the 
> SQL data type TIMESTAMP WITHOUT TIME ZONE. The purpose is to be able to use 
> {{Timestamp#getYear()}} and similar methods to implement SQL functions like 
> {{year}}.
> When the SQL session's time zone is a DST zone, such as America/Los_Angeles 
> that alternates between PST and PDT, there are times that cannot be 
> represented because the effective zone skips them.
> {code}
> hive> select TIMESTAMP '2015-03-08 02:10:00.101';
> 2015-03-08 03:10:00.101
> {code}
> Using UTC instead of the SQL session time zone as the underlying zone for a 
> java.sql.Timestamp avoids this bug, while still returning correct values for 
> {{getYear}} etc. Using UTC as the convenience representation (timestamp 
> without time zone has no real zone) would make timestamp calculations more 
> consistent and avoid similar problems in the future.
> Notably, this would break the {{unix_timestamp}} UDF that specifies the 
> result is with respect to ["the default timezone and default 
> locale"|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-DateFunctions].
>  That function would need to be updated to use the 
> {{System.getProperty("user.timezone")}} zone.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-12342) Set default value of hive.optimize.index.filter to true

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-12342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521849#comment-16521849
 ] 

Hive QA commented on HIVE-12342:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928914/HIVE-12342.21.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14603 tests 
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testParallelCompilation3 (batchId=247)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/12068/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12068/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12068/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12928914 - PreCommit-HIVE-Build

> Set default value of hive.optimize.index.filter to true
> ---
>
> Key: HIVE-12342
> URL: https://issues.apache.org/jira/browse/HIVE-12342
> Project: Hive
>  Issue Type: Task
>  Components: Configuration
>Reporter: Ashutosh Chauhan
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-12342.05.patch, HIVE-12342.06.patch, 
> HIVE-12342.07.patch, HIVE-12342.08.patch, HIVE-12342.09.patch, 
> HIVE-12342.1.patch, HIVE-12342.10.patch, HIVE-12342.11.patch, 
> HIVE-12342.12.patch, HIVE-12342.13.patch, HIVE-12342.14.patch, 
> HIVE-12342.15.patch, HIVE-12342.16.patch, HIVE-12342.17.patch, 
> HIVE-12342.18.patch, HIVE-12342.19.patch, HIVE-12342.2.patch, 
> HIVE-12342.20.patch, HIVE-12342.21.patch, HIVE-12342.3.patch, 
> HIVE-12342.4.patch, HIVE-12342.patch
>
>
> This configuration governs ppd for storage layer. When applicable, it will 
> always help. It should be on by default.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-12192) Hive should carry out timestamp computations in UTC

2018-06-24 Thread Ashutosh Chauhan (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-12192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-12192:

Resolution: Fixed
Status: Resolved  (was: Patch Available)

> Hive should carry out timestamp computations in UTC
> ---
>
> Key: HIVE-12192
> URL: https://issues.apache.org/jira/browse/HIVE-12192
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Ryan Blue
>Assignee: Jesus Camacho Rodriguez
>Priority: Blocker
>  Labels: timestamp
> Fix For: 3.1.0
>
> Attachments: HIVE-12192.01.patch, HIVE-12192.02.patch, 
> HIVE-12192.03.patch, HIVE-12192.04.patch, HIVE-12192.05.patch, 
> HIVE-12192.06.patch, HIVE-12192.07.patch, HIVE-12192.08.patch, 
> HIVE-12192.09.patch, HIVE-12192.10.patch, HIVE-12192.11.patch, 
> HIVE-12192.12.patch, HIVE-12192.13.patch, HIVE-12192.14.patch, 
> HIVE-12192.15.patch, HIVE-12192.16.patch, HIVE-12192.17.patch, 
> HIVE-12192.18.patch, HIVE-12192.19.patch, HIVE-12192.20.patch, 
> HIVE-12192.21.patch, HIVE-12192.22.patch, HIVE-12192.23.patch, 
> HIVE-12192.24.patch, HIVE-12192.25.patch, HIVE-12192.patch
>
>
> Hive currently uses the "local" time of a java.sql.Timestamp to represent the 
> SQL data type TIMESTAMP WITHOUT TIME ZONE. The purpose is to be able to use 
> {{Timestamp#getYear()}} and similar methods to implement SQL functions like 
> {{year}}.
> When the SQL session's time zone is a DST zone, such as America/Los_Angeles 
> that alternates between PST and PDT, there are times that cannot be 
> represented because the effective zone skips them.
> {code}
> hive> select TIMESTAMP '2015-03-08 02:10:00.101';
> 2015-03-08 03:10:00.101
> {code}
> Using UTC instead of the SQL session time zone as the underlying zone for a 
> java.sql.Timestamp avoids this bug, while still returning correct values for 
> {{getYear}} etc. Using UTC as the convenience representation (timestamp 
> without time zone has no real zone) would make timestamp calculations more 
> consistent and avoid similar problems in the future.
> Notably, this would break the {{unix_timestamp}} UDF that specifies the 
> result is with respect to ["the default timezone and default 
> locale"|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-DateFunctions].
>  That function would need to be updated to use the 
> {{System.getProperty("user.timezone")}} zone.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-19941) Row based Filters added via Hive Ranger policies are not pushed to druid

2018-06-24 Thread Ashutosh Chauhan (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ashutosh Chauhan updated HIVE-19941:

   Resolution: Fixed
Fix Version/s: 3.1.0
   Status: Resolved  (was: Patch Available)

Pushed to master and branch-3 Thanks, Nishant!

> Row based Filters added via Hive Ranger policies are not pushed to druid
> 
>
> Key: HIVE-19941
> URL: https://issues.apache.org/jira/browse/HIVE-19941
> Project: Hive
>  Issue Type: Bug
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Fix For: 3.1.0
>
> Attachments: HIVE-19941.1.patch, HIVE-19941.2.patch, HIVE-19941.patch
>
>
> Issue is that when applying table mask we add virtual columns, however 
> non-native tables do not have virtual columns, we need to skip adding virtual 
> columns when generating masking query. 
> Stack Trace - 
> {code} 
> org.apache.hadoop.hive.ql.parse.SemanticException: Line 1:79 Invalid table 
> alias or column reference 'BLOCK__OFFSET__INSIDE__FILE'
> : (possible column names are: __time, yearmonth, year, month, dayofmonth, 
> dayofweek, weekofyear, hour, minute, second, payment_typ
> e, fare_amount, surcharge, mta_tax, tip_amount, tolls_amount, total_amount, 
> trip_time)
> at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genAllExprNodeDesc(SemanticAnalyzer.java:11830)
>  ~[hive-exec-2.1.0.2.6.
> 4.0-91.jar:2.1.0.2.6.4.0-91]
> at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genExprNodeDesc(SemanticAnalyzer.java:11778)
>  ~[hive-exec-2.1.0.2.6.4.0
> -91.jar:2.1.0.2.6.4.0-91]
> at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genSelectLogicalPlan(CalcitePlanner.java:3780)
>  ~[hi
> ve-exec-2.1.0.2.6.4.0-91.jar:2.1.0.2.6.4.0-91]
> at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:4117)
>  ~[hive-exe
> c-2.1.0.2.6.4.0-91.jar:2.1.0.2.6.4.0-91]
> at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:4016)
>  ~[hive-exe
> c-2.1.0.2.6.4.0-91.jar:2.1.0.2.6.4.0-91]
> at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:4060)
>  ~[hive-exe
> c-2.1.0.2.6.4.0-91.jar:2.1.0.2.6.4.0-91]
> at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1340)
>  ~[hive-exec-2.1.0.2
> .6.4.0-91.jar:2.1.0.2.6.4.0-91]
> at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1277)
>  ~[hive-exec-2.1.0.2
> .6.4.0-91.jar:2.1.0.2.6.4.0-91]
> at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:113) 
> ~[calcite-core-1.10.0.2.6.4.0-91.jar:1.10.0.2.6.4.0-91
> ]
> at 
> org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:997)
>  ~[calcite-core-1.10.0.2.6.4.0-91.jar
> :1.10.0.2.6.4.0-91]
> at 
> org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:149) 
> ~[calcite-core-1.10.0.2.6.4.0-91.jar:1.10.0.2.6.4.
> 0-91]
> at 
> org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:106) 
> ~[calcite-core-1.10.0.2.6.4.0-91.jar:1.10.0.2.6.4.
> 0-91]
> at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.logicalPlan(CalcitePlanner.java:1082)
>  ~[hive-exec-2.1.0.2.6.4.0-91.jar:2
> .1.0.2.6.4.0-91]
> {code} 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-12342) Set default value of hive.optimize.index.filter to true

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-12342?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521835#comment-16521835
 ] 

Hive QA commented on HIVE-12342:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
7s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
15s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
43s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
32s{color} | {color:blue} common in master has 62 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
29s{color} | {color:blue} accumulo-handler in master has 21 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
24s{color} | {color:blue} contrib in master has 13 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
31s{color} | {color:blue} hbase-handler in master has 15 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 
58s{color} | {color:blue} ql in master has 2280 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
45s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 
30s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  2m 
16s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
46s{color} | {color:red} ql: The patch generated 1 new + 30 unchanged - 0 fixed 
= 31 total (was 30) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 1s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  6m 
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
45s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
12s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 15s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12068/dev-support/hive-personality.sh
 |
| git revision | master / b023f5d |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-12068/yetus/diff-checkstyle-ql.txt
 |
| modules | C: common accumulo-handler contrib hbase-handler ql U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-12068/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Set default value of hive.optimize.index.filter to true
> ---
>
> Key: HIVE-12342
> URL: https://issues.apache.org/jira/browse/HIVE-12342
> Project: Hive
>  Issue Type: Task
>  Components: Configuration
>Reporter: Ashutosh Chauhan
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-12342.05.patch, HIVE-12342.06.patch, 
> HIVE-12342.07.patch, HIVE-12342.08.patch, HIVE-12342.09.patch, 
> 

[jira] [Commented] (HIVE-19967) SMB Join : Need Optraits for PTFOperator ala GBY Op

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521808#comment-16521808
 ] 

Hive QA commented on HIVE-19967:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928909/HIVE-19967.2.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 6 failed/errored test(s), 14604 tests 
executed
*Failed tests:*
{noformat}
org.apache.hive.jdbc.TestActivePassiveHA.testActivePassiveHA (batchId=244)
org.apache.hive.jdbc.TestActivePassiveHA.testClientConnectionsOnFailover 
(batchId=244)
org.apache.hive.jdbc.TestActivePassiveHA.testConnectionActivePassiveHAServiceDiscovery
 (batchId=244)
org.apache.hive.jdbc.TestActivePassiveHA.testManualFailover (batchId=244)
org.apache.hive.jdbc.TestActivePassiveHA.testManualFailoverUnauthorized 
(batchId=244)
org.apache.hive.jdbc.TestJdbcWithMiniHS2.testParallelCompilation3 (batchId=247)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/12067/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12067/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12067/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 6 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12928909 - PreCommit-HIVE-Build

> SMB Join : Need Optraits for PTFOperator ala GBY Op
> ---
>
> Key: HIVE-19967
> URL: https://issues.apache.org/jira/browse/HIVE-19967
> Project: Hive
>  Issue Type: Task
>Reporter: Deepak Jaiswal
>Assignee: Deepak Jaiswal
>Priority: Major
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19967.1.patch, HIVE-19967.2.patch
>
>
> The SMB join on one or more PTF Ops should reset the optraits keys just like 
> GBY Op does.
> Currently there is no implementation of PTFOp optraits.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19972) Followup to HIVE-19928 : Fix the check for managed table

2018-06-24 Thread Deepak Jaiswal (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521794#comment-16521794
 ] 

Deepak Jaiswal commented on HIVE-19972:
---

committed to master, preparing for branch-3

> Followup to HIVE-19928 : Fix the check for managed table
> 
>
> Key: HIVE-19972
> URL: https://issues.apache.org/jira/browse/HIVE-19972
> Project: Hive
>  Issue Type: Bug
>Reporter: Deepak Jaiswal
>Assignee: Deepak Jaiswal
>Priority: Major
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19972.01-branch-3.patch, HIVE-19972.1.patch, 
> HIVE-19972.2.patch, HIVE-19972.3.patch
>
>
> The check for managed table should use ENUM comparison rather than string 
> comparison.
> The check in the patch will always return false, thus maintaining existing 
> behavior.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-19972) Followup to HIVE-19928 : Fix the check for managed table

2018-06-24 Thread Deepak Jaiswal (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deepak Jaiswal updated HIVE-19972:
--
Attachment: HIVE-19972.01-branch-3.patch

> Followup to HIVE-19928 : Fix the check for managed table
> 
>
> Key: HIVE-19972
> URL: https://issues.apache.org/jira/browse/HIVE-19972
> Project: Hive
>  Issue Type: Bug
>Reporter: Deepak Jaiswal
>Assignee: Deepak Jaiswal
>Priority: Major
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19972.01-branch-3.patch, HIVE-19972.1.patch, 
> HIVE-19972.2.patch, HIVE-19972.3.patch
>
>
> The check for managed table should use ENUM comparison rather than string 
> comparison.
> The check in the patch will always return false, thus maintaining existing 
> behavior.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-19267) Create/Replicate ACID Write event

2018-06-24 Thread mahesh kumar behera (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

mahesh kumar behera updated HIVE-19267:
---
Attachment: (was: HIVE-19267.18.patch)

> Create/Replicate ACID Write event
> -
>
> Key: HIVE-19267
> URL: https://issues.apache.org/jira/browse/HIVE-19267
> Project: Hive
>  Issue Type: Sub-task
>  Components: repl, Transactions
>Affects Versions: 3.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
>  Labels: ACID, DR, pull-request-available, replication
> Attachments: HIVE-19267.01-branch-3.patch, HIVE-19267.01.patch, 
> HIVE-19267.02.patch, HIVE-19267.03.patch, HIVE-19267.04.patch, 
> HIVE-19267.05.patch, HIVE-19267.06.patch, HIVE-19267.07.patch, 
> HIVE-19267.08.patch, HIVE-19267.09.patch, HIVE-19267.10.patch, 
> HIVE-19267.11.patch, HIVE-19267.12.patch, HIVE-19267.13.patch, 
> HIVE-19267.14.patch, HIVE-19267.15.patch, HIVE-19267.16.patch, 
> HIVE-19267.17.patch
>
>
>  
> h1. Replicate ACID write Events
>  * Create new EVENT_WRITE event with related message format to log the write 
> operations with in a txn along with data associated.
>  * Log this event when perform any writes (insert into, insert overwrite, 
> load table, delete, update, merge, truncate) on table/partition.
>  * If a single MERGE/UPDATE/INSERT/DELETE statement operates on multiple 
> partitions, then need to log one event per partition.
>  * DbNotificationListener should log this type of event to special metastore 
> table named "MTxnWriteNotificationLog".
>  * This table should maintain a map of txn ID against list of 
> tables/partitions written by given txn.
>  * The entry for a given txn should be removed by the cleaner thread that 
> removes the expired events from EventNotificationTable.
> h1. Replicate Commit Txn operation (with writes)
> Add new EVENT_COMMIT_TXN to log the metadata/data of all tables/partitions 
> modified within the txn.
> *Source warehouse:*
>  * This event should read the EVENT_WRITEs from "MTxnWriteNotificationLog" 
> metastore table to consolidate the list of tables/partitions modified within 
> this txn scope.
>  * Based on the list of tables/partitions modified and table Write ID, need 
> to compute the list of delta files added by this txn.
>  * Repl dump should read this message and dump the metadata and delta files 
> list.
> *Target warehouse:*
>  * Ensure snapshot isolation at target for on-going read txns which shouldn't 
> view the data replicated from committed txn. (Ensured with open and allocate 
> write ID events).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-18118) Explain Extended should indicate if a file being read is an EC file

2018-06-24 Thread Andrew Sherman (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-18118?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Sherman updated HIVE-18118:
--
Attachment: HIVE-18118.5.patch

> Explain Extended should indicate if a file being read is an EC file
> ---
>
> Key: HIVE-18118
> URL: https://issues.apache.org/jira/browse/HIVE-18118
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Sahil Takiar
>Assignee: Andrew Sherman
>Priority: Major
> Attachments: HIVE-18118.1.patch, HIVE-18118.2.patch, 
> HIVE-18118.3.patch, HIVE-18118.4.patch, HIVE-18118.5.patch
>
>
> We already print out the files Hive will read in the explain extended 
> command, we just have to modify it to say whether or not its an EC file.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19967) SMB Join : Need Optraits for PTFOperator ala GBY Op

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521791#comment-16521791
 ] 

Hive QA commented on HIVE-19967:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
48s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
5s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  4m 
10s{color} | {color:blue} ql in master has 2282 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
5s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m  
6s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
37s{color} | {color:red} ql: The patch generated 1 new + 14 unchanged - 1 fixed 
= 15 total (was 15) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m 
11s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
13s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 11s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12067/dev-support/hive-personality.sh
 |
| git revision | master / be1955f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-12067/yetus/diff-checkstyle-ql.txt
 |
| modules | C: itests ql U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-12067/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> SMB Join : Need Optraits for PTFOperator ala GBY Op
> ---
>
> Key: HIVE-19967
> URL: https://issues.apache.org/jira/browse/HIVE-19967
> Project: Hive
>  Issue Type: Task
>Reporter: Deepak Jaiswal
>Assignee: Deepak Jaiswal
>Priority: Major
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19967.1.patch, HIVE-19967.2.patch
>
>
> The SMB join on one or more PTF Ops should reset the optraits keys just like 
> GBY Op does.
> Currently there is no implementation of PTFOp optraits.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19671) Distribute by rand() can lead to data inconsistency

2018-06-24 Thread Rui Li (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521790#comment-16521790
 ] 

Rui Li commented on HIVE-19671:
---

We can check all RS and look for non-deterministic UDF in partition keys -- 
{{FunctionRegistry::isDeterministic}} can be used.
I noted Hive itself may also use non-deterministic partitioning, e.g. to handle 
skewed GBY, we first shuffle randomly to do partial aggregation. Do you think 
it makes sense to print a warning for that?

> Distribute by rand() can lead to data inconsistency
> ---
>
> Key: HIVE-19671
> URL: https://issues.apache.org/jira/browse/HIVE-19671
> Project: Hive
>  Issue Type: Bug
>Reporter: Rui Li
>Assignee: Rui Li
>Priority: Major
>
> Noticed the following queries can give different results:
> {code}
> select count(*) from tbl;
> select count(*) from (select * from tbl distribute by rand()) a;
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-19812) Disable external table replication by default via a configuration property

2018-06-24 Thread mahesh kumar behera (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

mahesh kumar behera updated HIVE-19812:
---
Attachment: HIVE-19812.07.patch

> Disable external table replication by default via a configuration property
> --
>
> Key: HIVE-19812
> URL: https://issues.apache.org/jira/browse/HIVE-19812
> Project: Hive
>  Issue Type: Task
>  Components: repl
>Affects Versions: 3.1.0, 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19812.01.patch, HIVE-19812.02.patch, 
> HIVE-19812.03.patch, HIVE-19812.04.patch, HIVE-19812.05.patch, 
> HIVE-19812.06-branch-3.patch, HIVE-19812.06.patch, HIVE-19812.07.patch
>
>
> use a hive config property to allow external table replication. set this 
> property by default to prevent external table replication.
> for metadata only hive repl always export metadata for external tables.
>  
> REPL_DUMP_EXTERNAL_TABLES("hive.repl.dump.include.external.tables", false,
> "Indicates if repl dump should include information about external tables. It 
> should be \n"
> + "used in conjunction with 'hive.repl.dump.metadata.only' set to false. if 
> 'hive.repl.dump.metadata.only' \n"
> + " is set to true then this config parameter has no effect as external table 
> meta data is flushed \n"
> + " always by default.")
> This should be done for only replication dump and not for export



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-19829) Incremental replication load should create tasks in execution phase rather than semantic phase

2018-06-24 Thread mahesh kumar behera (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

mahesh kumar behera updated HIVE-19829:
---
Attachment: HIVE-19829.08.patch

> Incremental replication load should create tasks in execution phase rather 
> than semantic phase
> --
>
> Key: HIVE-19829
> URL: https://issues.apache.org/jira/browse/HIVE-19829
> Project: Hive
>  Issue Type: Task
>  Components: repl
>Affects Versions: 3.1.0, 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
>  Labels: pull-request-available
> Fix For: 4.0.0
>
> Attachments: HIVE-19829.01.patch, HIVE-19829.02.patch, 
> HIVE-19829.03.patch, HIVE-19829.04.patch, HIVE-19829.06.patch, 
> HIVE-19829.07.patch, HIVE-19829.07.patch, HIVE-19829.08-branch-3.patch, 
> HIVE-19829.08.patch
>
>
> Split the incremental load into multiple iterations. In each iteration create 
> number of tasks equal to the configured value.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-19267) Create/Replicate ACID Write event

2018-06-24 Thread mahesh kumar behera (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

mahesh kumar behera updated HIVE-19267:
---
Attachment: HIVE-19267.18.patch

> Create/Replicate ACID Write event
> -
>
> Key: HIVE-19267
> URL: https://issues.apache.org/jira/browse/HIVE-19267
> Project: Hive
>  Issue Type: Sub-task
>  Components: repl, Transactions
>Affects Versions: 3.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
>  Labels: ACID, DR, pull-request-available, replication
> Attachments: HIVE-19267.01-branch-3.patch, HIVE-19267.01.patch, 
> HIVE-19267.02.patch, HIVE-19267.03.patch, HIVE-19267.04.patch, 
> HIVE-19267.05.patch, HIVE-19267.06.patch, HIVE-19267.07.patch, 
> HIVE-19267.08.patch, HIVE-19267.09.patch, HIVE-19267.10.patch, 
> HIVE-19267.11.patch, HIVE-19267.12.patch, HIVE-19267.13.patch, 
> HIVE-19267.14.patch, HIVE-19267.15.patch, HIVE-19267.16.patch, 
> HIVE-19267.17.patch, HIVE-19267.18.patch
>
>
>  
> h1. Replicate ACID write Events
>  * Create new EVENT_WRITE event with related message format to log the write 
> operations with in a txn along with data associated.
>  * Log this event when perform any writes (insert into, insert overwrite, 
> load table, delete, update, merge, truncate) on table/partition.
>  * If a single MERGE/UPDATE/INSERT/DELETE statement operates on multiple 
> partitions, then need to log one event per partition.
>  * DbNotificationListener should log this type of event to special metastore 
> table named "MTxnWriteNotificationLog".
>  * This table should maintain a map of txn ID against list of 
> tables/partitions written by given txn.
>  * The entry for a given txn should be removed by the cleaner thread that 
> removes the expired events from EventNotificationTable.
> h1. Replicate Commit Txn operation (with writes)
> Add new EVENT_COMMIT_TXN to log the metadata/data of all tables/partitions 
> modified within the txn.
> *Source warehouse:*
>  * This event should read the EVENT_WRITEs from "MTxnWriteNotificationLog" 
> metastore table to consolidate the list of tables/partitions modified within 
> this txn scope.
>  * Based on the list of tables/partitions modified and table Write ID, need 
> to compute the list of delta files added by this txn.
>  * Repl dump should read this message and dump the metadata and delta files 
> list.
> *Target warehouse:*
>  * Ensure snapshot isolation at target for on-going read txns which shouldn't 
> view the data replicated from committed txn. (Ensured with open and allocate 
> write ID events).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19481) sample10.q returns wrong results

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521774#comment-16521774
 ] 

Hive QA commented on HIVE-19481:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928908/HIVE-19481.2.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 14 failed/errored test(s), 14604 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestBeeLineDriver.testCliDriver[smb_mapjoin_11] 
(batchId=258)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[archive_excludeHadoop20] 
(batchId=68)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[masking_5] (batchId=30)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample10_mm] (batchId=58)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample6] (batchId=70)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample7] (batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[sample9] (batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[smb_mapjoin_11] 
(batchId=2)
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_basic]
 (batchId=258)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[infer_bucket_sort_bucketed_table]
 (batchId=187)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[sample2] 
(batchId=110)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[sample4] 
(batchId=114)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[sample6] 
(batchId=137)
org.apache.hadoop.hive.cli.TestSparkCliDriver.testCliDriver[sample7] 
(batchId=137)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/12066/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12066/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12066/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 14 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12928908 - PreCommit-HIVE-Build

> sample10.q returns wrong results 
> -
>
> Key: HIVE-19481
> URL: https://issues.apache.org/jira/browse/HIVE-19481
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Affects Versions: 3.0.0
>Reporter: Steve Yeom
>Assignee: Deepak Jaiswal
>Priority: Major
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19481.1.patch, HIVE-19481.2.patch
>
>
> Ran "mvn test -Dtest=TestMiniLlapLocalCliDriver -Dqfile=sample10.q " after 
> changing the table to be 
> insert-only transactional. 
> The following queries returns couple of rows whereis no row results returns 
> for non-ACID table. 
> query: select ds, count(1) from srcpartbucket tablesample (bucket 2 out of 4 
> on key) where ds is not null group by ds ORDER BY ds ASC
> 2008-04-08  14
> 2008-04-09  14
> ..
> query: select ds, count(1) from srcpartbucket tablesample (bucket 1 out of 2 
> on key) where ds is not null group by ds ORDER BY ds ASC
> 2008-04-08  4
> 2008-04-09  4



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19898) Disable TransactionalValidationListener when the table is not in the Hive catalog

2018-06-24 Thread Dongjoon Hyun (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521756#comment-16521756
 ] 

Dongjoon Hyun commented on HIVE-19898:
--

Oh.. Thank you, [~jdere].
cc [~bikassaha], [~kbadani].

> Disable TransactionalValidationListener when the table is not in the Hive 
> catalog
> -
>
> Key: HIVE-19898
> URL: https://issues.apache.org/jira/browse/HIVE-19898
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore, Standalone Metastore, Transactions
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19898.1.patch
>
>
> The TransactionalValidationListener does validation of tables specified as 
> transactional tables, as well as enforcing create.as.acid. While this can be 
> useful to Hive, this may not be useful to other catalogs which do not support 
> transactional tables, and would not benefit from being automatically tagged 
> as a transactional table. This should be changed so the 
> TransactionalValidationListener does not run for non-hive catalogs.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19481) sample10.q returns wrong results

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521754#comment-16521754
 ] 

Hive QA commented on HIVE-19481:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  8m 
 3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
6s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
38s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  4m  
2s{color} | {color:blue} ql in master has 2282 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
58s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m  
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
38s{color} | {color:red} ql: The patch generated 1 new + 35 unchanged - 0 fixed 
= 36 total (was 35) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m 
16s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
13s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 51s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12066/dev-support/hive-personality.sh
 |
| git revision | master / be1955f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-12066/yetus/diff-checkstyle-ql.txt
 |
| modules | C: ql U: ql |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-12066/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> sample10.q returns wrong results 
> -
>
> Key: HIVE-19481
> URL: https://issues.apache.org/jira/browse/HIVE-19481
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Affects Versions: 3.0.0
>Reporter: Steve Yeom
>Assignee: Deepak Jaiswal
>Priority: Major
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19481.1.patch, HIVE-19481.2.patch
>
>
> Ran "mvn test -Dtest=TestMiniLlapLocalCliDriver -Dqfile=sample10.q " after 
> changing the table to be 
> insert-only transactional. 
> The following queries returns couple of rows whereis no row results returns 
> for non-ACID table. 
> query: select ds, count(1) from srcpartbucket tablesample (bucket 2 out of 4 
> on key) where ds is not null group by ds ORDER BY ds ASC
> 2008-04-08  14
> 2008-04-09  14
> ..
> query: select ds, count(1) from srcpartbucket tablesample (bucket 1 out of 2 
> on key) where ds is not null group by ds ORDER BY ds ASC
> 2008-04-08  4
> 2008-04-09  4



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19970) Replication dump has a NPE when table is empty

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521747#comment-16521747
 ] 

Hive QA commented on HIVE-19970:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928907/HIVE-19970.01.patch

{color:green}SUCCESS:{color} +1 due to 3 test(s) being added or modified.

{color:green}SUCCESS:{color} +1 due to 14604 tests passed

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/12065/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12065/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12065/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12928907 - PreCommit-HIVE-Build

> Replication dump has a NPE when table is empty
> --
>
> Key: HIVE-19970
> URL: https://issues.apache.org/jira/browse/HIVE-19970
> Project: Hive
>  Issue Type: Task
>  Components: repl
>Affects Versions: 3.1.0, 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19970.01.patch
>
>
> if table directory or partition directory is missing ..dump is throwing NPE 
> instead of file missing exception.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-4367) enhance TRUNCATE syntex to drop data of external table

2018-06-24 Thread Jason Dere (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-4367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Dere updated HIVE-4367:
-
Attachment: HIVE-4367.5.patch

> enhance  TRUNCATE syntex  to drop data of external table
> 
>
> Key: HIVE-4367
> URL: https://issues.apache.org/jira/browse/HIVE-4367
> Project: Hive
>  Issue Type: Improvement
>  Components: Query Processor
>Affects Versions: 0.11.0
>Reporter: caofangkun
>Assignee: caofangkun
>Priority: Minor
> Attachments: HIVE-4367-1.patch, HIVE-4367.2.patch.txt, 
> HIVE-4367.3.patch, HIVE-4367.4.patch, HIVE-4367.5.patch
>
>
> In my use case ,
> sometimes I have to remove data of external tables to free up storage space 
> of the cluster .
> So it's necessary for to enhance the syntax like 
> "TRUNCATE TABLE srcpart_truncate PARTITION (dt='201130412') FORCE;"
> to remove data from EXTERNAL table.
> And I add a configuration property to enable remove data to Trash 
> 
>   hive.truncate.skiptrash
>   false
>   
>  if true will remove data to trash, else false drop data immediately
>   
> 
> For example :
> hive (default)> TRUNCATE TABLE external1 partition (ds='11'); 
> FAILED: Error in semantic analysis: Cannot truncate non-managed table 
> external1
> hive (default)> TRUNCATE TABLE external1 partition (ds='11') FORCE;
> [2013-04-16 17:15:52]: Compile Start 
> [2013-04-16 17:15:52]: Compile End
> [2013-04-16 17:15:52]: OK
> [2013-04-16 17:15:52]: Time taken: 0.413 seconds
> hive (default)> set hive.truncate.skiptrash;
> hive.truncate.skiptrash=false
> hive (default)> set hive.truncate.skiptrash=true; 
> hive (default)> TRUNCATE TABLE external1 partition (ds='12') FORCE;
> [2013-04-16 17:16:21]: Compile Start 
> [2013-04-16 17:16:21]: Compile End
> [2013-04-16 17:16:21]: OK
> [2013-04-16 17:16:21]: Time taken: 0.143 seconds
> hive (default)> dfs -ls /user/test/.Trash/Current/; 
> Found 1 items
> drwxr-xr-x -test supergroup 0 2013-04-16 17:06 /user/test/.Trash/Current/ds=11



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-19753) Strict managed tables mode in Hive

2018-06-24 Thread Jason Dere (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Dere updated HIVE-19753:
--
Fix Version/s: 3.1.0

> Strict managed tables mode in Hive
> --
>
> Key: HIVE-19753
> URL: https://issues.apache.org/jira/browse/HIVE-19753
> Project: Hive
>  Issue Type: Bug
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19753.01-branch-3.patch, HIVE-19753.1.patch, 
> HIVE-19753.2.patch, HIVE-19753.3.patch, HIVE-19753.4.patch, 
> HIVE-19753.5.patch, HIVE-19753.6.patch, HIVE-19753.7.patch
>
>
> Create a mode in Hive which enforces that all managed tables are 
> transactional (both full or insert-only tables allowed). Non-transactional 
> tables, as well as non-native tables, must be created as external tables when 
> this mode is enabled.
> The idea would be that in strict managed tables mode all of the data written 
> to managed tables would have been done through Hive.
> The mode would be enabled using config setting hive.strict.managed.tables.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-19336) Disable SMB/Bucketmap join for external tables

2018-06-24 Thread Jason Dere (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Dere updated HIVE-19336:
--
Fix Version/s: 3.1.0

> Disable SMB/Bucketmap join for external tables
> --
>
> Key: HIVE-19336
> URL: https://issues.apache.org/jira/browse/HIVE-19336
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19336.01-branch-3.patch, HIVE-19336.1.patch, 
> HIVE-19336.2.patch, HIVE-19336.3.patch, HIVE-19336.4.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19970) Replication dump has a NPE when table is empty

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521738#comment-16521738
 ] 

Hive QA commented on HIVE-19970:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
21s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
38s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 
59s{color} | {color:blue} ql in master has 2282 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m  
8s{color} | {color:blue} standalone-metastore in master has 228 extant Findbugs 
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  2m 
18s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  2m 
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  2m 
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  2m 
21s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
23s{color} | {color:red} itests/hive-unit: The patch generated 2 new + 631 
unchanged - 0 fixed = 633 total (was 631) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
22s{color} | {color:red} standalone-metastore: The patch generated 1 new + 467 
unchanged - 0 fixed = 468 total (was 467) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  8m  
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  2m 
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
12s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 40m 23s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12065/dev-support/hive-personality.sh
 |
| git revision | master / be1955f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-12065/yetus/diff-checkstyle-itests_hive-unit.txt
 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-12065/yetus/diff-checkstyle-standalone-metastore.txt
 |
| modules | C: itests/hive-unit ql standalone-metastore U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-12065/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Replication dump has a NPE when table is empty
> --
>
> Key: HIVE-19970
> URL: https://issues.apache.org/jira/browse/HIVE-19970
> Project: Hive
>  Issue Type: Task
>  Components: repl
>Affects Versions: 3.1.0, 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
>  Labels: pull-request-available
> Fix For: 3.1.0, 4.0.0
>
> 

[jira] [Updated] (HIVE-19336) Disable SMB/Bucketmap join for external tables

2018-06-24 Thread Jason Dere (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Dere updated HIVE-19336:
--
Attachment: (was: HIVE-19336.01-branch-3.patch)

> Disable SMB/Bucketmap join for external tables
> --
>
> Key: HIVE-19336
> URL: https://issues.apache.org/jira/browse/HIVE-19336
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-19336.01-branch-3.patch, HIVE-19336.1.patch, 
> HIVE-19336.2.patch, HIVE-19336.3.patch, HIVE-19336.4.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-19336) Disable SMB/Bucketmap join for external tables

2018-06-24 Thread Jason Dere (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19336?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Dere updated HIVE-19336:
--
Attachment: HIVE-19336.01-branch-3.patch

> Disable SMB/Bucketmap join for external tables
> --
>
> Key: HIVE-19336
> URL: https://issues.apache.org/jira/browse/HIVE-19336
> Project: Hive
>  Issue Type: Sub-task
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-19336.01-branch-3.patch, HIVE-19336.1.patch, 
> HIVE-19336.2.patch, HIVE-19336.3.patch, HIVE-19336.4.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19972) Followup to HIVE-19928 : Fix the check for managed table

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521712#comment-16521712
 ] 

Hive QA commented on HIVE-19972:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928960/HIVE-19972.3.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:green}SUCCESS:{color} +1 due to 14603 tests passed

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/12064/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12064/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12064/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12928960 - PreCommit-HIVE-Build

> Followup to HIVE-19928 : Fix the check for managed table
> 
>
> Key: HIVE-19972
> URL: https://issues.apache.org/jira/browse/HIVE-19972
> Project: Hive
>  Issue Type: Bug
>Reporter: Deepak Jaiswal
>Assignee: Deepak Jaiswal
>Priority: Major
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19972.1.patch, HIVE-19972.2.patch, 
> HIVE-19972.3.patch
>
>
> The check for managed table should use ENUM comparison rather than string 
> comparison.
> The check in the patch will always return false, thus maintaining existing 
> behavior.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19801) JDBC: Add some missing classes to jdbc standalone jar and remove hbase classes

2018-06-24 Thread Thejas M Nair (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521711#comment-16521711
 ] 

Thejas M Nair commented on HIVE-19801:
--

Thanks [~prasanth_j]!


> JDBC: Add some missing classes to jdbc standalone jar and remove hbase classes
> --
>
> Key: HIVE-19801
> URL: https://issues.apache.org/jira/browse/HIVE-19801
> Project: Hive
>  Issue Type: Bug
>  Components: JDBC
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Vaibhav Gumashta
>Assignee: Thejas M Nair
>Priority: Major
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19801-branch-3.patch, HIVE-19801.1-branch-3.patch, 
> HIVE-19801.1-branch-3.patch, HIVE-19801.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-19965) Make HiveEndPoint use IMetaStoreClient.add_partition

2018-06-24 Thread Eugene Koifman (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eugene Koifman updated HIVE-19965:
--
Status: Open  (was: Patch Available)

> Make HiveEndPoint use IMetaStoreClient.add_partition
> 
>
> Key: HIVE-19965
> URL: https://issues.apache.org/jira/browse/HIVE-19965
> Project: Hive
>  Issue Type: Bug
>  Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Blocker
> Attachments: HIVE-19965.01-branch-3.patch, HIVE-19965.01.patch, 
> HIVE-19965.02-branch-3.patch
>
>
> it currently uses "alter table add partition if exists..."
> which since HIVE-18814 requires X lock on the table which blocks other 
> streaming writers from making progress.  
> HIVE-19961 to investigate if X lock can be relaxed, etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19972) Followup to HIVE-19928 : Fix the check for managed table

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521710#comment-16521710
 ] 

Hive QA commented on HIVE-19972:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
43s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 
59s{color} | {color:blue} ql in master has 2282 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m  
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m  
3s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m 
11s{color} | {color:green} ql generated 0 new + 2280 unchanged - 2 fixed = 2280 
total (was 2282) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
56s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
13s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 25s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12064/dev-support/hive-personality.sh
 |
| git revision | master / be1955f |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-12064/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Followup to HIVE-19928 : Fix the check for managed table
> 
>
> Key: HIVE-19972
> URL: https://issues.apache.org/jira/browse/HIVE-19972
> Project: Hive
>  Issue Type: Bug
>Reporter: Deepak Jaiswal
>Assignee: Deepak Jaiswal
>Priority: Major
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19972.1.patch, HIVE-19972.2.patch, 
> HIVE-19972.3.patch
>
>
> The check for managed table should use ENUM comparison rather than string 
> comparison.
> The check in the patch will always return false, thus maintaining existing 
> behavior.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-19711) Refactor Hive Schema Tool

2018-06-24 Thread Miklos Gergely (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-19711:
--
Status: Open  (was: Patch Available)

> Refactor Hive Schema Tool
> -
>
> Key: HIVE-19711
> URL: https://issues.apache.org/jira/browse/HIVE-19711
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Affects Versions: 3.0.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-19711.01.patch, HIVE-19711.02.patch, 
> HIVE-19711.03.patch, HIVE-19711.04.patch, HIVE-19711.05.patch, 
> HIVE-19711.06.patch, HIVE-19711.07.patch
>
>
> HiveSchemaTool is an 1500 lines long class trying to do everything It shold 
> be cut into multiple classes doing smaller components.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-19711) Refactor Hive Schema Tool

2018-06-24 Thread Miklos Gergely (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-19711:
--
Attachment: HIVE-19711.07.patch

> Refactor Hive Schema Tool
> -
>
> Key: HIVE-19711
> URL: https://issues.apache.org/jira/browse/HIVE-19711
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Affects Versions: 3.0.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-19711.01.patch, HIVE-19711.02.patch, 
> HIVE-19711.03.patch, HIVE-19711.04.patch, HIVE-19711.05.patch, 
> HIVE-19711.06.patch, HIVE-19711.07.patch
>
>
> HiveSchemaTool is an 1500 lines long class trying to do everything It shold 
> be cut into multiple classes doing smaller components.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-19711) Refactor Hive Schema Tool

2018-06-24 Thread Miklos Gergely (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Miklos Gergely updated HIVE-19711:
--
Status: Patch Available  (was: Open)

> Refactor Hive Schema Tool
> -
>
> Key: HIVE-19711
> URL: https://issues.apache.org/jira/browse/HIVE-19711
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Affects Versions: 3.0.0
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Minor
> Fix For: 4.0.0
>
> Attachments: HIVE-19711.01.patch, HIVE-19711.02.patch, 
> HIVE-19711.03.patch, HIVE-19711.04.patch, HIVE-19711.05.patch, 
> HIVE-19711.06.patch, HIVE-19711.07.patch
>
>
> HiveSchemaTool is an 1500 lines long class trying to do everything It shold 
> be cut into multiple classes doing smaller components.
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19941) Row based Filters added via Hive Ranger policies are not pushed to druid

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521708#comment-16521708
 ] 

Hive QA commented on HIVE-19941:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928904/HIVE-19941.2.patch

{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 14603 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.client.TestRuntimeStats.testCleanup[Remote] 
(batchId=213)
org.apache.hadoop.hive.ql.TestTxnCommandsWithSplitUpdateAndVectorization.testMergeOnTezEdges
 (batchId=291)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/12063/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12063/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12063/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12928904 - PreCommit-HIVE-Build

> Row based Filters added via Hive Ranger policies are not pushed to druid
> 
>
> Key: HIVE-19941
> URL: https://issues.apache.org/jira/browse/HIVE-19941
> Project: Hive
>  Issue Type: Bug
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Attachments: HIVE-19941.1.patch, HIVE-19941.2.patch, HIVE-19941.patch
>
>
> Issue is that when applying table mask we add virtual columns, however 
> non-native tables do not have virtual columns, we need to skip adding virtual 
> columns when generating masking query. 
> Stack Trace - 
> {code} 
> org.apache.hadoop.hive.ql.parse.SemanticException: Line 1:79 Invalid table 
> alias or column reference 'BLOCK__OFFSET__INSIDE__FILE'
> : (possible column names are: __time, yearmonth, year, month, dayofmonth, 
> dayofweek, weekofyear, hour, minute, second, payment_typ
> e, fare_amount, surcharge, mta_tax, tip_amount, tolls_amount, total_amount, 
> trip_time)
> at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genAllExprNodeDesc(SemanticAnalyzer.java:11830)
>  ~[hive-exec-2.1.0.2.6.
> 4.0-91.jar:2.1.0.2.6.4.0-91]
> at 
> org.apache.hadoop.hive.ql.parse.SemanticAnalyzer.genExprNodeDesc(SemanticAnalyzer.java:11778)
>  ~[hive-exec-2.1.0.2.6.4.0
> -91.jar:2.1.0.2.6.4.0-91]
> at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genSelectLogicalPlan(CalcitePlanner.java:3780)
>  ~[hi
> ve-exec-2.1.0.2.6.4.0-91.jar:2.1.0.2.6.4.0-91]
> at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:4117)
>  ~[hive-exe
> c-2.1.0.2.6.4.0-91.jar:2.1.0.2.6.4.0-91]
> at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:4016)
>  ~[hive-exe
> c-2.1.0.2.6.4.0-91.jar:2.1.0.2.6.4.0-91]
> at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.genLogicalPlan(CalcitePlanner.java:4060)
>  ~[hive-exe
> c-2.1.0.2.6.4.0-91.jar:2.1.0.2.6.4.0-91]
> at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1340)
>  ~[hive-exec-2.1.0.2
> .6.4.0-91.jar:2.1.0.2.6.4.0-91]
> at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner$CalcitePlannerAction.apply(CalcitePlanner.java:1277)
>  ~[hive-exec-2.1.0.2
> .6.4.0-91.jar:2.1.0.2.6.4.0-91]
> at org.apache.calcite.tools.Frameworks$1.apply(Frameworks.java:113) 
> ~[calcite-core-1.10.0.2.6.4.0-91.jar:1.10.0.2.6.4.0-91
> ]
> at 
> org.apache.calcite.prepare.CalcitePrepareImpl.perform(CalcitePrepareImpl.java:997)
>  ~[calcite-core-1.10.0.2.6.4.0-91.jar
> :1.10.0.2.6.4.0-91]
> at 
> org.apache.calcite.tools.Frameworks.withPrepare(Frameworks.java:149) 
> ~[calcite-core-1.10.0.2.6.4.0-91.jar:1.10.0.2.6.4.
> 0-91]
> at 
> org.apache.calcite.tools.Frameworks.withPlanner(Frameworks.java:106) 
> ~[calcite-core-1.10.0.2.6.4.0-91.jar:1.10.0.2.6.4.
> 0-91]
> at 
> org.apache.hadoop.hive.ql.parse.CalcitePlanner.logicalPlan(CalcitePlanner.java:1082)
>  ~[hive-exec-2.1.0.2.6.4.0-91.jar:2
> .1.0.2.6.4.0-91]
> {code} 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19951) Vectorization: Need to disable encoded LLAP I/O for ORC when there is data type conversion (Schema Evolution)

2018-06-24 Thread Matt McCline (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521701#comment-16521701
 ] 

Matt McCline commented on HIVE-19951:
-

Wait a minute -- orc_ppd_schema_evol_3a is configured for MiniLlap not 
MiniLlapLocal...

> Vectorization: Need to disable encoded LLAP I/O for ORC when there is data 
> type conversion  (Schema Evolution)
> --
>
> Key: HIVE-19951
> URL: https://issues.apache.org/jira/browse/HIVE-19951
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-19951.01.patch, HIVE-19951.02.patch, 
> HIVE-19951.03.patch
>
>
> Currently, reading encoded ORC data does not support data type conversion.  
> So, encoded reading and cache populating needs to be disabled.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19941) Row based Filters added via Hive Ranger policies are not pushed to druid

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521699#comment-16521699
 ] 

Hive QA commented on HIVE-19941:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
33s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
12s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
55s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
45s{color} | {color:blue} itests/util in master has 52 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 
59s{color} | {color:blue} ql in master has 2282 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
8s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
31s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
44s{color} | {color:red} ql: The patch generated 1 new + 482 unchanged - 0 
fixed = 483 total (was 482) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m 
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
12s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 28m  5s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12063/dev-support/hive-personality.sh
 |
| git revision | master / ba210c9 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-12063/yetus/diff-checkstyle-ql.txt
 |
| modules | C: itests itests/util ql U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-12063/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Row based Filters added via Hive Ranger policies are not pushed to druid
> 
>
> Key: HIVE-19941
> URL: https://issues.apache.org/jira/browse/HIVE-19941
> Project: Hive
>  Issue Type: Bug
>Reporter: Nishant Bangarwa
>Assignee: Nishant Bangarwa
>Priority: Major
> Attachments: HIVE-19941.1.patch, HIVE-19941.2.patch, HIVE-19941.patch
>
>
> Issue is that when applying table mask we add virtual columns, however 
> non-native tables do not have virtual columns, we need to skip adding virtual 
> columns when generating masking query. 
> Stack Trace - 
> {code} 
> org.apache.hadoop.hive.ql.parse.SemanticException: Line 1:79 Invalid table 
> alias or column reference 'BLOCK__OFFSET__INSIDE__FILE'
> : (possible column names are: __time, yearmonth, year, month, dayofmonth, 
> dayofweek, weekofyear, hour, minute, second, payment_typ
> e, fare_amount, surcharge, mta_tax, tip_amount, tolls_amount, total_amount, 
> 

[jira] [Updated] (HIVE-19972) Followup to HIVE-19928 : Fix the check for managed table

2018-06-24 Thread Deepak Jaiswal (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Deepak Jaiswal updated HIVE-19972:
--
Attachment: HIVE-19972.3.patch

> Followup to HIVE-19928 : Fix the check for managed table
> 
>
> Key: HIVE-19972
> URL: https://issues.apache.org/jira/browse/HIVE-19972
> Project: Hive
>  Issue Type: Bug
>Reporter: Deepak Jaiswal
>Assignee: Deepak Jaiswal
>Priority: Major
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19972.1.patch, HIVE-19972.2.patch, 
> HIVE-19972.3.patch
>
>
> The check for managed table should use ENUM comparison rather than string 
> comparison.
> The check in the patch will always return false, thus maintaining existing 
> behavior.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19972) Followup to HIVE-19928 : Fix the check for managed table

2018-06-24 Thread Deepak Jaiswal (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521697#comment-16521697
 ] 

Deepak Jaiswal commented on HIVE-19972:
---

Flaky tests strike again.

> Followup to HIVE-19928 : Fix the check for managed table
> 
>
> Key: HIVE-19972
> URL: https://issues.apache.org/jira/browse/HIVE-19972
> Project: Hive
>  Issue Type: Bug
>Reporter: Deepak Jaiswal
>Assignee: Deepak Jaiswal
>Priority: Major
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19972.1.patch, HIVE-19972.2.patch, 
> HIVE-19972.3.patch
>
>
> The check for managed table should use ENUM comparison rather than string 
> comparison.
> The check in the patch will always return false, thus maintaining existing 
> behavior.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19951) Vectorization: Need to disable encoded LLAP I/O for ORC when there is data type conversion (Schema Evolution)

2018-06-24 Thread Matt McCline (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521692#comment-16521692
 ] 

Matt McCline commented on HIVE-19951:
-

I can't produce a stable orc_ppd_schema_evol_3a  Q file result on my laptop 
that passes Hive QA.

> Vectorization: Need to disable encoded LLAP I/O for ORC when there is data 
> type conversion  (Schema Evolution)
> --
>
> Key: HIVE-19951
> URL: https://issues.apache.org/jira/browse/HIVE-19951
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-19951.01.patch, HIVE-19951.02.patch, 
> HIVE-19951.03.patch
>
>
> Currently, reading encoded ORC data does not support data type conversion.  
> So, encoded reading and cache populating needs to be disabled.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19951) Vectorization: Need to disable encoded LLAP I/O for ORC when there is data type conversion (Schema Evolution)

2018-06-24 Thread Prasanth Jayachandran (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521691#comment-16521691
 ] 

Prasanth Jayachandran commented on HIVE-19951:
--

orc_ppd_schema_evol_3a is a stable test. The test diff here is expected since 
llap io is completely disable for schema evolution. 

> Vectorization: Need to disable encoded LLAP I/O for ORC when there is data 
> type conversion  (Schema Evolution)
> --
>
> Key: HIVE-19951
> URL: https://issues.apache.org/jira/browse/HIVE-19951
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-19951.01.patch, HIVE-19951.02.patch, 
> HIVE-19951.03.patch
>
>
> Currently, reading encoded ORC data does not support data type conversion.  
> So, encoded reading and cache populating needs to be disabled.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-19973) Enable materialized view rewriting by default

2018-06-24 Thread Jesus Camacho Rodriguez (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jesus Camacho Rodriguez updated HIVE-19973:
---
   Resolution: Fixed
Fix Version/s: 4.0.0
   3.1.0
   Status: Resolved  (was: Patch Available)

Pushed to master, branch-3.

> Enable materialized view rewriting by default
> -
>
> Key: HIVE-19973
> URL: https://issues.apache.org/jira/browse/HIVE-19973
> Project: Hive
>  Issue Type: Bug
>  Components: Materialized views
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19973.patch
>
>
> Change property value for {{hive.materializedview.rewriting}} to {{true}}. 
> For tests, it is already {{true}} by default.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19951) Vectorization: Need to disable encoded LLAP I/O for ORC when there is data type conversion (Schema Evolution)

2018-06-24 Thread Matt McCline (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521685#comment-16521685
 ] 

Matt McCline commented on HIVE-19951:
-

[~prasanth_j] Is the orc_ppd_schema_evol_3a test failure related to the 
counters and is it a stable test?

> Vectorization: Need to disable encoded LLAP I/O for ORC when there is data 
> type conversion  (Schema Evolution)
> --
>
> Key: HIVE-19951
> URL: https://issues.apache.org/jira/browse/HIVE-19951
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-19951.01.patch, HIVE-19951.02.patch, 
> HIVE-19951.03.patch
>
>
> Currently, reading encoded ORC data does not support data type conversion.  
> So, encoded reading and cache populating needs to be disabled.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-19956) Include yarn registry classes to jdbc standalone jar

2018-06-24 Thread Prasanth Jayachandran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasanth Jayachandran updated HIVE-19956:
-
   Resolution: Fixed
Fix Version/s: 3.1.0
   Status: Resolved  (was: Patch Available)

Committed to branch-3 as well.

> Include yarn registry classes to jdbc standalone jar
> 
>
> Key: HIVE-19956
> URL: https://issues.apache.org/jira/browse/HIVE-19956
> Project: Hive
>  Issue Type: Bug
>  Components: JDBC
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19956.1.patch, HIVE-19956.2.patch, 
> HIVE-19956.3.patch
>
>
> HS2 Active/Passive HA requires some yarn registry classes. Include it in JDBC 
> standalone jar. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-19801) JDBC: Add some missing classes to jdbc standalone jar and remove hbase classes

2018-06-24 Thread Prasanth Jayachandran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasanth Jayachandran updated HIVE-19801:
-
   Resolution: Fixed
Fix Version/s: 3.1.0
   Status: Resolved  (was: Patch Available)

Committed to branch-3 as well. 

> JDBC: Add some missing classes to jdbc standalone jar and remove hbase classes
> --
>
> Key: HIVE-19801
> URL: https://issues.apache.org/jira/browse/HIVE-19801
> Project: Hive
>  Issue Type: Bug
>  Components: JDBC
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Vaibhav Gumashta
>Assignee: Thejas M Nair
>Priority: Major
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19801-branch-3.patch, HIVE-19801.1-branch-3.patch, 
> HIVE-19801.1-branch-3.patch, HIVE-19801.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19801) JDBC: Add some missing classes to jdbc standalone jar and remove hbase classes

2018-06-24 Thread Prasanth Jayachandran (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521682#comment-16521682
 ] 

Prasanth Jayachandran commented on HIVE-19801:
--

Test failures look unrelated. HIVE-19965 also has same set of failures. 

> JDBC: Add some missing classes to jdbc standalone jar and remove hbase classes
> --
>
> Key: HIVE-19801
> URL: https://issues.apache.org/jira/browse/HIVE-19801
> Project: Hive
>  Issue Type: Bug
>  Components: JDBC
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Vaibhav Gumashta
>Assignee: Thejas M Nair
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-19801-branch-3.patch, HIVE-19801.1-branch-3.patch, 
> HIVE-19801.1-branch-3.patch, HIVE-19801.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-18729) Druid Time column type

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-18729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521681#comment-16521681
 ] 

Hive QA commented on HIVE-18729:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/1292/HIVE-18729.branch-3.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/12062/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12062/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12062/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL 
https://issues.apache.org/jira/secure/attachment/1292/HIVE-18729.branch-3.patch
 was found in seen patch url's cache and a test was probably run already on it. 
Aborting...
{noformat}

This message is automatically generated.

ATTACHMENT ID: 1292 - PreCommit-HIVE-Build

> Druid Time column type
> --
>
> Key: HIVE-18729
> URL: https://issues.apache.org/jira/browse/HIVE-18729
> Project: Hive
>  Issue Type: Task
>  Components: Druid integration
>Reporter: slim bouguerra
>Assignee: Jesus Camacho Rodriguez
>Priority: Blocker
> Attachments: HIVE-18729.branch-3.patch, HIVE-18729.patch
>
>
> I have talked Offline with [~jcamachorodriguez] about this and agreed that 
> the best way to go is to support both cases where Druid time column can be 
> Timestamp or Timestamp with local time zone. 
> In fact, for the Hive-Druid internal table, this makes perfect sense since we 
> have Hive metadata about the time column during the CTAS statement then we 
> can handle both cases as we do for another type of storage eg ORC.
> For the Druid external tables, we can have a default type and allow the user 
> to override that via table properties. 
> CC [~ashutoshc] and [~nishantbangarwa]. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19926) Remove deprecated hcatalog streaming

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521680#comment-16521680
 ] 

Hive QA commented on HIVE-19926:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928884/HIVE-19926.3.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 16 failed/errored test(s), 14447 tests 
executed
*Failed tests:*
{noformat}
TestAcidTableSerializer - did not produce a TEST-*.xml file (likely timed out) 
(batchId=209)
TestBucketIdResolverImpl - did not produce a TEST-*.xml file (likely timed out) 
(batchId=209)
TestDelimitedInputWriter - did not produce a TEST-*.xml file (likely timed out) 
(batchId=209)
TestGroupingValidator - did not produce a TEST-*.xml file (likely timed out) 
(batchId=209)
TestHeartbeatTimerTask - did not produce a TEST-*.xml file (likely timed out) 
(batchId=209)
TestLock - did not produce a TEST-*.xml file (likely timed out) (batchId=209)
TestMetaStorePartitionHelper - did not produce a TEST-*.xml file (likely timed 
out) (batchId=209)
TestMutations - did not produce a TEST-*.xml file (likely timed out) 
(batchId=209)
TestMutatorClient - did not produce a TEST-*.xml file (likely timed out) 
(batchId=209)
TestMutatorCoordinator - did not produce a TEST-*.xml file (likely timed out) 
(batchId=209)
TestMutatorImpl - did not produce a TEST-*.xml file (likely timed out) 
(batchId=209)
TestRecordInspectorImpl - did not produce a TEST-*.xml file (likely timed out) 
(batchId=209)
TestSequenceValidator - did not produce a TEST-*.xml file (likely timed out) 
(batchId=209)
TestStreaming - did not produce a TEST-*.xml file (likely timed out) 
(batchId=209)
TestTransaction - did not produce a TEST-*.xml file (likely timed out) 
(batchId=209)
TestWarehousePartitionHelper - did not produce a TEST-*.xml file (likely timed 
out) (batchId=209)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/12061/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12061/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12061/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 16 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12928884 - PreCommit-HIVE-Build

> Remove deprecated hcatalog streaming
> 
>
> Key: HIVE-19926
> URL: https://issues.apache.org/jira/browse/HIVE-19926
> Project: Hive
>  Issue Type: Improvement
>  Components: Streaming
>Affects Versions: 4.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19926.1.patch, HIVE-19926.2.patch, 
> HIVE-19926.3.patch
>
>
> hcatalog streaming is deprecated in 3.0.0. We should remove it in 4.0.0.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-19027) Make materializations invalidation cache work with multiple active remote metastores

2018-06-24 Thread Jesus Camacho Rodriguez (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jesus Camacho Rodriguez updated HIVE-19027:
---
Attachment: HIVE-19027.02.patch

> Make materializations invalidation cache work with multiple active remote 
> metastores
> 
>
> Key: HIVE-19027
> URL: https://issues.apache.org/jira/browse/HIVE-19027
> Project: Hive
>  Issue Type: Improvement
>  Components: Materialized views
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Attachments: HIVE-19027.01.patch, HIVE-19027.02.patch
>
>
> The main points:
>  - Only MVs stored in transactional tables can have a time window value of 0. 
> Those are the only MVs that can be guaranteed to not be outdated when a query 
> is executed, if we use custom storage handlers to store the materialized 
> view, we cannot make any promises.
>  - For MVs that +cannot be outdated+, we do not check the metastore. Instead, 
> comparison is based on valid write id lists.
>  - For MVs that +can be outdated+, we still rely on the invalidation cache.
>  ** The window for valid outdated MVs can be specified in intervals of 1 
> minute (less than that, it is difficult to have any guarantees about whether 
> the MV is actually outdated by less than a minute or not).
>  ** The async loading is done every interval / 2 (or probably better, we can 
> make it configurable).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19926) Remove deprecated hcatalog streaming

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521608#comment-16521608
 ] 

Hive QA commented on HIVE-19926:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
34s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 
22s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  1m 
 9s{color} | {color:green} master passed {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  1m 
38s{color} | {color:red} branch/hcatalog no findbugs output file 
(hcatalog/target/findbugsXml.xml) {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
27s{color} | {color:blue} hcatalog/streaming in master has 11 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
37s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs 
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m 
17s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
18s{color} | {color:green} hcatalog: The patch generated 0 new + 0 unchanged - 
617 fixed = 0 total (was 617) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
17s{color} | {color:red} itests/hive-unit: The patch generated 28 new + 134 
unchanged - 36 fixed = 162 total (was 170) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
 9s{color} | {color:green} The patch packaging passed checkstyle {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
4s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  1m 
27s{color} | {color:red} patch/hcatalog no findbugs output file 
(hcatalog/target/findbugsXml.xml) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
29s{color} | {color:green} hcatalog generated 0 new + 3 unchanged - 50 fixed = 
3 total (was 53) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
21s{color} | {color:green} hive-unit in the patch passed. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
10s{color} | {color:green} packaging in the patch passed. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
12s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 25s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  xml  compile  findbugs  
checkstyle  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12061/dev-support/hive-personality.sh
 |
| git revision | master / 97a6e16 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| findbugs | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-12061/yetus/branch-findbugs-hcatalog.txt
 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-12061/yetus/diff-checkstyle-itests_hive-unit.txt
 |
| findbugs | 

[jira] [Commented] (HIVE-19801) JDBC: Add some missing classes to jdbc standalone jar and remove hbase classes

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521597#comment-16521597
 ] 

Hive QA commented on HIVE-19801:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928885/HIVE-19801.1-branch-3.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 14366 tests 
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=256)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=256)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=256)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed 
out) (batchId=256)
TestTezPerfCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=256)
org.apache.hadoop.hive.ql.TestWarehouseExternalDir.testManagedPaths 
(batchId=233)
org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.testImpersonation 
(batchId=242)
org.apache.hive.spark.client.rpc.TestRpc.testServerPort (batchId=308)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/12060/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12060/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12060/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12928885 - PreCommit-HIVE-Build

> JDBC: Add some missing classes to jdbc standalone jar and remove hbase classes
> --
>
> Key: HIVE-19801
> URL: https://issues.apache.org/jira/browse/HIVE-19801
> Project: Hive
>  Issue Type: Bug
>  Components: JDBC
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Vaibhav Gumashta
>Assignee: Thejas M Nair
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-19801-branch-3.patch, HIVE-19801.1-branch-3.patch, 
> HIVE-19801.1-branch-3.patch, HIVE-19801.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19964) Apply resource plan fails if trigger expression has quotes

2018-06-24 Thread Prasanth Jayachandran (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521581#comment-16521581
 ] 

Prasanth Jayachandran commented on HIVE-19964:
--

Patch from different Jira was uploaded mistakenly. Uploading correct patch. 

> Apply resource plan fails if trigger expression has quotes
> --
>
> Key: HIVE-19964
> URL: https://issues.apache.org/jira/browse/HIVE-19964
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Aswathy Chellammal Sreekumar
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19964.1.patch, HIVE-19964.2.patch
>
>
> {code:java}
> 0: jdbc:hive2://localhost:1> CREATE TRIGGER global.big_hdfs_read WHEN 
> HDFS_BYTES_READ > '300kb' DO KILL;
> INFO : Compiling 
> command(queryId=pjayachandran_20180621131017_72b1441b-d790-4db7-83ca-479735843890):
>  CREATE TRIGGER global.big_hdfs_read WHEN HDFS_BYTES_READ > '300kb' DO KILL
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling 
> command(queryId=pjayachandran_20180621131017_72b1441b-d790-4db7-83ca-479735843890);
>  Time taken: 0.015 seconds
> INFO : Executing 
> command(queryId=pjayachandran_20180621131017_72b1441b-d790-4db7-83ca-479735843890):
>  CREATE TRIGGER global.big_hdfs_read WHEN HDFS_BYTES_READ > '300kb' DO KILL
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing 
> command(queryId=pjayachandran_20180621131017_72b1441b-d790-4db7-83ca-479735843890);
>  Time taken: 0.025 seconds
> INFO : OK
> No rows affected (0.054 seconds)
> 0: jdbc:hive2://localhost:1> ALTER TRIGGER global.big_hdfs_read ADD TO 
> UNMANAGED;
> INFO : Compiling 
> command(queryId=pjayachandran_20180621131031_dd489324-db23-412f-9409-32ba697a10e5):
>  ALTER TRIGGER global.big_hdfs_read ADD TO UNMANAGED
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling 
> command(queryId=pjayachandran_20180621131031_dd489324-db23-412f-9409-32ba697a10e5);
>  Time taken: 0.014 seconds
> INFO : Executing 
> command(queryId=pjayachandran_20180621131031_dd489324-db23-412f-9409-32ba697a10e5):
>  ALTER TRIGGER global.big_hdfs_read ADD TO UNMANAGED
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing 
> command(queryId=pjayachandran_20180621131031_dd489324-db23-412f-9409-32ba697a10e5);
>  Time taken: 0.029 seconds
> INFO : OK
> No rows affected (0.054 seconds)
> 0: jdbc:hive2://localhost:1> ALTER RESOURCE PLAN global ENABLE;
> INFO : Compiling 
> command(queryId=pjayachandran_20180621131036_26a5f4f3-91e3-4bec-ab42-800adb90104e):
>  ALTER RESOURCE PLAN global ENABLE
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling 
> command(queryId=pjayachandran_20180621131036_26a5f4f3-91e3-4bec-ab42-800adb90104e);
>  Time taken: 0.012 seconds
> INFO : Executing 
> command(queryId=pjayachandran_20180621131036_26a5f4f3-91e3-4bec-ab42-800adb90104e):
>  ALTER RESOURCE PLAN global ENABLE
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing 
> command(queryId=pjayachandran_20180621131036_26a5f4f3-91e3-4bec-ab42-800adb90104e);
>  Time taken: 0.021 seconds
> INFO : OK
> No rows affected (0.045 seconds)
> 0: jdbc:hive2://localhost:1> ALTER RESOURCE PLAN global ACTIVATE;
> INFO : Compiling 
> command(queryId=pjayachandran_20180621131037_551b2af0-321b-4638-8ac0-76771a159f4b):
>  ALTER RESOURCE PLAN global ACTIVATE
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling 
> command(queryId=pjayachandran_20180621131037_551b2af0-321b-4638-8ac0-76771a159f4b);
>  Time taken: 0.017 seconds
> INFO : Executing 
> command(queryId=pjayachandran_20180621131037_551b2af0-321b-4638-8ac0-76771a159f4b):
>  ALTER RESOURCE PLAN global ACTIVATE
> INFO : Starting task [Stage-0:DDL] in serial mode
> ERROR : FAILED: Execution Error, return code 1 from 
> org.apache.hadoop.hive.ql.exec.DDLTask. Invalid expression: HDFS_BYTES_READ > 
> 300kb
> INFO : Completed executing 
> command(queryId=pjayachandran_20180621131037_551b2af0-321b-4638-8ac0-76771a159f4b);
>  Time taken: 0.037 seconds
> Error: Error while processing statement: FAILED: Execution Error, return code 
> 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Invalid expression: 
> HDFS_BYTES_READ > 300kb (state=08S01,code=1){code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-19956) Include yarn registry classes to jdbc standalone jar

2018-06-24 Thread Prasanth Jayachandran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasanth Jayachandran updated HIVE-19956:
-
Fix Version/s: 4.0.0

> Include yarn registry classes to jdbc standalone jar
> 
>
> Key: HIVE-19956
> URL: https://issues.apache.org/jira/browse/HIVE-19956
> Project: Hive
>  Issue Type: Bug
>  Components: JDBC
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-19956.1.patch, HIVE-19956.2.patch, 
> HIVE-19956.3.patch
>
>
> HS2 Active/Passive HA requires some yarn registry classes. Include it in JDBC 
> standalone jar. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19956) Include yarn registry classes to jdbc standalone jar

2018-06-24 Thread Prasanth Jayachandran (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521580#comment-16521580
 ] 

Prasanth Jayachandran commented on HIVE-19956:
--

committed to master.

> Include yarn registry classes to jdbc standalone jar
> 
>
> Key: HIVE-19956
> URL: https://issues.apache.org/jira/browse/HIVE-19956
> Project: Hive
>  Issue Type: Bug
>  Components: JDBC
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-19956.1.patch, HIVE-19956.2.patch, 
> HIVE-19956.3.patch
>
>
> HS2 Active/Passive HA requires some yarn registry classes. Include it in JDBC 
> standalone jar. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19956) Include yarn registry classes to jdbc standalone jar

2018-06-24 Thread Prasanth Jayachandran (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521579#comment-16521579
 ] 

Prasanth Jayachandran commented on HIVE-19956:
--

branch-3 commit is pending HIVE-19801 because of conflicts. 

> Include yarn registry classes to jdbc standalone jar
> 
>
> Key: HIVE-19956
> URL: https://issues.apache.org/jira/browse/HIVE-19956
> Project: Hive
>  Issue Type: Bug
>  Components: JDBC
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-19956.1.patch, HIVE-19956.2.patch, 
> HIVE-19956.3.patch
>
>
> HS2 Active/Passive HA requires some yarn registry classes. Include it in JDBC 
> standalone jar. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19801) JDBC: Add some missing classes to jdbc standalone jar and remove hbase classes

2018-06-24 Thread Prasanth Jayachandran (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521577#comment-16521577
 ] 

Prasanth Jayachandran commented on HIVE-19801:
--

The patch was not picked. Trying with different name.

> JDBC: Add some missing classes to jdbc standalone jar and remove hbase classes
> --
>
> Key: HIVE-19801
> URL: https://issues.apache.org/jira/browse/HIVE-19801
> Project: Hive
>  Issue Type: Bug
>  Components: JDBC
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Vaibhav Gumashta
>Assignee: Thejas M Nair
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-19801-branch-3.patch, HIVE-19801.1-branch-3.patch, 
> HIVE-19801.1-branch-3.patch, HIVE-19801.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Assigned] (HIVE-19801) JDBC: Add some missing classes to jdbc standalone jar and remove hbase classes

2018-06-24 Thread Prasanth Jayachandran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasanth Jayachandran reassigned HIVE-19801:


Assignee: Thejas M Nair  (was: Prasanth Jayachandran)

> JDBC: Add some missing classes to jdbc standalone jar and remove hbase classes
> --
>
> Key: HIVE-19801
> URL: https://issues.apache.org/jira/browse/HIVE-19801
> Project: Hive
>  Issue Type: Bug
>  Components: JDBC
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Vaibhav Gumashta
>Assignee: Thejas M Nair
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-19801-branch-3.patch, HIVE-19801.1-branch-3.patch, 
> HIVE-19801.1-branch-3.patch, HIVE-19801.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-19801) JDBC: Add some missing classes to jdbc standalone jar and remove hbase classes

2018-06-24 Thread Prasanth Jayachandran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasanth Jayachandran updated HIVE-19801:
-
Attachment: HIVE-19801-branch-3.patch

> JDBC: Add some missing classes to jdbc standalone jar and remove hbase classes
> --
>
> Key: HIVE-19801
> URL: https://issues.apache.org/jira/browse/HIVE-19801
> Project: Hive
>  Issue Type: Bug
>  Components: JDBC
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Vaibhav Gumashta
>Assignee: Prasanth Jayachandran
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-19801-branch-3.patch, HIVE-19801.1-branch-3.patch, 
> HIVE-19801.1-branch-3.patch, HIVE-19801.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Assigned] (HIVE-19801) JDBC: Add some missing classes to jdbc standalone jar and remove hbase classes

2018-06-24 Thread Prasanth Jayachandran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasanth Jayachandran reassigned HIVE-19801:


Assignee: Prasanth Jayachandran  (was: Thejas M Nair)

> JDBC: Add some missing classes to jdbc standalone jar and remove hbase classes
> --
>
> Key: HIVE-19801
> URL: https://issues.apache.org/jira/browse/HIVE-19801
> Project: Hive
>  Issue Type: Bug
>  Components: JDBC
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Vaibhav Gumashta
>Assignee: Prasanth Jayachandran
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-19801.1-branch-3.patch, 
> HIVE-19801.1-branch-3.patch, HIVE-19801.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-19964) Apply resource plan fails if trigger expression has quotes

2018-06-24 Thread Prasanth Jayachandran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasanth Jayachandran updated HIVE-19964:
-
Attachment: HIVE-19964.2.patch

> Apply resource plan fails if trigger expression has quotes
> --
>
> Key: HIVE-19964
> URL: https://issues.apache.org/jira/browse/HIVE-19964
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Aswathy Chellammal Sreekumar
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19964.1.patch, HIVE-19964.2.patch
>
>
> {code:java}
> 0: jdbc:hive2://localhost:1> CREATE TRIGGER global.big_hdfs_read WHEN 
> HDFS_BYTES_READ > '300kb' DO KILL;
> INFO : Compiling 
> command(queryId=pjayachandran_20180621131017_72b1441b-d790-4db7-83ca-479735843890):
>  CREATE TRIGGER global.big_hdfs_read WHEN HDFS_BYTES_READ > '300kb' DO KILL
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling 
> command(queryId=pjayachandran_20180621131017_72b1441b-d790-4db7-83ca-479735843890);
>  Time taken: 0.015 seconds
> INFO : Executing 
> command(queryId=pjayachandran_20180621131017_72b1441b-d790-4db7-83ca-479735843890):
>  CREATE TRIGGER global.big_hdfs_read WHEN HDFS_BYTES_READ > '300kb' DO KILL
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing 
> command(queryId=pjayachandran_20180621131017_72b1441b-d790-4db7-83ca-479735843890);
>  Time taken: 0.025 seconds
> INFO : OK
> No rows affected (0.054 seconds)
> 0: jdbc:hive2://localhost:1> ALTER TRIGGER global.big_hdfs_read ADD TO 
> UNMANAGED;
> INFO : Compiling 
> command(queryId=pjayachandran_20180621131031_dd489324-db23-412f-9409-32ba697a10e5):
>  ALTER TRIGGER global.big_hdfs_read ADD TO UNMANAGED
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling 
> command(queryId=pjayachandran_20180621131031_dd489324-db23-412f-9409-32ba697a10e5);
>  Time taken: 0.014 seconds
> INFO : Executing 
> command(queryId=pjayachandran_20180621131031_dd489324-db23-412f-9409-32ba697a10e5):
>  ALTER TRIGGER global.big_hdfs_read ADD TO UNMANAGED
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing 
> command(queryId=pjayachandran_20180621131031_dd489324-db23-412f-9409-32ba697a10e5);
>  Time taken: 0.029 seconds
> INFO : OK
> No rows affected (0.054 seconds)
> 0: jdbc:hive2://localhost:1> ALTER RESOURCE PLAN global ENABLE;
> INFO : Compiling 
> command(queryId=pjayachandran_20180621131036_26a5f4f3-91e3-4bec-ab42-800adb90104e):
>  ALTER RESOURCE PLAN global ENABLE
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling 
> command(queryId=pjayachandran_20180621131036_26a5f4f3-91e3-4bec-ab42-800adb90104e);
>  Time taken: 0.012 seconds
> INFO : Executing 
> command(queryId=pjayachandran_20180621131036_26a5f4f3-91e3-4bec-ab42-800adb90104e):
>  ALTER RESOURCE PLAN global ENABLE
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing 
> command(queryId=pjayachandran_20180621131036_26a5f4f3-91e3-4bec-ab42-800adb90104e);
>  Time taken: 0.021 seconds
> INFO : OK
> No rows affected (0.045 seconds)
> 0: jdbc:hive2://localhost:1> ALTER RESOURCE PLAN global ACTIVATE;
> INFO : Compiling 
> command(queryId=pjayachandran_20180621131037_551b2af0-321b-4638-8ac0-76771a159f4b):
>  ALTER RESOURCE PLAN global ACTIVATE
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling 
> command(queryId=pjayachandran_20180621131037_551b2af0-321b-4638-8ac0-76771a159f4b);
>  Time taken: 0.017 seconds
> INFO : Executing 
> command(queryId=pjayachandran_20180621131037_551b2af0-321b-4638-8ac0-76771a159f4b):
>  ALTER RESOURCE PLAN global ACTIVATE
> INFO : Starting task [Stage-0:DDL] in serial mode
> ERROR : FAILED: Execution Error, return code 1 from 
> org.apache.hadoop.hive.ql.exec.DDLTask. Invalid expression: HDFS_BYTES_READ > 
> 300kb
> INFO : Completed executing 
> command(queryId=pjayachandran_20180621131037_551b2af0-321b-4638-8ac0-76771a159f4b);
>  Time taken: 0.037 seconds
> Error: Error while processing statement: FAILED: Execution Error, return code 
> 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Invalid expression: 
> HDFS_BYTES_READ > 300kb (state=08S01,code=1){code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Assigned] (HIVE-19964) Apply resource plan fails if trigger expression has quotes

2018-06-24 Thread Prasanth Jayachandran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasanth Jayachandran reassigned HIVE-19964:


Assignee: Prasanth Jayachandran  (was: Sergey Shelukhin)

> Apply resource plan fails if trigger expression has quotes
> --
>
> Key: HIVE-19964
> URL: https://issues.apache.org/jira/browse/HIVE-19964
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Aswathy Chellammal Sreekumar
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19964.1.patch
>
>
> {code:java}
> 0: jdbc:hive2://localhost:1> CREATE TRIGGER global.big_hdfs_read WHEN 
> HDFS_BYTES_READ > '300kb' DO KILL;
> INFO : Compiling 
> command(queryId=pjayachandran_20180621131017_72b1441b-d790-4db7-83ca-479735843890):
>  CREATE TRIGGER global.big_hdfs_read WHEN HDFS_BYTES_READ > '300kb' DO KILL
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling 
> command(queryId=pjayachandran_20180621131017_72b1441b-d790-4db7-83ca-479735843890);
>  Time taken: 0.015 seconds
> INFO : Executing 
> command(queryId=pjayachandran_20180621131017_72b1441b-d790-4db7-83ca-479735843890):
>  CREATE TRIGGER global.big_hdfs_read WHEN HDFS_BYTES_READ > '300kb' DO KILL
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing 
> command(queryId=pjayachandran_20180621131017_72b1441b-d790-4db7-83ca-479735843890);
>  Time taken: 0.025 seconds
> INFO : OK
> No rows affected (0.054 seconds)
> 0: jdbc:hive2://localhost:1> ALTER TRIGGER global.big_hdfs_read ADD TO 
> UNMANAGED;
> INFO : Compiling 
> command(queryId=pjayachandran_20180621131031_dd489324-db23-412f-9409-32ba697a10e5):
>  ALTER TRIGGER global.big_hdfs_read ADD TO UNMANAGED
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling 
> command(queryId=pjayachandran_20180621131031_dd489324-db23-412f-9409-32ba697a10e5);
>  Time taken: 0.014 seconds
> INFO : Executing 
> command(queryId=pjayachandran_20180621131031_dd489324-db23-412f-9409-32ba697a10e5):
>  ALTER TRIGGER global.big_hdfs_read ADD TO UNMANAGED
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing 
> command(queryId=pjayachandran_20180621131031_dd489324-db23-412f-9409-32ba697a10e5);
>  Time taken: 0.029 seconds
> INFO : OK
> No rows affected (0.054 seconds)
> 0: jdbc:hive2://localhost:1> ALTER RESOURCE PLAN global ENABLE;
> INFO : Compiling 
> command(queryId=pjayachandran_20180621131036_26a5f4f3-91e3-4bec-ab42-800adb90104e):
>  ALTER RESOURCE PLAN global ENABLE
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling 
> command(queryId=pjayachandran_20180621131036_26a5f4f3-91e3-4bec-ab42-800adb90104e);
>  Time taken: 0.012 seconds
> INFO : Executing 
> command(queryId=pjayachandran_20180621131036_26a5f4f3-91e3-4bec-ab42-800adb90104e):
>  ALTER RESOURCE PLAN global ENABLE
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing 
> command(queryId=pjayachandran_20180621131036_26a5f4f3-91e3-4bec-ab42-800adb90104e);
>  Time taken: 0.021 seconds
> INFO : OK
> No rows affected (0.045 seconds)
> 0: jdbc:hive2://localhost:1> ALTER RESOURCE PLAN global ACTIVATE;
> INFO : Compiling 
> command(queryId=pjayachandran_20180621131037_551b2af0-321b-4638-8ac0-76771a159f4b):
>  ALTER RESOURCE PLAN global ACTIVATE
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling 
> command(queryId=pjayachandran_20180621131037_551b2af0-321b-4638-8ac0-76771a159f4b);
>  Time taken: 0.017 seconds
> INFO : Executing 
> command(queryId=pjayachandran_20180621131037_551b2af0-321b-4638-8ac0-76771a159f4b):
>  ALTER RESOURCE PLAN global ACTIVATE
> INFO : Starting task [Stage-0:DDL] in serial mode
> ERROR : FAILED: Execution Error, return code 1 from 
> org.apache.hadoop.hive.ql.exec.DDLTask. Invalid expression: HDFS_BYTES_READ > 
> 300kb
> INFO : Completed executing 
> command(queryId=pjayachandran_20180621131037_551b2af0-321b-4638-8ac0-76771a159f4b);
>  Time taken: 0.037 seconds
> Error: Error while processing statement: FAILED: Execution Error, return code 
> 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Invalid expression: 
> HDFS_BYTES_READ > 300kb (state=08S01,code=1){code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19801) JDBC: Add some missing classes to jdbc standalone jar and remove hbase classes

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521576#comment-16521576
 ] 

Hive QA commented on HIVE-19801:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
| {color:red}-1{color} | {color:red} patch {color} | {color:red}  0m 13s{color} 
| {color:red} 
/data/hiveptest/logs/PreCommit-HIVE-Build-12060/patches/PreCommit-HIVE-Build-12060.patch
 does not apply to master. Rebase required? Wrong Branch? See 
http://cwiki.apache.org/confluence/display/Hive/HowToContribute for help. 
{color} |
\\
\\
|| Subsystem || Report/Notes ||
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-12060/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> JDBC: Add some missing classes to jdbc standalone jar and remove hbase classes
> --
>
> Key: HIVE-19801
> URL: https://issues.apache.org/jira/browse/HIVE-19801
> Project: Hive
>  Issue Type: Bug
>  Components: JDBC
>Affects Versions: 3.0.0, 3.1.0
>Reporter: Vaibhav Gumashta
>Assignee: Thejas M Nair
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-19801.1-branch-3.patch, 
> HIVE-19801.1-branch-3.patch, HIVE-19801.1.patch
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Assigned] (HIVE-19964) Apply resource plan fails if trigger expression has quotes

2018-06-24 Thread Prasanth Jayachandran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasanth Jayachandran reassigned HIVE-19964:


Assignee: Sergey Shelukhin  (was: Prasanth Jayachandran)

> Apply resource plan fails if trigger expression has quotes
> --
>
> Key: HIVE-19964
> URL: https://issues.apache.org/jira/browse/HIVE-19964
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Aswathy Chellammal Sreekumar
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-19964.1.patch
>
>
> {code:java}
> 0: jdbc:hive2://localhost:1> CREATE TRIGGER global.big_hdfs_read WHEN 
> HDFS_BYTES_READ > '300kb' DO KILL;
> INFO : Compiling 
> command(queryId=pjayachandran_20180621131017_72b1441b-d790-4db7-83ca-479735843890):
>  CREATE TRIGGER global.big_hdfs_read WHEN HDFS_BYTES_READ > '300kb' DO KILL
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling 
> command(queryId=pjayachandran_20180621131017_72b1441b-d790-4db7-83ca-479735843890);
>  Time taken: 0.015 seconds
> INFO : Executing 
> command(queryId=pjayachandran_20180621131017_72b1441b-d790-4db7-83ca-479735843890):
>  CREATE TRIGGER global.big_hdfs_read WHEN HDFS_BYTES_READ > '300kb' DO KILL
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing 
> command(queryId=pjayachandran_20180621131017_72b1441b-d790-4db7-83ca-479735843890);
>  Time taken: 0.025 seconds
> INFO : OK
> No rows affected (0.054 seconds)
> 0: jdbc:hive2://localhost:1> ALTER TRIGGER global.big_hdfs_read ADD TO 
> UNMANAGED;
> INFO : Compiling 
> command(queryId=pjayachandran_20180621131031_dd489324-db23-412f-9409-32ba697a10e5):
>  ALTER TRIGGER global.big_hdfs_read ADD TO UNMANAGED
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling 
> command(queryId=pjayachandran_20180621131031_dd489324-db23-412f-9409-32ba697a10e5);
>  Time taken: 0.014 seconds
> INFO : Executing 
> command(queryId=pjayachandran_20180621131031_dd489324-db23-412f-9409-32ba697a10e5):
>  ALTER TRIGGER global.big_hdfs_read ADD TO UNMANAGED
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing 
> command(queryId=pjayachandran_20180621131031_dd489324-db23-412f-9409-32ba697a10e5);
>  Time taken: 0.029 seconds
> INFO : OK
> No rows affected (0.054 seconds)
> 0: jdbc:hive2://localhost:1> ALTER RESOURCE PLAN global ENABLE;
> INFO : Compiling 
> command(queryId=pjayachandran_20180621131036_26a5f4f3-91e3-4bec-ab42-800adb90104e):
>  ALTER RESOURCE PLAN global ENABLE
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling 
> command(queryId=pjayachandran_20180621131036_26a5f4f3-91e3-4bec-ab42-800adb90104e);
>  Time taken: 0.012 seconds
> INFO : Executing 
> command(queryId=pjayachandran_20180621131036_26a5f4f3-91e3-4bec-ab42-800adb90104e):
>  ALTER RESOURCE PLAN global ENABLE
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing 
> command(queryId=pjayachandran_20180621131036_26a5f4f3-91e3-4bec-ab42-800adb90104e);
>  Time taken: 0.021 seconds
> INFO : OK
> No rows affected (0.045 seconds)
> 0: jdbc:hive2://localhost:1> ALTER RESOURCE PLAN global ACTIVATE;
> INFO : Compiling 
> command(queryId=pjayachandran_20180621131037_551b2af0-321b-4638-8ac0-76771a159f4b):
>  ALTER RESOURCE PLAN global ACTIVATE
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling 
> command(queryId=pjayachandran_20180621131037_551b2af0-321b-4638-8ac0-76771a159f4b);
>  Time taken: 0.017 seconds
> INFO : Executing 
> command(queryId=pjayachandran_20180621131037_551b2af0-321b-4638-8ac0-76771a159f4b):
>  ALTER RESOURCE PLAN global ACTIVATE
> INFO : Starting task [Stage-0:DDL] in serial mode
> ERROR : FAILED: Execution Error, return code 1 from 
> org.apache.hadoop.hive.ql.exec.DDLTask. Invalid expression: HDFS_BYTES_READ > 
> 300kb
> INFO : Completed executing 
> command(queryId=pjayachandran_20180621131037_551b2af0-321b-4638-8ac0-76771a159f4b);
>  Time taken: 0.037 seconds
> Error: Error while processing statement: FAILED: Execution Error, return code 
> 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Invalid expression: 
> HDFS_BYTES_READ > 300kb (state=08S01,code=1){code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-19964) Apply resource plan fails if trigger expression has quotes

2018-06-24 Thread Prasanth Jayachandran (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Prasanth Jayachandran updated HIVE-19964:
-
Attachment: (was: HIVE-19956.2.patch)

> Apply resource plan fails if trigger expression has quotes
> --
>
> Key: HIVE-19964
> URL: https://issues.apache.org/jira/browse/HIVE-19964
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Aswathy Chellammal Sreekumar
>Assignee: Sergey Shelukhin
>Priority: Major
> Attachments: HIVE-19964.1.patch
>
>
> {code:java}
> 0: jdbc:hive2://localhost:1> CREATE TRIGGER global.big_hdfs_read WHEN 
> HDFS_BYTES_READ > '300kb' DO KILL;
> INFO : Compiling 
> command(queryId=pjayachandran_20180621131017_72b1441b-d790-4db7-83ca-479735843890):
>  CREATE TRIGGER global.big_hdfs_read WHEN HDFS_BYTES_READ > '300kb' DO KILL
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling 
> command(queryId=pjayachandran_20180621131017_72b1441b-d790-4db7-83ca-479735843890);
>  Time taken: 0.015 seconds
> INFO : Executing 
> command(queryId=pjayachandran_20180621131017_72b1441b-d790-4db7-83ca-479735843890):
>  CREATE TRIGGER global.big_hdfs_read WHEN HDFS_BYTES_READ > '300kb' DO KILL
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing 
> command(queryId=pjayachandran_20180621131017_72b1441b-d790-4db7-83ca-479735843890);
>  Time taken: 0.025 seconds
> INFO : OK
> No rows affected (0.054 seconds)
> 0: jdbc:hive2://localhost:1> ALTER TRIGGER global.big_hdfs_read ADD TO 
> UNMANAGED;
> INFO : Compiling 
> command(queryId=pjayachandran_20180621131031_dd489324-db23-412f-9409-32ba697a10e5):
>  ALTER TRIGGER global.big_hdfs_read ADD TO UNMANAGED
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling 
> command(queryId=pjayachandran_20180621131031_dd489324-db23-412f-9409-32ba697a10e5);
>  Time taken: 0.014 seconds
> INFO : Executing 
> command(queryId=pjayachandran_20180621131031_dd489324-db23-412f-9409-32ba697a10e5):
>  ALTER TRIGGER global.big_hdfs_read ADD TO UNMANAGED
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing 
> command(queryId=pjayachandran_20180621131031_dd489324-db23-412f-9409-32ba697a10e5);
>  Time taken: 0.029 seconds
> INFO : OK
> No rows affected (0.054 seconds)
> 0: jdbc:hive2://localhost:1> ALTER RESOURCE PLAN global ENABLE;
> INFO : Compiling 
> command(queryId=pjayachandran_20180621131036_26a5f4f3-91e3-4bec-ab42-800adb90104e):
>  ALTER RESOURCE PLAN global ENABLE
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling 
> command(queryId=pjayachandran_20180621131036_26a5f4f3-91e3-4bec-ab42-800adb90104e);
>  Time taken: 0.012 seconds
> INFO : Executing 
> command(queryId=pjayachandran_20180621131036_26a5f4f3-91e3-4bec-ab42-800adb90104e):
>  ALTER RESOURCE PLAN global ENABLE
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing 
> command(queryId=pjayachandran_20180621131036_26a5f4f3-91e3-4bec-ab42-800adb90104e);
>  Time taken: 0.021 seconds
> INFO : OK
> No rows affected (0.045 seconds)
> 0: jdbc:hive2://localhost:1> ALTER RESOURCE PLAN global ACTIVATE;
> INFO : Compiling 
> command(queryId=pjayachandran_20180621131037_551b2af0-321b-4638-8ac0-76771a159f4b):
>  ALTER RESOURCE PLAN global ACTIVATE
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling 
> command(queryId=pjayachandran_20180621131037_551b2af0-321b-4638-8ac0-76771a159f4b);
>  Time taken: 0.017 seconds
> INFO : Executing 
> command(queryId=pjayachandran_20180621131037_551b2af0-321b-4638-8ac0-76771a159f4b):
>  ALTER RESOURCE PLAN global ACTIVATE
> INFO : Starting task [Stage-0:DDL] in serial mode
> ERROR : FAILED: Execution Error, return code 1 from 
> org.apache.hadoop.hive.ql.exec.DDLTask. Invalid expression: HDFS_BYTES_READ > 
> 300kb
> INFO : Completed executing 
> command(queryId=pjayachandran_20180621131037_551b2af0-321b-4638-8ac0-76771a159f4b);
>  Time taken: 0.037 seconds
> Error: Error while processing statement: FAILED: Execution Error, return code 
> 1 from org.apache.hadoop.hive.ql.exec.DDLTask. Invalid expression: 
> HDFS_BYTES_READ > 300kb (state=08S01,code=1){code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19956) Include yarn registry classes to jdbc standalone jar

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521571#comment-16521571
 ] 

Hive QA commented on HIVE-19956:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928883/HIVE-19956.3.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:green}SUCCESS:{color} +1 due to 14602 tests passed

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/12059/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12059/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12059/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12928883 - PreCommit-HIVE-Build

> Include yarn registry classes to jdbc standalone jar
> 
>
> Key: HIVE-19956
> URL: https://issues.apache.org/jira/browse/HIVE-19956
> Project: Hive
>  Issue Type: Bug
>  Components: JDBC
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19956.1.patch, HIVE-19956.2.patch, 
> HIVE-19956.3.patch
>
>
> HS2 Active/Passive HA requires some yarn registry classes. Include it in JDBC 
> standalone jar. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-19027) Make materializations invalidation cache work with multiple active remote metastores

2018-06-24 Thread Jesus Camacho Rodriguez (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19027?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jesus Camacho Rodriguez updated HIVE-19027:
---
Attachment: (was: HIVE-19027.01.patch)

> Make materializations invalidation cache work with multiple active remote 
> metastores
> 
>
> Key: HIVE-19027
> URL: https://issues.apache.org/jira/browse/HIVE-19027
> Project: Hive
>  Issue Type: Improvement
>  Components: Materialized views
>Affects Versions: 3.0.0
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Critical
> Attachments: HIVE-19027.01.patch
>
>
> The main points:
>  - Only MVs stored in transactional tables can have a time window value of 0. 
> Those are the only MVs that can be guaranteed to not be outdated when a query 
> is executed, if we use custom storage handlers to store the materialized 
> view, we cannot make any promises.
>  - For MVs that +cannot be outdated+, we do not check the metastore. Instead, 
> comparison is based on valid write id lists.
>  - For MVs that +can be outdated+, we still rely on the invalidation cache.
>  ** The window for valid outdated MVs can be specified in intervals of 1 
> minute (less than that, it is difficult to have any guarantees about whether 
> the MV is actually outdated by less than a minute or not).
>  ** The async loading is done every interval / 2 (or probably better, we can 
> make it configurable).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-12192) Hive should carry out timestamp computations in UTC

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-12192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521567#comment-16521567
 ] 

Hive QA commented on HIVE-12192:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
38s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m 
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 10m 
58s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
21s{color} | {color:blue} storage-api in master has 48 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
31s{color} | {color:blue} common in master has 62 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
38s{color} | {color:blue} serde in master has 190 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
30s{color} | {color:blue} accumulo-handler in master has 21 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
27s{color} | {color:blue} druid-handler in master has 13 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
30s{color} | {color:blue} hbase-handler in master has 15 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
36s{color} | {color:blue} hcatalog/core in master has 33 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
25s{color} | {color:blue} hcatalog/hcatalog-pig-adapter in master has 2 extant 
Findbugs warnings. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red}  4m  
6s{color} | {color:red} branch/itests/hive-jmh cannot run convertXmlToText from 
findbugs {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
37s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
28s{color} | {color:blue} llap-common in master has 71 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
39s{color} | {color:blue} llap-server in master has 83 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 
57s{color} | {color:blue} ql in master has 2282 extant Findbugs warnings. 
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 
11s{color} | {color:blue} standalone-metastore in master has 228 extant 
Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 10m 
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
7s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m 
36s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m 
49s{color} | {color:red} accumulo-handler in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m 
31s{color} | {color:red} core in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m 
22s{color} | {color:red} hive-jmh in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m 
43s{color} | {color:red} hive-unit in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m 
20s{color} | {color:red} llap-common in the patch failed. {color} |
| {color:red}-1{color} | {color:red} compile {color} | {color:red}  0m 
44s{color} | {color:red} ql in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red}  0m 49s{color} 
| {color:red} accumulo-handler in the patch failed. {color} |
| {color:red}-1{color} | {color:red} javac {color} | {color:red}  0m 

[jira] [Updated] (HIVE-19753) Strict managed tables mode in Hive

2018-06-24 Thread Jason Dere (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Dere updated HIVE-19753:
--
  Resolution: Fixed
   Fix Version/s: 4.0.0
Target Version/s: 3.1.0
  Status: Resolved  (was: Patch Available)

Committed to master
Uploaded patch for branch-3, awaiting test results

> Strict managed tables mode in Hive
> --
>
> Key: HIVE-19753
> URL: https://issues.apache.org/jira/browse/HIVE-19753
> Project: Hive
>  Issue Type: Bug
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-19753.01-branch-3.patch, HIVE-19753.1.patch, 
> HIVE-19753.2.patch, HIVE-19753.3.patch, HIVE-19753.4.patch, 
> HIVE-19753.5.patch, HIVE-19753.6.patch, HIVE-19753.7.patch
>
>
> Create a mode in Hive which enforces that all managed tables are 
> transactional (both full or insert-only tables allowed). Non-transactional 
> tables, as well as non-native tables, must be created as external tables when 
> this mode is enabled.
> The idea would be that in strict managed tables mode all of the data written 
> to managed tables would have been done through Hive.
> The mode would be enabled using config setting hive.strict.managed.tables.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-19753) Strict managed tables mode in Hive

2018-06-24 Thread Jason Dere (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Dere updated HIVE-19753:
--
Attachment: HIVE-19753.01-branch-3.patch

> Strict managed tables mode in Hive
> --
>
> Key: HIVE-19753
> URL: https://issues.apache.org/jira/browse/HIVE-19753
> Project: Hive
>  Issue Type: Bug
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Attachments: HIVE-19753.01-branch-3.patch, HIVE-19753.1.patch, 
> HIVE-19753.2.patch, HIVE-19753.3.patch, HIVE-19753.4.patch, 
> HIVE-19753.5.patch, HIVE-19753.6.patch, HIVE-19753.7.patch
>
>
> Create a mode in Hive which enforces that all managed tables are 
> transactional (both full or insert-only tables allowed). Non-transactional 
> tables, as well as non-native tables, must be created as external tables when 
> this mode is enabled.
> The idea would be that in strict managed tables mode all of the data written 
> to managed tables would have been done through Hive.
> The mode would be enabled using config setting hive.strict.managed.tables.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19956) Include yarn registry classes to jdbc standalone jar

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521561#comment-16521561
 ] 

Hive QA commented on HIVE-19956:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 10m 
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
22s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
52s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
21s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
0s{color} | {color:red} The patch 1 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
15s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
15s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 32s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  xml  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12059/dev-support/hive-personality.sh
 |
| git revision | master / 0911864 |
| Default Java | 1.8.0_111 |
| whitespace | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-12059/yetus/whitespace-tabs.txt
 |
| modules | C: jdbc U: jdbc |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-12059/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Include yarn registry classes to jdbc standalone jar
> 
>
> Key: HIVE-19956
> URL: https://issues.apache.org/jira/browse/HIVE-19956
> Project: Hive
>  Issue Type: Bug
>  Components: JDBC
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Prasanth Jayachandran
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19956.1.patch, HIVE-19956.2.patch, 
> HIVE-19956.3.patch
>
>
> HS2 Active/Passive HA requires some yarn registry classes. Include it in JDBC 
> standalone jar. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19898) Disable TransactionalValidationListener when the table is not in the Hive catalog

2018-06-24 Thread Jason Dere (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521560#comment-16521560
 ] 

Jason Dere commented on HIVE-19898:
---

Looks like this was somehow removed from branch-3 by the following commit:

{noformat}
commit 90b442c1b54f786558e04cec60715756e7accd1d
Author: Deepak Jaiswal 
Date:   Mon Jun 18 10:57:53 2018 -0700

Revert "Merge branch 'branch-3' of 
http://git-wip-us.apache.org/repos/asf/hive into branch-3"

This reverts commit 9d80c2d6a1a7bf378d2c6a28006d09f3c3643e5f, reversing
changes made to 6f9a76744ea3acb015a72e8d1e94a335ac80f42f.
{noformat}

I've re-applied the patch back to branch-3.

> Disable TransactionalValidationListener when the table is not in the Hive 
> catalog
> -
>
> Key: HIVE-19898
> URL: https://issues.apache.org/jira/browse/HIVE-19898
> Project: Hive
>  Issue Type: Bug
>  Components: Metastore, Standalone Metastore, Transactions
>Reporter: Jason Dere
>Assignee: Jason Dere
>Priority: Major
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19898.1.patch
>
>
> The TransactionalValidationListener does validation of tables specified as 
> transactional tables, as well as enforcing create.as.acid. While this can be 
> useful to Hive, this may not be useful to other catalogs which do not support 
> transactional tables, and would not benefit from being automatically tagged 
> as a transactional table. This should be changed so the 
> TransactionalValidationListener does not run for non-hive catalogs.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-12192) Hive should carry out timestamp computations in UTC

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-12192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521557#comment-16521557
 ] 

Hive QA commented on HIVE-12192:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928922/HIVE-12192.25.patch

{color:green}SUCCESS:{color} +1 due to 79 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 14578 tests 
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=258)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=258)
TestMinimrCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=258)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/12058/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12058/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12058/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12928922 - PreCommit-HIVE-Build

> Hive should carry out timestamp computations in UTC
> ---
>
> Key: HIVE-12192
> URL: https://issues.apache.org/jira/browse/HIVE-12192
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Ryan Blue
>Assignee: Jesus Camacho Rodriguez
>Priority: Blocker
>  Labels: timestamp
> Fix For: 3.1.0
>
> Attachments: HIVE-12192.01.patch, HIVE-12192.02.patch, 
> HIVE-12192.03.patch, HIVE-12192.04.patch, HIVE-12192.05.patch, 
> HIVE-12192.06.patch, HIVE-12192.07.patch, HIVE-12192.08.patch, 
> HIVE-12192.09.patch, HIVE-12192.10.patch, HIVE-12192.11.patch, 
> HIVE-12192.12.patch, HIVE-12192.13.patch, HIVE-12192.14.patch, 
> HIVE-12192.15.patch, HIVE-12192.16.patch, HIVE-12192.17.patch, 
> HIVE-12192.18.patch, HIVE-12192.19.patch, HIVE-12192.20.patch, 
> HIVE-12192.21.patch, HIVE-12192.22.patch, HIVE-12192.23.patch, 
> HIVE-12192.24.patch, HIVE-12192.25.patch, HIVE-12192.patch
>
>
> Hive currently uses the "local" time of a java.sql.Timestamp to represent the 
> SQL data type TIMESTAMP WITHOUT TIME ZONE. The purpose is to be able to use 
> {{Timestamp#getYear()}} and similar methods to implement SQL functions like 
> {{year}}.
> When the SQL session's time zone is a DST zone, such as America/Los_Angeles 
> that alternates between PST and PDT, there are times that cannot be 
> represented because the effective zone skips them.
> {code}
> hive> select TIMESTAMP '2015-03-08 02:10:00.101';
> 2015-03-08 03:10:00.101
> {code}
> Using UTC instead of the SQL session time zone as the underlying zone for a 
> java.sql.Timestamp avoids this bug, while still returning correct values for 
> {{getYear}} etc. Using UTC as the convenience representation (timestamp 
> without time zone has no real zone) would make timestamp calculations more 
> consistent and avoid similar problems in the future.
> Notably, this would break the {{unix_timestamp}} UDF that specifies the 
> result is with respect to ["the default timezone and default 
> locale"|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-DateFunctions].
>  That function would need to be updated to use the 
> {{System.getProperty("user.timezone")}} zone.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19922) TestMiniDruidKafkaCliDriver[druidkafkamini_basic] is flaky

2018-06-24 Thread Jason Dere (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521552#comment-16521552
 ] 

Jason Dere commented on HIVE-19922:
---

+1
This should be disabled and fixed in followup

> TestMiniDruidKafkaCliDriver[druidkafkamini_basic] is flaky
> --
>
> Key: HIVE-19922
> URL: https://issues.apache.org/jira/browse/HIVE-19922
> Project: Hive
>  Issue Type: Bug
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-19922.2.patch, HIVE-19922.3.patch, 
> HIVE-19922.4.patch, HIVE-19922.patch
>
>
> Consistently failing in the last 4 runs.
> See:
> [https://builds.apache.org/job/PreCommit-HIVE-Build/11824/testReport/org.apache.hadoop.hive.cli/TestMiniDruidKafkaCliDriver/testCliDriver_druidkafkamini_basic_/history/]
> Can not reproduce the failure locally :(
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-18705) Improve HiveMetaStoreClient.dropDatabase

2018-06-24 Thread Adam Szita (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-18705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adam Szita updated HIVE-18705:
--
Status: Patch Available  (was: In Progress)

> Improve HiveMetaStoreClient.dropDatabase
> 
>
> Key: HIVE-18705
> URL: https://issues.apache.org/jira/browse/HIVE-18705
> Project: Hive
>  Issue Type: Improvement
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-18705.0.patch, HIVE-18705.1.patch, 
> HIVE-18705.2.patch, HIVE-18705.4.patch, HIVE-18705.5.patch, 
> HIVE-18705.6.patch, HIVE-18705.7.patch, HIVE-18705.8.patch
>
>
> {{HiveMetaStoreClient.dropDatabase}} has a strange implementation to ensure 
> dealing with client side hooks (for non-native tables e.g. HBase). Currently 
> it starts by retrieving all the tables from HMS, and then sends {{dropTable}} 
> calls to HMS table-by-table. At the end a {{dropDatabase}} just to be sure :) 
> I believe this could be refactored so that it speeds up the dropDB in 
> situations where the average table count per DB is very high.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-18705) Improve HiveMetaStoreClient.dropDatabase

2018-06-24 Thread Adam Szita (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-18705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adam Szita updated HIVE-18705:
--
Attachment: HIVE-18705.8.patch

> Improve HiveMetaStoreClient.dropDatabase
> 
>
> Key: HIVE-18705
> URL: https://issues.apache.org/jira/browse/HIVE-18705
> Project: Hive
>  Issue Type: Improvement
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-18705.0.patch, HIVE-18705.1.patch, 
> HIVE-18705.2.patch, HIVE-18705.4.patch, HIVE-18705.5.patch, 
> HIVE-18705.6.patch, HIVE-18705.7.patch, HIVE-18705.8.patch
>
>
> {{HiveMetaStoreClient.dropDatabase}} has a strange implementation to ensure 
> dealing with client side hooks (for non-native tables e.g. HBase). Currently 
> it starts by retrieving all the tables from HMS, and then sends {{dropTable}} 
> calls to HMS table-by-table. At the end a {{dropDatabase}} just to be sure :) 
> I believe this could be refactored so that it speeds up the dropDB in 
> situations where the average table count per DB is very high.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-18705) Improve HiveMetaStoreClient.dropDatabase

2018-06-24 Thread Adam Szita (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-18705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adam Szita updated HIVE-18705:
--
Status: In Progress  (was: Patch Available)

> Improve HiveMetaStoreClient.dropDatabase
> 
>
> Key: HIVE-18705
> URL: https://issues.apache.org/jira/browse/HIVE-18705
> Project: Hive
>  Issue Type: Improvement
>Reporter: Adam Szita
>Assignee: Adam Szita
>Priority: Major
> Attachments: HIVE-18705.0.patch, HIVE-18705.1.patch, 
> HIVE-18705.2.patch, HIVE-18705.4.patch, HIVE-18705.5.patch, 
> HIVE-18705.6.patch, HIVE-18705.7.patch, HIVE-18705.8.patch
>
>
> {{HiveMetaStoreClient.dropDatabase}} has a strange implementation to ensure 
> dealing with client side hooks (for non-native tables e.g. HBase). Currently 
> it starts by retrieving all the tables from HMS, and then sends {{dropTable}} 
> calls to HMS table-by-table. At the end a {{dropDatabase}} just to be sure :) 
> I believe this could be refactored so that it speeds up the dropDB in 
> situations where the average table count per DB is very high.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19964) Apply resource plan fails if trigger expression has quotes

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521531#comment-16521531
 ] 

Hive QA commented on HIVE-19964:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928881/HIVE-19956.2.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:green}SUCCESS:{color} +1 due to 14594 tests passed

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/12057/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12057/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12057/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12928881 - PreCommit-HIVE-Build

> Apply resource plan fails if trigger expression has quotes
> --
>
> Key: HIVE-19964
> URL: https://issues.apache.org/jira/browse/HIVE-19964
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Aswathy Chellammal Sreekumar
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19956.2.patch, HIVE-19964.1.patch
>
>
> {code:java}
> 0: jdbc:hive2://localhost:1> CREATE TRIGGER global.big_hdfs_read WHEN 
> HDFS_BYTES_READ > '300kb' DO KILL;
> INFO : Compiling 
> command(queryId=pjayachandran_20180621131017_72b1441b-d790-4db7-83ca-479735843890):
>  CREATE TRIGGER global.big_hdfs_read WHEN HDFS_BYTES_READ > '300kb' DO KILL
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling 
> command(queryId=pjayachandran_20180621131017_72b1441b-d790-4db7-83ca-479735843890);
>  Time taken: 0.015 seconds
> INFO : Executing 
> command(queryId=pjayachandran_20180621131017_72b1441b-d790-4db7-83ca-479735843890):
>  CREATE TRIGGER global.big_hdfs_read WHEN HDFS_BYTES_READ > '300kb' DO KILL
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing 
> command(queryId=pjayachandran_20180621131017_72b1441b-d790-4db7-83ca-479735843890);
>  Time taken: 0.025 seconds
> INFO : OK
> No rows affected (0.054 seconds)
> 0: jdbc:hive2://localhost:1> ALTER TRIGGER global.big_hdfs_read ADD TO 
> UNMANAGED;
> INFO : Compiling 
> command(queryId=pjayachandran_20180621131031_dd489324-db23-412f-9409-32ba697a10e5):
>  ALTER TRIGGER global.big_hdfs_read ADD TO UNMANAGED
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling 
> command(queryId=pjayachandran_20180621131031_dd489324-db23-412f-9409-32ba697a10e5);
>  Time taken: 0.014 seconds
> INFO : Executing 
> command(queryId=pjayachandran_20180621131031_dd489324-db23-412f-9409-32ba697a10e5):
>  ALTER TRIGGER global.big_hdfs_read ADD TO UNMANAGED
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing 
> command(queryId=pjayachandran_20180621131031_dd489324-db23-412f-9409-32ba697a10e5);
>  Time taken: 0.029 seconds
> INFO : OK
> No rows affected (0.054 seconds)
> 0: jdbc:hive2://localhost:1> ALTER RESOURCE PLAN global ENABLE;
> INFO : Compiling 
> command(queryId=pjayachandran_20180621131036_26a5f4f3-91e3-4bec-ab42-800adb90104e):
>  ALTER RESOURCE PLAN global ENABLE
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling 
> command(queryId=pjayachandran_20180621131036_26a5f4f3-91e3-4bec-ab42-800adb90104e);
>  Time taken: 0.012 seconds
> INFO : Executing 
> command(queryId=pjayachandran_20180621131036_26a5f4f3-91e3-4bec-ab42-800adb90104e):
>  ALTER RESOURCE PLAN global ENABLE
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing 
> command(queryId=pjayachandran_20180621131036_26a5f4f3-91e3-4bec-ab42-800adb90104e);
>  Time taken: 0.021 seconds
> INFO : OK
> No rows affected (0.045 seconds)
> 0: jdbc:hive2://localhost:1> ALTER RESOURCE PLAN global ACTIVATE;
> INFO : Compiling 
> command(queryId=pjayachandran_20180621131037_551b2af0-321b-4638-8ac0-76771a159f4b):
>  ALTER RESOURCE PLAN global ACTIVATE
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling 
> command(queryId=pjayachandran_20180621131037_551b2af0-321b-4638-8ac0-76771a159f4b);
>  Time taken: 0.017 seconds
> INFO : Executing 
> 

[jira] [Updated] (HIVE-12192) Hive should carry out timestamp computations in UTC

2018-06-24 Thread Jesus Camacho Rodriguez (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-12192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jesus Camacho Rodriguez updated HIVE-12192:
---
Attachment: HIVE-12192.25.patch

> Hive should carry out timestamp computations in UTC
> ---
>
> Key: HIVE-12192
> URL: https://issues.apache.org/jira/browse/HIVE-12192
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Ryan Blue
>Assignee: Jesus Camacho Rodriguez
>Priority: Blocker
>  Labels: timestamp
> Fix For: 3.1.0
>
> Attachments: HIVE-12192.01.patch, HIVE-12192.02.patch, 
> HIVE-12192.03.patch, HIVE-12192.04.patch, HIVE-12192.05.patch, 
> HIVE-12192.06.patch, HIVE-12192.07.patch, HIVE-12192.08.patch, 
> HIVE-12192.09.patch, HIVE-12192.10.patch, HIVE-12192.11.patch, 
> HIVE-12192.12.patch, HIVE-12192.13.patch, HIVE-12192.14.patch, 
> HIVE-12192.15.patch, HIVE-12192.16.patch, HIVE-12192.17.patch, 
> HIVE-12192.18.patch, HIVE-12192.19.patch, HIVE-12192.20.patch, 
> HIVE-12192.21.patch, HIVE-12192.22.patch, HIVE-12192.23.patch, 
> HIVE-12192.24.patch, HIVE-12192.25.patch, HIVE-12192.patch
>
>
> Hive currently uses the "local" time of a java.sql.Timestamp to represent the 
> SQL data type TIMESTAMP WITHOUT TIME ZONE. The purpose is to be able to use 
> {{Timestamp#getYear()}} and similar methods to implement SQL functions like 
> {{year}}.
> When the SQL session's time zone is a DST zone, such as America/Los_Angeles 
> that alternates between PST and PDT, there are times that cannot be 
> represented because the effective zone skips them.
> {code}
> hive> select TIMESTAMP '2015-03-08 02:10:00.101';
> 2015-03-08 03:10:00.101
> {code}
> Using UTC instead of the SQL session time zone as the underlying zone for a 
> java.sql.Timestamp avoids this bug, while still returning correct values for 
> {{getYear}} etc. Using UTC as the convenience representation (timestamp 
> without time zone has no real zone) would make timestamp calculations more 
> consistent and avoid similar problems in the future.
> Notably, this would break the {{unix_timestamp}} UDF that specifies the 
> result is with respect to ["the default timezone and default 
> locale"|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-DateFunctions].
>  That function would need to be updated to use the 
> {{System.getProperty("user.timezone")}} zone.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19890) ACID: Inherit bucket-id from original ROW_ID for delete deltas

2018-06-24 Thread Eugene Koifman (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521520#comment-16521520
 ] 

Eugene Koifman commented on HIVE-19890:
---

branch-3 patch for HIVE-19965 has all the same failures

> ACID: Inherit bucket-id from original ROW_ID for delete deltas
> --
>
> Key: HIVE-19890
> URL: https://issues.apache.org/jira/browse/HIVE-19890
> Project: Hive
>  Issue Type: Bug
>  Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Attachments: HIVE-19890.1.patch, HIVE-19890.2.patch, 
> HIVE-19890.3.patch, HIVE-19890.4-branch-3.patch
>
>
> The ACID delete deltas for unbucketed tables are written to arbitrary files, 
> which should instead be shuffled using the bucket-id instead of hash(ROW__ID).



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19965) Make HiveEndPoint use IMetaStoreClient.add_partition

2018-06-24 Thread Eugene Koifman (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521519#comment-16521519
 ] 

Eugene Koifman commented on HIVE-19965:
---

complete unrelated HIVE-19890 patch HIVE-19890.4-branch-3.patch has all the 
same failures

> Make HiveEndPoint use IMetaStoreClient.add_partition
> 
>
> Key: HIVE-19965
> URL: https://issues.apache.org/jira/browse/HIVE-19965
> Project: Hive
>  Issue Type: Bug
>  Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Blocker
> Attachments: HIVE-19965.01-branch-3.patch, HIVE-19965.01.patch, 
> HIVE-19965.02-branch-3.patch
>
>
> it currently uses "alter table add partition if exists..."
> which since HIVE-18814 requires X lock on the table which blocks other 
> streaming writers from making progress.  
> HIVE-19961 to investigate if X lock can be relaxed, etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-19965) Make HiveEndPoint use IMetaStoreClient.add_partition

2018-06-24 Thread Eugene Koifman (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eugene Koifman updated HIVE-19965:
--
Attachment: HIVE-19965.02-branch-3.patch

> Make HiveEndPoint use IMetaStoreClient.add_partition
> 
>
> Key: HIVE-19965
> URL: https://issues.apache.org/jira/browse/HIVE-19965
> Project: Hive
>  Issue Type: Bug
>  Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Blocker
> Attachments: HIVE-19965.01-branch-3.patch, HIVE-19965.01.patch, 
> HIVE-19965.02-branch-3.patch
>
>
> it currently uses "alter table add partition if exists..."
> which since HIVE-18814 requires X lock on the table which blocks other 
> streaming writers from making progress.  
> HIVE-19961 to investigate if X lock can be relaxed, etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19964) Apply resource plan fails if trigger expression has quotes

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521515#comment-16521515
 ] 

Hive QA commented on HIVE-19964:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
17s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red}  0m  
0s{color} | {color:red} The patch 1 line(s) with tabs. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green}  0m  
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
13s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black}  8m 55s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  xml  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12057/dev-support/hive-personality.sh
 |
| git revision | master / 4b7f88a |
| Default Java | 1.8.0_111 |
| whitespace | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-12057/yetus/whitespace-tabs.txt
 |
| modules | C: jdbc U: jdbc |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-12057/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Apply resource plan fails if trigger expression has quotes
> --
>
> Key: HIVE-19964
> URL: https://issues.apache.org/jira/browse/HIVE-19964
> Project: Hive
>  Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0
>Reporter: Aswathy Chellammal Sreekumar
>Assignee: Prasanth Jayachandran
>Priority: Major
> Attachments: HIVE-19956.2.patch, HIVE-19964.1.patch
>
>
> {code:java}
> 0: jdbc:hive2://localhost:1> CREATE TRIGGER global.big_hdfs_read WHEN 
> HDFS_BYTES_READ > '300kb' DO KILL;
> INFO : Compiling 
> command(queryId=pjayachandran_20180621131017_72b1441b-d790-4db7-83ca-479735843890):
>  CREATE TRIGGER global.big_hdfs_read WHEN HDFS_BYTES_READ > '300kb' DO KILL
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling 
> command(queryId=pjayachandran_20180621131017_72b1441b-d790-4db7-83ca-479735843890);
>  Time taken: 0.015 seconds
> INFO : Executing 
> command(queryId=pjayachandran_20180621131017_72b1441b-d790-4db7-83ca-479735843890):
>  CREATE TRIGGER global.big_hdfs_read WHEN HDFS_BYTES_READ > '300kb' DO KILL
> INFO : Starting task [Stage-0:DDL] in serial mode
> INFO : Completed executing 
> command(queryId=pjayachandran_20180621131017_72b1441b-d790-4db7-83ca-479735843890);
>  Time taken: 0.025 seconds
> INFO : OK
> No rows affected (0.054 seconds)
> 0: jdbc:hive2://localhost:1> ALTER TRIGGER global.big_hdfs_read ADD TO 
> UNMANAGED;
> INFO : Compiling 
> command(queryId=pjayachandran_20180621131031_dd489324-db23-412f-9409-32ba697a10e5):
>  ALTER TRIGGER global.big_hdfs_read ADD TO UNMANAGED
> INFO : Semantic Analysis Completed (retrial = false)
> INFO : Returning Hive schema: Schema(fieldSchemas:null, properties:null)
> INFO : Completed compiling 
> command(queryId=pjayachandran_20180621131031_dd489324-db23-412f-9409-32ba697a10e5);
>  Time taken: 0.014 seconds
> INFO : Executing 
> 

[jira] [Updated] (HIVE-12192) Hive should carry out timestamp computations in UTC

2018-06-24 Thread Jesus Camacho Rodriguez (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-12192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jesus Camacho Rodriguez updated HIVE-12192:
---
Attachment: HIVE-12192.24.patch

> Hive should carry out timestamp computations in UTC
> ---
>
> Key: HIVE-12192
> URL: https://issues.apache.org/jira/browse/HIVE-12192
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Ryan Blue
>Assignee: Jesus Camacho Rodriguez
>Priority: Blocker
>  Labels: timestamp
> Fix For: 3.1.0
>
> Attachments: HIVE-12192.01.patch, HIVE-12192.02.patch, 
> HIVE-12192.03.patch, HIVE-12192.04.patch, HIVE-12192.05.patch, 
> HIVE-12192.06.patch, HIVE-12192.07.patch, HIVE-12192.08.patch, 
> HIVE-12192.09.patch, HIVE-12192.10.patch, HIVE-12192.11.patch, 
> HIVE-12192.12.patch, HIVE-12192.13.patch, HIVE-12192.14.patch, 
> HIVE-12192.15.patch, HIVE-12192.16.patch, HIVE-12192.17.patch, 
> HIVE-12192.18.patch, HIVE-12192.19.patch, HIVE-12192.20.patch, 
> HIVE-12192.21.patch, HIVE-12192.22.patch, HIVE-12192.23.patch, 
> HIVE-12192.24.patch, HIVE-12192.patch
>
>
> Hive currently uses the "local" time of a java.sql.Timestamp to represent the 
> SQL data type TIMESTAMP WITHOUT TIME ZONE. The purpose is to be able to use 
> {{Timestamp#getYear()}} and similar methods to implement SQL functions like 
> {{year}}.
> When the SQL session's time zone is a DST zone, such as America/Los_Angeles 
> that alternates between PST and PDT, there are times that cannot be 
> represented because the effective zone skips them.
> {code}
> hive> select TIMESTAMP '2015-03-08 02:10:00.101';
> 2015-03-08 03:10:00.101
> {code}
> Using UTC instead of the SQL session time zone as the underlying zone for a 
> java.sql.Timestamp avoids this bug, while still returning correct values for 
> {{getYear}} etc. Using UTC as the convenience representation (timestamp 
> without time zone has no real zone) would make timestamp calculations more 
> consistent and avoid similar problems in the future.
> Notably, this would break the {{unix_timestamp}} UDF that specifies the 
> result is with respect to ["the default timezone and default 
> locale"|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-DateFunctions].
>  That function would need to be updated to use the 
> {{System.getProperty("user.timezone")}} zone.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19965) Make HiveEndPoint use IMetaStoreClient.add_partition

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521512#comment-16521512
 ] 

Hive QA commented on HIVE-19965:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928880/HIVE-19965.01-branch-3.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 10 failed/errored test(s), 14366 tests 
executed
*Failed tests:*
{noformat}
TestBeeLineDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=256)
TestDummy - did not produce a TEST-*.xml file (likely timed out) (batchId=256)
TestMiniDruidCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=256)
TestMiniDruidKafkaCliDriver - did not produce a TEST-*.xml file (likely timed 
out) (batchId=256)
TestTezPerfCliDriver - did not produce a TEST-*.xml file (likely timed out) 
(batchId=256)
org.apache.hadoop.hive.accumulo.mr.TestHiveAccumuloTypes.testBinaryTypes 
(batchId=191)
org.apache.hadoop.hive.cli.TestNegativeCliDriver.testCliDriver[date_literal3] 
(batchId=95)
org.apache.hadoop.hive.ql.TestWarehouseExternalDir.testManagedPaths 
(batchId=233)
org.apache.hive.service.TestHS2ImpersonationWithRemoteMS.testImpersonation 
(batchId=242)
org.apache.hive.spark.client.rpc.TestRpc.testServerPort (batchId=308)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/12056/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12056/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12056/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 10 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12928880 - PreCommit-HIVE-Build

> Make HiveEndPoint use IMetaStoreClient.add_partition
> 
>
> Key: HIVE-19965
> URL: https://issues.apache.org/jira/browse/HIVE-19965
> Project: Hive
>  Issue Type: Bug
>  Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Blocker
> Attachments: HIVE-19965.01-branch-3.patch, HIVE-19965.01.patch
>
>
> it currently uses "alter table add partition if exists..."
> which since HIVE-18814 requires X lock on the table which blocks other 
> streaming writers from making progress.  
> HIVE-19961 to investigate if X lock can be relaxed, etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19973) Enable materialized view rewriting by default

2018-06-24 Thread Ashutosh Chauhan (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521510#comment-16521510
 ] 

Ashutosh Chauhan commented on HIVE-19973:
-

+1

> Enable materialized view rewriting by default
> -
>
> Key: HIVE-19973
> URL: https://issues.apache.org/jira/browse/HIVE-19973
> Project: Hive
>  Issue Type: Bug
>  Components: Materialized views
>Reporter: Jesus Camacho Rodriguez
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Attachments: HIVE-19973.patch
>
>
> Change property value for {{hive.materializedview.rewriting}} to {{true}}. 
> For tests, it is already {{true}} by default.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (HIVE-19924) Tag distcp jobs run by Repl Load

2018-06-24 Thread mahesh kumar behera (JIRA)


 [ 
https://issues.apache.org/jira/browse/HIVE-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

mahesh kumar behera updated HIVE-19924:
---
Attachment: HIVE-19924.01.patch

> Tag distcp jobs run by Repl Load
> 
>
> Key: HIVE-19924
> URL: https://issues.apache.org/jira/browse/HIVE-19924
> Project: Hive
>  Issue Type: Task
>  Components: repl
>Affects Versions: 3.1.0, 4.0.0
>Reporter: mahesh kumar behera
>Assignee: mahesh kumar behera
>Priority: Major
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19924.01.patch
>
>
> Add tags in jobconf for distcp related jobs started by replication. This will 
> allow hive to kill these jobs in case beacon retries, or hs2 dies and beacon 
> issues a kill command.
>  * one of the tags should definitely be the query_id that starts the job : 
> With this flow beacon before retrying the bootstrap load, will issue a kill 
> command to hs2 with the query id of the previous issued command. hs2 will 
> then kill an running jobs on yarn tagged with the Query_id.
>  * To get around the additional failure point as mentioned above. The jobs 
> can be tagged with an additional unique tag_id provided by Beacon in the WITH 
> clause in repl load command to be used to tag distcp jobs ). Enhance the kill 
> api to take the tag as input and kill jobs associated with that tag. Problem 
> here is how do we validate the association of the tag with a hive query id to 
> make sure this api is not used to kill jobs run by other components, however 
> we can provide this capability to only admins and should be ok in that case.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19965) Make HiveEndPoint use IMetaStoreClient.add_partition

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521497#comment-16521497
 ] 

Hive QA commented on HIVE-19965:


| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  6m 
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
14s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
26s{color} | {color:blue} hcatalog/streaming in master has 11 extant Findbugs 
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
19s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red}  0m 
14s{color} | {color:red} hcatalog/streaming: The patch generated 9 new + 313 
unchanged - 11 fixed = 322 total (was 324) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  0m 
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
12s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
12s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 10m  7s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12056/dev-support/hive-personality.sh
 |
| git revision | master / 4b7f88a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-12056/yetus/diff-checkstyle-hcatalog_streaming.txt
 |
| modules | C: hcatalog/streaming U: hcatalog/streaming |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-12056/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Make HiveEndPoint use IMetaStoreClient.add_partition
> 
>
> Key: HIVE-19965
> URL: https://issues.apache.org/jira/browse/HIVE-19965
> Project: Hive
>  Issue Type: Bug
>  Components: Transactions
>Affects Versions: 3.0.0
>Reporter: Eugene Koifman
>Assignee: Eugene Koifman
>Priority: Blocker
> Attachments: HIVE-19965.01-branch-3.patch, HIVE-19965.01.patch
>
>
> it currently uses "alter table add partition if exists..."
> which since HIVE-18814 requires X lock on the table which blocks other 
> streaming writers from making progress.  
> HIVE-19961 to investigate if X lock can be relaxed, etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-12192) Hive should carry out timestamp computations in UTC

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-12192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521494#comment-16521494
 ] 

Hive QA commented on HIVE-12192:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928882/HIVE-12192.23.patch

{color:red}ERROR:{color} -1 due to build exiting with an error

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/12055/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12055/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12055/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Tests exited with: Exception: Patch URL 
https://issues.apache.org/jira/secure/attachment/12928882/HIVE-12192.23.patch 
was found in seen patch url's cache and a test was probably run already on it. 
Aborting...
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12928882 - PreCommit-HIVE-Build

> Hive should carry out timestamp computations in UTC
> ---
>
> Key: HIVE-12192
> URL: https://issues.apache.org/jira/browse/HIVE-12192
> Project: Hive
>  Issue Type: Sub-task
>  Components: Hive
>Reporter: Ryan Blue
>Assignee: Jesus Camacho Rodriguez
>Priority: Blocker
>  Labels: timestamp
> Fix For: 3.1.0
>
> Attachments: HIVE-12192.01.patch, HIVE-12192.02.patch, 
> HIVE-12192.03.patch, HIVE-12192.04.patch, HIVE-12192.05.patch, 
> HIVE-12192.06.patch, HIVE-12192.07.patch, HIVE-12192.08.patch, 
> HIVE-12192.09.patch, HIVE-12192.10.patch, HIVE-12192.11.patch, 
> HIVE-12192.12.patch, HIVE-12192.13.patch, HIVE-12192.14.patch, 
> HIVE-12192.15.patch, HIVE-12192.16.patch, HIVE-12192.17.patch, 
> HIVE-12192.18.patch, HIVE-12192.19.patch, HIVE-12192.20.patch, 
> HIVE-12192.21.patch, HIVE-12192.22.patch, HIVE-12192.23.patch, 
> HIVE-12192.patch
>
>
> Hive currently uses the "local" time of a java.sql.Timestamp to represent the 
> SQL data type TIMESTAMP WITHOUT TIME ZONE. The purpose is to be able to use 
> {{Timestamp#getYear()}} and similar methods to implement SQL functions like 
> {{year}}.
> When the SQL session's time zone is a DST zone, such as America/Los_Angeles 
> that alternates between PST and PDT, there are times that cannot be 
> represented because the effective zone skips them.
> {code}
> hive> select TIMESTAMP '2015-03-08 02:10:00.101';
> 2015-03-08 03:10:00.101
> {code}
> Using UTC instead of the SQL session time zone as the underlying zone for a 
> java.sql.Timestamp avoids this bug, while still returning correct values for 
> {{getYear}} etc. Using UTC as the convenience representation (timestamp 
> without time zone has no real zone) would make timestamp calculations more 
> consistent and avoid similar problems in the future.
> Notably, this would break the {{unix_timestamp}} UDF that specifies the 
> result is with respect to ["the default timezone and default 
> locale"|https://cwiki.apache.org/confluence/display/Hive/LanguageManual+UDF#LanguageManualUDF-DateFunctions].
>  That function would need to be updated to use the 
> {{System.getProperty("user.timezone")}} zone.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-18882) Do Not Hide Exception in Hive Metastore Client Connection

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-18882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521493#comment-16521493
 ] 

Hive QA commented on HIVE-18882:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928876/HIVE-18882.1.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 9 failed/errored test(s), 14517 tests 
executed
*Failed tests:*
{noformat}
TestBeeLineExceptionHandling - did not produce a TEST-*.xml file (likely timed 
out) (batchId=193)
TestBeeLineHistory - did not produce a TEST-*.xml file (likely timed out) 
(batchId=193)
TestBeelineArgParsing - did not produce a TEST-*.xml file (likely timed out) 
(batchId=193)
TestClientCommandHookFactory - did not produce a TEST-*.xml file (likely timed 
out) (batchId=193)
TestHiveCli - did not produce a TEST-*.xml file (likely timed out) (batchId=193)
TestHiveSchemaTool - did not produce a TEST-*.xml file (likely timed out) 
(batchId=193)
TestIncrementalRows - did not produce a TEST-*.xml file (likely timed out) 
(batchId=193)
TestShutdownHook - did not produce a TEST-*.xml file (likely timed out) 
(batchId=193)
TestTableOutputFormat - did not produce a TEST-*.xml file (likely timed out) 
(batchId=193)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/12054/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12054/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12054/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 9 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12928876 - PreCommit-HIVE-Build

> Do Not Hide Exception in Hive Metastore Client Connection
> -
>
> Key: HIVE-18882
> URL: https://issues.apache.org/jira/browse/HIVE-18882
> Project: Hive
>  Issue Type: Improvement
>  Components: Standalone Metastore
>Affects Versions: 3.0.0
>Reporter: BELUGA BEHR
>Assignee: Manoj Narayanan
>Priority: Minor
>  Labels: noob
> Attachments: HIVE-18882.1.patch, HIVE-18882.patch
>
>
> [https://github.com/apache/hive/blob/4047befe48c8f762c58d8854e058385c1df151c6/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java#L526-L531]
>  
> {code:java}
> if (LOG.isDebugEnabled()) {
>   LOG.warn("Failed to connect to the MetaStore Server...", e);
> } else {
>   // Don't print full exception trace if DEBUG is not on.
>   LOG.warn("Failed to connect to the MetaStore Server...");
> }
> {code}
> I do not understand the logic here.  I always want to see the reason for the 
> failure. Otherwise, I do not know why it is failing unless I restart the 
> server with debug logging enabled.  By that point, the error may have 
> cleared.  Please just use the Exception in the WARN output without adding 
> confusing logging for debugging.  This is never an expected behavior... that 
> enabling debug would change a _warn_ level log message.
> Also... please remove the ellipsis, they add no value. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-18882) Do Not Hide Exception in Hive Metastore Client Connection

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-18882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521484#comment-16521484
 ] 

Hive QA commented on HIVE-18882:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
42s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
41s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
18s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m  
8s{color} | {color:blue} standalone-metastore in master has 228 extant Findbugs 
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  0m 
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  0m 
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  0m 
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  0m 
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  3m  
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
0s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
13s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 19m 16s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12054/dev-support/hive-personality.sh
 |
| git revision | master / 4b7f88a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore U: standalone-metastore |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-12054/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Do Not Hide Exception in Hive Metastore Client Connection
> -
>
> Key: HIVE-18882
> URL: https://issues.apache.org/jira/browse/HIVE-18882
> Project: Hive
>  Issue Type: Improvement
>  Components: Standalone Metastore
>Affects Versions: 3.0.0
>Reporter: BELUGA BEHR
>Assignee: Manoj Narayanan
>Priority: Minor
>  Labels: noob
> Attachments: HIVE-18882.1.patch, HIVE-18882.patch
>
>
> [https://github.com/apache/hive/blob/4047befe48c8f762c58d8854e058385c1df151c6/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java#L526-L531]
>  
> {code:java}
> if (LOG.isDebugEnabled()) {
>   LOG.warn("Failed to connect to the MetaStore Server...", e);
> } else {
>   // Don't print full exception trace if DEBUG is not on.
>   LOG.warn("Failed to connect to the MetaStore Server...");
> }
> {code}
> I do not understand the logic here.  I always want to see the reason for the 
> failure. Otherwise, I do not know why it is failing unless I restart the 
> server with debug logging enabled.  By that point, the error may have 
> cleared.  Please just use the Exception in the WARN output without adding 
> confusing logging for debugging.  This is never an expected behavior... that 
> enabling debug would change a _warn_ level log message.
> Also... please remove the ellipsis, they add no value. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19951) Vectorization: Need to disable encoded LLAP I/O for ORC when there is data type conversion (Schema Evolution)

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521479#comment-16521479
 ] 

Hive QA commented on HIVE-19951:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928862/HIVE-19951.03.patch

{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 14595 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[orc_ppd_schema_evol_3a]
 (batchId=151)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/12053/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12053/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12053/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12928862 - PreCommit-HIVE-Build

> Vectorization: Need to disable encoded LLAP I/O for ORC when there is data 
> type conversion  (Schema Evolution)
> --
>
> Key: HIVE-19951
> URL: https://issues.apache.org/jira/browse/HIVE-19951
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-19951.01.patch, HIVE-19951.02.patch, 
> HIVE-19951.03.patch
>
>
> Currently, reading encoded ORC data does not support data type conversion.  
> So, encoded reading and cache populating needs to be disabled.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19951) Vectorization: Need to disable encoded LLAP I/O for ORC when there is data type conversion (Schema Evolution)

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521470#comment-16521470
 ] 

Hive QA commented on HIVE-19951:


| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green}  0m  
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m 
40s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  7m 
 0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
23s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
48s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  0m 
40s{color} | {color:blue} llap-server in master has 83 extant Findbugs 
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue}  3m 
48s{color} | {color:blue} ql in master has 2282 extant Findbugs warnings. 
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue}  0m  
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green}  1m 
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green}  1m 
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green}  1m 
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green}  0m 
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green}  0m 
 0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green}  4m 
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green}  1m  
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green}  0m 
12s{color} | {color:green} The patch does not generate ASF License warnings. 
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m  2s{color} | 
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests |  asflicense  javac  javadoc  findbugs  checkstyle  compile  |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian 
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality | 
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-12053/dev-support/hive-personality.sh
 |
| git revision | master / 4b7f88a |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: itests llap-server ql U: . |
| Console output | 
http://104.198.109.242/logs//PreCommit-HIVE-Build-12053/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |


This message was automatically generated.



> Vectorization: Need to disable encoded LLAP I/O for ORC when there is data 
> type conversion  (Schema Evolution)
> --
>
> Key: HIVE-19951
> URL: https://issues.apache.org/jira/browse/HIVE-19951
> Project: Hive
>  Issue Type: Bug
>  Components: Hive
>Reporter: Matt McCline
>Assignee: Matt McCline
>Priority: Critical
> Attachments: HIVE-19951.01.patch, HIVE-19951.02.patch, 
> HIVE-19951.03.patch
>
>
> Currently, reading encoded ORC data does not support data type conversion.  
> So, encoded reading and cache populating needs to be disabled.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (HIVE-19972) Followup to HIVE-19928 : Fix the check for managed table

2018-06-24 Thread Hive QA (JIRA)


[ 
https://issues.apache.org/jira/browse/HIVE-19972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16521467#comment-16521467
 ] 

Hive QA commented on HIVE-19972:




Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12928906/HIVE-19972.2.patch

{color:red}ERROR:{color} -1 due to no test(s) being added or modified.

{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 14594 tests 
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_basic]
 (batchId=257)
org.apache.hadoop.hive.cli.TestTezPerfCliDriver.testCliDriver[query64] 
(batchId=257)
{noformat}

Test results: 
https://builds.apache.org/job/PreCommit-HIVE-Build/12052/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/12052/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-12052/

Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}

This message is automatically generated.

ATTACHMENT ID: 12928906 - PreCommit-HIVE-Build

> Followup to HIVE-19928 : Fix the check for managed table
> 
>
> Key: HIVE-19972
> URL: https://issues.apache.org/jira/browse/HIVE-19972
> Project: Hive
>  Issue Type: Bug
>Reporter: Deepak Jaiswal
>Assignee: Deepak Jaiswal
>Priority: Major
> Fix For: 3.1.0, 4.0.0
>
> Attachments: HIVE-19972.1.patch, HIVE-19972.2.patch
>
>
> The check for managed table should use ENUM comparison rather than string 
> comparison.
> The check in the patch will always return false, thus maintaining existing 
> behavior.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


  1   2   >