[jira] [Assigned] (HIVE-20318) NullPointerException when union with lateral view
[
https://issues.apache.org/jira/browse/HIVE-20318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryu Kobayashi reassigned HIVE-20318:
Assignee: Ryu Kobayashi
> NullPointerException when union with lateral view
> -
>
> Key: HIVE-20318
> URL: https://issues.apache.org/jira/browse/HIVE-20318
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 3.0.0, 2.3.2, 3.1.0, 3.1.1
> Environment: Run on MR, hadoop 2.7.3 and Tez
>Reporter: Wang Yan
>Assignee: Ryu Kobayashi
>Priority: Blocker
>
> The following sql throws NullPointerException.
> This sql is not table/data specific and can be run directly.
> {code:sql}
> WITH
> t1 AS (SELECT 0 AS c1),
> t2 AS (
> SELECT 0 AS c1
> FROM (
> SELECT COLLECT_SET('line') AS c2
>) t3 lateral VIEW explode(ARRAY("a")) er AS c3
> )
> SELECT c1
> FROM t1
> UNION ALL
> SELECT c1
> FROM t2
> {code}
> This is the exception.
> {code:java}
> 2018-04-20 01:53:50,845 WARN [Thread-5] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.RuntimeException: Hive Runtime Error
> while closing operators
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:207)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:3901)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:1020)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:697)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:189)
> ... 8 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20318) NullPointerException when union with lateral view
[
https://issues.apache.org/jira/browse/HIVE-20318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryu Kobayashi updated HIVE-20318:
-
Attachment: HIVE-20318.01.patch
Status: Patch Available (was: In Progress)
This problem is caused by HIVE-16227. The problem is the case of SELECT instead
of multi-insert as above.
> NullPointerException when union with lateral view
> -
>
> Key: HIVE-20318
> URL: https://issues.apache.org/jira/browse/HIVE-20318
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 3.1.1, 3.1.0, 2.3.2, 3.0.0
> Environment: Run on MR, hadoop 2.7.3 and Tez
>Reporter: Wang Yan
>Assignee: Ryu Kobayashi
>Priority: Blocker
> Attachments: HIVE-20318.01.patch
>
>
> The following sql throws NullPointerException.
> This sql is not table/data specific and can be run directly.
> {code:sql}
> WITH
> t1 AS (SELECT 0 AS c1),
> t2 AS (
> SELECT 0 AS c1
> FROM (
> SELECT COLLECT_SET('line') AS c2
>) t3 lateral VIEW explode(ARRAY("a")) er AS c3
> )
> SELECT c1
> FROM t1
> UNION ALL
> SELECT c1
> FROM t2
> {code}
> This is the exception.
> {code:java}
> 2018-04-20 01:53:50,845 WARN [Thread-5] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.RuntimeException: Hive Runtime Error
> while closing operators
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:207)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:3901)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:1020)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:697)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:189)
> ... 8 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Issue Comment Deleted] (HIVE-20318) NullPointerException when union with lateral view
[
https://issues.apache.org/jira/browse/HIVE-20318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryu Kobayashi updated HIVE-20318:
-
Comment: was deleted
(was: This problem is caused by HIVE-16227. The problem is the case of SELECT
instead of multi-insert as above.)
> NullPointerException when union with lateral view
> -
>
> Key: HIVE-20318
> URL: https://issues.apache.org/jira/browse/HIVE-20318
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 3.0.0, 2.3.2, 3.1.0, 3.1.1
> Environment: Run on MR, hadoop 2.7.3 and Tez
>Reporter: Wang Yan
>Assignee: Ryu Kobayashi
>Priority: Blocker
>
> The following sql throws NullPointerException.
> This sql is not table/data specific and can be run directly.
> {code:sql}
> WITH
> t1 AS (SELECT 0 AS c1),
> t2 AS (
> SELECT 0 AS c1
> FROM (
> SELECT COLLECT_SET('line') AS c2
>) t3 lateral VIEW explode(ARRAY("a")) er AS c3
> )
> SELECT c1
> FROM t1
> UNION ALL
> SELECT c1
> FROM t2
> {code}
> This is the exception.
> {code:java}
> 2018-04-20 01:53:50,845 WARN [Thread-5] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.RuntimeException: Hive Runtime Error
> while closing operators
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:207)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:3901)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:1020)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:697)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:189)
> ... 8 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work started] (HIVE-20318) NullPointerException when union with lateral view
[
https://issues.apache.org/jira/browse/HIVE-20318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-20318 started by Ryu Kobayashi.
> NullPointerException when union with lateral view
> -
>
> Key: HIVE-20318
> URL: https://issues.apache.org/jira/browse/HIVE-20318
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 3.0.0, 2.3.2, 3.1.0, 3.1.1
> Environment: Run on MR, hadoop 2.7.3 and Tez
>Reporter: Wang Yan
>Assignee: Ryu Kobayashi
>Priority: Blocker
>
> The following sql throws NullPointerException.
> This sql is not table/data specific and can be run directly.
> {code:sql}
> WITH
> t1 AS (SELECT 0 AS c1),
> t2 AS (
> SELECT 0 AS c1
> FROM (
> SELECT COLLECT_SET('line') AS c2
>) t3 lateral VIEW explode(ARRAY("a")) er AS c3
> )
> SELECT c1
> FROM t1
> UNION ALL
> SELECT c1
> FROM t2
> {code}
> This is the exception.
> {code:java}
> 2018-04-20 01:53:50,845 WARN [Thread-5] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.RuntimeException: Hive Runtime Error
> while closing operators
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:207)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:3901)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:1020)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:697)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:189)
> ... 8 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20318) NullPointerException when union with lateral view
[
https://issues.apache.org/jira/browse/HIVE-20318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryu Kobayashi updated HIVE-20318:
-
Attachment: (was: HIVE-20318.01.patch)
> NullPointerException when union with lateral view
> -
>
> Key: HIVE-20318
> URL: https://issues.apache.org/jira/browse/HIVE-20318
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 3.0.0, 2.3.2, 3.1.0, 3.1.1
> Environment: Run on MR, hadoop 2.7.3 and Tez
>Reporter: Wang Yan
>Assignee: Ryu Kobayashi
>Priority: Blocker
>
> The following sql throws NullPointerException.
> This sql is not table/data specific and can be run directly.
> {code:sql}
> WITH
> t1 AS (SELECT 0 AS c1),
> t2 AS (
> SELECT 0 AS c1
> FROM (
> SELECT COLLECT_SET('line') AS c2
>) t3 lateral VIEW explode(ARRAY("a")) er AS c3
> )
> SELECT c1
> FROM t1
> UNION ALL
> SELECT c1
> FROM t2
> {code}
> This is the exception.
> {code:java}
> 2018-04-20 01:53:50,845 WARN [Thread-5] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.RuntimeException: Hive Runtime Error
> while closing operators
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:207)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:3901)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:1020)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:697)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:189)
> ... 8 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20318) NullPointerException when union with lateral view
[
https://issues.apache.org/jira/browse/HIVE-20318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740112#comment-16740112
]
Ryu Kobayashi commented on HIVE-20318:
--
This problem is caused by HIVE-16227. The problem is the case of SELECT instead
of multi-insert as above.
> NullPointerException when union with lateral view
> -
>
> Key: HIVE-20318
> URL: https://issues.apache.org/jira/browse/HIVE-20318
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 3.0.0, 2.3.2, 3.1.0, 3.1.1
> Environment: Run on MR, hadoop 2.7.3 and Tez
>Reporter: Wang Yan
>Assignee: Ryu Kobayashi
>Priority: Blocker
> Attachments: HIVE-20318.01.patch
>
>
> The following sql throws NullPointerException.
> This sql is not table/data specific and can be run directly.
> {code:sql}
> WITH
> t1 AS (SELECT 0 AS c1),
> t2 AS (
> SELECT 0 AS c1
> FROM (
> SELECT COLLECT_SET('line') AS c2
>) t3 lateral VIEW explode(ARRAY("a")) er AS c3
> )
> SELECT c1
> FROM t1
> UNION ALL
> SELECT c1
> FROM t2
> {code}
> This is the exception.
> {code:java}
> 2018-04-20 01:53:50,845 WARN [Thread-5] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.RuntimeException: Hive Runtime Error
> while closing operators
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:207)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:3901)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:1020)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:697)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:189)
> ... 8 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20318) NullPointerException when union with lateral view
[
https://issues.apache.org/jira/browse/HIVE-20318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryu Kobayashi updated HIVE-20318:
-
Attachment: HIVE-20318.01.patch
> NullPointerException when union with lateral view
> -
>
> Key: HIVE-20318
> URL: https://issues.apache.org/jira/browse/HIVE-20318
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 3.0.0, 2.3.2, 3.1.0, 3.1.1
> Environment: Run on MR, hadoop 2.7.3 and Tez
>Reporter: Wang Yan
>Assignee: Ryu Kobayashi
>Priority: Blocker
> Attachments: HIVE-20318.01.patch
>
>
> The following sql throws NullPointerException.
> This sql is not table/data specific and can be run directly.
> {code:sql}
> WITH
> t1 AS (SELECT 0 AS c1),
> t2 AS (
> SELECT 0 AS c1
> FROM (
> SELECT COLLECT_SET('line') AS c2
>) t3 lateral VIEW explode(ARRAY("a")) er AS c3
> )
> SELECT c1
> FROM t1
> UNION ALL
> SELECT c1
> FROM t2
> {code}
> This is the exception.
> {code:java}
> 2018-04-20 01:53:50,845 WARN [Thread-5] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.RuntimeException: Hive Runtime Error
> while closing operators
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:207)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:3901)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:1020)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:697)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:189)
> ... 8 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[
https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740109#comment-16740109
]
Hive QA commented on HIVE-20801:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954529/HIVE-20801.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 15713 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15588/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15588/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15588/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954529 - PreCommit-HIVE-Build
> ACID: Allow DbTxnManager to ignore non-ACID table locking
> -
>
> Key: HIVE-20801
> URL: https://issues.apache.org/jira/browse/HIVE-20801
> Project: Hive
> Issue Type: Bug
> Components: Locking, Transactions
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Labels: Branch3Candidate, TODOC
> Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch,
> HIVE-20801.2.patch, HIVE-20801.3.patch
>
>
> Enabling ACIDv1 on a cluster produces a central locking bottleneck for all
> table types, which is not always the intention.
> The Hive locking for non-acid tables are advisory (i.e a client can
> write/read without locking), which means that the implementation does not
> offer strong consistency despite the lock manager consuming resources
> centrally.
> Disabling this lock acquisition would improve the performance of non-ACID
> tables co-existing with a globally configured DbTxnManager implementation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20318) NullPointerException when union with lateral view
[
https://issues.apache.org/jira/browse/HIVE-20318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryu Kobayashi updated HIVE-20318:
-
Affects Version/s: 3.0.0
3.1.0
3.1.1
> NullPointerException when union with lateral view
> -
>
> Key: HIVE-20318
> URL: https://issues.apache.org/jira/browse/HIVE-20318
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 3.0.0, 2.3.2, 3.1.0, 3.1.1
> Environment: Run on MR, hadoop 2.7.3
>Reporter: Wang Yan
>Priority: Blocker
>
> The following sql throws NullPointerException.
> This sql is not table/data specific and can be run directly.
> {code:sql}
> WITH
> t1 AS (SELECT 0 AS c1),
> t2 AS (
> SELECT 0 AS c1
> FROM (
> SELECT COLLECT_SET('line') AS c2
>) t3 lateral VIEW explode(ARRAY("a")) er AS c3
> )
> SELECT c1
> FROM t1
> UNION ALL
> SELECT c1
> FROM t2
> {code}
> This is the exception.
> {code:java}
> 2018-04-20 01:53:50,845 WARN [Thread-5] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.RuntimeException: Hive Runtime Error
> while closing operators
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:207)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:3901)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:1020)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:697)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:189)
> ... 8 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20318) NullPointerException when union with lateral view
[
https://issues.apache.org/jira/browse/HIVE-20318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ryu Kobayashi updated HIVE-20318:
-
Environment: Run on MR, hadoop 2.7.3 and Tez (was: Run on MR, hadoop 2.7.3)
> NullPointerException when union with lateral view
> -
>
> Key: HIVE-20318
> URL: https://issues.apache.org/jira/browse/HIVE-20318
> Project: Hive
> Issue Type: Bug
> Components: Query Processor
>Affects Versions: 3.0.0, 2.3.2, 3.1.0, 3.1.1
> Environment: Run on MR, hadoop 2.7.3 and Tez
>Reporter: Wang Yan
>Priority: Blocker
>
> The following sql throws NullPointerException.
> This sql is not table/data specific and can be run directly.
> {code:sql}
> WITH
> t1 AS (SELECT 0 AS c1),
> t2 AS (
> SELECT 0 AS c1
> FROM (
> SELECT COLLECT_SET('line') AS c2
>) t3 lateral VIEW explode(ARRAY("a")) er AS c3
> )
> SELECT c1
> FROM t1
> UNION ALL
> SELECT c1
> FROM t2
> {code}
> This is the exception.
> {code:java}
> 2018-04-20 01:53:50,845 WARN [Thread-5] org.apache.hadoop.mapred.YarnChild:
> Exception running child : java.lang.RuntimeException: Hive Runtime Error
> while closing operators
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:207)
> at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:61)
> at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:453)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:343)
> at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1698)
> at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.lang.NullPointerException
> at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:3901)
> at
> org.apache.hadoop.hive.ql.exec.FileSinkOperator.closeOp(FileSinkOperator.java:1020)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:697)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
> at org.apache.hadoop.hive.ql.exec.Operator.close(Operator.java:711)
> at org.apache.hadoop.hive.ql.exec.mr.ExecMapper.close(ExecMapper.java:189)
> ... 8 more
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[
https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740085#comment-16740085
]
Hive QA commented on HIVE-20801:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
31s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
16s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
38s{color} | {color:blue} ql in master has 2309 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
41s{color} | {color:red} ql: The patch generated 6 new + 542 unchanged - 0
fixed = 548 total (was 542) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 10s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15588/dev-support/hive-personality.sh
|
| git revision | master / f713140 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15588/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15588/yetus/whitespace-eol.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15588/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ACID: Allow DbTxnManager to ignore non-ACID table locking
> -
>
> Key: HIVE-20801
> URL: https://issues.apache.org/jira/browse/HIVE-20801
> Project: Hive
> Issue Type: Bug
> Components: Locking, Transactions
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Labels: Branch3Candidate, TODOC
> Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch,
> HIVE-20801.2.patch, HIVE-20801.3.patch
>
>
> Enabling ACIDv1 on a cluster produces a central locking bottleneck for all
> table types, which is not always the intention.
> The Hive locking for non-acid tables are advisory (i.e a client can
> write/read without locking), which means that the implementation does not
> offer strong consistency despite the lock
[jira] [Updated] (HIVE-21111) ConditionalTask cannot be cast to MapRedTask
[
https://issues.apache.org/jira/browse/HIVE-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhuwei updated HIVE-2:
--
Status: Patch Available (was: Open)
> ConditionalTask cannot be cast to MapRedTask
>
>
> Key: HIVE-2
> URL: https://issues.apache.org/jira/browse/HIVE-2
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Affects Versions: 2.3.4, 3.1.1, 2.1.1
>Reporter: zhuwei
>Assignee: zhuwei
>Priority: Major
> Attachments: HIVE-2.1.patch
>
>
> We met error like this in our product environment:
> java.lang.ClassCastException: org.apache.hadoop.hive.ql.exec.ConditionalTask
> cannot be cast to org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> at
> org.apache.hadoop.hive.ql.optimizer.physical.AbstractJoinTaskDispatcher.dispatch(AbstractJoinTaskDispatcher.java:173)
>
> There is a bug in function
> org.apache.hadoop.hive.ql.optimizer.physical.AbstractJoinTaskDispatcher.dispatch:
> if (tsk.isMapRedTask()) {
> Task newTask = this.processCurrentTask((MapRedTask)
> tsk,
> ((ConditionalTask) currTask), physicalContext.getContext());
> walkerCtx.addToDispatchList(newTask);
> }
> In the above code, when tsk is instance of ConditionalTask,
> tsk.isMapRedTask() still can be true, but it cannot be cast to MapRedTask.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21111) ConditionalTask cannot be cast to MapRedTask
[
https://issues.apache.org/jira/browse/HIVE-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhuwei updated HIVE-2:
--
Status: Open (was: Patch Available)
> ConditionalTask cannot be cast to MapRedTask
>
>
> Key: HIVE-2
> URL: https://issues.apache.org/jira/browse/HIVE-2
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Affects Versions: 2.3.4, 3.1.1, 2.1.1
>Reporter: zhuwei
>Assignee: zhuwei
>Priority: Major
> Attachments: HIVE-2.1.patch
>
>
> We met error like this in our product environment:
> java.lang.ClassCastException: org.apache.hadoop.hive.ql.exec.ConditionalTask
> cannot be cast to org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> at
> org.apache.hadoop.hive.ql.optimizer.physical.AbstractJoinTaskDispatcher.dispatch(AbstractJoinTaskDispatcher.java:173)
>
> There is a bug in function
> org.apache.hadoop.hive.ql.optimizer.physical.AbstractJoinTaskDispatcher.dispatch:
> if (tsk.isMapRedTask()) {
> Task newTask = this.processCurrentTask((MapRedTask)
> tsk,
> ((ConditionalTask) currTask), physicalContext.getContext());
> walkerCtx.addToDispatchList(newTask);
> }
> In the above code, when tsk is instance of ConditionalTask,
> tsk.isMapRedTask() still can be true, but it cannot be cast to MapRedTask.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[ https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-20801: --- Attachment: HIVE-20801.3.patch > ACID: Allow DbTxnManager to ignore non-ACID table locking > - > > Key: HIVE-20801 > URL: https://issues.apache.org/jira/browse/HIVE-20801 > Project: Hive > Issue Type: Bug > Components: Locking, Transactions >Affects Versions: 4.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Labels: Branch3Candidate, TODOC > Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch, > HIVE-20801.2.patch, HIVE-20801.3.patch > > > Enabling ACIDv1 on a cluster produces a central locking bottleneck for all > table types, which is not always the intention. > The Hive locking for non-acid tables are advisory (i.e a client can > write/read without locking), which means that the implementation does not > offer strong consistency despite the lock manager consuming resources > centrally. > Disabling this lock acquisition would improve the performance of non-ACID > tables co-existing with a globally configured DbTxnManager implementation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-20776) Run HMS filterHooks on server-side in addition to client-side
[
https://issues.apache.org/jira/browse/HIVE-20776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740062#comment-16740062
]
Hive QA commented on HIVE-20776:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954512/HIVE-20776.011.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15716 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_groupby_reduce]
(batchId=61)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15587/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15587/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15587/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954512 - PreCommit-HIVE-Build
> Run HMS filterHooks on server-side in addition to client-side
> -
>
> Key: HIVE-20776
> URL: https://issues.apache.org/jira/browse/HIVE-20776
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Karthik Manamcheri
>Assignee: Na Li
>Priority: Major
> Attachments: HIVE-20776.001.patch, HIVE-20776.003.patch,
> HIVE-20776.004.patch, HIVE-20776.005.patch, HIVE-20776.006.patch,
> HIVE-20776.007.patch, HIVE-20776.007.patch, HIVE-20776.008.patch,
> HIVE-20776.009.patch, HIVE-20776.009.patch, HIVE-20776.010.patch,
> HIVE-20776.011.patch, HIVE-20776.011.patch
>
>
> In HMS, I noticed that all the filter hooks are applied on the client side
> (in HiveMetaStoreClient.java). Is there any reason why we can't apply the
> filters on the server-side?
> Motivation: Some newer apache projects such as Kudu use HMS for metadata
> storage. Kudu is not completely Java-based and there are interaction points
> where they have C++ clients. In such cases, it would be ideal to have
> consistent behavior from HMS side as far as filters, etc are concerned.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21115) Add support for object versions in metastore
[
https://issues.apache.org/jira/browse/HIVE-21115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740045#comment-16740045
]
Vihang Karajgaonkar commented on HIVE-21115:
[~thejas] [~alangates] Do you have any thoughts or concerns with the above
approach? If there are easier ways to do this I would be happy to use them
instead of datanucleus based approach.
> Add support for object versions in metastore
>
>
> Key: HIVE-21115
> URL: https://issues.apache.org/jira/browse/HIVE-21115
> Project: Hive
> Issue Type: Improvement
>Reporter: Vihang Karajgaonkar
>Priority: Major
>
> Currently, metastore objects are identified uniquely by their names (eg.
> catName, dbName and tblName for a table is unique). Once a table or partition
> is created it could be altered in many ways. There is no good way currently
> to identify the version of the object once it is altered. For example,
> suppose there are two clients (Hive and Impala) using the same metastore.
> Once some alter operations are performed by a client, another client which
> wants to do a alter operation has no good way to know if the object which it
> has is the same as the one stored in metastore. Metastore updates the
> {{transient_lastDdlTime}} every time there is a DDL operation on the object.
> However, this value cannot be relied for all the clients since after
> HIVE-1768 metastore updates the value only when it is not set in the
> parameters. It is possible that a client which alters the object state, does
> not remove the {{transient_lastDdlTime}} and metastore will not update it.
> Secondly, if there is a clock skew between multiple HMS instances when HMS-HA
> is configured, time values cannot be relied on to find out the sequence of
> alter operations on a given object.
> This JIRA propose to use JDO versioning support by Datanucleus
> http://www.datanucleus.org/products/accessplatform_4_2/jdo/versioning.html to
> generate a incrementing sequence number every time a object is altered. The
> value of this object can be set as one of the values in the parameters. The
> advantage of using Datanucleus the versioning can be done across HMS
> instances as part of the database transaction and it should work for all the
> supported databases.
> In theory such a version can be used to detect if the client is presenting a
> object which is "stale" when issuing a alter request. Metastore can choose to
> reject such a alter request since the client may be caching a old version of
> the object and any alter operation on such stale object can potentially
> overwrite previous operations. However, this is can be done in a separate
> JIRA.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20776) Run HMS filterHooks on server-side in addition to client-side
[
https://issues.apache.org/jira/browse/HIVE-20776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740037#comment-16740037
]
Hive QA commented on HIVE-20776:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
13s{color} | {color:blue} standalone-metastore/metastore-common in master has
29 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
1s{color} | {color:blue} standalone-metastore/metastore-server in master has
188 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
56s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 20m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15587/dev-support/hive-personality.sh
|
| git revision | master / f713140 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore/metastore-common
standalone-metastore/metastore-server U: standalone-metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15587/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Run HMS filterHooks on server-side in addition to client-side
> -
>
> Key: HIVE-20776
> URL: https://issues.apache.org/jira/browse/HIVE-20776
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Karthik Manamcheri
>Assignee: Na Li
>Priority: Major
> Attachments: HIVE-20776.001.patch, HIVE-20776.003.patch,
> HIVE-20776.004.patch, HIVE-20776.005.patch, HIVE-20776.006.patch,
> HIVE-20776.007.patch, HIVE-20776.007.patch, HIVE-20776.008.patch,
> HIVE-20776.009.patch, HIVE-20776.009.patch, HIVE-20776.010.patch,
> HIVE-20776.011.patch, HIVE-20776.011.patch
>
>
> In HMS, I noticed that all the filter hooks are applied on the client side
> (in HiveMetaStoreClient.java). Is there any reason why we can't apply the
> filters on the server-side?
> Motivation: Some newer apache projects such as Kudu use HMS for metadata
> storage. Kudu is not completely Java-based and there are interaction points
> where
[jira] [Commented] (HIVE-17020) Aggressive RS dedup can incorrectly remove OP tree branch
[
https://issues.apache.org/jira/browse/HIVE-17020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16740019#comment-16740019
]
Hive QA commented on HIVE-17020:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954504/HIVE-17020.5.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 15712 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[implicit_cast_during_insert]
(batchId=55)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[merge3] (batchId=63)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[reducesink_dedup]
(batchId=165)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15586/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15586/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15586/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954504 - PreCommit-HIVE-Build
> Aggressive RS dedup can incorrectly remove OP tree branch
> -
>
> Key: HIVE-17020
> URL: https://issues.apache.org/jira/browse/HIVE-17020
> Project: Hive
> Issue Type: Bug
>Reporter: Rui Li
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-17020.1.patch, HIVE-17020.2.patch,
> HIVE-17020.3.patch, HIVE-17020.4.patch, HIVE-17020.5.patch
>
>
> Suppose we have an OP tree like this:
> {noformat}
> ...
> |
> RS[1]
> |
> SEL[2]
> /\
> SEL[3] SEL[4]
> | |
> RS[5] FS[6]
> |
> ...
> {noformat}
> When doing aggressive RS dedup, we'll remove all the operators between RS5
> and RS1, and thus the branch containing FS6 is lost.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-17020) Aggressive RS dedup can incorrectly remove OP tree branch
[
https://issues.apache.org/jira/browse/HIVE-17020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739994#comment-16739994
]
Hive QA commented on HIVE-17020:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
32s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
39s{color} | {color:blue} ql in master has 2309 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} ql: The patch generated 0 new + 10 unchanged - 1
fixed = 10 total (was 11) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
55s{color} | {color:red} ql generated 1 new + 2309 unchanged - 0 fixed = 2310
total (was 2309) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 12s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Dead store to numPartAndBuck in
org.apache.hadoop.hive.ql.optimizer.SortedDynPartitionOptimizer$SortedDynamicPartitionProc.getReduceSinkOp(List,
List, List, List, ArrayList, ArrayList, int, Operator, AcidUtils$Operation)
At
SortedDynPartitionOptimizer.java:org.apache.hadoop.hive.ql.optimizer.SortedDynPartitionOptimizer$SortedDynamicPartitionProc.getReduceSinkOp(List,
List, List, List, ArrayList, ArrayList, int, Operator, AcidUtils$Operation)
At SortedDynPartitionOptimizer.java:[line 493] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15586/dev-support/hive-personality.sh
|
| git revision | master / f713140 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15586/yetus/new-findbugs-ql.html
|
| modules | C: ql itests U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15586/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Aggressive RS dedup can incorrectly remove OP tree branch
> -
>
> Key: HIVE-17020
> URL: https://issues.apache.org/jira/browse/HIVE-17020
> Project: Hive
> Issue Type: Bug
>Reporter: Rui Li
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-17020.1.patch, HIVE-17020.2.patch,
> HIVE-17020.3.patch, HIVE-17020.4.patch, HIVE-17020.5.patch
>
>
> Suppose we have an OP tree like this:
> {noformat}
> ...
> |
> RS[1]
> |
> SEL[2]
> /\
>
[jira] [Commented] (HIVE-21077) Database and catalogs should have creation time
[
https://issues.apache.org/jira/browse/HIVE-21077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739982#comment-16739982
]
Hive QA commented on HIVE-21077:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954502/HIVE-21077.06.patch
{color:green}SUCCESS:{color} +1 due to 4 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 15716 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.llap.security.TestLlapSignerImpl.testSigning
(batchId=336)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15585/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15585/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15585/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954502 - PreCommit-HIVE-Build
> Database and catalogs should have creation time
> ---
>
> Key: HIVE-21077
> URL: https://issues.apache.org/jira/browse/HIVE-21077
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-21077.01.patch, HIVE-21077.02.patch,
> HIVE-21077.03.patch, HIVE-21077.04.patch, HIVE-21077.05.patch,
> HIVE-21077.06.patch
>
>
> Currently, database do not have creation time like we have for tables and
> partitions.
> {noformat}
> // namespace for tables
> struct Database {
> 1: string name,
> 2: string description,
> 3: string locationUri,
> 4: map parameters, // properties associated with the
> database
> 5: optional PrincipalPrivilegeSet privileges,
> 6: optional string ownerName,
> 7: optional PrincipalType ownerType,
> 8: optional string catalogName
> }
> {noformat}
> Currently, without creationTime there is no way to identify if the copy of
> Database which a client has is the same as the one on the server if the name
> is same. Without object ids creationTime value is the only way currently to
> identify uniquely a instance of metastore object. It would be good to have
> Database creation time as well.
> Same applies for catalogs as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20776) Run HMS filterHooks on server-side in addition to client-side
[ https://issues.apache.org/jira/browse/HIVE-20776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated HIVE-20776: - Attachment: HIVE-20776.011.patch > Run HMS filterHooks on server-side in addition to client-side > - > > Key: HIVE-20776 > URL: https://issues.apache.org/jira/browse/HIVE-20776 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Karthik Manamcheri >Assignee: Na Li >Priority: Major > Attachments: HIVE-20776.001.patch, HIVE-20776.003.patch, > HIVE-20776.004.patch, HIVE-20776.005.patch, HIVE-20776.006.patch, > HIVE-20776.007.patch, HIVE-20776.007.patch, HIVE-20776.008.patch, > HIVE-20776.009.patch, HIVE-20776.009.patch, HIVE-20776.010.patch, > HIVE-20776.011.patch, HIVE-20776.011.patch > > > In HMS, I noticed that all the filter hooks are applied on the client side > (in HiveMetaStoreClient.java). Is there any reason why we can't apply the > filters on the server-side? > Motivation: Some newer apache projects such as Kudu use HMS for metadata > storage. Kudu is not completely Java-based and there are interaction points > where they have C++ clients. In such cases, it would be ideal to have > consistent behavior from HMS side as far as filters, etc are concerned. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21077) Database and catalogs should have creation time
[
https://issues.apache.org/jira/browse/HIVE-21077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739958#comment-16739958
]
Hive QA commented on HIVE-21077:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
40s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
16s{color} | {color:blue} standalone-metastore/metastore-common in master has
29 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
4s{color} | {color:blue} standalone-metastore/metastore-server in master has
188 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 4 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
14s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 53s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15585/dev-support/hive-personality.sh
|
| git revision | master / f713140 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15585/yetus/whitespace-eol.txt
|
| modules | C: standalone-metastore/metastore-common
standalone-metastore/metastore-server U: standalone-metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15585/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Database and catalogs should have creation time
> ---
>
> Key: HIVE-21077
> URL: https://issues.apache.org/jira/browse/HIVE-21077
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-21077.01.patch, HIVE-21077.02.patch,
> HIVE-21077.03.patch, HIVE-21077.04.patch, HIVE-21077.05.patch,
> HIVE-21077.06.patch
>
>
> Currently, database do not have creation time like we have for tables and
> partitions.
> {noformat}
> // namespace for tables
> struct Database {
> 1: string name,
> 2: string description,
> 3: string locationUri,
> 4: map parameters, // properties associated with the
> database
> 5: optional PrincipalPrivilegeSet privileges,
>
[jira] [Commented] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[
https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739937#comment-16739937
]
Hive QA commented on HIVE-20801:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954488/HIVE-20801.2.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 15713 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[show_conf] (batchId=27)
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_basic]
(batchId=275)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15584/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15584/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15584/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954488 - PreCommit-HIVE-Build
> ACID: Allow DbTxnManager to ignore non-ACID table locking
> -
>
> Key: HIVE-20801
> URL: https://issues.apache.org/jira/browse/HIVE-20801
> Project: Hive
> Issue Type: Bug
> Components: Locking, Transactions
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Labels: Branch3Candidate, TODOC
> Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch,
> HIVE-20801.2.patch
>
>
> Enabling ACIDv1 on a cluster produces a central locking bottleneck for all
> table types, which is not always the intention.
> The Hive locking for non-acid tables are advisory (i.e a client can
> write/read without locking), which means that the implementation does not
> offer strong consistency despite the lock manager consuming resources
> centrally.
> Disabling this lock acquisition would improve the performance of non-ACID
> tables co-existing with a globally configured DbTxnManager implementation.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[
https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739917#comment-16739917
]
Hive QA commented on HIVE-20801:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
47s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
47s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
41s{color} | {color:blue} ql in master has 2309 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
5s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
26s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
45s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
17s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 6 new + 542 unchanged - 0
fixed = 548 total (was 542) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
7s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 4s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15584/dev-support/hive-personality.sh
|
| git revision | master / f713140 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15584/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15584/yetus/whitespace-eol.txt
|
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15584/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ACID: Allow DbTxnManager to ignore non-ACID table locking
> -
>
> Key: HIVE-20801
> URL: https://issues.apache.org/jira/browse/HIVE-20801
> Project: Hive
> Issue Type: Bug
> Components: Locking, Transactions
>Affects Versions: 4.0.0
>Reporter: Gopal V
>Assignee: Gopal V
>Priority: Major
> Labels: Branch3Candidate, TODOC
> Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch,
> HIVE-20801.2.patch
>
>
> Enabling ACIDv1 on a cluster produces a central locking bottleneck for all
> table types, which is not always the intention.
> The Hive locking for non-acid tables are advisory (i.e a client can
> write/read without locking), which means that the implementation does not
> offer strong consistency despite the lock manager consuming
[jira] [Updated] (HIVE-17020) Aggressive RS dedup can incorrectly remove OP tree branch
[
https://issues.apache.org/jira/browse/HIVE-17020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-17020:
---
Status: Open (was: Patch Available)
> Aggressive RS dedup can incorrectly remove OP tree branch
> -
>
> Key: HIVE-17020
> URL: https://issues.apache.org/jira/browse/HIVE-17020
> Project: Hive
> Issue Type: Bug
>Reporter: Rui Li
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-17020.1.patch, HIVE-17020.2.patch,
> HIVE-17020.3.patch, HIVE-17020.4.patch, HIVE-17020.5.patch
>
>
> Suppose we have an OP tree like this:
> {noformat}
> ...
> |
> RS[1]
> |
> SEL[2]
> /\
> SEL[3] SEL[4]
> | |
> RS[5] FS[6]
> |
> ...
> {noformat}
> When doing aggressive RS dedup, we'll remove all the operators between RS5
> and RS1, and thus the branch containing FS6 is lost.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20776) Run HMS filterHooks on server-side in addition to client-side
[
https://issues.apache.org/jira/browse/HIVE-20776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739906#comment-16739906
]
Hive QA commented on HIVE-20776:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954486/HIVE-20776.011.patch
{color:green}SUCCESS:{color} +1 due to 2 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 15700 tests
executed
*Failed tests:*
{noformat}
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=251)
TestReplAcidTablesWithJsonMessage - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15583/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15583/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15583/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954486 - PreCommit-HIVE-Build
> Run HMS filterHooks on server-side in addition to client-side
> -
>
> Key: HIVE-20776
> URL: https://issues.apache.org/jira/browse/HIVE-20776
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Karthik Manamcheri
>Assignee: Na Li
>Priority: Major
> Attachments: HIVE-20776.001.patch, HIVE-20776.003.patch,
> HIVE-20776.004.patch, HIVE-20776.005.patch, HIVE-20776.006.patch,
> HIVE-20776.007.patch, HIVE-20776.007.patch, HIVE-20776.008.patch,
> HIVE-20776.009.patch, HIVE-20776.009.patch, HIVE-20776.010.patch,
> HIVE-20776.011.patch
>
>
> In HMS, I noticed that all the filter hooks are applied on the client side
> (in HiveMetaStoreClient.java). Is there any reason why we can't apply the
> filters on the server-side?
> Motivation: Some newer apache projects such as Kudu use HMS for metadata
> storage. Kudu is not completely Java-based and there are interaction points
> where they have C++ clients. In such cases, it would be ideal to have
> consistent behavior from HMS side as far as filters, etc are concerned.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17020) Aggressive RS dedup can incorrectly remove OP tree branch
[
https://issues.apache.org/jira/browse/HIVE-17020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-17020:
---
Status: Patch Available (was: Open)
> Aggressive RS dedup can incorrectly remove OP tree branch
> -
>
> Key: HIVE-17020
> URL: https://issues.apache.org/jira/browse/HIVE-17020
> Project: Hive
> Issue Type: Bug
>Reporter: Rui Li
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-17020.1.patch, HIVE-17020.2.patch,
> HIVE-17020.3.patch, HIVE-17020.4.patch, HIVE-17020.5.patch
>
>
> Suppose we have an OP tree like this:
> {noformat}
> ...
> |
> RS[1]
> |
> SEL[2]
> /\
> SEL[3] SEL[4]
> | |
> RS[5] FS[6]
> |
> ...
> {noformat}
> When doing aggressive RS dedup, we'll remove all the operators between RS5
> and RS1, and thus the branch containing FS6 is lost.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-17020) Aggressive RS dedup can incorrectly remove OP tree branch
[
https://issues.apache.org/jira/browse/HIVE-17020?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vineet Garg updated HIVE-17020:
---
Attachment: HIVE-17020.5.patch
> Aggressive RS dedup can incorrectly remove OP tree branch
> -
>
> Key: HIVE-17020
> URL: https://issues.apache.org/jira/browse/HIVE-17020
> Project: Hive
> Issue Type: Bug
>Reporter: Rui Li
>Assignee: Vineet Garg
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-17020.1.patch, HIVE-17020.2.patch,
> HIVE-17020.3.patch, HIVE-17020.4.patch, HIVE-17020.5.patch
>
>
> Suppose we have an OP tree like this:
> {noformat}
> ...
> |
> RS[1]
> |
> SEL[2]
> /\
> SEL[3] SEL[4]
> | |
> RS[5] FS[6]
> |
> ...
> {noformat}
> When doing aggressive RS dedup, we'll remove all the operators between RS5
> and RS1, and thus the branch containing FS6 is lost.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21114) Create read-only transactions
[
https://issues.apache.org/jira/browse/HIVE-21114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-21114:
--
Description:
With HIVE-21036 we have a way to indicate that a txn is read only.
We should (at least in auto-commit mode) determine if the single stmt is a read
and mark the txn accordingly.
Then we can optimize {{TxnHandler.commitTxn()}} so that it doesn't do any
checks in write_set etc.
{{TxnHandler.commitTxn()}} already starts with {{lockTransactionRecord(stmt,
txnid, TXN_OPEN)}} so it can read the txn type in the same SQL stmt.
HiveOperation only has QUERY, which includes Insert and Select, so this
requires figuring out how to determine if a query is a SELECT. By the time
{{Driver.openTransaction();}} is called, we have already parsed the query so
there should be a way to know if the statement only reads.
For multi-stmt txns (once these are supported) we should allow user to indicate
that a txn is read-only and then not allow any statements that can make
modifications in this txn. This should be a different jira.
cc [~ikryvenko]
was:
With HIVE-21036 we have a way to indicate that a txn is read only.
We should (at least in auto-commit mode) determine if the single stmt is a read
and mark the txn accordingly.
Then we can optimize {{TxnHandler.commitTxn()}} so that it doesn't do any
checks in write_set etc.
HiveOperation only has QUERY, which includes Insert and Select, so this
requires figuring out how to determine if a query is a SELECT. By the time
{{Driver.openTransaction();}} is called, we have already parsed the query so
there should be a way to know if the statement only reads.
For multi-stmt txns (once these are supported) we should allow user to indicate
that a txn is read-only and then not allow any statements that can make
modifications in this txn. This should be a different jira.
cc [~ikryvenko]
> Create read-only transactions
> -
>
> Key: HIVE-21114
> URL: https://issues.apache.org/jira/browse/HIVE-21114
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 4.0.0
>Reporter: Eugene Koifman
>Priority: Major
>
> With HIVE-21036 we have a way to indicate that a txn is read only.
> We should (at least in auto-commit mode) determine if the single stmt is a
> read and mark the txn accordingly.
> Then we can optimize {{TxnHandler.commitTxn()}} so that it doesn't do any
> checks in write_set etc.
> {{TxnHandler.commitTxn()}} already starts with {{lockTransactionRecord(stmt,
> txnid, TXN_OPEN)}} so it can read the txn type in the same SQL stmt.
> HiveOperation only has QUERY, which includes Insert and Select, so this
> requires figuring out how to determine if a query is a SELECT. By the time
> {{Driver.openTransaction();}} is called, we have already parsed the query so
> there should be a way to know if the statement only reads.
> For multi-stmt txns (once these are supported) we should allow user to
> indicate that a txn is read-only and then not allow any statements that can
> make modifications in this txn. This should be a different jira.
> cc [~ikryvenko]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21077) Database and catalogs should have creation time
[
https://issues.apache.org/jira/browse/HIVE-21077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739885#comment-16739885
]
Vihang Karajgaonkar commented on HIVE-21077:
The last set of timeouts are most likely due to excessive time taken by
TestReplAcidTablesWithJsonMessage test. I have added it to skip batching.
> Database and catalogs should have creation time
> ---
>
> Key: HIVE-21077
> URL: https://issues.apache.org/jira/browse/HIVE-21077
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-21077.01.patch, HIVE-21077.02.patch,
> HIVE-21077.03.patch, HIVE-21077.04.patch, HIVE-21077.05.patch
>
>
> Currently, database do not have creation time like we have for tables and
> partitions.
> {noformat}
> // namespace for tables
> struct Database {
> 1: string name,
> 2: string description,
> 3: string locationUri,
> 4: map parameters, // properties associated with the
> database
> 5: optional PrincipalPrivilegeSet privileges,
> 6: optional string ownerName,
> 7: optional PrincipalType ownerType,
> 8: optional string catalogName
> }
> {noformat}
> Currently, without creationTime there is no way to identify if the copy of
> Database which a client has is the same as the one on the server if the name
> is same. Without object ids creationTime value is the only way currently to
> identify uniquely a instance of metastore object. It would be good to have
> Database creation time as well.
> Same applies for catalogs as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21077) Database and catalogs should have creation time
[
https://issues.apache.org/jira/browse/HIVE-21077?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vihang Karajgaonkar updated HIVE-21077:
---
Attachment: HIVE-21077.06.patch
> Database and catalogs should have creation time
> ---
>
> Key: HIVE-21077
> URL: https://issues.apache.org/jira/browse/HIVE-21077
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-21077.01.patch, HIVE-21077.02.patch,
> HIVE-21077.03.patch, HIVE-21077.04.patch, HIVE-21077.05.patch,
> HIVE-21077.06.patch
>
>
> Currently, database do not have creation time like we have for tables and
> partitions.
> {noformat}
> // namespace for tables
> struct Database {
> 1: string name,
> 2: string description,
> 3: string locationUri,
> 4: map parameters, // properties associated with the
> database
> 5: optional PrincipalPrivilegeSet privileges,
> 6: optional string ownerName,
> 7: optional PrincipalType ownerType,
> 8: optional string catalogName
> }
> {noformat}
> Currently, without creationTime there is no way to identify if the copy of
> Database which a client has is the same as the one on the server if the name
> is same. Without object ids creationTime value is the only way currently to
> identify uniquely a instance of metastore object. It would be good to have
> Database creation time as well.
> Same applies for catalogs as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21077) Database and catalogs should have creation time
[
https://issues.apache.org/jira/browse/HIVE-21077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739887#comment-16739887
]
Vihang Karajgaonkar commented on HIVE-21077:
Reattached the same patch as v5 to trigger tests after skipping
TestReplAcidTablesWithJsonMessage
> Database and catalogs should have creation time
> ---
>
> Key: HIVE-21077
> URL: https://issues.apache.org/jira/browse/HIVE-21077
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Vihang Karajgaonkar
>Assignee: Vihang Karajgaonkar
>Priority: Major
> Attachments: HIVE-21077.01.patch, HIVE-21077.02.patch,
> HIVE-21077.03.patch, HIVE-21077.04.patch, HIVE-21077.05.patch,
> HIVE-21077.06.patch
>
>
> Currently, database do not have creation time like we have for tables and
> partitions.
> {noformat}
> // namespace for tables
> struct Database {
> 1: string name,
> 2: string description,
> 3: string locationUri,
> 4: map parameters, // properties associated with the
> database
> 5: optional PrincipalPrivilegeSet privileges,
> 6: optional string ownerName,
> 7: optional PrincipalType ownerType,
> 8: optional string catalogName
> }
> {noformat}
> Currently, without creationTime there is no way to identify if the copy of
> Database which a client has is the same as the one on the server if the name
> is same. Without object ids creationTime value is the only way currently to
> identify uniquely a instance of metastore object. It would be good to have
> Database creation time as well.
> Same applies for catalogs as well.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21036) extend OpenTxnRequest with transaction type
[
https://issues.apache.org/jira/browse/HIVE-21036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Eugene Koifman updated HIVE-21036:
--
Resolution: Fixed
Fix Version/s: 4.0.0
Release Note: n/a
Status: Resolved (was: Patch Available)
committed to master
thanks Igor for the contribution
> extend OpenTxnRequest with transaction type
> ---
>
> Key: HIVE-21036
> URL: https://issues.apache.org/jira/browse/HIVE-21036
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 4.0.0
>Reporter: Eugene Koifman
>Assignee: Igor Kryvenko
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21036.01.patch, HIVE-21036.02.patch,
> HIVE-21036.03.patch, HIVE-21036.04.patch
>
>
> There is a {{TXN_TYPE}} field in {{TXNS}} table.
> There is {{TxnHandler.TxnType}} with legal values. It would be useful to
> TxnType a {{Thrift}}, add a new {{COMPACTION}} type object and allow setting
> it in {{OpenTxnRequest}}.
> Since HIVE-20823 compactor starts a txn and should set this.
> Down the road we may want to set READ_ONLY either based on parsing of the
> query or user input which can make {{TxnHandler.commitTxn}} faster.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-20776) Run HMS filterHooks on server-side in addition to client-side
[
https://issues.apache.org/jira/browse/HIVE-20776?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739875#comment-16739875
]
Hive QA commented on HIVE-20776:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
37s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
24s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
18s{color} | {color:blue} standalone-metastore/metastore-common in master has
29 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
2s{color} | {color:blue} standalone-metastore/metastore-server in master has
188 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
9s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
47s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 52s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15583/dev-support/hive-personality.sh
|
| git revision | master / dcc8950 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore/metastore-common
standalone-metastore/metastore-server U: standalone-metastore |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15583/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Run HMS filterHooks on server-side in addition to client-side
> -
>
> Key: HIVE-20776
> URL: https://issues.apache.org/jira/browse/HIVE-20776
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Reporter: Karthik Manamcheri
>Assignee: Na Li
>Priority: Major
> Attachments: HIVE-20776.001.patch, HIVE-20776.003.patch,
> HIVE-20776.004.patch, HIVE-20776.005.patch, HIVE-20776.006.patch,
> HIVE-20776.007.patch, HIVE-20776.007.patch, HIVE-20776.008.patch,
> HIVE-20776.009.patch, HIVE-20776.009.patch, HIVE-20776.010.patch,
> HIVE-20776.011.patch
>
>
> In HMS, I noticed that all the filter hooks are applied on the client side
> (in HiveMetaStoreClient.java). Is there any reason why we can't apply the
> filters on the server-side?
> Motivation: Some newer apache projects such as Kudu use HMS for metadata
> storage. Kudu is not completely Java-based and there are interaction points
> where they have C++
[jira] [Commented] (HIVE-20699) Query based compactor for full CRUD Acid tables
[
https://issues.apache.org/jira/browse/HIVE-20699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739856#comment-16739856
]
Hive QA commented on HIVE-20699:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954471/HIVE-20699.3.patch
{color:red}ERROR:{color} -1 due to build exiting with an error
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15582/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15582/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15582/
Messages:
{noformat}
This message was trimmed, see log for full details
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/eclipse/jetty/jetty-xml/9.3.25.v20180904/jetty-xml-9.3.25.v20180904.jar(org/eclipse/jetty/xml/XmlConfiguration.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/slf4j/jul-to-slf4j/1.7.10/jul-to-slf4j-1.7.10.jar(org/slf4j/bridge/SLF4JBridgeHandler.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar(javax/servlet/DispatcherType.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar(javax/servlet/Filter.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar(javax/servlet/FilterChain.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar(javax/servlet/FilterConfig.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar(javax/servlet/ServletException.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar(javax/servlet/ServletRequest.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar(javax/servlet/ServletResponse.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar(javax/servlet/annotation/WebFilter.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar(javax/servlet/http/HttpServletRequest.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar(javax/servlet/http/HttpServletResponse.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/classification/target/hive-classification-4.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/common/classification/InterfaceAudience$LimitedPrivate.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/classification/target/hive-classification-4.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/common/classification/InterfaceStability$Unstable.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/io/ByteArrayOutputStream.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/io/OutputStream.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/io/Closeable.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/lang/AutoCloseable.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/io/Flushable.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(javax/xml/bind/annotation/XmlRootElement.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/commons/commons-exec/1.1/commons-exec-1.1.jar(org/apache/commons/exec/ExecuteException.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/security/PrivilegedExceptionAction.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/util/concurrent/ExecutionException.class)]]
[loading
ZipFileIndexFileObject[/usr/lib/jvm/java-8-openjdk-amd64/jre/lib/rt.jar(java/util/concurrent/TimeoutException.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/maven/org/apache/hadoop/hadoop-common/3.1.0/hadoop-common-3.1.0.jar(org/apache/hadoop/fs/FileSystem.class)]]
[loading
ZipFileIndexFileObject[/data/hiveptest/working/apache-github-source-source/shims/common/target/hive-shims-common-4.0.0-SNAPSHOT.jar(org/apache/hadoop/hive/shims/HadoopShimsSecure.class)]]
[loading
[jira] [Commented] (HIVE-21036) extend OpenTxnRequest with transaction type
[
https://issues.apache.org/jira/browse/HIVE-21036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739854#comment-16739854
]
Hive QA commented on HIVE-21036:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954458/HIVE-21036.04.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 15697 tests
executed
*Failed tests:*
{noformat}
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=251)
TestReplAcidTablesWithJsonMessage - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15581/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15581/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15581/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954458 - PreCommit-HIVE-Build
> extend OpenTxnRequest with transaction type
> ---
>
> Key: HIVE-21036
> URL: https://issues.apache.org/jira/browse/HIVE-21036
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 4.0.0
>Reporter: Eugene Koifman
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-21036.01.patch, HIVE-21036.02.patch,
> HIVE-21036.03.patch, HIVE-21036.04.patch
>
>
> There is a {{TXN_TYPE}} field in {{TXNS}} table.
> There is {{TxnHandler.TxnType}} with legal values. It would be useful to
> TxnType a {{Thrift}}, add a new {{COMPACTION}} type object and allow setting
> it in {{OpenTxnRequest}}.
> Since HIVE-20823 compactor starts a txn and should set this.
> Down the road we may want to set READ_ONLY either based on parsing of the
> query or user input which can make {{TxnHandler.commitTxn}} faster.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21036) extend OpenTxnRequest with transaction type
[
https://issues.apache.org/jira/browse/HIVE-21036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739832#comment-16739832
]
Hive QA commented on HIVE-21036:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
46s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
23s{color} | {color:blue} standalone-metastore/metastore-common in master has
29 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
6s{color} | {color:blue} standalone-metastore/metastore-server in master has
188 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
45s{color} | {color:blue} ql in master has 2309 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
3s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 2m
14s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
51s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
51s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 6 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 7m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 2m
2s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 36m 26s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15581/dev-support/hive-personality.sh
|
| git revision | master / dcc8950 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15581/yetus/whitespace-eol.txt
|
| modules | C: standalone-metastore/metastore-common
standalone-metastore/metastore-server ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15581/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> extend OpenTxnRequest with transaction type
> ---
>
> Key: HIVE-21036
> URL: https://issues.apache.org/jira/browse/HIVE-21036
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 4.0.0
>Reporter: Eugene Koifman
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-21036.01.patch, HIVE-21036.02.patch,
> HIVE-21036.03.patch, HIVE-21036.04.patch
>
>
> There is a {{TXN_TYPE}} field in {{TXNS}} table.
> There is {{TxnHandler.TxnType}} with legal values. It would be useful to
> TxnType a {{Thrift}}, add a new {{COMPACTION}} type object and allow setting
> it in {{OpenTxnRequest}}.
> Since
[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called
[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739823#comment-16739823 ] Eugene Koifman commented on HIVE-21052: --- it would be useful to add some tests of partitioned tables, with > 1 partition column AcidUtils.list() - how does this work when there are very many files (which I think would be common here)? Should it use some form of RemoteFileIterator? e.g. FileUtils.RemoteIteratorWithFilter > Make sure transactions get cleaned if they are aborted before addPartitions > is called > - > > Key: HIVE-21052 > URL: https://issues.apache.org/jira/browse/HIVE-21052 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.1.1 >Reporter: Jaume M >Assignee: Jaume M >Priority: Critical > Attachments: Aborted Txn w_Direct Write.pdf, HIVE-21052.1.patch, > HIVE-21052.2.patch > > > If the transaction is aborted between openTxn and addPartitions and data has > been written on the table the transaction manager will think it's an empty > transaction and no cleaning will be done. > This is currently an issue in the streaming API and in micromanaged tables. > As proposed by [~ekoifman] this can be solved by: > * Writing an entry with a special marker to TXN_COMPONENTS at openTxn and > when addPartitions is called remove this entry from TXN_COMPONENTS and add > the corresponding partition entry to TXN_COMPONENTS. > * If the cleaner finds and entry with a special marker in TXN_COMPONENTS that > specifies that a transaction was opened and it was aborted it must generate > jobs for the worker for every possible partition available. > cc [~ewohlstadter] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21048) Remove needless org.mortbay.jetty from hadoop exclusions
[
https://issues.apache.org/jira/browse/HIVE-21048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739815#comment-16739815
]
Hive QA commented on HIVE-21048:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954437/HIVE-21048.08.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 52 failed/errored test(s), 15696 tests
executed
*Failed tests:*
{noformat}
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=251)
TestReplAcidTablesWithJsonMessage - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_avro]
(batchId=273)
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_basic]
(batchId=273)
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_csv]
(batchId=273)
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_delimited]
(batchId=273)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testAlterPartition
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testAlterTable
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testAlterTableCascade
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testAlterViewParititon
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testColumnStatistics
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testComplexTable
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testComplexTypeApi
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testConcurrentMetastores
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testCreateAndGetTableWithDriver
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testCreateTableSettingId
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testDBLocationChange
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testDBOwner
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testDBOwnerChange
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testDatabase
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testDatabaseLocation
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testDatabaseLocationWithPermissionProblems
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testDropDatabaseCascadeMVMultiDB
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testDropTable
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testFilterLastPartition
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testFilterSinglePartition
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testFunctionWithResources
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testGetConfigValue
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testGetMetastoreUuid
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testGetPartitionsWithSpec
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testGetSchemaWithNoClassDefFoundError
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testGetTableObjects
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testGetUUIDInParallel
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testJDOPersistanceManagerCleanup
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testListPartitionNames
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testListPartitions
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testListPartitionsWihtLimitEnabled
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testNameMethods
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testPartition
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testPartitionFilter
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testRenamePartition
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testRetriableClientWithConnLifetime
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testSimpleFunction
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testSimpleTable
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testSimpleTypeApi
(batchId=219)
org.apache.hadoop.hive.metastore.TestSetUGIOnOnlyClient.testStatsFastTrivial
[jira] [Comment Edited] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called
[
https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739799#comment-16739799
]
Eugene Koifman edited comment on HIVE-21052 at 1/10/19 9:52 PM:
I don't think testing the manually is sufficient.
See {{HiveConf.HIVETESTMODEROLLBACKTXN}},
{{HiveConf.HIVETESTMODEFAILCOMPACTION}} etc
You can add a similar one to inject a fault during a unit test to throw at the
start of {{TxnHandler.addDynamicPartitions()}} for example
Could you explain you overall strategy?
I would've thought {{TxnHandler.enqueueLockWithRetry()}} has all the relevant
info about whether it's a DP write or not.
Is it not possible to check if {{lc.getOperationType()}} is I/U/D and
{{lc.isIsDynamicPartitionWrite()}}, then get the (dbname,table name) from
LockComonent
and create the 'p' type entry in TXN_COMPONENTS. just make sure there is 1 per
table.
The same method has logic (via SQL query) find writeID associated with this
txnId.
Does this work?
was (Author: ekoifman):
I don't think testing the manually is sufficient.
See {{HiveConf.HIVETESTMODEROLLBACKTXN}},
{{HiveConf.HIVETESTMODEFAILCOMPACTION}} etc
You can add a similar one to inject a fault during a unit test to throw at the
start of {{TxnHandler.addDynamicPartitions()}} for example
> Make sure transactions get cleaned if they are aborted before addPartitions
> is called
> -
>
> Key: HIVE-21052
> URL: https://issues.apache.org/jira/browse/HIVE-21052
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.1.1
>Reporter: Jaume M
>Assignee: Jaume M
>Priority: Critical
> Attachments: Aborted Txn w_Direct Write.pdf, HIVE-21052.1.patch,
> HIVE-21052.2.patch
>
>
> If the transaction is aborted between openTxn and addPartitions and data has
> been written on the table the transaction manager will think it's an empty
> transaction and no cleaning will be done.
> This is currently an issue in the streaming API and in micromanaged tables.
> As proposed by [~ekoifman] this can be solved by:
> * Writing an entry with a special marker to TXN_COMPONENTS at openTxn and
> when addPartitions is called remove this entry from TXN_COMPONENTS and add
> the corresponding partition entry to TXN_COMPONENTS.
> * If the cleaner finds and entry with a special marker in TXN_COMPONENTS that
> specifies that a transaction was opened and it was aborted it must generate
> jobs for the worker for every possible partition available.
> cc [~ewohlstadter]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[ https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-20801: --- Attachment: HIVE-20801.2.patch > ACID: Allow DbTxnManager to ignore non-ACID table locking > - > > Key: HIVE-20801 > URL: https://issues.apache.org/jira/browse/HIVE-20801 > Project: Hive > Issue Type: Bug > Components: Locking, Transactions >Affects Versions: 4.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Labels: Branch3Candidate, TODOC > Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch, > HIVE-20801.2.patch > > > Enabling ACIDv1 on a cluster produces a central locking bottleneck for all > table types, which is not always the intention. > The Hive locking for non-acid tables are advisory (i.e a client can > write/read without locking), which means that the implementation does not > offer strong consistency despite the lock manager consuming resources > centrally. > Disabling this lock acquisition would improve the performance of non-ACID > tables co-existing with a globally configured DbTxnManager implementation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[ https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-20801: --- Status: Open (was: Patch Available) > ACID: Allow DbTxnManager to ignore non-ACID table locking > - > > Key: HIVE-20801 > URL: https://issues.apache.org/jira/browse/HIVE-20801 > Project: Hive > Issue Type: Bug > Components: Locking, Transactions >Affects Versions: 4.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Labels: Branch3Candidate, TODOC > Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch > > > Enabling ACIDv1 on a cluster produces a central locking bottleneck for all > table types, which is not always the intention. > The Hive locking for non-acid tables are advisory (i.e a client can > write/read without locking), which means that the implementation does not > offer strong consistency despite the lock manager consuming resources > centrally. > Disabling this lock acquisition would improve the performance of non-ACID > tables co-existing with a globally configured DbTxnManager implementation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21048) Remove needless org.mortbay.jetty from hadoop exclusions
[
https://issues.apache.org/jira/browse/HIVE-21048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739806#comment-16739806
]
Hive QA commented on HIVE-21048:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
32s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
29s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 9m
39s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
23s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 11m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 9m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 9m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
11s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 9m
36s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 60m 23s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15580/dev-support/hive-personality.sh
|
| git revision | master / dcc8950 |
| Default Java | 1.8.0_111 |
| modules | C: storage-api common llap-tez ql service jdbc hcatalog
hcatalog/core hcatalog/hcatalog-pig-adapter hcatalog/webhcat/svr .
itests/qtest-druid U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15580/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Remove needless org.mortbay.jetty from hadoop exclusions
>
>
> Key: HIVE-21048
> URL: https://issues.apache.org/jira/browse/HIVE-21048
> Project: Hive
> Issue Type: Bug
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Major
> Attachments: HIVE-21048.01.patch, HIVE-21048.02.patch,
> HIVE-21048.03.patch, HIVE-21048.04.patch, HIVE-21048.05.patch,
> HIVE-21048.06.patch, HIVE-21048.07.patch, HIVE-21048.08.patch,
> HIVE-21048.08.patch, dep.out
>
>
> During HIVE-20638 i found that org.mortbay.jetty exclusions from e.g. hadoop
> don't take effect, as the actual groupId of jetty is org.eclipse.jetty for
> most of the current projects, please find attachment (example for hive
> commons project).
> https://en.wikipedia.org/wiki/Jetty_(web_server)#History
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20801) ACID: Allow DbTxnManager to ignore non-ACID table locking
[ https://issues.apache.org/jira/browse/HIVE-20801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gopal V updated HIVE-20801: --- Status: Patch Available (was: Open) > ACID: Allow DbTxnManager to ignore non-ACID table locking > - > > Key: HIVE-20801 > URL: https://issues.apache.org/jira/browse/HIVE-20801 > Project: Hive > Issue Type: Bug > Components: Locking, Transactions >Affects Versions: 4.0.0 >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Labels: Branch3Candidate, TODOC > Attachments: HIVE-20801.1.patch, HIVE-20801.2.patch, > HIVE-20801.2.patch > > > Enabling ACIDv1 on a cluster produces a central locking bottleneck for all > table types, which is not always the intention. > The Hive locking for non-acid tables are advisory (i.e a client can > write/read without locking), which means that the implementation does not > offer strong consistency despite the lock manager consuming resources > centrally. > Disabling this lock acquisition would improve the performance of non-ACID > tables co-existing with a globally configured DbTxnManager implementation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called
[
https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739799#comment-16739799
]
Eugene Koifman commented on HIVE-21052:
---
I don't think testing the manually is sufficient.
See {{HiveConf.HIVETESTMODEROLLBACKTXN}},
{{HiveConf.HIVETESTMODEFAILCOMPACTION}} etc
You can add a similar one to inject a fault during a unit test to throw at the
start of {{TxnHandler.addDynamicPartitions()}} for example
> Make sure transactions get cleaned if they are aborted before addPartitions
> is called
> -
>
> Key: HIVE-21052
> URL: https://issues.apache.org/jira/browse/HIVE-21052
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 3.1.1
>Reporter: Jaume M
>Assignee: Jaume M
>Priority: Critical
> Attachments: Aborted Txn w_Direct Write.pdf, HIVE-21052.1.patch,
> HIVE-21052.2.patch
>
>
> If the transaction is aborted between openTxn and addPartitions and data has
> been written on the table the transaction manager will think it's an empty
> transaction and no cleaning will be done.
> This is currently an issue in the streaming API and in micromanaged tables.
> As proposed by [~ekoifman] this can be solved by:
> * Writing an entry with a special marker to TXN_COMPONENTS at openTxn and
> when addPartitions is called remove this entry from TXN_COMPONENTS and add
> the corresponding partition entry to TXN_COMPONENTS.
> * If the cleaner finds and entry with a special marker in TXN_COMPONENTS that
> specifies that a transaction was opened and it was aborted it must generate
> jobs for the worker for every possible partition available.
> cc [~ewohlstadter]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20776) Run HMS filterHooks on server-side in addition to client-side
[ https://issues.apache.org/jira/browse/HIVE-20776?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Na Li updated HIVE-20776: - Attachment: HIVE-20776.011.patch > Run HMS filterHooks on server-side in addition to client-side > - > > Key: HIVE-20776 > URL: https://issues.apache.org/jira/browse/HIVE-20776 > Project: Hive > Issue Type: Improvement > Components: Standalone Metastore >Reporter: Karthik Manamcheri >Assignee: Na Li >Priority: Major > Attachments: HIVE-20776.001.patch, HIVE-20776.003.patch, > HIVE-20776.004.patch, HIVE-20776.005.patch, HIVE-20776.006.patch, > HIVE-20776.007.patch, HIVE-20776.007.patch, HIVE-20776.008.patch, > HIVE-20776.009.patch, HIVE-20776.009.patch, HIVE-20776.010.patch, > HIVE-20776.011.patch > > > In HMS, I noticed that all the filter hooks are applied on the client side > (in HiveMetaStoreClient.java). Is there any reason why we can't apply the > filters on the server-side? > Motivation: Some newer apache projects such as Kudu use HMS for metadata > storage. Kudu is not completely Java-based and there are interaction points > where they have C++ clients. In such cases, it would be ideal to have > consistent behavior from HMS side as far as filters, etc are concerned. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21036) extend OpenTxnRequest with transaction type
[
https://issues.apache.org/jira/browse/HIVE-21036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739772#comment-16739772
]
Eugene Koifman commented on HIVE-21036:
---
+1 patch 4 pending tests
> extend OpenTxnRequest with transaction type
> ---
>
> Key: HIVE-21036
> URL: https://issues.apache.org/jira/browse/HIVE-21036
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 4.0.0
>Reporter: Eugene Koifman
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-21036.01.patch, HIVE-21036.02.patch,
> HIVE-21036.03.patch, HIVE-21036.04.patch
>
>
> There is a {{TXN_TYPE}} field in {{TXNS}} table.
> There is {{TxnHandler.TxnType}} with legal values. It would be useful to
> TxnType a {{Thrift}}, add a new {{COMPACTION}} type object and allow setting
> it in {{OpenTxnRequest}}.
> Since HIVE-20823 compactor starts a txn and should set this.
> Down the road we may want to set READ_ONLY either based on parsing of the
> query or user input which can make {{TxnHandler.commitTxn}} faster.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21095) 'Show create table' should not display a time zone for timestamp with local time zone
[
https://issues.apache.org/jira/browse/HIVE-21095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739727#comment-16739727
]
Hive QA commented on HIVE-21095:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
35s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
44s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
39s{color} | {color:blue} serde in master has 198 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
52s{color} | {color:blue} ql in master has 2309 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
40s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
9s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 27m 15s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15579/dev-support/hive-personality.sh
|
| git revision | master / dcc8950 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: serde ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15579/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> 'Show create table' should not display a time zone for timestamp with local
> time zone
> -
>
> Key: HIVE-21095
> URL: https://issues.apache.org/jira/browse/HIVE-21095
> Project: Hive
> Issue Type: Improvement
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-21095.1.patch, HIVE-21095.2.patch,
> HIVE-21095.2.patch, HIVE-21095.2.patch
>
>
> SHOW CREATE TABLE shows the time zone that the table was created in (if it
> contains a TIMESTAMPTZ column). This is also misleading, since it has nothing
> to do with the actual data or server or user time zone.
> e.g.
> {code:java}
> hive> set time zone America/Los_Angeles;
> hive> create table text_local (ts timestamp with local time zone) stored as
> textfile;
> hive> show create table text_local;
> CREATE TABLE `text_local`(
> `ts` timestamp with local time zone('America/Los_Angeles'))
> {code}
> should be:
> {code:java}
> hive> show create table text_local;
> CREATE TABLE `text_local`(
> `ts` timestamp with local time zone)
> {code}
> This was discussed in the community doc
[jira] [Commented] (HIVE-21095) 'Show create table' should not display a time zone for timestamp with local time zone
[
https://issues.apache.org/jira/browse/HIVE-21095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739759#comment-16739759
]
Hive QA commented on HIVE-21095:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954432/HIVE-21095.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 15696 tests
executed
*Failed tests:*
{noformat}
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=251)
TestReplAcidTablesWithJsonMessage - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15579/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15579/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15579/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954432 - PreCommit-HIVE-Build
> 'Show create table' should not display a time zone for timestamp with local
> time zone
> -
>
> Key: HIVE-21095
> URL: https://issues.apache.org/jira/browse/HIVE-21095
> Project: Hive
> Issue Type: Improvement
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-21095.1.patch, HIVE-21095.2.patch,
> HIVE-21095.2.patch, HIVE-21095.2.patch
>
>
> SHOW CREATE TABLE shows the time zone that the table was created in (if it
> contains a TIMESTAMPTZ column). This is also misleading, since it has nothing
> to do with the actual data or server or user time zone.
> e.g.
> {code:java}
> hive> set time zone America/Los_Angeles;
> hive> create table text_local (ts timestamp with local time zone) stored as
> textfile;
> hive> show create table text_local;
> CREATE TABLE `text_local`(
> `ts` timestamp with local time zone('America/Los_Angeles'))
> {code}
> should be:
> {code:java}
> hive> show create table text_local;
> CREATE TABLE `text_local`(
> `ts` timestamp with local time zone)
> {code}
> This was discussed in the community doc [Consistent timestamp types in Hadoop
> SQL
> engines|https://docs.google.com/document/d/1gNRww9mZJcHvUDCXklzjFEQGpefsuR_akCDfWsdE35Q/edit?usp=sharing]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20699) Query based compactor for full CRUD Acid tables
[
https://issues.apache.org/jira/browse/HIVE-20699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vaibhav Gumashta updated HIVE-20699:
Status: Patch Available (was: Open)
> Query based compactor for full CRUD Acid tables
> ---
>
> Key: HIVE-20699
> URL: https://issues.apache.org/jira/browse/HIVE-20699
> Project: Hive
> Issue Type: New Feature
> Components: Transactions
>Affects Versions: 3.1.0
>Reporter: Eugene Koifman
>Assignee: Vaibhav Gumashta
>Priority: Major
> Attachments: HIVE-20699.1.patch, HIVE-20699.1.patch,
> HIVE-20699.2.patch, HIVE-20699.3.patch
>
>
> Currently the Acid compactor is implemented as generated MR job
> ({{CompactorMR.java}}).
> It could also be expressed as a Hive query that reads from a given partition
> and writes data back to the same partition. This will merge the deltas and
> 'apply' the delete events. The simplest would be to just use Insert
> Overwrite but that will change all ROW__IDs which we don't want.
> Need to implement this in a way that preserves ROW__IDs and creates a new
> {{base_x}} directory to handle Major compaction.
> Minor compaction will be investigated separately.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20699) Query based compactor for full CRUD Acid tables
[
https://issues.apache.org/jira/browse/HIVE-20699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vaibhav Gumashta updated HIVE-20699:
Attachment: (was: HIVE-20699.3.patch)
> Query based compactor for full CRUD Acid tables
> ---
>
> Key: HIVE-20699
> URL: https://issues.apache.org/jira/browse/HIVE-20699
> Project: Hive
> Issue Type: New Feature
> Components: Transactions
>Affects Versions: 3.1.0
>Reporter: Eugene Koifman
>Assignee: Vaibhav Gumashta
>Priority: Major
> Attachments: HIVE-20699.1.patch, HIVE-20699.1.patch,
> HIVE-20699.2.patch, HIVE-20699.3.patch
>
>
> Currently the Acid compactor is implemented as generated MR job
> ({{CompactorMR.java}}).
> It could also be expressed as a Hive query that reads from a given partition
> and writes data back to the same partition. This will merge the deltas and
> 'apply' the delete events. The simplest would be to just use Insert
> Overwrite but that will change all ROW__IDs which we don't want.
> Need to implement this in a way that preserves ROW__IDs and creates a new
> {{base_x}} directory to handle Major compaction.
> Minor compaction will be investigated separately.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-20699) Query based compactor for full CRUD Acid tables
[
https://issues.apache.org/jira/browse/HIVE-20699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vaibhav Gumashta updated HIVE-20699:
Attachment: HIVE-20699.3.patch
> Query based compactor for full CRUD Acid tables
> ---
>
> Key: HIVE-20699
> URL: https://issues.apache.org/jira/browse/HIVE-20699
> Project: Hive
> Issue Type: New Feature
> Components: Transactions
>Affects Versions: 3.1.0
>Reporter: Eugene Koifman
>Assignee: Vaibhav Gumashta
>Priority: Major
> Attachments: HIVE-20699.1.patch, HIVE-20699.1.patch,
> HIVE-20699.2.patch, HIVE-20699.3.patch
>
>
> Currently the Acid compactor is implemented as generated MR job
> ({{CompactorMR.java}}).
> It could also be expressed as a Hive query that reads from a given partition
> and writes data back to the same partition. This will merge the deltas and
> 'apply' the delete events. The simplest would be to just use Insert
> Overwrite but that will change all ROW__IDs which we don't want.
> Need to implement this in a way that preserves ROW__IDs and creates a new
> {{base_x}} directory to handle Major compaction.
> Minor compaction will be investigated separately.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21052) Make sure transactions get cleaned if they are aborted before addPartitions is called
[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jaume M updated HIVE-21052: --- Summary: Make sure transactions get cleaned if they are aborted before addPartitions is called (was: Make sure transaction get cleaned if they are aborted before addPartitions is called) > Make sure transactions get cleaned if they are aborted before addPartitions > is called > - > > Key: HIVE-21052 > URL: https://issues.apache.org/jira/browse/HIVE-21052 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.1.1 >Reporter: Jaume M >Assignee: Jaume M >Priority: Critical > Attachments: Aborted Txn w_Direct Write.pdf, HIVE-21052.1.patch, > HIVE-21052.2.patch > > > If the transaction is aborted between openTxn and addPartitions and data has > been written on the table the transaction manager will think it's an empty > transaction and no cleaning will be done. > This is currently an issue in the streaming API and in micromanaged tables. > As proposed by [~ekoifman] this can be solved by: > * Writing an entry with a special marker to TXN_COMPONENTS at openTxn and > when addPartitions is called remove this entry from TXN_COMPONENTS and add > the corresponding partition entry to TXN_COMPONENTS. > * If the cleaner finds and entry with a special marker in TXN_COMPONENTS that > specifies that a transaction was opened and it was aborted it must generate > jobs for the worker for every possible partition available. > cc [~ewohlstadter] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21086) Fetch task and MR job give back different results when multiple partitions point to the same location
[
https://issues.apache.org/jira/browse/HIVE-21086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739700#comment-16739700
]
Hive QA commented on HIVE-21086:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954425/HIVE-21086.03.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 15696 tests
executed
*Failed tests:*
{noformat}
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=251)
TestReplAcidTablesWithJsonMessage - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
org.apache.hive.minikdc.TestJdbcWithMiniKdcSQLAuthHttp.testAuthorization1
(batchId=274)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15578/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15578/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15578/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954425 - PreCommit-HIVE-Build
> Fetch task and MR job give back different results when multiple partitions
> point to the same location
> -
>
> Key: HIVE-21086
> URL: https://issues.apache.org/jira/browse/HIVE-21086
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21086.01.patch, HIVE-21086.02.patch,
> HIVE-21086.03.patch
>
>
> Create a table with 2 partitions where both partitions share the same
> location and inserting a single line to one of them.
> {code:sql}
> create table test (i int) partitioned by (j int);
> alter table test add partition (j=1);
> alter table test add partition (j=2) location '<>';
> insert into table test partition (j=1) values (1);{code}
> Make sure *hive.fetch.task.conversion=more*.
> {code:sql}
> set hive.fetch.task.conversion=more;{code}
> The result of a simple select * test:
> +---+---+
> |test.i |test.j |
> +---+---+
> |1 |1 |
> |1 |2 |
> +---+---+
> Change the *hive.fetch.task.conversion=none;*
> {code:sql}
> set hive.fetch.task.conversion=none;
> {code}
> The result of a simple select * test:
> +---+---+
> |test.i |test.j |
> +---+---+
> |1 |2 |
> +---+---+
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21086) Fetch task and MR job give back different results when multiple partitions point to the same location
[
https://issues.apache.org/jira/browse/HIVE-21086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739658#comment-16739658
]
Hive QA commented on HIVE-21086:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
8s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
1s{color} | {color:blue} ql in master has 2309 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
42s{color} | {color:red} ql: The patch generated 6 new + 367 unchanged - 4
fixed = 373 total (was 371) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
57s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 28s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15578/dev-support/hive-personality.sh
|
| git revision | master / dcc8950 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15578/yetus/diff-checkstyle-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15578/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Fetch task and MR job give back different results when multiple partitions
> point to the same location
> -
>
> Key: HIVE-21086
> URL: https://issues.apache.org/jira/browse/HIVE-21086
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21086.01.patch, HIVE-21086.02.patch,
> HIVE-21086.03.patch
>
>
> Create a table with 2 partitions where both partitions share the same
> location and inserting a single line to one of them.
> {code:sql}
> create table test (i int) partitioned by (j int);
> alter table test add partition (j=1);
> alter table test add partition (j=2) location '<>';
> insert into table test partition (j=1) values (1);{code}
> Make sure *hive.fetch.task.conversion=more*.
> {code:sql}
> set hive.fetch.task.conversion=more;{code}
> The result of a simple select * test:
> +---+---+
> |test.i |test.j |
> +---+---+
> |1 |1 |
> |1 |2 |
> +---+---+
> Change the *hive.fetch.task.conversion=none;*
> {code:sql}
> set hive.fetch.task.conversion=none;
> {code}
> The result of a simple select * test:
> +---+---+
> |test.i |test.j |
> +---+---+
> |1 |2 |
> +---+---+
--
This message was
[jira] [Commented] (HIVE-21001) Upgrade to calcite-1.18
[
https://issues.apache.org/jira/browse/HIVE-21001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739642#comment-16739642
]
Hive QA commented on HIVE-21001:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954415/HIVE-21001.05.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 148 failed/errored test(s), 15696 tests
executed
*Failed tests:*
{noformat}
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=251)
TestReplAcidTablesWithJsonMessage - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[constprog_when_case]
(batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[fold_eq_with_case_when]
(batchId=88)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[fold_to_null]
(batchId=74)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[folder_predicate]
(batchId=5)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[in_typecheck_char]
(batchId=51)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[in_typecheck_mixed]
(batchId=6)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_dml_6]
(batchId=34)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_dml_7]
(batchId=59)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[list_bucket_dml_8]
(batchId=78)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[parquet_vectorization_6]
(batchId=44)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[pcs] (batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_join_filter]
(batchId=62)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[ppd_udf_case]
(batchId=47)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[rand_partitionpruner3]
(batchId=86)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamp_ints_casts]
(batchId=1)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[udf_isops_simplify]
(batchId=35)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[union_offcbo]
(batchId=49)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_date_1]
(batchId=23)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vector_decimal_math_funcs]
(batchId=25)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorization_6]
(batchId=29)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_casts]
(batchId=89)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_math_funcs]
(batchId=22)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_string_funcs]
(batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[vectorized_timestamp_ints_casts]
(batchId=53)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[windowing_duplicate]
(batchId=34)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_extractTime]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_floorTime]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_mv]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[auto_sortmerge_join_16]
(batchId=174)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[bucketpruning1]
(batchId=183)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[kryo]
(batchId=170)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[lineage3]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[results_cache_2]
(batchId=181)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_multi]
(batchId=163)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_notin]
(batchId=178)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[subquery_null_agg]
(batchId=178)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_case_when_1]
(batchId=182)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_date_1]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_decimal_math_funcs]
(batchId=164)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_interval_2]
(batchId=177)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_6]
(batchId=165)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorization_short_regress]
(batchId=172)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_casts]
(batchId=182)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_math_funcs]
(batchId=164)
[jira] [Commented] (HIVE-21036) extend OpenTxnRequest with transaction type
[
https://issues.apache.org/jira/browse/HIVE-21036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739623#comment-16739623
]
Igor Kryvenko commented on HIVE-21036:
--
I understood you literally about required and default.
I've updated my patch, please take another look.
> extend OpenTxnRequest with transaction type
> ---
>
> Key: HIVE-21036
> URL: https://issues.apache.org/jira/browse/HIVE-21036
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 4.0.0
>Reporter: Eugene Koifman
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-21036.01.patch, HIVE-21036.02.patch,
> HIVE-21036.03.patch, HIVE-21036.04.patch
>
>
> There is a {{TXN_TYPE}} field in {{TXNS}} table.
> There is {{TxnHandler.TxnType}} with legal values. It would be useful to
> TxnType a {{Thrift}}, add a new {{COMPACTION}} type object and allow setting
> it in {{OpenTxnRequest}}.
> Since HIVE-20823 compactor starts a txn and should set this.
> Down the road we may want to set READ_ONLY either based on parsing of the
> query or user input which can make {{TxnHandler.commitTxn}} faster.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21001) Upgrade to calcite-1.18
[
https://issues.apache.org/jira/browse/HIVE-21001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739627#comment-16739627
]
Hive QA commented on HIVE-21001:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
21s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
27s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
30s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
27s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 7m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 7m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 7m
48s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 48m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15577/dev-support/hive-personality.sh
|
| git revision | master / dcc8950 |
| Default Java | 1.8.0_111 |
| modules | C: ql accumulo-handler hbase-handler . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15577/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Upgrade to calcite-1.18
> ---
>
> Key: HIVE-21001
> URL: https://issues.apache.org/jira/browse/HIVE-21001
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21001.01.patch, HIVE-21001.01.patch,
> HIVE-21001.02.patch, HIVE-21001.03.patch, HIVE-21001.04.patch,
> HIVE-21001.05.patch
>
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21036) extend OpenTxnRequest with transaction type
[
https://issues.apache.org/jira/browse/HIVE-21036?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Igor Kryvenko updated HIVE-21036:
-
Attachment: HIVE-21036.04.patch
> extend OpenTxnRequest with transaction type
> ---
>
> Key: HIVE-21036
> URL: https://issues.apache.org/jira/browse/HIVE-21036
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 4.0.0
>Reporter: Eugene Koifman
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-21036.01.patch, HIVE-21036.02.patch,
> HIVE-21036.03.patch, HIVE-21036.04.patch
>
>
> There is a {{TXN_TYPE}} field in {{TXNS}} table.
> There is {{TxnHandler.TxnType}} with legal values. It would be useful to
> TxnType a {{Thrift}}, add a new {{COMPACTION}} type object and allow setting
> it in {{OpenTxnRequest}}.
> Since HIVE-20823 compactor starts a txn and should set this.
> Down the road we may want to set READ_ONLY either based on parsing of the
> query or user input which can make {{TxnHandler.commitTxn}} faster.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18884) Simplify Logging in Hive Metastore Client
[
https://issues.apache.org/jira/browse/HIVE-18884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739590#comment-16739590
]
Hive QA commented on HIVE-18884:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954411/HIVE.18884.07.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 4 failed/errored test(s), 15696 tests
executed
*Failed tests:*
{noformat}
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=251)
TestReplAcidTablesWithJsonMessage - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[mapjoin_memcheck]
(batchId=45)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15576/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15576/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15576/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 4 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954411 - PreCommit-HIVE-Build
> Simplify Logging in Hive Metastore Client
> -
>
> Key: HIVE-18884
> URL: https://issues.apache.org/jira/browse/HIVE-18884
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: Mani M
>Priority: Minor
> Labels: logging, newbie, noob
> Attachments: HIVE.18884.02.patch, HIVE.18884.03.patch,
> HIVE.18884.04.patch, HIVE.18884.05.patch, HIVE.18884.06.patch,
> HIVE.18884.07.patch, HIVE.18884.patch
>
>
> https://github.com/apache/hive/blob/4047befe48c8f762c58d8854e058385c1df151c6/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
> The current logging is:
> {code}
> 2018-02-26 07:02:44,883 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Trying to connect to metastore with URI
> thrift://host.company.com:9083
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Connected to metastore.
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Opened a connection to metastore, current connections: 2
> {code}
> Please simplify to something like:
> {code}
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Opened a connection to the Metastore Server (URI
> thrift://host.company.com:9083), current connections: 2
> ... or ...
> 2018-02-26 07:02:44,892 ERROR hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Failed to connect to the Metastore Server (URI
> thrift://host.company.com:9083)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21036) extend OpenTxnRequest with transaction type
[
https://issues.apache.org/jira/browse/HIVE-21036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739586#comment-16739586
]
Eugene Koifman commented on HIVE-21036:
---
I don't understand your explanation about the constructors.
I meant something like "{{optional TxnType txn_type = TxnType.DEFAULT}}" in
Thrift definition.
Apparently you can't have an item both required and with default.
But this way it's always set to the most general transaction type and places
like Worker supply a more specific one.
> extend OpenTxnRequest with transaction type
> ---
>
> Key: HIVE-21036
> URL: https://issues.apache.org/jira/browse/HIVE-21036
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 4.0.0
>Reporter: Eugene Koifman
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-21036.01.patch, HIVE-21036.02.patch,
> HIVE-21036.03.patch
>
>
> There is a {{TXN_TYPE}} field in {{TXNS}} table.
> There is {{TxnHandler.TxnType}} with legal values. It would be useful to
> TxnType a {{Thrift}}, add a new {{COMPACTION}} type object and allow setting
> it in {{OpenTxnRequest}}.
> Since HIVE-20823 compactor starts a txn and should set this.
> Down the road we may want to set READ_ONLY either based on parsing of the
> query or user input which can make {{TxnHandler.commitTxn}} faster.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21036) extend OpenTxnRequest with transaction type
[
https://issues.apache.org/jira/browse/HIVE-21036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739573#comment-16739573
]
Igor Kryvenko commented on HIVE-21036:
--
```
could {{optional TxnType txn_type,}} in {{OpenTxnRequest}} be made required
with the default being TxnType.DEFAULT? Seems like you wouldn't need
{{rqst.isSetTxn_type()}} all over in that case.
```
Actually, No. We can have only 2 c'tors: the first one without parameters, in
which we set the TxnType to DEFAULT, and the second one with all required
parameters. And in the case of 2 c'tor we need to manually set the correct
TxnType in all places in the code. Also, in some places, we can't determine
correct TxnType and it might confuse someone who will read this code. I propose
to create a follow-up Jira to investigate it.
> extend OpenTxnRequest with transaction type
> ---
>
> Key: HIVE-21036
> URL: https://issues.apache.org/jira/browse/HIVE-21036
> Project: Hive
> Issue Type: Bug
> Components: Transactions
>Affects Versions: 4.0.0
>Reporter: Eugene Koifman
>Assignee: Igor Kryvenko
>Priority: Major
> Attachments: HIVE-21036.01.patch, HIVE-21036.02.patch,
> HIVE-21036.03.patch
>
>
> There is a {{TXN_TYPE}} field in {{TXNS}} table.
> There is {{TxnHandler.TxnType}} with legal values. It would be useful to
> TxnType a {{Thrift}}, add a new {{COMPACTION}} type object and allow setting
> it in {{OpenTxnRequest}}.
> Since HIVE-20823 compactor starts a txn and should set this.
> Down the road we may want to set READ_ONLY either based on parsing of the
> query or user input which can make {{TxnHandler.commitTxn}} faster.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-18884) Simplify Logging in Hive Metastore Client
[
https://issues.apache.org/jira/browse/HIVE-18884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739523#comment-16739523
]
Hive QA commented on HIVE-18884:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
6s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
13s{color} | {color:blue} standalone-metastore/metastore-common in master has
29 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
51s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
29s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
6s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 16m 20s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15576/dev-support/hive-personality.sh
|
| git revision | master / dcc8950 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore/metastore-common U:
standalone-metastore/metastore-common |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15576/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Simplify Logging in Hive Metastore Client
> -
>
> Key: HIVE-18884
> URL: https://issues.apache.org/jira/browse/HIVE-18884
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: Mani M
>Priority: Minor
> Labels: logging, newbie, noob
> Attachments: HIVE.18884.02.patch, HIVE.18884.03.patch,
> HIVE.18884.04.patch, HIVE.18884.05.patch, HIVE.18884.06.patch,
> HIVE.18884.07.patch, HIVE.18884.patch
>
>
> https://github.com/apache/hive/blob/4047befe48c8f762c58d8854e058385c1df151c6/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
> The current logging is:
> {code}
> 2018-02-26 07:02:44,883 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Trying to connect to metastore with URI
> thrift://host.company.com:9083
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Connected to metastore.
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Opened a connection to metastore, current connections: 2
> {code}
> Please simplify to something like:
> {code}
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Opened a connection to the Metastore Server (URI
> thrift://host.company.com:9083), current connections: 2
> ... or ...
> 2018-02-26 07:02:44,892 ERROR hive.metastore:
[jira] [Commented] (HIVE-21099) Do Not Print StackTraces to STDERR in ConditionalResolverMergeFiles
[
https://issues.apache.org/jira/browse/HIVE-21099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739500#comment-16739500
]
Hive QA commented on HIVE-21099:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954409/HIVE.21099.02.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 5 failed/errored test(s), 15694 tests
executed
*Failed tests:*
{noformat}
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=251)
TestReplAcidTablesWithJsonMessage - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
TestReplicationScenariosIncrementalLoadAcidTables - did not produce a
TEST-*.xml file (likely timed out) (batchId=249)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
org.apache.hive.jdbc.TestSSL.testMetastoreWithSSL (batchId=258)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15575/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15575/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15575/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 5 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954409 - PreCommit-HIVE-Build
> Do Not Print StackTraces to STDERR in ConditionalResolverMergeFiles
> ---
>
> Key: HIVE-21099
> URL: https://issues.apache.org/jira/browse/HIVE-21099
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Affects Versions: 3.0.0, 4.0.0
>Reporter: Mani M
>Assignee: Mani M
>Priority: Minor
> Labels: logging, newbie, noob
> Attachments: HIVE.21099.02.patch, HIVE.21099.patch
>
>
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/plan/ConditionalResolverMergeFiles.java]#L193
>
> {code:java}
> } catch (IOException e){
> e.printStackTrace();
> }
> {code}
> This is inline with the HIVE-20159 to print the error using the SLF4J. in
> Method getTasks
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21048) Remove needless org.mortbay.jetty from hadoop exclusions
[ https://issues.apache.org/jira/browse/HIVE-21048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-21048: Attachment: HIVE-21048.08.patch > Remove needless org.mortbay.jetty from hadoop exclusions > > > Key: HIVE-21048 > URL: https://issues.apache.org/jira/browse/HIVE-21048 > Project: Hive > Issue Type: Bug >Reporter: Laszlo Bodor >Assignee: Laszlo Bodor >Priority: Major > Attachments: HIVE-21048.01.patch, HIVE-21048.02.patch, > HIVE-21048.03.patch, HIVE-21048.04.patch, HIVE-21048.05.patch, > HIVE-21048.06.patch, HIVE-21048.07.patch, HIVE-21048.08.patch, > HIVE-21048.08.patch, dep.out > > > During HIVE-20638 i found that org.mortbay.jetty exclusions from e.g. hadoop > don't take effect, as the actual groupId of jetty is org.eclipse.jetty for > most of the current projects, please find attachment (example for hive > commons project). > https://en.wikipedia.org/wiki/Jetty_(web_server)#History -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21095) 'Show create table' should not display a time zone for timestamp with local time zone
[
https://issues.apache.org/jira/browse/HIVE-21095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karen Coppage updated HIVE-21095:
-
Description:
SHOW CREATE TABLE shows the time zone that the table was created in (if it
contains a TIMESTAMPTZ column). This is also misleading, since it has nothing
to do with the actual data or server or user time zone.
e.g.
{code:java}
hive> set time zone America/Los_Angeles;
hive> create table text_local (ts timestamp with local time zone) stored as
textfile;
hive> show create table text_local;
CREATE TABLE `text_local`(
`ts` timestamp with local time zone('America/Los_Angeles'))
{code}
should be:
{code:java}
hive> show create table text_local;
CREATE TABLE `text_local`(
`ts` timestamp with local time zone)
{code}
This was discussed in the community doc [Consistent timestamp types in Hadoop
SQL
engines|https://docs.google.com/document/d/1gNRww9mZJcHvUDCXklzjFEQGpefsuR_akCDfWsdE35Q/edit?usp=sharing]
was:
SHOW CREATE TABLE shows the time zone that the table was created in (if it
contains a TIMESTAMPTZ column). This is also misleading, since it might have
nothing to do with the actual data.
e.g.
{code:java}
hive> set time zone America/Los_Angeles;
hive> create table text_local (ts timestamp with local time zone) stored as
textfile;
hive> show create table text_local;
CREATE TABLE `text_local`(
`ts` timestamp with local time zone('America/Los_Angeles'))
{code}
should be:
{code:java}
hive> show create table text_local;
CREATE TABLE `text_local`(
`ts` timestamp with local time zone)
{code}
This was discussed in the community doc [Consistent timestamp types in Hadoop
SQL
engines|https://docs.google.com/document/d/1gNRww9mZJcHvUDCXklzjFEQGpefsuR_akCDfWsdE35Q/edit?usp=sharing]
> 'Show create table' should not display a time zone for timestamp with local
> time zone
> -
>
> Key: HIVE-21095
> URL: https://issues.apache.org/jira/browse/HIVE-21095
> Project: Hive
> Issue Type: Improvement
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-21095.1.patch, HIVE-21095.2.patch,
> HIVE-21095.2.patch, HIVE-21095.2.patch
>
>
> SHOW CREATE TABLE shows the time zone that the table was created in (if it
> contains a TIMESTAMPTZ column). This is also misleading, since it has nothing
> to do with the actual data or server or user time zone.
> e.g.
> {code:java}
> hive> set time zone America/Los_Angeles;
> hive> create table text_local (ts timestamp with local time zone) stored as
> textfile;
> hive> show create table text_local;
> CREATE TABLE `text_local`(
> `ts` timestamp with local time zone('America/Los_Angeles'))
> {code}
> should be:
> {code:java}
> hive> show create table text_local;
> CREATE TABLE `text_local`(
> `ts` timestamp with local time zone)
> {code}
> This was discussed in the community doc [Consistent timestamp types in Hadoop
> SQL
> engines|https://docs.google.com/document/d/1gNRww9mZJcHvUDCXklzjFEQGpefsuR_akCDfWsdE35Q/edit?usp=sharing]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21095) 'Show create table' should not display a time zone for timestamp with local time zone
[
https://issues.apache.org/jira/browse/HIVE-21095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karen Coppage updated HIVE-21095:
-
Status: Open (was: Patch Available)
> 'Show create table' should not display a time zone for timestamp with local
> time zone
> -
>
> Key: HIVE-21095
> URL: https://issues.apache.org/jira/browse/HIVE-21095
> Project: Hive
> Issue Type: Improvement
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-21095.1.patch, HIVE-21095.2.patch,
> HIVE-21095.2.patch, HIVE-21095.2.patch
>
>
> SHOW CREATE TABLE shows the time zone that the table was created in (if it
> contains a TIMESTAMPTZ column). This is also misleading, since it might have
> nothing to do with the actual data.
> e.g.
> {code:java}
> hive> set time zone America/Los_Angeles;
> hive> create table text_local (ts timestamp with local time zone) stored as
> textfile;
> hive> show create table text_local;
> CREATE TABLE `text_local`(
> `ts` timestamp with local time zone('America/Los_Angeles'))
> {code}
> should be:
> {code:java}
> hive> show create table text_local;
> CREATE TABLE `text_local`(
> `ts` timestamp with local time zone)
> {code}
> This was discussed in the community doc [Consistent timestamp types in Hadoop
> SQL
> engines|https://docs.google.com/document/d/1gNRww9mZJcHvUDCXklzjFEQGpefsuR_akCDfWsdE35Q/edit?usp=sharing]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21095) 'Show create table' should not display a time zone for timestamp with local time zone
[
https://issues.apache.org/jira/browse/HIVE-21095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karen Coppage updated HIVE-21095:
-
Attachment: HIVE-21095.2.patch
Status: Patch Available (was: Open)
> 'Show create table' should not display a time zone for timestamp with local
> time zone
> -
>
> Key: HIVE-21095
> URL: https://issues.apache.org/jira/browse/HIVE-21095
> Project: Hive
> Issue Type: Improvement
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-21095.1.patch, HIVE-21095.2.patch,
> HIVE-21095.2.patch, HIVE-21095.2.patch
>
>
> SHOW CREATE TABLE shows the time zone that the table was created in (if it
> contains a TIMESTAMPTZ column). This is also misleading, since it might have
> nothing to do with the actual data.
> e.g.
> {code:java}
> hive> set time zone America/Los_Angeles;
> hive> create table text_local (ts timestamp with local time zone) stored as
> textfile;
> hive> show create table text_local;
> CREATE TABLE `text_local`(
> `ts` timestamp with local time zone('America/Los_Angeles'))
> {code}
> should be:
> {code:java}
> hive> show create table text_local;
> CREATE TABLE `text_local`(
> `ts` timestamp with local time zone)
> {code}
> This was discussed in the community doc [Consistent timestamp types in Hadoop
> SQL
> engines|https://docs.google.com/document/d/1gNRww9mZJcHvUDCXklzjFEQGpefsuR_akCDfWsdE35Q/edit?usp=sharing]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21095) 'Show create table' should not display a time zone for timestamp with local time zone
[
https://issues.apache.org/jira/browse/HIVE-21095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karen Coppage updated HIVE-21095:
-
Attachment: (was: HIVE-21095.1.patch)
> 'Show create table' should not display a time zone for timestamp with local
> time zone
> -
>
> Key: HIVE-21095
> URL: https://issues.apache.org/jira/browse/HIVE-21095
> Project: Hive
> Issue Type: Improvement
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-21095.1.patch, HIVE-21095.2.patch,
> HIVE-21095.2.patch, HIVE-21095.2.patch
>
>
> SHOW CREATE TABLE shows the time zone that the table was created in (if it
> contains a TIMESTAMPTZ column). This is also misleading, since it might have
> nothing to do with the actual data.
> e.g.
> {code:java}
> hive> set time zone America/Los_Angeles;
> hive> create table text_local (ts timestamp with local time zone) stored as
> textfile;
> hive> show create table text_local;
> CREATE TABLE `text_local`(
> `ts` timestamp with local time zone('America/Los_Angeles'))
> {code}
> should be:
> {code:java}
> hive> show create table text_local;
> CREATE TABLE `text_local`(
> `ts` timestamp with local time zone)
> {code}
> This was discussed in the community doc [Consistent timestamp types in Hadoop
> SQL
> engines|https://docs.google.com/document/d/1gNRww9mZJcHvUDCXklzjFEQGpefsuR_akCDfWsdE35Q/edit?usp=sharing]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21111) ConditionalTask cannot be cast to MapRedTask
[
https://issues.apache.org/jira/browse/HIVE-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhuwei updated HIVE-2:
--
Affects Version/s: 3.1.1
2.3.4
> ConditionalTask cannot be cast to MapRedTask
>
>
> Key: HIVE-2
> URL: https://issues.apache.org/jira/browse/HIVE-2
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Affects Versions: 2.1.1, 3.1.1, 2.3.4
>Reporter: zhuwei
>Assignee: zhuwei
>Priority: Major
> Attachments: HIVE-2.1.patch
>
>
> We met error like this in our product environment:
> java.lang.ClassCastException: org.apache.hadoop.hive.ql.exec.ConditionalTask
> cannot be cast to org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> at
> org.apache.hadoop.hive.ql.optimizer.physical.AbstractJoinTaskDispatcher.dispatch(AbstractJoinTaskDispatcher.java:173)
>
> There is a bug in function
> org.apache.hadoop.hive.ql.optimizer.physical.AbstractJoinTaskDispatcher.dispatch:
> if (tsk.isMapRedTask()) {
> Task newTask = this.processCurrentTask((MapRedTask)
> tsk,
> ((ConditionalTask) currTask), physicalContext.getContext());
> walkerCtx.addToDispatchList(newTask);
> }
> In the above code, when tsk is instance of ConditionalTask,
> tsk.isMapRedTask() still can be true, but it cannot be cast to MapRedTask.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21099) Do Not Print StackTraces to STDERR in ConditionalResolverMergeFiles
[
https://issues.apache.org/jira/browse/HIVE-21099?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739446#comment-16739446
]
Hive QA commented on HIVE-21099:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
33s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
39s{color} | {color:blue} ql in master has 2309 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
53s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 58s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15575/dev-support/hive-personality.sh
|
| git revision | master / dcc8950 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15575/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Do Not Print StackTraces to STDERR in ConditionalResolverMergeFiles
> ---
>
> Key: HIVE-21099
> URL: https://issues.apache.org/jira/browse/HIVE-21099
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Affects Versions: 3.0.0, 4.0.0
>Reporter: Mani M
>Assignee: Mani M
>Priority: Minor
> Labels: logging, newbie, noob
> Attachments: HIVE.21099.02.patch, HIVE.21099.patch
>
>
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/plan/ConditionalResolverMergeFiles.java]#L193
>
> {code:java}
> } catch (IOException e){
> e.printStackTrace();
> }
> {code}
> This is inline with the HIVE-20159 to print the error using the SLF4J. in
> Method getTasks
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21086) Fetch task and MR job give back different results when multiple partitions point to the same location
[
https://issues.apache.org/jira/browse/HIVE-21086?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Laszlo Pinter updated HIVE-21086:
-
Attachment: HIVE-21086.03.patch
> Fetch task and MR job give back different results when multiple partitions
> point to the same location
> -
>
> Key: HIVE-21086
> URL: https://issues.apache.org/jira/browse/HIVE-21086
> Project: Hive
> Issue Type: Bug
> Components: Hive
>Affects Versions: 4.0.0
>Reporter: Laszlo Pinter
>Assignee: Laszlo Pinter
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21086.01.patch, HIVE-21086.02.patch,
> HIVE-21086.03.patch
>
>
> Create a table with 2 partitions where both partitions share the same
> location and inserting a single line to one of them.
> {code:sql}
> create table test (i int) partitioned by (j int);
> alter table test add partition (j=1);
> alter table test add partition (j=2) location '<>';
> insert into table test partition (j=1) values (1);{code}
> Make sure *hive.fetch.task.conversion=more*.
> {code:sql}
> set hive.fetch.task.conversion=more;{code}
> The result of a simple select * test:
> +---+---+
> |test.i |test.j |
> +---+---+
> |1 |1 |
> |1 |2 |
> +---+---+
> Change the *hive.fetch.task.conversion=none;*
> {code:sql}
> set hive.fetch.task.conversion=none;
> {code}
> The result of a simple select * test:
> +---+---+
> |test.i |test.j |
> +---+---+
> |1 |2 |
> +---+---+
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21111) ConditionalTask cannot be cast to MapRedTask
[
https://issues.apache.org/jira/browse/HIVE-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739413#comment-16739413
]
Hive QA commented on HIVE-2:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954270/HIVE-2.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 15696 tests
executed
*Failed tests:*
{noformat}
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=251)
TestReplAcidTablesWithJsonMessage - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15574/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15574/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15574/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954270 - PreCommit-HIVE-Build
> ConditionalTask cannot be cast to MapRedTask
>
>
> Key: HIVE-2
> URL: https://issues.apache.org/jira/browse/HIVE-2
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Affects Versions: 2.1.1
>Reporter: zhuwei
>Assignee: zhuwei
>Priority: Major
> Attachments: HIVE-2.1.patch
>
>
> We met error like this in our product environment:
> java.lang.ClassCastException: org.apache.hadoop.hive.ql.exec.ConditionalTask
> cannot be cast to org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> at
> org.apache.hadoop.hive.ql.optimizer.physical.AbstractJoinTaskDispatcher.dispatch(AbstractJoinTaskDispatcher.java:173)
>
> There is a bug in function
> org.apache.hadoop.hive.ql.optimizer.physical.AbstractJoinTaskDispatcher.dispatch:
> if (tsk.isMapRedTask()) {
> Task newTask = this.processCurrentTask((MapRedTask)
> tsk,
> ((ConditionalTask) currTask), physicalContext.getContext());
> walkerCtx.addToDispatchList(newTask);
> }
> In the above code, when tsk is instance of ConditionalTask,
> tsk.isMapRedTask() still can be true, but it cannot be cast to MapRedTask.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21111) ConditionalTask cannot be cast to MapRedTask
[
https://issues.apache.org/jira/browse/HIVE-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739375#comment-16739375
]
Hive QA commented on HIVE-2:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
30s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
35s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
37s{color} | {color:blue} ql in master has 2309 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
52s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
59s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
50s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 21m 57s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15574/dev-support/hive-personality.sh
|
| git revision | master / dcc8950 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15574/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ConditionalTask cannot be cast to MapRedTask
>
>
> Key: HIVE-2
> URL: https://issues.apache.org/jira/browse/HIVE-2
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Affects Versions: 2.1.1
>Reporter: zhuwei
>Assignee: zhuwei
>Priority: Major
> Attachments: HIVE-2.1.patch
>
>
> We met error like this in our product environment:
> java.lang.ClassCastException: org.apache.hadoop.hive.ql.exec.ConditionalTask
> cannot be cast to org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> at
> org.apache.hadoop.hive.ql.optimizer.physical.AbstractJoinTaskDispatcher.dispatch(AbstractJoinTaskDispatcher.java:173)
>
> There is a bug in function
> org.apache.hadoop.hive.ql.optimizer.physical.AbstractJoinTaskDispatcher.dispatch:
> if (tsk.isMapRedTask()) {
> Task newTask = this.processCurrentTask((MapRedTask)
> tsk,
> ((ConditionalTask) currTask), physicalContext.getContext());
> walkerCtx.addToDispatchList(newTask);
> }
> In the above code, when tsk is instance of ConditionalTask,
> tsk.isMapRedTask() still can be true, but it cannot be cast to MapRedTask.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21094) Store TIMESTAMP WITH LOCAL TIME ZONE in UTC instead of writer's time zone
[
https://issues.apache.org/jira/browse/HIVE-21094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739359#comment-16739359
]
Hive QA commented on HIVE-21094:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954404/HIVE-21094.1.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 17 failed/errored test(s), 15696 tests
executed
*Failed tests:*
{noformat}
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=251)
TestReplAcidTablesWithJsonMessage - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[localtimezone2]
(batchId=62)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[localtimezone]
(batchId=57)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamptz] (batchId=69)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamptz_1]
(batchId=63)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[timestamptz_3]
(batchId=60)
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[windowing_windowspec]
(batchId=19)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druid_timestamptz2]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_dynamic_partition]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_expressions]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_floorTime]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test1]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test_insert]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test_ts]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[default_constraint]
(batchId=169)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15573/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15573/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15573/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 17 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954404 - PreCommit-HIVE-Build
> Store TIMESTAMP WITH LOCAL TIME ZONE in UTC instead of writer's time zone
> -
>
> Key: HIVE-21094
> URL: https://issues.apache.org/jira/browse/HIVE-21094
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-21094.1.patch, HIVE-21094.1.patch,
> HIVE-21094.1.patch, HIVE-21094.1.patch, HIVE-21094.1.patch
>
>
> TIMESTAMP WITH LOCAL TIME ZONE (aka TIMESTAMPTZ) is stored in writer's local
> time, and the writer's zone is stored with it. When reading, the timestamp in
> reader local time + reader zone is displayed. This is misleading for the
> user, since it looks like all the data was written in the reader's time zone.
> TIMESTAMPTZ should be stored in UTC time and be displayed in reader local
> time (as it was before) but should not display the reader's time zone.
> This was discussed in the community doc [Consistent timestamp types in Hadoop
> SQL
> engines|https://docs.google.com/document/d/1gNRww9mZJcHvUDCXklzjFEQGpefsuR_akCDfWsdE35Q/edit?usp=sharing]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21001) Upgrade to calcite-1.18
[ https://issues.apache.org/jira/browse/HIVE-21001?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21001: Attachment: HIVE-21001.05.patch > Upgrade to calcite-1.18 > --- > > Key: HIVE-21001 > URL: https://issues.apache.org/jira/browse/HIVE-21001 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21001.01.patch, HIVE-21001.01.patch, > HIVE-21001.02.patch, HIVE-21001.03.patch, HIVE-21001.04.patch, > HIVE-21001.05.patch > > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21052) Make sure transaction get cleaned if they are aborted before addPartitions is called
[ https://issues.apache.org/jira/browse/HIVE-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739330#comment-16739330 ] Jaume M commented on HIVE-21052: Can you review [~ekoifman]? : https://reviews.apache.org/r/69704/ > Make sure transaction get cleaned if they are aborted before addPartitions is > called > > > Key: HIVE-21052 > URL: https://issues.apache.org/jira/browse/HIVE-21052 > Project: Hive > Issue Type: Bug > Components: Transactions >Affects Versions: 3.1.1 >Reporter: Jaume M >Assignee: Jaume M >Priority: Critical > Attachments: Aborted Txn w_Direct Write.pdf, HIVE-21052.1.patch, > HIVE-21052.2.patch > > > If the transaction is aborted between openTxn and addPartitions and data has > been written on the table the transaction manager will think it's an empty > transaction and no cleaning will be done. > This is currently an issue in the streaming API and in micromanaged tables. > As proposed by [~ekoifman] this can be solved by: > * Writing an entry with a special marker to TXN_COMPONENTS at openTxn and > when addPartitions is called remove this entry from TXN_COMPONENTS and add > the corresponding partition entry to TXN_COMPONENTS. > * If the cleaner finds and entry with a special marker in TXN_COMPONENTS that > specifies that a transaction was opened and it was aborted it must generate > jobs for the worker for every possible partition available. > cc [~ewohlstadter] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21094) Store TIMESTAMP WITH LOCAL TIME ZONE in UTC instead of writer's time zone
[
https://issues.apache.org/jira/browse/HIVE-21094?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739318#comment-16739318
]
Hive QA commented on HIVE-21094:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
24s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
28s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
23s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
31s{color} | {color:blue} common in master has 65 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} serde in master has 198 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
22s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
33s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
11s{color} | {color:green} common: The patch generated 0 new + 5 unchanged - 2
fixed = 5 total (was 7) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} The patch serde passed checkstyle {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 1m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
26s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 47s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15573/dev-support/hive-personality.sh
|
| git revision | master / dcc8950 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15573/yetus/whitespace-eol.txt
|
| modules | C: common serde U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15573/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Store TIMESTAMP WITH LOCAL TIME ZONE in UTC instead of writer's time zone
> -
>
> Key: HIVE-21094
> URL: https://issues.apache.org/jira/browse/HIVE-21094
> Project: Hive
> Issue Type: Improvement
> Components: Hive
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-21094.1.patch, HIVE-21094.1.patch,
> HIVE-21094.1.patch, HIVE-21094.1.patch, HIVE-21094.1.patch
>
>
> TIMESTAMP WITH LOCAL TIME ZONE (aka TIMESTAMPTZ) is stored in writer's local
> time, and the writer's zone is stored with it. When reading, the timestamp in
> reader local time + reader zone is displayed. This is misleading for the
> user, since it looks like all the data was
[jira] [Commented] (HIVE-21095) 'Show create table' should not display a time zone for timestamp with local time zone
[
https://issues.apache.org/jira/browse/HIVE-21095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739305#comment-16739305
]
Hive QA commented on HIVE-21095:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954403/HIVE-21095.2.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 15696 tests
executed
*Failed tests:*
{noformat}
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=251)
TestReplAcidTablesWithJsonMessage - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15572/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15572/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15572/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954403 - PreCommit-HIVE-Build
> 'Show create table' should not display a time zone for timestamp with local
> time zone
> -
>
> Key: HIVE-21095
> URL: https://issues.apache.org/jira/browse/HIVE-21095
> Project: Hive
> Issue Type: Improvement
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-21095.1.patch, HIVE-21095.1.patch,
> HIVE-21095.2.patch, HIVE-21095.2.patch
>
>
> SHOW CREATE TABLE shows the time zone that the table was created in (if it
> contains a TIMESTAMPTZ column). This is also misleading, since it might have
> nothing to do with the actual data.
> e.g.
> {code:java}
> hive> set time zone America/Los_Angeles;
> hive> create table text_local (ts timestamp with local time zone) stored as
> textfile;
> hive> show create table text_local;
> CREATE TABLE `text_local`(
> `ts` timestamp with local time zone('America/Los_Angeles'))
> {code}
> should be:
> {code:java}
> hive> show create table text_local;
> CREATE TABLE `text_local`(
> `ts` timestamp with local time zone)
> {code}
> This was discussed in the community doc [Consistent timestamp types in Hadoop
> SQL
> engines|https://docs.google.com/document/d/1gNRww9mZJcHvUDCXklzjFEQGpefsuR_akCDfWsdE35Q/edit?usp=sharing]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18884) Simplify Logging in Hive Metastore Client
[
https://issues.apache.org/jira/browse/HIVE-18884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mani M updated HIVE-18884:
--
Attachment: HIVE.18884.07.patch
> Simplify Logging in Hive Metastore Client
> -
>
> Key: HIVE-18884
> URL: https://issues.apache.org/jira/browse/HIVE-18884
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: Mani M
>Priority: Minor
> Labels: logging, newbie, noob
> Attachments: HIVE.18884.02.patch, HIVE.18884.03.patch,
> HIVE.18884.04.patch, HIVE.18884.05.patch, HIVE.18884.06.patch,
> HIVE.18884.07.patch, HIVE.18884.patch
>
>
> https://github.com/apache/hive/blob/4047befe48c8f762c58d8854e058385c1df151c6/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
> The current logging is:
> {code}
> 2018-02-26 07:02:44,883 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Trying to connect to metastore with URI
> thrift://host.company.com:9083
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Connected to metastore.
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Opened a connection to metastore, current connections: 2
> {code}
> Please simplify to something like:
> {code}
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Opened a connection to the Metastore Server (URI
> thrift://host.company.com:9083), current connections: 2
> ... or ...
> 2018-02-26 07:02:44,892 ERROR hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Failed to connect to the Metastore Server (URI
> thrift://host.company.com:9083)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18884) Simplify Logging in Hive Metastore Client
[
https://issues.apache.org/jira/browse/HIVE-18884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mani M updated HIVE-18884:
--
Status: Patch Available (was: In Progress)
Applied same patch for retesting
> Simplify Logging in Hive Metastore Client
> -
>
> Key: HIVE-18884
> URL: https://issues.apache.org/jira/browse/HIVE-18884
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: Mani M
>Priority: Minor
> Labels: logging, newbie, noob
> Attachments: HIVE.18884.02.patch, HIVE.18884.03.patch,
> HIVE.18884.04.patch, HIVE.18884.05.patch, HIVE.18884.06.patch,
> HIVE.18884.07.patch, HIVE.18884.patch
>
>
> https://github.com/apache/hive/blob/4047befe48c8f762c58d8854e058385c1df151c6/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
> The current logging is:
> {code}
> 2018-02-26 07:02:44,883 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Trying to connect to metastore with URI
> thrift://host.company.com:9083
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Connected to metastore.
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Opened a connection to metastore, current connections: 2
> {code}
> Please simplify to something like:
> {code}
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Opened a connection to the Metastore Server (URI
> thrift://host.company.com:9083), current connections: 2
> ... or ...
> 2018-02-26 07:02:44,892 ERROR hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Failed to connect to the Metastore Server (URI
> thrift://host.company.com:9083)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-18884) Simplify Logging in Hive Metastore Client
[
https://issues.apache.org/jira/browse/HIVE-18884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mani M updated HIVE-18884:
--
Status: In Progress (was: Patch Available)
For retesting
> Simplify Logging in Hive Metastore Client
> -
>
> Key: HIVE-18884
> URL: https://issues.apache.org/jira/browse/HIVE-18884
> Project: Hive
> Issue Type: Improvement
> Components: Standalone Metastore
>Affects Versions: 3.0.0, 4.0.0
>Reporter: BELUGA BEHR
>Assignee: Mani M
>Priority: Minor
> Labels: logging, newbie, noob
> Attachments: HIVE.18884.02.patch, HIVE.18884.03.patch,
> HIVE.18884.04.patch, HIVE.18884.05.patch, HIVE.18884.06.patch,
> HIVE.18884.patch
>
>
> https://github.com/apache/hive/blob/4047befe48c8f762c58d8854e058385c1df151c6/standalone-metastore/src/main/java/org/apache/hadoop/hive/metastore/HiveMetaStoreClient.java
> The current logging is:
> {code}
> 2018-02-26 07:02:44,883 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Trying to connect to metastore with URI
> thrift://host.company.com:9083
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Connected to metastore.
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Opened a connection to metastore, current connections: 2
> {code}
> Please simplify to something like:
> {code}
> 2018-02-26 07:02:44,892 INFO hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Opened a connection to the Metastore Server (URI
> thrift://host.company.com:9083), current connections: 2
> ... or ...
> 2018-02-26 07:02:44,892 ERROR hive.metastore: [HiveServer2-Handler-Pool:
> Thread-65]: Failed to connect to the Metastore Server (URI
> thrift://host.company.com:9083)
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21099) Do Not Print StackTraces to STDERR in ConditionalResolverMergeFiles
[
https://issues.apache.org/jira/browse/HIVE-21099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mani M updated HIVE-21099:
--
Status: Patch Available (was: In Progress)
> Do Not Print StackTraces to STDERR in ConditionalResolverMergeFiles
> ---
>
> Key: HIVE-21099
> URL: https://issues.apache.org/jira/browse/HIVE-21099
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Affects Versions: 3.0.0, 4.0.0
>Reporter: Mani M
>Assignee: Mani M
>Priority: Minor
> Labels: logging, newbie, noob
> Attachments: HIVE.21099.02.patch, HIVE.21099.patch
>
>
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/plan/ConditionalResolverMergeFiles.java]#L193
>
> {code:java}
> } catch (IOException e){
> e.printStackTrace();
> }
> {code}
> This is inline with the HIVE-20159 to print the error using the SLF4J. in
> Method getTasks
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21099) Do Not Print StackTraces to STDERR in ConditionalResolverMergeFiles
[
https://issues.apache.org/jira/browse/HIVE-21099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mani M updated HIVE-21099:
--
Attachment: HIVE.21099.02.patch
> Do Not Print StackTraces to STDERR in ConditionalResolverMergeFiles
> ---
>
> Key: HIVE-21099
> URL: https://issues.apache.org/jira/browse/HIVE-21099
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Affects Versions: 3.0.0, 4.0.0
>Reporter: Mani M
>Assignee: Mani M
>Priority: Minor
> Labels: logging, newbie, noob
> Attachments: HIVE.21099.02.patch, HIVE.21099.patch
>
>
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/plan/ConditionalResolverMergeFiles.java]#L193
>
> {code:java}
> } catch (IOException e){
> e.printStackTrace();
> }
> {code}
> This is inline with the HIVE-20159 to print the error using the SLF4J. in
> Method getTasks
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21099) Do Not Print StackTraces to STDERR in ConditionalResolverMergeFiles
[
https://issues.apache.org/jira/browse/HIVE-21099?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mani M updated HIVE-21099:
--
Status: In Progress (was: Patch Available)
Check Style issue
> Do Not Print StackTraces to STDERR in ConditionalResolverMergeFiles
> ---
>
> Key: HIVE-21099
> URL: https://issues.apache.org/jira/browse/HIVE-21099
> Project: Hive
> Issue Type: Improvement
> Components: Query Planning
>Affects Versions: 3.0.0, 4.0.0
>Reporter: Mani M
>Assignee: Mani M
>Priority: Minor
> Labels: logging, newbie, noob
> Attachments: HIVE.21099.patch
>
>
> [https://github.com/apache/hive/blob/master/ql/src/java/org/apache/hadoop/hive/ql/plan/ConditionalResolverMergeFiles.java]#L193
>
> {code:java}
> } catch (IOException e){
> e.printStackTrace();
> }
> {code}
> This is inline with the HIVE-20159 to print the error using the SLF4J. in
> Method getTasks
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21095) 'Show create table' should not display a time zone for timestamp with local time zone
[
https://issues.apache.org/jira/browse/HIVE-21095?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739277#comment-16739277
]
Hive QA commented on HIVE-21095:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
39s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
36s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
17s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} serde in master has 198 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
43s{color} | {color:blue} ql in master has 2309 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
8s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
39s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
19s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
48s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
36s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 26m 32s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15572/dev-support/hive-personality.sh
|
| git revision | master / dcc8950 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: serde ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15572/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> 'Show create table' should not display a time zone for timestamp with local
> time zone
> -
>
> Key: HIVE-21095
> URL: https://issues.apache.org/jira/browse/HIVE-21095
> Project: Hive
> Issue Type: Improvement
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-21095.1.patch, HIVE-21095.1.patch,
> HIVE-21095.2.patch, HIVE-21095.2.patch
>
>
> SHOW CREATE TABLE shows the time zone that the table was created in (if it
> contains a TIMESTAMPTZ column). This is also misleading, since it might have
> nothing to do with the actual data.
> e.g.
> {code:java}
> hive> set time zone America/Los_Angeles;
> hive> create table text_local (ts timestamp with local time zone) stored as
> textfile;
> hive> show create table text_local;
> CREATE TABLE `text_local`(
> `ts` timestamp with local time zone('America/Los_Angeles'))
> {code}
> should be:
> {code:java}
> hive> show create table text_local;
> CREATE TABLE `text_local`(
> `ts` timestamp with local time zone)
> {code}
> This was discussed in the community doc [Consistent timestamp types
[jira] [Updated] (HIVE-21111) ConditionalTask cannot be cast to MapRedTask
[
https://issues.apache.org/jira/browse/HIVE-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
zhuwei updated HIVE-2:
--
Status: Patch Available (was: Open)
> ConditionalTask cannot be cast to MapRedTask
>
>
> Key: HIVE-2
> URL: https://issues.apache.org/jira/browse/HIVE-2
> Project: Hive
> Issue Type: Bug
> Components: Physical Optimizer
>Affects Versions: 2.1.1
>Reporter: zhuwei
>Assignee: zhuwei
>Priority: Major
> Attachments: HIVE-2.1.patch
>
>
> We met error like this in our product environment:
> java.lang.ClassCastException: org.apache.hadoop.hive.ql.exec.ConditionalTask
> cannot be cast to org.apache.hadoop.hive.ql.exec.mr.MapRedTask
> at
> org.apache.hadoop.hive.ql.optimizer.physical.AbstractJoinTaskDispatcher.dispatch(AbstractJoinTaskDispatcher.java:173)
>
> There is a bug in function
> org.apache.hadoop.hive.ql.optimizer.physical.AbstractJoinTaskDispatcher.dispatch:
> if (tsk.isMapRedTask()) {
> Task newTask = this.processCurrentTask((MapRedTask)
> tsk,
> ((ConditionalTask) currTask), physicalContext.getContext());
> walkerCtx.addToDispatchList(newTask);
> }
> In the above code, when tsk is instance of ConditionalTask,
> tsk.isMapRedTask() still can be true, but it cannot be cast to MapRedTask.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21113) For HPL/SQL that contains boolean expression with NOT, incorrect SQL may be generated.
[ https://issues.apache.org/jira/browse/HIVE-21113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739267#comment-16739267 ] Baoning He commented on HIVE-21113: --- [~daijy] Could you please review the patch? Thank you! > For HPL/SQL that contains boolean expression with NOT, incorrect SQL may be > generated. > --- > > Key: HIVE-21113 > URL: https://issues.apache.org/jira/browse/HIVE-21113 > Project: Hive > Issue Type: Bug > Components: hpl/sql >Reporter: Baoning He >Assignee: Baoning He >Priority: Major > Attachments: HIVE-21113.1.patch > > > In HPL/SQL, ' SELECT * FROM a WHERE NOT (1 = 2) ' will generate to incorrect > SQL ' SELECT * FROM a WHERE (1 = 2) ', the 'NOT' in boolean expression is > missing. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (HIVE-21113) For HPL/SQL that contains boolean expression with NOT, incorrect SQL may be generated.
[
https://issues.apache.org/jira/browse/HIVE-21113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739256#comment-16739256
]
Hive QA commented on HIVE-21113:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12954399/HIVE-21113.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 15696 tests
executed
*Failed tests:*
{noformat}
TestAlterTableMetadata - did not produce a TEST-*.xml file (likely timed out)
(batchId=251)
TestReplAcidTablesWithJsonMessage - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
TestSemanticAnalyzerHookLoading - did not produce a TEST-*.xml file (likely
timed out) (batchId=251)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/15571/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/15571/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-15571/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12954399 - PreCommit-HIVE-Build
> For HPL/SQL that contains boolean expression with NOT, incorrect SQL may be
> generated.
> ---
>
> Key: HIVE-21113
> URL: https://issues.apache.org/jira/browse/HIVE-21113
> Project: Hive
> Issue Type: Bug
> Components: hpl/sql
>Reporter: Baoning He
>Assignee: Baoning He
>Priority: Major
> Attachments: HIVE-21113.1.patch
>
>
> In HPL/SQL, ' SELECT * FROM a WHERE NOT (1 = 2) ' will generate to incorrect
> SQL ' SELECT * FROM a WHERE (1 = 2) ', the 'NOT' in boolean expression is
> missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Work stopped] (HIVE-21094) Store TIMESTAMP WITH LOCAL TIME ZONE in UTC instead of writer's time zone
[ https://issues.apache.org/jira/browse/HIVE-21094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-21094 stopped by Karen Coppage. > Store TIMESTAMP WITH LOCAL TIME ZONE in UTC instead of writer's time zone > - > > Key: HIVE-21094 > URL: https://issues.apache.org/jira/browse/HIVE-21094 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Attachments: HIVE-21094.1.patch, HIVE-21094.1.patch, > HIVE-21094.1.patch, HIVE-21094.1.patch, HIVE-21094.1.patch > > > TIMESTAMP WITH LOCAL TIME ZONE (aka TIMESTAMPTZ) is stored in writer's local > time, and the writer's zone is stored with it. When reading, the timestamp in > reader local time + reader zone is displayed. This is misleading for the > user, since it looks like all the data was written in the reader's time zone. > TIMESTAMPTZ should be stored in UTC time and be displayed in reader local > time (as it was before) but should not display the reader's time zone. > This was discussed in the community doc [Consistent timestamp types in Hadoop > SQL > engines|https://docs.google.com/document/d/1gNRww9mZJcHvUDCXklzjFEQGpefsuR_akCDfWsdE35Q/edit?usp=sharing] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21094) Store TIMESTAMP WITH LOCAL TIME ZONE in UTC instead of writer's time zone
[ https://issues.apache.org/jira/browse/HIVE-21094?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karen Coppage updated HIVE-21094: - Attachment: HIVE-21094.1.patch Status: Patch Available (was: Open) > Store TIMESTAMP WITH LOCAL TIME ZONE in UTC instead of writer's time zone > - > > Key: HIVE-21094 > URL: https://issues.apache.org/jira/browse/HIVE-21094 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Karen Coppage >Assignee: Karen Coppage >Priority: Major > Attachments: HIVE-21094.1.patch, HIVE-21094.1.patch, > HIVE-21094.1.patch, HIVE-21094.1.patch, HIVE-21094.1.patch > > > TIMESTAMP WITH LOCAL TIME ZONE (aka TIMESTAMPTZ) is stored in writer's local > time, and the writer's zone is stored with it. When reading, the timestamp in > reader local time + reader zone is displayed. This is misleading for the > user, since it looks like all the data was written in the reader's time zone. > TIMESTAMPTZ should be stored in UTC time and be displayed in reader local > time (as it was before) but should not display the reader's time zone. > This was discussed in the community doc [Consistent timestamp types in Hadoop > SQL > engines|https://docs.google.com/document/d/1gNRww9mZJcHvUDCXklzjFEQGpefsuR_akCDfWsdE35Q/edit?usp=sharing] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (HIVE-21095) 'Show create table' should not display a time zone for timestamp with local time zone
[
https://issues.apache.org/jira/browse/HIVE-21095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karen Coppage updated HIVE-21095:
-
Attachment: HIVE-21095.2.patch
Status: Patch Available (was: Open)
> 'Show create table' should not display a time zone for timestamp with local
> time zone
> -
>
> Key: HIVE-21095
> URL: https://issues.apache.org/jira/browse/HIVE-21095
> Project: Hive
> Issue Type: Improvement
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-21095.1.patch, HIVE-21095.1.patch,
> HIVE-21095.2.patch, HIVE-21095.2.patch
>
>
> SHOW CREATE TABLE shows the time zone that the table was created in (if it
> contains a TIMESTAMPTZ column). This is also misleading, since it might have
> nothing to do with the actual data.
> e.g.
> {code:java}
> hive> set time zone America/Los_Angeles;
> hive> create table text_local (ts timestamp with local time zone) stored as
> textfile;
> hive> show create table text_local;
> CREATE TABLE `text_local`(
> `ts` timestamp with local time zone('America/Los_Angeles'))
> {code}
> should be:
> {code:java}
> hive> show create table text_local;
> CREATE TABLE `text_local`(
> `ts` timestamp with local time zone)
> {code}
> This was discussed in the community doc [Consistent timestamp types in Hadoop
> SQL
> engines|https://docs.google.com/document/d/1gNRww9mZJcHvUDCXklzjFEQGpefsuR_akCDfWsdE35Q/edit?usp=sharing]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21095) 'Show create table' should not display a time zone for timestamp with local time zone
[
https://issues.apache.org/jira/browse/HIVE-21095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Karen Coppage updated HIVE-21095:
-
Status: Open (was: Patch Available)
> 'Show create table' should not display a time zone for timestamp with local
> time zone
> -
>
> Key: HIVE-21095
> URL: https://issues.apache.org/jira/browse/HIVE-21095
> Project: Hive
> Issue Type: Improvement
>Reporter: Karen Coppage
>Assignee: Karen Coppage
>Priority: Major
> Attachments: HIVE-21095.1.patch, HIVE-21095.1.patch,
> HIVE-21095.2.patch, HIVE-21095.2.patch
>
>
> SHOW CREATE TABLE shows the time zone that the table was created in (if it
> contains a TIMESTAMPTZ column). This is also misleading, since it might have
> nothing to do with the actual data.
> e.g.
> {code:java}
> hive> set time zone America/Los_Angeles;
> hive> create table text_local (ts timestamp with local time zone) stored as
> textfile;
> hive> show create table text_local;
> CREATE TABLE `text_local`(
> `ts` timestamp with local time zone('America/Los_Angeles'))
> {code}
> should be:
> {code:java}
> hive> show create table text_local;
> CREATE TABLE `text_local`(
> `ts` timestamp with local time zone)
> {code}
> This was discussed in the community doc [Consistent timestamp types in Hadoop
> SQL
> engines|https://docs.google.com/document/d/1gNRww9mZJcHvUDCXklzjFEQGpefsuR_akCDfWsdE35Q/edit?usp=sharing]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (HIVE-21113) For HPL/SQL that contains boolean expression with NOT, incorrect SQL may be generated.
[
https://issues.apache.org/jira/browse/HIVE-21113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16739210#comment-16739210
]
Hive QA commented on HIVE-21113:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
45s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
41s{color} | {color:blue} hplsql in master has 176 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
17s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
9s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 0m
46s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
16s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 12m 31s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.36-1+deb8u1 (2016-09-03) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-15571/dev-support/hive-personality.sh
|
| git revision | master / dcc8950 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: hplsql U: hplsql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-15571/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> For HPL/SQL that contains boolean expression with NOT, incorrect SQL may be
> generated.
> ---
>
> Key: HIVE-21113
> URL: https://issues.apache.org/jira/browse/HIVE-21113
> Project: Hive
> Issue Type: Bug
> Components: hpl/sql
>Reporter: Baoning He
>Assignee: Baoning He
>Priority: Major
> Attachments: HIVE-21113.1.patch
>
>
> In HPL/SQL, ' SELECT * FROM a WHERE NOT (1 = 2) ' will generate to incorrect
> SQL ' SELECT * FROM a WHERE (1 = 2) ', the 'NOT' in boolean expression is
> missing.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Updated] (HIVE-21113) For HPL/SQL that contains boolean expression with NOT, incorrect SQL may be generated.
[ https://issues.apache.org/jira/browse/HIVE-21113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Baoning He updated HIVE-21113: -- Attachment: HIVE-21113.1.patch Status: Patch Available (was: Open) > For HPL/SQL that contains boolean expression with NOT, incorrect SQL may be > generated. > --- > > Key: HIVE-21113 > URL: https://issues.apache.org/jira/browse/HIVE-21113 > Project: Hive > Issue Type: Bug > Components: hpl/sql >Reporter: Baoning He >Assignee: Baoning He >Priority: Major > Attachments: HIVE-21113.1.patch > > > In HPL/SQL, ' SELECT * FROM a WHERE NOT (1 = 2) ' will generate to incorrect > SQL ' SELECT * FROM a WHERE (1 = 2) ', the 'NOT' in boolean expression is > missing. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (HIVE-21113) For HPL/SQL that contains boolean expression with NOT, incorrect SQL may be generated.
[ https://issues.apache.org/jira/browse/HIVE-21113?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Baoning He reassigned HIVE-21113: - > For HPL/SQL that contains boolean expression with NOT, incorrect SQL may be > generated. > --- > > Key: HIVE-21113 > URL: https://issues.apache.org/jira/browse/HIVE-21113 > Project: Hive > Issue Type: Bug > Components: hpl/sql >Reporter: Baoning He >Assignee: Baoning He >Priority: Major > > In HPL/SQL, ' SELECT * FROM a WHERE NOT (1 = 2) ' will generate to incorrect > SQL ' SELECT * FROM a WHERE (1 = 2) ', the 'NOT' in boolean expression is > missing. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
