[jira] [Updated] (HIVE-22188) Clean up the SemanticAnalyzerFactory
[ https://issues.apache.org/jira/browse/HIVE-22188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22188: -- Attachment: HIVE-22188.01.patch > Clean up the SemanticAnalyzerFactory > > > Key: HIVE-22188 > URL: https://issues.apache.org/jira/browse/HIVE-22188 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-semanticanalyzer > Fix For: 4.0.0 > > Attachments: HIVE-22188.01.patch > > > SemanticAnalyzerFactory contains some huge maps that assign tokentypes to > hive operations. Some operations share a token type for doing them on a > table, or on a partition, which leads to some more overcomplication. > Two things need to be done: > # Each operation must have it's own token. > # The HiveOperation enum constants themselves must declare which token they > are related to, and not some external source should declare information about > them. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22188) Clean up the SemanticAnalyzerFactory
[ https://issues.apache.org/jira/browse/HIVE-22188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22188: -- Attachment: (was: HIVE-22188.01.patch) > Clean up the SemanticAnalyzerFactory > > > Key: HIVE-22188 > URL: https://issues.apache.org/jira/browse/HIVE-22188 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-semanticanalyzer > Fix For: 4.0.0 > > Attachments: HIVE-22188.01.patch > > > SemanticAnalyzerFactory contains some huge maps that assign tokentypes to > hive operations. Some operations share a token type for doing them on a > table, or on a partition, which leads to some more overcomplication. > Two things need to be done: > # Each operation must have it's own token. > # The HiveOperation enum constants themselves must declare which token they > are related to, and not some external source should declare information about > them. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22189) HMS Translation: Enforce strict locations for managed vs external tables.
[ https://issues.apache.org/jira/browse/HIVE-22189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam updated HIVE-22189: - Status: Patch Available (was: Open) > HMS Translation: Enforce strict locations for managed vs external tables. > - > > Key: HIVE-22189 > URL: https://issues.apache.org/jira/browse/HIVE-22189 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Major > Attachments: HIVE-22189.patch > > > Currently, HMS allows flexibility with location of a table. External tables > can be located within Hive managed warehouse space and managed tables can be > located within the external warehouse directory if the user chooses to do so. > There are certain advantages to restrict such flexibility. We could have > different encryption policies for different warehouses, different replication > policies etc. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22189) HMS Translation: Enforce strict locations for managed vs external tables.
[ https://issues.apache.org/jira/browse/HIVE-22189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam updated HIVE-22189: - Attachment: HIVE-22189.patch > HMS Translation: Enforce strict locations for managed vs external tables. > - > > Key: HIVE-22189 > URL: https://issues.apache.org/jira/browse/HIVE-22189 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Major > Attachments: HIVE-22189.patch > > > Currently, HMS allows flexibility with location of a table. External tables > can be located within Hive managed warehouse space and managed tables can be > located within the external warehouse directory if the user chooses to do so. > There are certain advantages to restrict such flexibility. We could have > different encryption policies for different warehouses, different replication > policies etc. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (HIVE-22189) HMS Translation: Enforce strict locations for managed vs external tables.
[ https://issues.apache.org/jira/browse/HIVE-22189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Naveen Gangam reassigned HIVE-22189: > HMS Translation: Enforce strict locations for managed vs external tables. > - > > Key: HIVE-22189 > URL: https://issues.apache.org/jira/browse/HIVE-22189 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Major > > Currently, HMS allows flexibility with location of a table. External tables > can be located within Hive managed warehouse space and managed tables can be > located within the external warehouse directory if the user chooses to do so. > There are certain advantages to restrict such flexibility. We could have > different encryption policies for different warehouses, different replication > policies etc. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[
https://issues.apache.org/jira/browse/HIVE-22170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16927197#comment-16927197
]
Hive QA commented on HIVE-22170:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12980002/HIVE-22170.04.patch
{color:green}SUCCESS:{color} +1 due to 23 test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16751 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18532/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18532/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18532/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12980002 - PreCommit-HIVE-Build
> from_unixtime and unix_timestamp should use user session time zone
> --
>
> Key: HIVE-22170
> URL: https://issues.apache.org/jira/browse/HIVE-22170
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0, 3.2.0, 3.1.1, 3.1.2
>Reporter: Riju Trivedi
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22170.01.patch, HIVE-22170.02.patch,
> HIVE-22170.03.patch, HIVE-22170.04.patch
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> According to documentation, that is the expected behavior (since session time
> zone was not present, system time zone was being used previously). This was
> incorrectly changed by HIVE-12192 / HIVE-20007. This JIRA should fix this
> issue.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[
https://issues.apache.org/jira/browse/HIVE-22170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16927180#comment-16927180
]
Hive QA commented on HIVE-22170:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
52s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 1m
12s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
48s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
0s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 1m
10s{color} | {color:red} ql: The patch generated 17 new + 4140 unchanged - 3
fixed = 4157 total (was 4143) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch has 2 line(s) that end in whitespace. Use git
apply --whitespace=fix <>. Refer https://git-scm.com/docs/git-apply
{color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
55s{color} | {color:red} ql generated 2 new + 2243 unchanged - 3 fixed = 2245
total (was 2246) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 6s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Class
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorUDFUnixTimeStampDate
defines non-transient non-serializable instance field date In
VectorUDFUnixTimeStampDate.java:instance field date In
VectorUDFUnixTimeStampDate.java |
| | Class
org.apache.hadoop.hive.ql.exec.vector.expressions.VectorUDFUnixTimeStampTimestamp
defines non-transient non-serializable instance field timestamp In
VectorUDFUnixTimeStampTimestamp.java:instance field timestamp In
VectorUDFUnixTimeStampTimestamp.java |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18532/dev-support/hive-personality.sh
|
| git revision | master / 1dfa2d8 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18532/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18532/yetus/whitespace-eol.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18532/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18532/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> from_unixtime and unix_timestamp should use user session time zone
> --
>
> Key: HIVE-22170
> URL: https://issues.apache.org/jira/browse/HIVE-22170
> Project: Hive
> Issue Type: Bug
>Affects Versions: 3.1.0, 4.0.0, 3.2.0, 3.1.1, 3.1.2
>Reporter: Riju Trivedi
>Assignee: Jesus Camacho Rodriguez
>Priority: Major
> Labels: pull-request-available
> Attachments: HIVE-22170.01.patch, HIVE-22170.02.patch,
> HIVE-22170.03.patch, HIVE-22170.04.patch
>
> Time Spent: 10m
> Remaining
[jira] [Commented] (HIVE-22184) Extract LockedDriverState from Driver
[
https://issues.apache.org/jira/browse/HIVE-22184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16927163#comment-16927163
]
Hive QA commented on HIVE-22184:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/1298/HIVE-22184.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16751 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestCliDriver.testCliDriver[updateBasicStats]
(batchId=69)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18531/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18531/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18531/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 1298 - PreCommit-HIVE-Build
> Extract LockedDriverState from Driver
> -
>
> Key: HIVE-22184
> URL: https://issues.apache.org/jira/browse/HIVE-22184
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Minor
> Labels: refactor-driver
> Attachments: HIVE-22184.01.patch
>
>
> LockedDriverState is a nested class within Driver, while it is used outside
> of it as well, and it is complex enough to be a class on it's own.
> DriverState should be it's nested class, and transitions / locking should be
> facilitated by functions within it.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22184) Extract LockedDriverState from Driver
[
https://issues.apache.org/jira/browse/HIVE-22184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16927137#comment-16927137
]
Hive QA commented on HIVE-22184:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
55s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
43s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
40s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
39s{color} | {color:red} ql: The patch generated 1 new + 348 unchanged - 4
fixed = 349 total (was 352) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 3m
53s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
54s{color} | {color:red} ql generated 1 new + 99 unchanged - 1 fixed = 100
total (was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 51s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18531/dev-support/hive-personality.sh
|
| git revision | master / 1dfa2d8 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18531/yetus/diff-checkstyle-ql.txt
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18531/yetus/diff-javadoc-javadoc-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18531/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Extract LockedDriverState from Driver
> -
>
> Key: HIVE-22184
> URL: https://issues.apache.org/jira/browse/HIVE-22184
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Minor
> Labels: refactor-driver
> Attachments: HIVE-22184.01.patch
>
>
> LockedDriverState is a nested class within Driver, while it is used outside
> of it as well, and it is complex enough to be a class on it's own.
> DriverState should be it's nested class, and transitions / locking should be
> facilitated by functions within it.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Work logged] (HIVE-22179) Break up DDLSemanticAnalyzer - extract Function related analyzers
[
https://issues.apache.org/jira/browse/HIVE-22179?focusedWorklogId=310192=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-310192
]
ASF GitHub Bot logged work on HIVE-22179:
-
Author: ASF GitHub Bot
Created on: 10/Sep/19 23:57
Start Date: 10/Sep/19 23:57
Worklog Time Spent: 10m
Work Description: jcamachor commented on pull request #763: HIVE-22179
Break up DDLSemanticAnalyzer - extract Function related analyzers
URL: https://github.com/apache/hive/pull/763#discussion_r323009350
##
File path:
ql/src/java/org/apache/hadoop/hive/ql/ddl/function/createfunction/CreateFunctionAnalyzer.java
##
@@ -0,0 +1,108 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.hadoop.hive.ql.ddl.function.createfunction;
+
+import java.util.ArrayList;
+import java.util.List;
+import java.util.Map;
+
+import org.apache.hadoop.hive.metastore.api.ResourceType;
+import org.apache.hadoop.hive.metastore.api.ResourceUri;
+import org.apache.hadoop.hive.ql.QueryState;
+import org.apache.hadoop.hive.ql.exec.FunctionUtils;
+import org.apache.hadoop.hive.ql.exec.TaskFactory;
+import org.apache.hadoop.hive.ql.ddl.DDLSemanticAnalyzerFactory.DDLType;
+import org.apache.hadoop.hive.ql.ddl.function.AbstractFunctionAnalyzer;
+import org.apache.hadoop.hive.ql.ddl.DDLWork;
+import org.apache.hadoop.hive.ql.parse.ASTNode;
+import org.apache.hadoop.hive.ql.parse.HiveParser;
+import org.apache.hadoop.hive.ql.parse.SemanticException;
+import org.apache.hadoop.hive.ql.plan.PlanUtils;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+
+import com.google.common.collect.ImmutableMap;
+
+/**
+ * Analyzer for function creation commands.
+ */
+@DDLType(type=HiveParser.TOK_CREATEFUNCTION)
+public class CreateFunctionAnalyzer extends AbstractFunctionAnalyzer {
+ private static final Logger SESISON_STATE_LOG =
LoggerFactory.getLogger("SessionState");
Review comment:
typo : `SESISON` -> `SESSION`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Issue Time Tracking
---
Worklog Id: (was: 310192)
Time Spent: 40m (was: 0.5h)
> Break up DDLSemanticAnalyzer - extract Function related analyzers
> -
>
> Key: HIVE-22179
> URL: https://issues.apache.org/jira/browse/HIVE-22179
> Project: Hive
> Issue Type: Sub-task
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: pull-request-available, refactor-ddl
> Attachments: HIVE-22179.01.patch
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is
> to refactor it in order to have everything cut into more handleable classes
> under the package org.apache.hadoop.hive.ql.exec.ddl:
> * have a separate class for each analyzers
> * have a package for each operation, containing an analyzer, a description,
> and an operation, so the amount of classes under a package is more manageable
> Step #2: extract all the functin related analyzers from DDLSemanticAnalyzer
> and FunctionSemanticAnalyzer and MacroSemanticAnalyzer, and move them under
> the new package.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Work logged] (HIVE-22179) Break up DDLSemanticAnalyzer - extract Function related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22179?focusedWorklogId=310191=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-310191 ] ASF GitHub Bot logged work on HIVE-22179: - Author: ASF GitHub Bot Created on: 10/Sep/19 23:57 Start Date: 10/Sep/19 23:57 Worklog Time Spent: 10m Work Description: jcamachor commented on pull request #763: HIVE-22179 Break up DDLSemanticAnalyzer - extract Function related analyzers URL: https://github.com/apache/hive/pull/763#discussion_r323008424 ## File path: ql/src/java/org/apache/hadoop/hive/ql/ddl/function/createfunction/CreateFunctionAnalyzer.java ## @@ -0,0 +1,108 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hadoop.hive.ql.ddl.function.createfunction; Review comment: `createfunction` -> `create` ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 310191) Time Spent: 0.5h (was: 20m) > Break up DDLSemanticAnalyzer - extract Function related analyzers > - > > Key: HIVE-22179 > URL: https://issues.apache.org/jira/browse/HIVE-22179 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22179.01.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #2: extract all the functin related analyzers from DDLSemanticAnalyzer > and FunctionSemanticAnalyzer and MacroSemanticAnalyzer, and move them under > the new package. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (HIVE-22179) Break up DDLSemanticAnalyzer - extract Function related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22179?focusedWorklogId=310195=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-310195 ] ASF GitHub Bot logged work on HIVE-22179: - Author: ASF GitHub Bot Created on: 10/Sep/19 23:57 Start Date: 10/Sep/19 23:57 Worklog Time Spent: 10m Work Description: jcamachor commented on pull request #763: HIVE-22179 Break up DDLSemanticAnalyzer - extract Function related analyzers URL: https://github.com/apache/hive/pull/763#discussion_r323009037 ## File path: ql/src/java/org/apache/hadoop/hive/ql/ddl/function/showfunctions/ShowFunctionsAnalyzer.java ## @@ -0,0 +1,59 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hadoop.hive.ql.ddl.function.showfunctions; Review comment: `showfunctions` -> `show` ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 310195) Time Spent: 1h (was: 50m) > Break up DDLSemanticAnalyzer - extract Function related analyzers > - > > Key: HIVE-22179 > URL: https://issues.apache.org/jira/browse/HIVE-22179 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22179.01.patch > > Time Spent: 1h > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #2: extract all the functin related analyzers from DDLSemanticAnalyzer > and FunctionSemanticAnalyzer and MacroSemanticAnalyzer, and move them under > the new package. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (HIVE-22179) Break up DDLSemanticAnalyzer - extract Function related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22179?focusedWorklogId=310197=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-310197 ] ASF GitHub Bot logged work on HIVE-22179: - Author: ASF GitHub Bot Created on: 10/Sep/19 23:57 Start Date: 10/Sep/19 23:57 Worklog Time Spent: 10m Work Description: jcamachor commented on pull request #763: HIVE-22179 Break up DDLSemanticAnalyzer - extract Function related analyzers URL: https://github.com/apache/hive/pull/763#discussion_r323008042 ## File path: ql/src/java/org/apache/hadoop/hive/ql/parse/repl/load/message/DropFunctionHandler.java ## @@ -19,7 +19,7 @@ import org.apache.hadoop.hive.metastore.messaging.DropFunctionMessage; import org.apache.hadoop.hive.ql.ddl.DDLWork; -import org.apache.hadoop.hive.ql.ddl.function.DropFunctionDesc; +import org.apache.hadoop.hive.ql.ddl.function.dropfunction.DropFunctionDesc; Review comment: `dropfunction` -> `drop` ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 310197) Time Spent: 1h (was: 50m) > Break up DDLSemanticAnalyzer - extract Function related analyzers > - > > Key: HIVE-22179 > URL: https://issues.apache.org/jira/browse/HIVE-22179 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22179.01.patch > > Time Spent: 1h > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #2: extract all the functin related analyzers from DDLSemanticAnalyzer > and FunctionSemanticAnalyzer and MacroSemanticAnalyzer, and move them under > the new package. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (HIVE-22179) Break up DDLSemanticAnalyzer - extract Function related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22179?focusedWorklogId=310189=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-310189 ] ASF GitHub Bot logged work on HIVE-22179: - Author: ASF GitHub Bot Created on: 10/Sep/19 23:57 Start Date: 10/Sep/19 23:57 Worklog Time Spent: 10m Work Description: jcamachor commented on pull request #763: HIVE-22179 Break up DDLSemanticAnalyzer - extract Function related analyzers URL: https://github.com/apache/hive/pull/763#discussion_r323008795 ## File path: ql/src/java/org/apache/hadoop/hive/ql/ddl/function/dropfunction/DropFunctionAnalyzer.java ## @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hadoop.hive.ql.ddl.function.dropfunction; Review comment: `dropfunction` -> `drop` ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 310189) Time Spent: 20m (was: 10m) > Break up DDLSemanticAnalyzer - extract Function related analyzers > - > > Key: HIVE-22179 > URL: https://issues.apache.org/jira/browse/HIVE-22179 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22179.01.patch > > Time Spent: 20m > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #2: extract all the functin related analyzers from DDLSemanticAnalyzer > and FunctionSemanticAnalyzer and MacroSemanticAnalyzer, and move them under > the new package. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (HIVE-22179) Break up DDLSemanticAnalyzer - extract Function related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22179?focusedWorklogId=310190=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-310190 ] ASF GitHub Bot logged work on HIVE-22179: - Author: ASF GitHub Bot Created on: 10/Sep/19 23:57 Start Date: 10/Sep/19 23:57 Worklog Time Spent: 10m Work Description: jcamachor commented on pull request #763: HIVE-22179 Break up DDLSemanticAnalyzer - extract Function related analyzers URL: https://github.com/apache/hive/pull/763#discussion_r323008871 ## File path: ql/src/java/org/apache/hadoop/hive/ql/ddl/function/dropmacro/DropMacroAnalyzer.java ## @@ -0,0 +1,67 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hadoop.hive.ql.ddl.function.dropmacro; Review comment: `dropmacro` -> `macro.drop` ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 310190) Time Spent: 0.5h (was: 20m) > Break up DDLSemanticAnalyzer - extract Function related analyzers > - > > Key: HIVE-22179 > URL: https://issues.apache.org/jira/browse/HIVE-22179 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22179.01.patch > > Time Spent: 0.5h > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #2: extract all the functin related analyzers from DDLSemanticAnalyzer > and FunctionSemanticAnalyzer and MacroSemanticAnalyzer, and move them under > the new package. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (HIVE-22179) Break up DDLSemanticAnalyzer - extract Function related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22179?focusedWorklogId=310194=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-310194 ] ASF GitHub Bot logged work on HIVE-22179: - Author: ASF GitHub Bot Created on: 10/Sep/19 23:57 Start Date: 10/Sep/19 23:57 Worklog Time Spent: 10m Work Description: jcamachor commented on pull request #763: HIVE-22179 Break up DDLSemanticAnalyzer - extract Function related analyzers URL: https://github.com/apache/hive/pull/763#discussion_r323008559 ## File path: ql/src/java/org/apache/hadoop/hive/ql/ddl/function/createmacro/CreateMacroAnalyzer.java ## @@ -0,0 +1,139 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hadoop.hive.ql.ddl.function.createmacro; Review comment: `createmacro` -> `macro.create` ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 310194) Time Spent: 50m (was: 40m) > Break up DDLSemanticAnalyzer - extract Function related analyzers > - > > Key: HIVE-22179 > URL: https://issues.apache.org/jira/browse/HIVE-22179 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22179.01.patch > > Time Spent: 50m > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #2: extract all the functin related analyzers from DDLSemanticAnalyzer > and FunctionSemanticAnalyzer and MacroSemanticAnalyzer, and move them under > the new package. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (HIVE-22179) Break up DDLSemanticAnalyzer - extract Function related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22179?focusedWorklogId=310196=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-310196 ] ASF GitHub Bot logged work on HIVE-22179: - Author: ASF GitHub Bot Created on: 10/Sep/19 23:57 Start Date: 10/Sep/19 23:57 Worklog Time Spent: 10m Work Description: jcamachor commented on pull request #763: HIVE-22179 Break up DDLSemanticAnalyzer - extract Function related analyzers URL: https://github.com/apache/hive/pull/763#discussion_r323008960 ## File path: ql/src/java/org/apache/hadoop/hive/ql/ddl/function/reloadfunctions/ReloadFunctionsAnalyzer.java ## @@ -0,0 +1,43 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hadoop.hive.ql.ddl.function.reloadfunctions; Review comment: `reloadfunctions` -> `reload` ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 310196) Time Spent: 1h (was: 50m) > Break up DDLSemanticAnalyzer - extract Function related analyzers > - > > Key: HIVE-22179 > URL: https://issues.apache.org/jira/browse/HIVE-22179 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22179.01.patch > > Time Spent: 1h > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #2: extract all the functin related analyzers from DDLSemanticAnalyzer > and FunctionSemanticAnalyzer and MacroSemanticAnalyzer, and move them under > the new package. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (HIVE-22179) Break up DDLSemanticAnalyzer - extract Function related analyzers
[ https://issues.apache.org/jira/browse/HIVE-22179?focusedWorklogId=310193=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-310193 ] ASF GitHub Bot logged work on HIVE-22179: - Author: ASF GitHub Bot Created on: 10/Sep/19 23:57 Start Date: 10/Sep/19 23:57 Worklog Time Spent: 10m Work Description: jcamachor commented on pull request #763: HIVE-22179 Break up DDLSemanticAnalyzer - extract Function related analyzers URL: https://github.com/apache/hive/pull/763#discussion_r323008710 ## File path: ql/src/java/org/apache/hadoop/hive/ql/ddl/function/descfunction/DescFunctionAnalyzer.java ## @@ -0,0 +1,58 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.hadoop.hive.ql.ddl.function.descfunction; Review comment: `descfunction` -> `desc` ? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 310193) Time Spent: 50m (was: 40m) > Break up DDLSemanticAnalyzer - extract Function related analyzers > - > > Key: HIVE-22179 > URL: https://issues.apache.org/jira/browse/HIVE-22179 > Project: Hive > Issue Type: Sub-task >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Attachments: HIVE-22179.01.patch > > Time Spent: 50m > Remaining Estimate: 0h > > DDLSemanticAnalyzer is a huge class, more than 4000 lines long. The goal is > to refactor it in order to have everything cut into more handleable classes > under the package org.apache.hadoop.hive.ql.exec.ddl: > * have a separate class for each analyzers > * have a package for each operation, containing an analyzer, a description, > and an operation, so the amount of classes under a package is more manageable > Step #2: extract all the functin related analyzers from DDLSemanticAnalyzer > and FunctionSemanticAnalyzer and MacroSemanticAnalyzer, and move them under > the new package. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22188) Clean up the SemanticAnalyzerFactory
[
https://issues.apache.org/jira/browse/HIVE-22188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16927120#comment-16927120
]
Hive QA commented on HIVE-22188:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979997/HIVE-22188.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16751 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestHiveMetaStoreAlterColumnPar.org.apache.hadoop.hive.metastore.TestHiveMetaStoreAlterColumnPar
(batchId=246)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18530/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18530/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18530/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979997 - PreCommit-HIVE-Build
> Clean up the SemanticAnalyzerFactory
>
>
> Key: HIVE-22188
> URL: https://issues.apache.org/jira/browse/HIVE-22188
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: refactor-semanticanalyzer
> Fix For: 4.0.0
>
> Attachments: HIVE-22188.01.patch
>
>
> SemanticAnalyzerFactory contains some huge maps that assign tokentypes to
> hive operations. Some operations share a token type for doing them on a
> table, or on a partition, which leads to some more overcomplication.
> Two things need to be done:
> # Each operation must have it's own token.
> # The HiveOperation enum constants themselves must declare which token they
> are related to, and not some external source should declare information about
> them.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22174) Clean up Drop Partition
[ https://issues.apache.org/jira/browse/HIVE-22174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16927119#comment-16927119 ] Jesus Camacho Rodriguez commented on HIVE-22174: +1 > Clean up Drop Partition > --- > > Key: HIVE-22174 > URL: https://issues.apache.org/jira/browse/HIVE-22174 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: pull-request-available, refactor-ddl > Fix For: 4.0.0 > > Attachments: HIVE-22174.01.patch, HIVE-22174.02.patch > > Time Spent: 10m > Remaining Estimate: 0h > > There are many complex drop partition and drop partitions operations in > Hive.java, most of them are not used by anyone inside or outside of Hive. The > unused codes should be removed, and the remaining code should be as simple as > possible, calling dropPartition(s) operation at the metastore client. Also > the AlterTableDropPartitionDesc class should not be exposed here. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[ https://issues.apache.org/jira/browse/HIVE-22170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jesus Camacho Rodriguez updated HIVE-22170: --- Attachment: HIVE-22170.04.patch > from_unixtime and unix_timestamp should use user session time zone > -- > > Key: HIVE-22170 > URL: https://issues.apache.org/jira/browse/HIVE-22170 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0, 4.0.0, 3.2.0, 3.1.1, 3.1.2 >Reporter: Riju Trivedi >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22170.01.patch, HIVE-22170.02.patch, > HIVE-22170.03.patch, HIVE-22170.04.patch > > Time Spent: 10m > Remaining Estimate: 0h > > According to documentation, that is the expected behavior (since session time > zone was not present, system time zone was being used previously). This was > incorrectly changed by HIVE-12192 / HIVE-20007. This JIRA should fix this > issue. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[ https://issues.apache.org/jira/browse/HIVE-22170?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16927093#comment-16927093 ] Jesus Camacho Rodriguez commented on HIVE-22170: [~klcopp], [~szita], can you take a look? https://github.com/apache/hive/pull/764 Thanks > from_unixtime and unix_timestamp should use user session time zone > -- > > Key: HIVE-22170 > URL: https://issues.apache.org/jira/browse/HIVE-22170 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0, 4.0.0, 3.2.0, 3.1.1, 3.1.2 >Reporter: Riju Trivedi >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22170.01.patch, HIVE-22170.02.patch, > HIVE-22170.03.patch > > Time Spent: 10m > Remaining Estimate: 0h > > According to documentation, that is the expected behavior (since session time > zone was not present, system time zone was being used previously). This was > incorrectly changed by HIVE-12192 / HIVE-20007. This JIRA should fix this > issue. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work logged] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[ https://issues.apache.org/jira/browse/HIVE-22170?focusedWorklogId=310166=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-310166 ] ASF GitHub Bot logged work on HIVE-22170: - Author: ASF GitHub Bot Created on: 10/Sep/19 22:55 Start Date: 10/Sep/19 22:55 Worklog Time Spent: 10m Work Description: jcamachor commented on pull request #764: HIVE-22170 URL: https://github.com/apache/hive/pull/764 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org Issue Time Tracking --- Worklog Id: (was: 310166) Remaining Estimate: 0h Time Spent: 10m > from_unixtime and unix_timestamp should use user session time zone > -- > > Key: HIVE-22170 > URL: https://issues.apache.org/jira/browse/HIVE-22170 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0, 4.0.0, 3.2.0, 3.1.1, 3.1.2 >Reporter: Riju Trivedi >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22170.01.patch, HIVE-22170.02.patch, > HIVE-22170.03.patch > > Time Spent: 10m > Remaining Estimate: 0h > > According to documentation, that is the expected behavior (since session time > zone was not present, system time zone was being used previously). This was > incorrectly changed by HIVE-12192 / HIVE-20007. This JIRA should fix this > issue. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22170) from_unixtime and unix_timestamp should use user session time zone
[ https://issues.apache.org/jira/browse/HIVE-22170?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ASF GitHub Bot updated HIVE-22170: -- Labels: pull-request-available (was: ) > from_unixtime and unix_timestamp should use user session time zone > -- > > Key: HIVE-22170 > URL: https://issues.apache.org/jira/browse/HIVE-22170 > Project: Hive > Issue Type: Bug >Affects Versions: 3.1.0, 4.0.0, 3.2.0, 3.1.1, 3.1.2 >Reporter: Riju Trivedi >Assignee: Jesus Camacho Rodriguez >Priority: Major > Labels: pull-request-available > Attachments: HIVE-22170.01.patch, HIVE-22170.02.patch, > HIVE-22170.03.patch > > > According to documentation, that is the expected behavior (since session time > zone was not present, system time zone was being used previously). This was > incorrectly changed by HIVE-12192 / HIVE-20007. This JIRA should fix this > issue. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22188) Clean up the SemanticAnalyzerFactory
[
https://issues.apache.org/jira/browse/HIVE-22188?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16927091#comment-16927091
]
Hive QA commented on HIVE-22188:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 7m
53s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
44s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
40s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
53s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
44s{color} | {color:red} ql: The patch generated 9 new + 715 unchanged - 45
fixed = 724 total (was 760) {color} |
| {color:red}-1{color} | {color:red} whitespace {color} | {color:red} 0m
0s{color} | {color:red} The patch 2 line(s) with tabs. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 3m
53s{color} | {color:red} ql generated 2 new + 2229 unchanged - 17 fixed = 2231
total (was 2246) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
12s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 22m 51s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:ql |
| | Should org.apache.hadoop.hive.ql.parse.HiveParser$DFA236 be a _static_
inner class? At HiveParser.java:inner class? At HiveParser.java:[lines
48714-48727] |
| | org.apache.hadoop.hive.ql.plan.HiveOperation.getTokens() may expose
internal representation by returning HiveOperation.tokens At
HiveOperation.java:by returning HiveOperation.tokens At
HiveOperation.java:[line 245] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18530/dev-support/hive-personality.sh
|
| git revision | master / 1dfa2d8 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18530/yetus/diff-checkstyle-ql.txt
|
| whitespace |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18530/yetus/whitespace-tabs.txt
|
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18530/yetus/new-findbugs-ql.html
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18530/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Clean up the SemanticAnalyzerFactory
>
>
> Key: HIVE-22188
> URL: https://issues.apache.org/jira/browse/HIVE-22188
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Major
> Labels: refactor-semanticanalyzer
> Fix For: 4.0.0
>
> Attachments: HIVE-22188.01.patch
>
>
> SemanticAnalyzerFactory contains some huge maps that assign tokentypes to
> hive operations. Some operations share a token type for doing them on a
> table, or on a partition, which leads to some more overcomplication.
> Two things need to be done:
> # Each operation must have it's own token.
> # The HiveOperation enum constants themselves must declare which token they
[jira] [Commented] (HIVE-20284) In strict mode, if constant propagation is enable, the partition filter may be folded before partition pruner lead to error "No partition predicate for Alias"
[
https://issues.apache.org/jira/browse/HIVE-20284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16927078#comment-16927078
]
Hive QA commented on HIVE-20284:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12934014/HIVE-20284.3.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16741 tests
executed
*Failed tests:*
{noformat}
TestJdbcWithMiniLlapArrow - did not produce a TEST-*.xml file (likely timed
out) (batchId=284)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18529/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18529/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18529/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12934014 - PreCommit-HIVE-Build
> In strict mode, if constant propagation is enable, the partition filter may
> be folded before partition pruner lead to error "No partition predicate for
> Alias"
>
>
> Key: HIVE-20284
> URL: https://issues.apache.org/jira/browse/HIVE-20284
> Project: Hive
> Issue Type: Improvement
> Components: CLI
>Affects Versions: 1.2.1, 2.3.3
>Reporter: Hui Huang
>Priority: Trivial
> Fix For: 4.0.0
>
> Attachments: HIVE-20284.1.patch, HIVE-20284.2.patch,
> HIVE-20284.3.patch, HIVE-20284.patch
>
>
> In strict mode and the hive.optimize.constant.propagation is set to true, the
> following sql will failed:
> {code:java}
> hive> desc employee_part;
> OK
> col_name data_type comment
> eid int
> name string
> dept string
> year string
> month string
> # Partition Information
> # col_namedata_type comment
> year string
> month string
> Time taken: 0.564 seconds, Fetched: 11 row(s)
> hive> set hive.mapred.mode=strict;
> hive> select * from employee_part where false and concat(year,month)='201807';
> FAILED: SemanticException Queries against partitioned tables without a
> partition filter are disabled for safety reasons. If you know what you are
> doing, please sethive.strict.checks.large.query to false and that
> hive.mapred.mode is not set to 'strict' to proceed. Note that if you may get
> errors or incorrect results if you make a mistake while using some of the
> unsafe features. No partition predicatie for Alias "employee_part" Table
> "employee_part"
> {code}
> The above error message is confusing because the expression
> concat(year,month)='201807' is a partition filter。
> The reason is during logic optimization, the ConstantPropagate optimizer is
> running before partitionPruner optimizer, when found a express like 'false
> and concat(year,month)=', the express is replaced with 'fasle' , and the
> partition filter is dropped. So the PartitionPruner can not get the partition
> filter.
> Users can remove the constant express that always has true/false values to
> work around.
> When views used, if some columns are constant values, users will be confusing.
> So we should add some more message in the error msg returned.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-20113) Shuffle avoidance: Disable 1-1 edges for sorted shuffle
[ https://issues.apache.org/jira/browse/HIVE-20113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16927068#comment-16927068 ] Jesus Camacho Rodriguez commented on HIVE-20113: +1 (pending tests) [~vgarg], [~gopalv], what is the remaining work that we should do to enable this back? Can we link any new / existing JIRA to this one? Thanks > Shuffle avoidance: Disable 1-1 edges for sorted shuffle > > > Key: HIVE-20113 > URL: https://issues.apache.org/jira/browse/HIVE-20113 > Project: Hive > Issue Type: Bug > Components: Tez >Reporter: Gopal V >Assignee: Gopal V >Priority: Major > Labels: Branch3Candidate > Attachments: HIVE-20113.1.patch, HIVE-20113.2.patch, > HIVE-20113.3.patch, HIVE-20113.4.patch, HIVE-20113.4.patch, > HIVE-20113.5.patch, HIVE-20113.6.patch, HIVE-20113.7.patch > > > The sorted shuffle avoidance can have some issues when the shuffle data gets > broken up into multiple chunks on disk. > The 1-1 edge cannot skip the tez final merge - there's no reason for 1-1 to > have a final merge at all, it should open a single compressed file and write > a single index entry. > Until the shuffle issue is resolved & a lot more testing, it is prudent to > disable the optimization for sorted shuffle edges and stop rewriting the > RS(sorted) = = = RS(sorted) into RS(sorted) = = = RS(FORWARD). -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22184) Extract LockedDriverState from Driver
[ https://issues.apache.org/jira/browse/HIVE-22184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22184: -- Attachment: HIVE-22184.01.patch > Extract LockedDriverState from Driver > - > > Key: HIVE-22184 > URL: https://issues.apache.org/jira/browse/HIVE-22184 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Minor > Labels: refactor-driver > Attachments: HIVE-22184.01.patch > > > LockedDriverState is a nested class within Driver, while it is used outside > of it as well, and it is complex enough to be a class on it's own. > DriverState should be it's nested class, and transitions / locking should be > facilitated by functions within it. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22184) Extract LockedDriverState from Driver
[ https://issues.apache.org/jira/browse/HIVE-22184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22184: -- Attachment: (was: HIVE-22184.01.patch) > Extract LockedDriverState from Driver > - > > Key: HIVE-22184 > URL: https://issues.apache.org/jira/browse/HIVE-22184 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Minor > Labels: refactor-driver > Attachments: HIVE-22184.01.patch > > > LockedDriverState is a nested class within Driver, while it is used outside > of it as well, and it is complex enough to be a class on it's own. > DriverState should be it's nested class, and transitions / locking should be > facilitated by functions within it. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22158) HMS Translation layer - Disallow non-ACID MANAGED tables.
[ https://issues.apache.org/jira/browse/HIVE-22158?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16927051#comment-16927051 ] Naveen Gangam commented on HIVE-22158: -- Tests are now clean. So +1 for me. I am awaiting review from other reviewers. [~thejas] Can you review this please? Thanks > HMS Translation layer - Disallow non-ACID MANAGED tables. > - > > Key: HIVE-22158 > URL: https://issues.apache.org/jira/browse/HIVE-22158 > Project: Hive > Issue Type: Sub-task > Components: Hive >Affects Versions: 4.0.0 >Reporter: Naveen Gangam >Assignee: Naveen Gangam >Priority: Major > Attachments: HIVE-22158.1.patch, HIVE-22158.1.patch, > HIVE-22158.2.patch > > > In the recent commits, we have allowed non-ACID MANAGED tables to be created > by clients that have some form of ACID WRITE capabilities. > I think it would make sense to disallow this entirely. MANAGED tables should > be ACID tables only. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-20284) In strict mode, if constant propagation is enable, the partition filter may be folded before partition pruner lead to error "No partition predicate for Alias"
[
https://issues.apache.org/jira/browse/HIVE-20284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926910#comment-16926910
]
Hive QA commented on HIVE-20284:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
14s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
11s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
30s{color} | {color:blue} common in master has 61 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
39s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
6s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
24s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
34s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
18s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
49s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 25m 46s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18529/dev-support/hive-personality.sh
|
| git revision | master / 1dfa2d8 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: common ql U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18529/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> In strict mode, if constant propagation is enable, the partition filter may
> be folded before partition pruner lead to error "No partition predicate for
> Alias"
>
>
> Key: HIVE-20284
> URL: https://issues.apache.org/jira/browse/HIVE-20284
> Project: Hive
> Issue Type: Improvement
> Components: CLI
>Affects Versions: 1.2.1, 2.3.3
>Reporter: Hui Huang
>Priority: Trivial
> Fix For: 4.0.0
>
> Attachments: HIVE-20284.1.patch, HIVE-20284.2.patch,
> HIVE-20284.3.patch, HIVE-20284.patch
>
>
> In strict mode and the hive.optimize.constant.propagation is set to true, the
> following sql will failed:
> {code:java}
> hive> desc employee_part;
> OK
> col_name data_type comment
> eid int
> name string
> dept string
> year string
> month string
> # Partition Information
> # col_namedata_type comment
> year string
> month
[jira] [Updated] (HIVE-22188) Clean up the SemanticAnalyzerFactory
[ https://issues.apache.org/jira/browse/HIVE-22188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22188: -- Status: Patch Available (was: Open) > Clean up the SemanticAnalyzerFactory > > > Key: HIVE-22188 > URL: https://issues.apache.org/jira/browse/HIVE-22188 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-semanticanalyzer > Fix For: 4.0.0 > > Attachments: HIVE-22188.01.patch > > > SemanticAnalyzerFactory contains some huge maps that assign tokentypes to > hive operations. Some operations share a token type for doing them on a > table, or on a partition, which leads to some more overcomplication. > Two things need to be done: > # Each operation must have it's own token. > # The HiveOperation enum constants themselves must declare which token they > are related to, and not some external source should declare information about > them. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22188) Clean up the SemanticAnalyzerFactory
[ https://issues.apache.org/jira/browse/HIVE-22188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22188: -- Attachment: HIVE-22188.01.patch > Clean up the SemanticAnalyzerFactory > > > Key: HIVE-22188 > URL: https://issues.apache.org/jira/browse/HIVE-22188 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-semanticanalyzer > Fix For: 4.0.0 > > Attachments: HIVE-22188.01.patch > > > SemanticAnalyzerFactory contains some huge maps that assign tokentypes to > hive operations. Some operations share a token type for doing them on a > table, or on a partition, which leads to some more overcomplication. > Two things need to be done: > # Each operation must have it's own token. > # The HiveOperation enum constants themselves must declare which token they > are related to, and not some external source should declare information about > them. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926888#comment-16926888
]
Hive QA commented on HIVE-21508:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979972/HIVE-21508.2.branch-2.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 8 failed/errored test(s), 10587 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[dynamic_semijoin_reduction]
(batchId=151)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[mergejoin]
(batchId=153)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_dynamic_semijoin_reduction2]
(batchId=150)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_dynamic_semijoin_reduction]
(batchId=142)
org.apache.hadoop.hive.cli.TestSparkCliDriver.org.apache.hadoop.hive.cli.TestSparkCliDriver
(batchId=108)
org.apache.hadoop.hive.ql.TestTxnCommands2.testNonAcidToAcidConversion02
(batchId=264)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdate.testNonAcidToAcidConversion02
(batchId=276)
org.apache.hadoop.hive.ql.TestTxnCommands2WithSplitUpdateAndVectorization.testNonAcidToAcidConversion02
(batchId=273)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18528/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18528/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18528/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 8 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979972 - PreCommit-HIVE-Build
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21508.1.patch, HIVE-21508.2.branch-2.3.patch,
> HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21884) Scheduled query support
[
https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926878#comment-16926878
]
Hive QA commented on HIVE-21884:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 1m
41s{color} | {color:blue} Maven dependency ordering for branch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 6m
50s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 5m
30s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
33s{color} | {color:blue} standalone-metastore/metastore-common in master has
32 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
35s{color} | {color:blue} common in master has 61 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
9s{color} | {color:blue} standalone-metastore/metastore-server in master has
181 extant Findbugs warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
59s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
38s{color} | {color:blue} service in master has 48 extant Findbugs warnings.
{color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
37s{color} | {color:blue} itests/hive-unit in master has 2 extant Findbugs
warnings. {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 0m
45s{color} | {color:blue} itests/util in master has 44 extant Findbugs
warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 11m
31s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:blue}0{color} | {color:blue} mvndep {color} | {color:blue} 0m
25s{color} | {color:blue} Maven dependency ordering for patch {color} |
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 12m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 10m
24s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 10m
24s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
35s{color} | {color:red} standalone-metastore: The patch generated 19 new +
1745 unchanged - 15 fixed = 1764 total (was 1760) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} standalone-metastore/metastore-common: The patch
generated 1 new + 495 unchanged - 0 fixed = 496 total (was 495) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
14s{color} | {color:red} common: The patch generated 2 new + 367 unchanged - 0
fixed = 369 total (was 367) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
8s{color} | {color:green} The patch metastore passed checkstyle {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
26s{color} | {color:red} standalone-metastore/metastore-server: The patch
generated 18 new + 1250 unchanged - 15 fixed = 1268 total (was 1265) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
38s{color} | {color:red} ql: The patch generated 62 new + 141 unchanged - 0
fixed = 203 total (was 141) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} service: The patch generated 0 new + 38 unchanged -
1 fixed = 38 total (was 39) {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 2m
9s{color} | {color:red} root: The patch generated 100 new + 2338 unchanged - 16
fixed = 2438 total (was 2354) {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
10s{color} | {color:green} The patch hcatalog-unit passed checkstyle {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
16s{color} | {color:red} itests/hive-unit: The patch generated 17 new + 0
unchanged - 0 fixed = 17
[jira] [Commented] (HIVE-22183) Prepare for ZooKeeper 3.5 upgrade
[
https://issues.apache.org/jira/browse/HIVE-22183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926871#comment-16926871
]
David Mollitor commented on HIVE-22183:
---
Hello [~pvary] What components in Hive are using ZK? I thought ZK-Hive
integration was going away in favor of a RDBMS system.
> Prepare for ZooKeeper 3.5 upgrade
> -
>
> Key: HIVE-22183
> URL: https://issues.apache.org/jira/browse/HIVE-22183
> Project: Hive
> Issue Type: Task
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-22183.patch
>
>
> Tried to run some tests with 3.5 ZooKeeper, and got the following exception:
> {code:java}
> java.io.IOException: Waiting for startup of standalone server
> at
> org.apache.hive.testutils.MiniZooKeeperCluster.startup(MiniZooKeeperCluster.java:253)
> at
> org.apache.hive.testutils.MiniZooKeeperCluster.startup(MiniZooKeeperCluster.java:184)
> at
> org.apache.hive.testutils.HiveTestEnvSetup$SetupZookeeper.beforeClass(HiveTestEnvSetup.java:187)
> at
> org.apache.hive.testutils.HiveTestEnvSetup.before(HiveTestEnvSetup.java:269)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:46)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413)
> Options {code}
> We uses "stat" 4 letter command to check if the ZooKeeper cluster is running.
> In ZooKeeper 3.5 we need to whitelist the commands we use.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21884) Scheduled query support
[
https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926863#comment-16926863
]
Hive QA commented on HIVE-21884:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979970/HIVE-21844.08.patch
{color:green}SUCCESS:{color} +1 due to 9 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 29 failed/errored test(s), 16785 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[resourceplan]
(batchId=175)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[strict_managed_tables_sysdb]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb]
(batchId=171)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[sysdb_schq]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[tez_join_hash]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vector_udf_string_to_boolean]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[vectorized_parquet]
(batchId=176)
org.apache.hadoop.hive.cli.TestMiniLlapLocalCliDriver.testCliDriver[windowing]
(batchId=176)
org.apache.hadoop.hive.metastore.TestMetaStoreMetrics.org.apache.hadoop.hive.metastore.TestMetaStoreMetrics
(batchId=242)
org.apache.hadoop.hive.metastore.TestTransactionalValidationListener.testCreateAsAcid[Embedded]
(batchId=246)
org.apache.hadoop.hive.metastore.TestTransactionalValidationListener.testCreateAsAcid[Remote]
(batchId=246)
org.apache.hadoop.hive.metastore.tools.schematool.TestSchemaToolForMetastore.testMetastoreDbPropertiesAfterUpgrade
(batchId=227)
org.apache.hadoop.hive.metastore.tools.schematool.TestSchemaToolForMetastore.testValidateSchemaTables
(batchId=227)
org.apache.hadoop.hive.ql.parse.authorization.plugin.sqlstd.TestOperation2Privilege.checkHiveOperationTypeMatch
(batchId=327)
org.apache.hadoop.hive.ql.schq.TestScheduledQueryService.testScheduledQueryExecution
(batchId=348)
org.apache.hadoop.hive.ql.schq.TestScheduledQueryStatements.testCreateFromNonDefaultDatabase
(batchId=348)
org.apache.hadoop.hive.schq.TestScheduledQueryIntegration.testScheduledQueryExecutionImpersonation
(batchId=277)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testCancelRenewTokenFlow
(batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testConnection
(batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testIsValid (batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testIsValidNeg
(batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testNegativeProxyAuth
(batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testNegativeTokenAuth
(batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testNoKrbSASLTokenAuthNeg
(batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testNonKrbSASLAuth
(batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testNonKrbSASLFullNameAuth
(batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testProxyAuth
(batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testRenewDelegationToken
(batchId=298)
org.apache.hive.minikdc.TestJdbcNonKrbSASLWithMiniKdc.testTokenAuth
(batchId=298)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18527/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18527/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18527/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 29 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979970 - PreCommit-HIVE-Build
> Scheduled query support
> ---
>
> Key: HIVE-21884
> URL: https://issues.apache.org/jira/browse/HIVE-21884
> Project: Hive
> Issue Type: Improvement
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch,
> HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch,
> HIVE-21884.01.patch, HIVE-21884.02.patch, HIVE-21884.03.patch, Scheduled
> queries2.pdf
>
>
> design document:
> https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit#
> in case the google doc is not reachable: [^Scheduled queries2.pdf]
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (HIVE-22188) Clean up the SemanticAnalyzerFactory
[ https://issues.apache.org/jira/browse/HIVE-22188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely reassigned HIVE-22188: - > Clean up the SemanticAnalyzerFactory > > > Key: HIVE-22188 > URL: https://issues.apache.org/jira/browse/HIVE-22188 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-semanticanalyzer > Fix For: 4.0.0 > > > SemanticAnalyzerFactory contains some huge maps that assign tokentypes to > hive operations. Some operations share a token type for doing them on a > table, or on a partition, which leads to some more overcomplication. > Two things need to be done: > # Each operation must have it's own token. > # The HiveOperation enum constants themselves must declare which token they > are related to, and not some external source should declare information about > them. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22187) Refactor the SemanticAnalyzers
[ https://issues.apache.org/jira/browse/HIVE-22187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22187: -- Description: The Semantic Analyzers are generally huge classes, many thousand of lines each. They should be put into more manageable structures. > Refactor the SemanticAnalyzers > -- > > Key: HIVE-22187 > URL: https://issues.apache.org/jira/browse/HIVE-22187 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-semanticanalyzer > Fix For: 4.0.0 > > > The Semantic Analyzers are generally huge classes, many thousand of lines > each. They should be put into more manageable structures. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22187) Refactor the SemanticAnalyzers
[ https://issues.apache.org/jira/browse/HIVE-22187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22187: -- Component/s: Hive > Refactor the SemanticAnalyzers > -- > > Key: HIVE-22187 > URL: https://issues.apache.org/jira/browse/HIVE-22187 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Assigned] (HIVE-22187) Refactor the SemanticAnalyzers
[ https://issues.apache.org/jira/browse/HIVE-22187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely reassigned HIVE-22187: - Assignee: Miklos Gergely > Refactor the SemanticAnalyzers > -- > > Key: HIVE-22187 > URL: https://issues.apache.org/jira/browse/HIVE-22187 > Project: Hive > Issue Type: Improvement >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22187) Refactor the SemanticAnalyzers
[ https://issues.apache.org/jira/browse/HIVE-22187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22187: -- Fix Version/s: 4.0.0 > Refactor the SemanticAnalyzers > -- > > Key: HIVE-22187 > URL: https://issues.apache.org/jira/browse/HIVE-22187 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-semanticanalyzer > Fix For: 4.0.0 > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22187) Refactor the SemanticAnalyzers
[ https://issues.apache.org/jira/browse/HIVE-22187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22187: -- Labels: refactor-semanticanalyzer (was: ) > Refactor the SemanticAnalyzers > -- > > Key: HIVE-22187 > URL: https://issues.apache.org/jira/browse/HIVE-22187 > Project: Hive > Issue Type: Improvement > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Major > Labels: refactor-semanticanalyzer > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22185) HADOOP-15832 will cause problems with tests using MiniYarn clusters
[
https://issues.apache.org/jira/browse/HIVE-22185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926829#comment-16926829
]
Hive QA commented on HIVE-22185:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979969/HIVE-22185.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16751 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18526/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18526/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18526/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979969 - PreCommit-HIVE-Build
> HADOOP-15832 will cause problems with tests using MiniYarn clusters
> ---
>
> Key: HIVE-22185
> URL: https://issues.apache.org/jira/browse/HIVE-22185
> Project: Hive
> Issue Type: Task
> Components: Test
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-22185.patch
>
>
> HADOOP-15832 changed the way to depend on bouncycastle jars.
> This will break tests depending on it like:
> *
> org.apache.hive.hcatalog.mapreduce.TestHCatMultiOutputFormat.org.apache.hive.hcatalog.mapreduce.TestHCatMultiOutputFormat
> *
> org.apache.hive.hcatalog.mapreduce.TestHCatMultiOutputFormat.org.apache.hive.hcatalog.mapreduce.TestHCatMultiOutputFormat
> *
> org.apache.hive.hcatalog.mapreduce.TestHCatPartitionPublish.org.apache.hive.hcatalog.mapreduce.TestHCatPartitionPublish
> *
> org.apache.hive.hcatalog.mapreduce.TestMultiOutputFormat.org.apache.hive.hcatalog.mapreduce.TestMultiOutputFormat
> Example exception:
> {code:java}
> java.lang.NoClassDefFoundError:
> org/bouncycastle/operator/OperatorCreationException
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:814)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1245)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:321)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster.initResourceManager(MiniYARNCluster.java:348)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster.access$200(MiniYARNCluster.java:128)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.serviceInit(MiniYARNCluster.java:497)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster.serviceInit(MiniYARNCluster.java:316)
> at
> org.apache.hadoop.mapreduce.v2.MiniMRYarnCluster.serviceInit(MiniMRYarnCluster.java:206)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.mapred.MiniMRClientClusterFactory.create(MiniMRClientClusterFactory.java:79)
> at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:188)
> at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:176)
> at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:168)
> at
> org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:129) {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22185) HADOOP-15832 will cause problems with tests using MiniYarn clusters
[
https://issues.apache.org/jira/browse/HIVE-22185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926802#comment-16926802
]
Hive QA commented on HIVE-22185:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
18s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
22s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
2s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
13s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 10m 21s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18526/dev-support/hive-personality.sh
|
| git revision | master / 1dfa2d8 |
| Default Java | 1.8.0_111 |
| modules | C: hcatalog/core U: hcatalog/core |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18526/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> HADOOP-15832 will cause problems with tests using MiniYarn clusters
> ---
>
> Key: HIVE-22185
> URL: https://issues.apache.org/jira/browse/HIVE-22185
> Project: Hive
> Issue Type: Task
> Components: Test
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-22185.patch
>
>
> HADOOP-15832 changed the way to depend on bouncycastle jars.
> This will break tests depending on it like:
> *
> org.apache.hive.hcatalog.mapreduce.TestHCatMultiOutputFormat.org.apache.hive.hcatalog.mapreduce.TestHCatMultiOutputFormat
> *
> org.apache.hive.hcatalog.mapreduce.TestHCatMultiOutputFormat.org.apache.hive.hcatalog.mapreduce.TestHCatMultiOutputFormat
> *
> org.apache.hive.hcatalog.mapreduce.TestHCatPartitionPublish.org.apache.hive.hcatalog.mapreduce.TestHCatPartitionPublish
> *
> org.apache.hive.hcatalog.mapreduce.TestMultiOutputFormat.org.apache.hive.hcatalog.mapreduce.TestMultiOutputFormat
> Example exception:
> {code:java}
> java.lang.NoClassDefFoundError:
> org/bouncycastle/operator/OperatorCreationException
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:814)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1245)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:321)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster.initResourceManager(MiniYARNCluster.java:348)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster.access$200(MiniYARNCluster.java:128)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.serviceInit(MiniYARNCluster.java:497)
> at
>
[jira] [Commented] (HIVE-22184) Extract LockedDriverState from Driver
[
https://issues.apache.org/jira/browse/HIVE-22184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926793#comment-16926793
]
Hive QA commented on HIVE-22184:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979967/HIVE-22184.01.patch
{color:green}SUCCESS:{color} +1 due to 1 test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 2 failed/errored test(s), 16747 tests
executed
*Failed tests:*
{noformat}
TestStatsReplicationScenariosACIDNoAutogather - did not produce a TEST-*.xml
file (likely timed out) (batchId=254)
org.apache.hadoop.hive.cli.TestMiniTezCliDriver.testCliDriver[hybridgrace_hashjoin_2]
(batchId=111)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18525/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18525/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18525/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 2 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979967 - PreCommit-HIVE-Build
> Extract LockedDriverState from Driver
> -
>
> Key: HIVE-22184
> URL: https://issues.apache.org/jira/browse/HIVE-22184
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Minor
> Labels: refactor-driver
> Attachments: HIVE-22184.01.patch
>
>
> LockedDriverState is a nested class within Driver, while it is used outside
> of it as well, and it is complex enough to be a class on it's own.
> DriverState should be it's nested class, and transitions / locking should be
> facilitated by functions within it.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22184) Extract LockedDriverState from Driver
[
https://issues.apache.org/jira/browse/HIVE-22184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926768#comment-16926768
]
Hive QA commented on HIVE-22184:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
32s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
42s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
52s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} checkstyle {color} | {color:red} 0m
43s{color} | {color:red} ql: The patch generated 3 new + 348 unchanged - 4
fixed = 351 total (was 352) {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
2s{color} | {color:green} the patch passed {color} |
| {color:red}-1{color} | {color:red} javadoc {color} | {color:red} 0m
58s{color} | {color:red} ql generated 1 new + 99 unchanged - 1 fixed = 100
total (was 100) {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 23m 55s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18525/dev-support/hive-personality.sh
|
| git revision | master / 1dfa2d8 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| checkstyle |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18525/yetus/diff-checkstyle-ql.txt
|
| javadoc |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18525/yetus/diff-javadoc-javadoc-ql.txt
|
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18525/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Extract LockedDriverState from Driver
> -
>
> Key: HIVE-22184
> URL: https://issues.apache.org/jira/browse/HIVE-22184
> Project: Hive
> Issue Type: Sub-task
> Components: Hive
>Reporter: Miklos Gergely
>Assignee: Miklos Gergely
>Priority: Minor
> Labels: refactor-driver
> Attachments: HIVE-22184.01.patch
>
>
> LockedDriverState is a nested class within Driver, while it is used outside
> of it as well, and it is complex enough to be a class on it's own.
> DriverState should be it's nested class, and transitions / locking should be
> facilitated by functions within it.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22183) Prepare for ZooKeeper 3.5 upgrade
[
https://issues.apache.org/jira/browse/HIVE-22183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926740#comment-16926740
]
Hive QA commented on HIVE-22183:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979962/HIVE-22183.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16751 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18524/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18524/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18524/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979962 - PreCommit-HIVE-Build
> Prepare for ZooKeeper 3.5 upgrade
> -
>
> Key: HIVE-22183
> URL: https://issues.apache.org/jira/browse/HIVE-22183
> Project: Hive
> Issue Type: Task
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-22183.patch
>
>
> Tried to run some tests with 3.5 ZooKeeper, and got the following exception:
> {code:java}
> java.io.IOException: Waiting for startup of standalone server
> at
> org.apache.hive.testutils.MiniZooKeeperCluster.startup(MiniZooKeeperCluster.java:253)
> at
> org.apache.hive.testutils.MiniZooKeeperCluster.startup(MiniZooKeeperCluster.java:184)
> at
> org.apache.hive.testutils.HiveTestEnvSetup$SetupZookeeper.beforeClass(HiveTestEnvSetup.java:187)
> at
> org.apache.hive.testutils.HiveTestEnvSetup.before(HiveTestEnvSetup.java:269)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:46)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413)
> Options {code}
> We uses "stat" 4 letter command to check if the ZooKeeper cluster is running.
> In ZooKeeper 3.5 we need to whitelist the commands we use.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22182) SemanticAnalyzer populates map which is not used at all
[
https://issues.apache.org/jira/browse/HIVE-22182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Akos Dombi updated HIVE-22182:
--
Attachment: HIVE-22182.patch
> SemanticAnalyzer populates map which is not used at all
> ---
>
> Key: HIVE-22182
> URL: https://issues.apache.org/jira/browse/HIVE-22182
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Akos Dombi
>Assignee: Akos Dombi
>Priority: Major
> Attachments: HIVE-22182.patch
>
>
> The {{SemanticAnalyzer}} class has two map fields which are populated but
> never queried:
> * {{topToTable}}
> * {{topToTableProps}}
> Last usage was removed in HIVE-11752.
> h3. Purpose of this task
> - Remove unused {{topToTable}} and {{topToTableProps}} fields
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-20284) In strict mode, if constant propagation is enable, the partition filter may be folded before partition pruner lead to error "No partition predicate for Alias"
[
https://issues.apache.org/jira/browse/HIVE-20284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Simon Poortman updated HIVE-20284:
--
Description:
In strict mode and the hive.optimize.constant.propagation is set to true, the
following sql will failed:
{code:java}
hive> desc employee_part;
OK
col_namedata_type comment
eid int
namestring
deptstring
yearstring
month string
# Partition Information
# col_name data_type comment
yearstring
month string
Time taken: 0.564 seconds, Fetched: 11 row(s)
hive> set hive.mapred.mode=strict;
hive> select * from employee_part where false and concat(year,month)='201807';
FAILED: SemanticException Queries against partitioned tables without a
partition filter are disabled for safety reasons. If you know what you are
doing, please sethive.strict.checks.large.query to false and that
hive.mapred.mode is not set to 'strict' to proceed. Note that if you may get
errors or incorrect results if you make a mistake while using some of the
unsafe features. No partition predicatie for Alias "employee_part" Table
"employee_part"
{code}
The above error message is confusing because the expression
concat(year,month)='201807' is a partition filter。
The reason is during logic optimization, the ConstantPropagate optimizer is
running before partitionPruner optimizer, when found a express like 'false and
concat(year,month)=', the express is replaced with 'fasle' , and the
partition filter is dropped. So the PartitionPruner can not get the partition
filter.
Users can remove the constant express that always has true/false values to work
around.
When views used, if some columns are constant values, users will be confusing.
So we should add some more message in the error msg returned.
was:
In strict mode and the hive.optimize.constant.propagation is set to true, the
following sql will failed:
{code:java}
hive> desc employee_part;
OK
col_namedata_type comment
eid int
namestring
deptstring
yearstring
month string
# Partition Information
# col_name data_type comment
yearstring
month string
Time taken: 0.564 seconds, Fetched: 11 row(s)
hive> set hive.mapred.mode=strict;
hive> select * from employee_part where false and concat(year,month)='201807';
FAILED: SemanticException Queries against partitioned tables without a
partition filter are disabled for safety reasons. If you know what you are
doing, please sethive.strict.checks.large.query to false and that
hive.mapred.mode is not set to 'strict' to proceed. Note that if you may get
errors or incorrect results if you make a mistake while using some of the
unsafe features. No partition predicate for Alias "employee_part" Table
"employee_part"
{code}
The above error message is confusing because the expression
concat(year,month)='201807' is a partition filter。
The reason is during logic optimization, the ConstantPropagate optimizer is
running before partitionPruner optimizer, when found a express like 'false and
concat(year,month)=', the express is replaced with 'fasle' , and the
partition filter is dropped. So the PartitionPruner can not get the partition
filter.
Users can remove the constant express that always has true/false values to work
around.
When views used, if some columns are constant values, users will be confusing.
So we should add some more message in the error msg returned.
> In strict mode, if constant propagation is enable, the partition filter may
> be folded before partition pruner lead to error "No partition predicate for
> Alias"
>
>
> Key: HIVE-20284
> URL: https://issues.apache.org/jira/browse/HIVE-20284
> Project: Hive
> Issue Type: Improvement
> Components: CLI
>Affects Versions: 1.2.1, 2.3.3
>Reporter: Hui Huang
>Priority: Trivial
> Fix For: 4.0.0
>
> Attachments: HIVE-20284.1.patch, HIVE-20284.2.patch,
> HIVE-20284.3.patch, HIVE-20284.patch
>
>
> In strict mode and the hive.optimize.constant.propagation is set to true, the
> following sql will failed:
> {code:java}
> hive> desc employee_part;
> OK
> col_name data_type comment
> eid int
> name string
> dept string
> year string
> month string
> # Partition Information
> # col_namedata_type comment
> year
[jira] [Commented] (HIVE-22183) Prepare for ZooKeeper 3.5 upgrade
[
https://issues.apache.org/jira/browse/HIVE-22183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926718#comment-16926718
]
Hive QA commented on HIVE-22183:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
1s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
52s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
26s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
37s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
8s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 17s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18524/dev-support/hive-personality.sh
|
| git revision | master / 1dfa2d8 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18524/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Prepare for ZooKeeper 3.5 upgrade
> -
>
> Key: HIVE-22183
> URL: https://issues.apache.org/jira/browse/HIVE-22183
> Project: Hive
> Issue Type: Task
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-22183.patch
>
>
> Tried to run some tests with 3.5 ZooKeeper, and got the following exception:
> {code:java}
> java.io.IOException: Waiting for startup of standalone server
> at
> org.apache.hive.testutils.MiniZooKeeperCluster.startup(MiniZooKeeperCluster.java:253)
> at
> org.apache.hive.testutils.MiniZooKeeperCluster.startup(MiniZooKeeperCluster.java:184)
> at
> org.apache.hive.testutils.HiveTestEnvSetup$SetupZookeeper.beforeClass(HiveTestEnvSetup.java:187)
> at
> org.apache.hive.testutils.HiveTestEnvSetup.before(HiveTestEnvSetup.java:269)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:46)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)
> at
>
[jira] [Commented] (HIVE-22038) Fix memory related sideeffects of opening/closing sessions
[
https://issues.apache.org/jira/browse/HIVE-22038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926694#comment-16926694
]
Hive QA commented on HIVE-22038:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979954/HIVE-22038.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 36 failed/errored test(s), 16751 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druid_materialized_view_rewrite_ssb]
(batchId=198)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druid_timeseries]
(batchId=198)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druid_timestamptz2]
(batchId=198)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druid_topn]
(batchId=198)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_dynamic_partition]
(batchId=198)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_expressions]
(batchId=198)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_extractTime]
(batchId=198)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_floorTime]
(batchId=198)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_mv]
(batchId=198)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test1]
(batchId=198)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test_alter]
(batchId=198)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test_insert]
(batchId=198)
org.apache.hadoop.hive.cli.TestMiniDruidCliDriver.testCliDriver[druidmini_test_ts]
(batchId=198)
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_avro]
(batchId=297)
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_basic]
(batchId=297)
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_csv]
(batchId=297)
org.apache.hadoop.hive.cli.TestMiniDruidKafkaCliDriver.testCliDriver[druidkafkamini_delimited]
(batchId=297)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[cttl]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[table_nonprintable]
(batchId=157)
org.apache.hadoop.hive.cli.TestMiniLlapCliDriver.testCliDriver[temp_table_external]
(batchId=158)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[constprog_semijoin]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[external_table_with_space_in_location_path]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[infer_bucket_sort_map_operators]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[infer_bucket_sort_merge]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge4]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[orc_merge_incompat1]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[root_dir_external_table]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_2]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_3]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_dynamic_partition_pruning_4]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[spark_multi_insert_parallel_orderby]
(batchId=195)
org.apache.hadoop.hive.cli.TestMiniSparkOnYarnCliDriver.testCliDriver[vector_outer_join0]
(batchId=195)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[ct_noperm_loc]
(batchId=104)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[ctas_noperm_loc]
(batchId=104)
org.apache.hadoop.hive.cli.TestNegativeMinimrCliDriver.testCliDriver[table_nonprintable_negative]
(batchId=104)
org.apache.hadoop.hive.ql.stats.TestStatsUpdaterThread.testQueueingWithThreads
(batchId=319)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18523/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18523/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18523/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 36 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID:
[jira] [Updated] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ana Jalba updated HIVE-21508:
-
Attachment: HIVE-21508.2.branch-2.3.patch
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21508.1.patch, HIVE-21508.2.branch-2.3.patch,
> HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22038) Fix memory related sideeffects of opening/closing sessions
[
https://issues.apache.org/jira/browse/HIVE-22038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926664#comment-16926664
]
Hive QA commented on HIVE-22038:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
56s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
1s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
40s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 3m
51s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
23s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
38s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
57s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 11s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18523/dev-support/hive-personality.sh
|
| git revision | master / 1dfa2d8 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18523/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Fix memory related sideeffects of opening/closing sessions
> --
>
> Key: HIVE-22038
> URL: https://issues.apache.org/jira/browse/HIVE-22038
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22038.01.patch
>
>
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926661#comment-16926661
]
Ana Jalba commented on HIVE-21508:
--
[~pvary]: Thanks, that sounds great! I'll continue with the patch for
branch-2.3.
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21508.1.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21884) Scheduled query support
[ https://issues.apache.org/jira/browse/HIVE-21884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-21884: Attachment: HIVE-21844.08.patch > Scheduled query support > --- > > Key: HIVE-21884 > URL: https://issues.apache.org/jira/browse/HIVE-21884 > Project: Hive > Issue Type: Improvement >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-21844.04.patch, HIVE-21844.05.patch, > HIVE-21844.06.patch, HIVE-21844.07.patch, HIVE-21844.08.patch, > HIVE-21884.01.patch, HIVE-21884.02.patch, HIVE-21884.03.patch, Scheduled > queries2.pdf > > > design document: > https://docs.google.com/document/d/1mJSFdJi_1cbxJTXC9QvGw2rQ3zzJkNfxOO6b5esmyCE/edit# > in case the google doc is not reachable: [^Scheduled queries2.pdf] -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22183) Prepare for ZooKeeper 3.5 upgrade
[
https://issues.apache.org/jira/browse/HIVE-22183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926628#comment-16926628
]
Adam Szita commented on HIVE-22183:
---
+1 pending tests
> Prepare for ZooKeeper 3.5 upgrade
> -
>
> Key: HIVE-22183
> URL: https://issues.apache.org/jira/browse/HIVE-22183
> Project: Hive
> Issue Type: Task
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-22183.patch
>
>
> Tried to run some tests with 3.5 ZooKeeper, and got the following exception:
> {code:java}
> java.io.IOException: Waiting for startup of standalone server
> at
> org.apache.hive.testutils.MiniZooKeeperCluster.startup(MiniZooKeeperCluster.java:253)
> at
> org.apache.hive.testutils.MiniZooKeeperCluster.startup(MiniZooKeeperCluster.java:184)
> at
> org.apache.hive.testutils.HiveTestEnvSetup$SetupZookeeper.beforeClass(HiveTestEnvSetup.java:187)
> at
> org.apache.hive.testutils.HiveTestEnvSetup.before(HiveTestEnvSetup.java:269)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:46)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413)
> Options {code}
> We uses "stat" 4 letter command to check if the ZooKeeper cluster is running.
> In ZooKeeper 3.5 we need to whitelist the commands we use.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22185) HADOOP-15832 will cause problems with tests using MiniYarn clusters
[
https://issues.apache.org/jira/browse/HIVE-22185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926625#comment-16926625
]
Adam Szita commented on HIVE-22185:
---
+1 pending tests
> HADOOP-15832 will cause problems with tests using MiniYarn clusters
> ---
>
> Key: HIVE-22185
> URL: https://issues.apache.org/jira/browse/HIVE-22185
> Project: Hive
> Issue Type: Task
> Components: Test
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-22185.patch
>
>
> HADOOP-15832 changed the way to depend on bouncycastle jars.
> This will break tests depending on it like:
> *
> org.apache.hive.hcatalog.mapreduce.TestHCatMultiOutputFormat.org.apache.hive.hcatalog.mapreduce.TestHCatMultiOutputFormat
> *
> org.apache.hive.hcatalog.mapreduce.TestHCatMultiOutputFormat.org.apache.hive.hcatalog.mapreduce.TestHCatMultiOutputFormat
> *
> org.apache.hive.hcatalog.mapreduce.TestHCatPartitionPublish.org.apache.hive.hcatalog.mapreduce.TestHCatPartitionPublish
> *
> org.apache.hive.hcatalog.mapreduce.TestMultiOutputFormat.org.apache.hive.hcatalog.mapreduce.TestMultiOutputFormat
> Example exception:
> {code:java}
> java.lang.NoClassDefFoundError:
> org/bouncycastle/operator/OperatorCreationException
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:814)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1245)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:321)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster.initResourceManager(MiniYARNCluster.java:348)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster.access$200(MiniYARNCluster.java:128)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.serviceInit(MiniYARNCluster.java:497)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster.serviceInit(MiniYARNCluster.java:316)
> at
> org.apache.hadoop.mapreduce.v2.MiniMRYarnCluster.serviceInit(MiniMRYarnCluster.java:206)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.mapred.MiniMRClientClusterFactory.create(MiniMRClientClusterFactory.java:79)
> at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:188)
> at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:176)
> at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:168)
> at
> org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:129) {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22185) HADOOP-15832 will cause problems with tests using MiniYarn clusters
[
https://issues.apache.org/jira/browse/HIVE-22185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Vary updated HIVE-22185:
--
Status: Patch Available (was: Open)
> HADOOP-15832 will cause problems with tests using MiniYarn clusters
> ---
>
> Key: HIVE-22185
> URL: https://issues.apache.org/jira/browse/HIVE-22185
> Project: Hive
> Issue Type: Task
> Components: Test
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-22185.patch
>
>
> HADOOP-15832 changed the way to depend on bouncycastle jars.
> This will break tests depending on it like:
> *
> org.apache.hive.hcatalog.mapreduce.TestHCatMultiOutputFormat.org.apache.hive.hcatalog.mapreduce.TestHCatMultiOutputFormat
> *
> org.apache.hive.hcatalog.mapreduce.TestHCatMultiOutputFormat.org.apache.hive.hcatalog.mapreduce.TestHCatMultiOutputFormat
> *
> org.apache.hive.hcatalog.mapreduce.TestHCatPartitionPublish.org.apache.hive.hcatalog.mapreduce.TestHCatPartitionPublish
> *
> org.apache.hive.hcatalog.mapreduce.TestMultiOutputFormat.org.apache.hive.hcatalog.mapreduce.TestMultiOutputFormat
> Example exception:
> {code:java}
> java.lang.NoClassDefFoundError:
> org/bouncycastle/operator/OperatorCreationException
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:814)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1245)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:321)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster.initResourceManager(MiniYARNCluster.java:348)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster.access$200(MiniYARNCluster.java:128)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.serviceInit(MiniYARNCluster.java:497)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster.serviceInit(MiniYARNCluster.java:316)
> at
> org.apache.hadoop.mapreduce.v2.MiniMRYarnCluster.serviceInit(MiniMRYarnCluster.java:206)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.mapred.MiniMRClientClusterFactory.create(MiniMRClientClusterFactory.java:79)
> at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:188)
> at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:176)
> at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:168)
> at
> org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:129) {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22185) HADOOP-15832 will cause problems with tests using MiniYarn clusters
[
https://issues.apache.org/jira/browse/HIVE-22185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Vary updated HIVE-22185:
--
Attachment: HIVE-22185.patch
> HADOOP-15832 will cause problems with tests using MiniYarn clusters
> ---
>
> Key: HIVE-22185
> URL: https://issues.apache.org/jira/browse/HIVE-22185
> Project: Hive
> Issue Type: Task
> Components: Test
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-22185.patch
>
>
> HADOOP-15832 changed the way to depend on bouncycastle jars.
> This will break tests depending on it like:
> *
> org.apache.hive.hcatalog.mapreduce.TestHCatMultiOutputFormat.org.apache.hive.hcatalog.mapreduce.TestHCatMultiOutputFormat
> *
> org.apache.hive.hcatalog.mapreduce.TestHCatMultiOutputFormat.org.apache.hive.hcatalog.mapreduce.TestHCatMultiOutputFormat
> *
> org.apache.hive.hcatalog.mapreduce.TestHCatPartitionPublish.org.apache.hive.hcatalog.mapreduce.TestHCatPartitionPublish
> *
> org.apache.hive.hcatalog.mapreduce.TestMultiOutputFormat.org.apache.hive.hcatalog.mapreduce.TestMultiOutputFormat
> Example exception:
> {code:java}
> java.lang.NoClassDefFoundError:
> org/bouncycastle/operator/OperatorCreationException
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:814)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1245)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:321)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster.initResourceManager(MiniYARNCluster.java:348)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster.access$200(MiniYARNCluster.java:128)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.serviceInit(MiniYARNCluster.java:497)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster.serviceInit(MiniYARNCluster.java:316)
> at
> org.apache.hadoop.mapreduce.v2.MiniMRYarnCluster.serviceInit(MiniMRYarnCluster.java:206)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.mapred.MiniMRClientClusterFactory.create(MiniMRClientClusterFactory.java:79)
> at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:188)
> at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:176)
> at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:168)
> at
> org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:129) {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (HIVE-22185) HADOOP-15832 will cause problems with tests using MiniYarn clusters
[
https://issues.apache.org/jira/browse/HIVE-22185?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Vary reassigned HIVE-22185:
-
> HADOOP-15832 will cause problems with tests using MiniYarn clusters
> ---
>
> Key: HIVE-22185
> URL: https://issues.apache.org/jira/browse/HIVE-22185
> Project: Hive
> Issue Type: Task
> Components: Test
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
>
> HADOOP-15832 changed the way to depend on bouncycastle jars.
> This will break tests depending on it like:
> *
> org.apache.hive.hcatalog.mapreduce.TestHCatMultiOutputFormat.org.apache.hive.hcatalog.mapreduce.TestHCatMultiOutputFormat
> *
> org.apache.hive.hcatalog.mapreduce.TestHCatMultiOutputFormat.org.apache.hive.hcatalog.mapreduce.TestHCatMultiOutputFormat
> *
> org.apache.hive.hcatalog.mapreduce.TestHCatPartitionPublish.org.apache.hive.hcatalog.mapreduce.TestHCatPartitionPublish
> *
> org.apache.hive.hcatalog.mapreduce.TestMultiOutputFormat.org.apache.hive.hcatalog.mapreduce.TestMultiOutputFormat
> Example exception:
> {code:java}
> java.lang.NoClassDefFoundError:
> org/bouncycastle/operator/OperatorCreationException
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$RMActiveServices.serviceInit(ResourceManager.java:814)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.createAndInitActiveServices(ResourceManager.java:1245)
> at
> org.apache.hadoop.yarn.server.resourcemanager.ResourceManager.serviceInit(ResourceManager.java:321)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster.initResourceManager(MiniYARNCluster.java:348)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster.access$200(MiniYARNCluster.java:128)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster$ResourceManagerWrapper.serviceInit(MiniYARNCluster.java:497)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.service.CompositeService.serviceInit(CompositeService.java:108)
> at
> org.apache.hadoop.yarn.server.MiniYARNCluster.serviceInit(MiniYARNCluster.java:316)
> at
> org.apache.hadoop.mapreduce.v2.MiniMRYarnCluster.serviceInit(MiniMRYarnCluster.java:206)
> at
> org.apache.hadoop.service.AbstractService.init(AbstractService.java:164)
> at
> org.apache.hadoop.mapred.MiniMRClientClusterFactory.create(MiniMRClientClusterFactory.java:79)
> at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:188)
> at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:176)
> at org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:168)
> at
> org.apache.hadoop.mapred.MiniMRCluster.(MiniMRCluster.java:129) {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926614#comment-16926614
]
Peter Vary commented on HIVE-21508:
---
+1 for HIVE-21508.1.patch
[~ananamj]: We need to 1 day after the +1, and then I can commit.
Thanks,
Peter
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21508.1.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21604) preCommit job should not be triggered on non-patch attachments
[ https://issues.apache.org/jira/browse/HIVE-21604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926615#comment-16926615 ] Zoltan Haindrich commented on HIVE-21604: - +1 > preCommit job should not be triggered on non-patch attachments > -- > > Key: HIVE-21604 > URL: https://issues.apache.org/jira/browse/HIVE-21604 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Laszlo Bodor >Assignee: Laszlo Bodor >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21604.01.patch, HIVE-21604.01.patch, files.txt > > > latest example: HIVE-21604 > https://issues.apache.org/jira/browse/HIVE-14669?focusedCommentId=16815520=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16815520 > I think one should be able to upload any kind of attachments (e.g. > screenshot) without triggering the precommit job > 2 possible ways: > 1. strict: enable only .patch (should work) > (2. lenient: introduce blacklist, .png ...) -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22184) Extract LockedDriverState from Driver
[ https://issues.apache.org/jira/browse/HIVE-22184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22184: -- Attachment: HIVE-22184.01.patch > Extract LockedDriverState from Driver > - > > Key: HIVE-22184 > URL: https://issues.apache.org/jira/browse/HIVE-22184 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Minor > Labels: refactor-driver > Attachments: HIVE-22184.01.patch > > > LockedDriverState is a nested class within Driver, while it is used outside > of it as well, and it is complex enough to be a class on it's own. > DriverState should be it's nested class, and transitions / locking should be > facilitated by functions within it. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22184) Extract LockedDriverState from Driver
[ https://issues.apache.org/jira/browse/HIVE-22184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely updated HIVE-22184: -- Status: Patch Available (was: Open) > Extract LockedDriverState from Driver > - > > Key: HIVE-22184 > URL: https://issues.apache.org/jira/browse/HIVE-22184 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Minor > Labels: refactor-driver > Attachments: HIVE-22184.01.patch > > > LockedDriverState is a nested class within Driver, while it is used outside > of it as well, and it is complex enough to be a class on it's own. > DriverState should be it's nested class, and transitions / locking should be > facilitated by functions within it. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-15408) Support auto-casting non-boolean where condition into boolean
[
https://issues.apache.org/jira/browse/HIVE-15408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926591#comment-16926591
]
Hive QA commented on HIVE-15408:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
38s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
39s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 4m
10s{color} | {color:blue} ql in master has 2246 extant Findbugs warnings.
{color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
58s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
27s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
4s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 4m
10s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 1m
1s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
16s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 24m 48s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18522/dev-support/hive-personality.sh
|
| git revision | master / 1dfa2d8 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18522/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> Support auto-casting non-boolean where condition into boolean
> -
>
> Key: HIVE-15408
> URL: https://issues.apache.org/jira/browse/HIVE-15408
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Minor
> Attachments: HIVE-15408.01.patch, HIVE-15408.02.patch
>
>
> Hive should support
> {code}
> select 1 where 1
> {code}
> currently it throws an execption (at least with mr)...complaining about
> Integer / Boolean casting difficulties.
> Current behaviour is a bit inconsistent; because using tez / vectorization
> this is supported.
> related tickets:
> HIVE-15089
> HIVE-1478
> HIVE-13659
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (HIVE-22184) Extract LockedDriverState from Driver
[ https://issues.apache.org/jira/browse/HIVE-22184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Miklos Gergely reassigned HIVE-22184: - > Extract LockedDriverState from Driver > - > > Key: HIVE-22184 > URL: https://issues.apache.org/jira/browse/HIVE-22184 > Project: Hive > Issue Type: Sub-task > Components: Hive >Reporter: Miklos Gergely >Assignee: Miklos Gergely >Priority: Minor > Labels: refactor-driver > > LockedDriverState is a nested class within Driver, while it is used outside > of it as well, and it is complex enough to be a class on it's own. > DriverState should be it's nested class, and transitions / locking should be > facilitated by functions within it. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926561#comment-16926561
]
Ana Jalba commented on HIVE-21508:
--
This looks like it passed. [~pvary], is the "_-1 due to no test(s) being added
or modified_" ok for this change?
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21508.1.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926560#comment-16926560
]
Hive QA commented on HIVE-21508:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979945/HIVE-21508.1.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16751 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18521/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18521/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18521/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979945 - PreCommit-HIVE-Build
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21508.1.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22183) Prepare for ZooKeeper 3.5 upgrade
[
https://issues.apache.org/jira/browse/HIVE-22183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Vary updated HIVE-22183:
--
Attachment: HIVE-22183.patch
> Prepare for ZooKeeper 3.5 upgrade
> -
>
> Key: HIVE-22183
> URL: https://issues.apache.org/jira/browse/HIVE-22183
> Project: Hive
> Issue Type: Task
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
> Attachments: HIVE-22183.patch
>
>
> Tried to run some tests with 3.5 ZooKeeper, and got the following exception:
> {code:java}
> java.io.IOException: Waiting for startup of standalone server
> at
> org.apache.hive.testutils.MiniZooKeeperCluster.startup(MiniZooKeeperCluster.java:253)
> at
> org.apache.hive.testutils.MiniZooKeeperCluster.startup(MiniZooKeeperCluster.java:184)
> at
> org.apache.hive.testutils.HiveTestEnvSetup$SetupZookeeper.beforeClass(HiveTestEnvSetup.java:187)
> at
> org.apache.hive.testutils.HiveTestEnvSetup.before(HiveTestEnvSetup.java:269)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:46)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413)
> Options {code}
> We uses "stat" 4 letter command to check if the ZooKeeper cluster is running.
> In ZooKeeper 3.5 we need to whitelist the commands we use.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22183) Prepare for ZooKeeper 3.5 upgrade
[
https://issues.apache.org/jira/browse/HIVE-22183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Vary updated HIVE-22183:
--
Status: Patch Available (was: Open)
> Prepare for ZooKeeper 3.5 upgrade
> -
>
> Key: HIVE-22183
> URL: https://issues.apache.org/jira/browse/HIVE-22183
> Project: Hive
> Issue Type: Task
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
>
> Tried to run some tests with 3.5 ZooKeeper, and got the following exception:
> {code:java}
> java.io.IOException: Waiting for startup of standalone server
> at
> org.apache.hive.testutils.MiniZooKeeperCluster.startup(MiniZooKeeperCluster.java:253)
> at
> org.apache.hive.testutils.MiniZooKeeperCluster.startup(MiniZooKeeperCluster.java:184)
> at
> org.apache.hive.testutils.HiveTestEnvSetup$SetupZookeeper.beforeClass(HiveTestEnvSetup.java:187)
> at
> org.apache.hive.testutils.HiveTestEnvSetup.before(HiveTestEnvSetup.java:269)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:46)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413)
> Options {code}
> We uses "stat" 4 letter command to check if the ZooKeeper cluster is running.
> In ZooKeeper 3.5 we need to whitelist the commands we use.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (HIVE-22183) Prepare for ZooKeeper 3.5 upgrade
[
https://issues.apache.org/jira/browse/HIVE-22183?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Peter Vary reassigned HIVE-22183:
-
> Prepare for ZooKeeper 3.5 upgrade
> -
>
> Key: HIVE-22183
> URL: https://issues.apache.org/jira/browse/HIVE-22183
> Project: Hive
> Issue Type: Task
>Reporter: Peter Vary
>Assignee: Peter Vary
>Priority: Major
>
> Tried to run some tests with 3.5 ZooKeeper, and got the following exception:
> {code:java}
> java.io.IOException: Waiting for startup of standalone server
> at
> org.apache.hive.testutils.MiniZooKeeperCluster.startup(MiniZooKeeperCluster.java:253)
> at
> org.apache.hive.testutils.MiniZooKeeperCluster.startup(MiniZooKeeperCluster.java:184)
> at
> org.apache.hive.testutils.HiveTestEnvSetup$SetupZookeeper.beforeClass(HiveTestEnvSetup.java:187)
> at
> org.apache.hive.testutils.HiveTestEnvSetup.before(HiveTestEnvSetup.java:269)
> at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:46)
> at org.junit.rules.RunRules.evaluate(RunRules.java:20)
> at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:365)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:273)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
> at
> org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:159)
> at
> org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:379)
> at
> org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:340)
> at
> org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:125)
> at
> org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:413)
> Options {code}
> We uses "stat" 4 letter command to check if the ZooKeeper cluster is running.
> In ZooKeeper 3.5 we need to whitelist the commands we use.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926526#comment-16926526
]
Hive QA commented on HIVE-21508:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
46s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 2m
36s{color} | {color:blue} standalone-metastore/metastore-common in master has
32 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
56s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
32s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
31s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
12s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} findbugs {color} | {color:green} 2m
41s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
14s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 18m 30s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18521/dev-support/hive-personality.sh
|
| git revision | master / 1dfa2d8 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| modules | C: standalone-metastore/metastore-common U:
standalone-metastore/metastore-common |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18521/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21508.1.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of
[jira] [Commented] (HIVE-15956) StackOverflowError when drop lots of partitions
[
https://issues.apache.org/jira/browse/HIVE-15956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926512#comment-16926512
]
Hive QA commented on HIVE-15956:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979943/HIVE-15956.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16751 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18520/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18520/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18520/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979943 - PreCommit-HIVE-Build
> StackOverflowError when drop lots of partitions
> ---
>
> Key: HIVE-15956
> URL: https://issues.apache.org/jira/browse/HIVE-15956
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 1.3.0, 2.2.0
>Reporter: Niklaus Xiao
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-15956.2.patch, HIVE-15956.3.patch, HIVE-15956.patch
>
>
> Repro steps:
> 1. Create partitioned table and add 1 partitions
> {code}
> create table test_partition(id int) partitioned by (dt int);
> alter table test_partition add partition(dt=1);
> alter table test_partition add partition(dt=3);
> alter table test_partition add partition(dt=4);
> ...
> alter table test_partition add partition(dt=1);
> {code}
> 2. Drop 9000 partitions:
> {code}
> alter table test_partition drop partition(dt<9000);
> {code}
> Step 2 will fail with StackOverflowError:
> {code}
> Exception in thread "pool-7-thread-161" java.lang.StackOverflowError
> at
> org.datanucleus.query.expression.ExpressionCompiler.isOperator(ExpressionCompiler.java:819)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileOrAndExpression(ExpressionCompiler.java:190)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileExpression(ExpressionCompiler.java:179)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileOrAndExpression(ExpressionCompiler.java:192)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileExpression(ExpressionCompiler.java:179)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileOrAndExpression(ExpressionCompiler.java:192)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileExpression(ExpressionCompiler.java:179)
> {code}
> {code}
> Exception in thread "pool-7-thread-198" java.lang.StackOverflowError
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:83)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22038) Fix memory related sideeffects of opening/closing sessions
[ https://issues.apache.org/jira/browse/HIVE-22038?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926510#comment-16926510 ] Zoltan Haindrich commented on HIVE-22038: - this patch was included in HIVE-21884; but there were some odd test failures...lets see if this is the one causing them... > Fix memory related sideeffects of opening/closing sessions > -- > > Key: HIVE-22038 > URL: https://issues.apache.org/jira/browse/HIVE-22038 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22038.01.patch > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22038) Fix memory related sideeffects of opening/closing sessions
[ https://issues.apache.org/jira/browse/HIVE-22038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22038: Status: Patch Available (was: In Progress) > Fix memory related sideeffects of opening/closing sessions > -- > > Key: HIVE-22038 > URL: https://issues.apache.org/jira/browse/HIVE-22038 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22038.01.patch > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Work started] (HIVE-22038) Fix memory related sideeffects of opening/closing sessions
[ https://issues.apache.org/jira/browse/HIVE-22038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Work on HIVE-22038 started by Zoltan Haindrich. --- > Fix memory related sideeffects of opening/closing sessions > -- > > Key: HIVE-22038 > URL: https://issues.apache.org/jira/browse/HIVE-22038 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22038.01.patch > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-22038) Fix memory related sideeffects of opening/closing sessions
[ https://issues.apache.org/jira/browse/HIVE-22038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zoltan Haindrich updated HIVE-22038: Attachment: HIVE-22038.01.patch > Fix memory related sideeffects of opening/closing sessions > -- > > Key: HIVE-22038 > URL: https://issues.apache.org/jira/browse/HIVE-22038 > Project: Hive > Issue Type: Bug >Reporter: Zoltan Haindrich >Assignee: Zoltan Haindrich >Priority: Major > Attachments: HIVE-22038.01.patch > > -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Commented] (HIVE-22126) hive-exec packaging should shade guava
[ https://issues.apache.org/jira/browse/HIVE-22126?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926490#comment-16926490 ] Adam Szita commented on HIVE-22126: --- [~lpinter], looks good to me, +1 > hive-exec packaging should shade guava > -- > > Key: HIVE-22126 > URL: https://issues.apache.org/jira/browse/HIVE-22126 > Project: Hive > Issue Type: Bug >Reporter: Vihang Karajgaonkar >Assignee: Laszlo Pinter >Priority: Major > Attachments: HIVE-22126.01.patch > > > The ql/pom.xml includes complete guava library into hive-exec.jar > https://github.com/apache/hive/blob/master/ql/pom.xml#L990 This causes a > problems for downstream clients of hive which have hive-exec.jar in their > classpath since they are pinned to the same guava version as that of hive. > We should shade guava classes so that other components which depend on > hive-exec can independently use a different version of guava as needed. -- This message was sent by Atlassian Jira (v8.3.2#803003)
[jira] [Updated] (HIVE-15408) Support auto-casting non-boolean where condition into boolean
[
https://issues.apache.org/jira/browse/HIVE-15408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-15408:
Attachment: HIVE-15408.02.patch
> Support auto-casting non-boolean where condition into boolean
> -
>
> Key: HIVE-15408
> URL: https://issues.apache.org/jira/browse/HIVE-15408
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Minor
> Attachments: HIVE-15408.01.patch, HIVE-15408.02.patch
>
>
> Hive should support
> {code}
> select 1 where 1
> {code}
> currently it throws an execption (at least with mr)...complaining about
> Integer / Boolean casting difficulties.
> Current behaviour is a bit inconsistent; because using tez / vectorization
> this is supported.
> related tickets:
> HIVE-15089
> HIVE-1478
> HIVE-13659
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-15956) StackOverflowError when drop lots of partitions
[
https://issues.apache.org/jira/browse/HIVE-15956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926482#comment-16926482
]
Hive QA commented on HIVE-15956:
| (x) *{color:red}-1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
19s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} master passed {color} |
| {color:blue}0{color} | {color:blue} findbugs {color} | {color:blue} 1m
6s{color} | {color:blue} standalone-metastore/metastore-server in master has
181 extant Findbugs warnings. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
19s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 0m
28s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 0m
25s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} checkstyle {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:red}-1{color} | {color:red} findbugs {color} | {color:red} 1m
18s{color} | {color:red} standalone-metastore/metastore-server generated 1 new
+ 179 unchanged - 2 fixed = 180 total (was 181) {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
20s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 13m 59s{color} |
{color:black} {color} |
\\
\\
|| Reason || Tests ||
| FindBugs | module:standalone-metastore/metastore-server |
| | org.apache.hadoop.hive.metastore.ObjectStore$18.run(List) concatenates
strings using + in a loop At ObjectStore.java:in a loop At
ObjectStore.java:[line 8771] |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc findbugs checkstyle compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18520/dev-support/hive-personality.sh
|
| git revision | master / 1dfa2d8 |
| Default Java | 1.8.0_111 |
| findbugs | v3.0.0 |
| findbugs |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18520/yetus/new-findbugs-standalone-metastore_metastore-server.html
|
| modules | C: standalone-metastore/metastore-server U:
standalone-metastore/metastore-server |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18520/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> StackOverflowError when drop lots of partitions
> ---
>
> Key: HIVE-15956
> URL: https://issues.apache.org/jira/browse/HIVE-15956
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 1.3.0, 2.2.0
>Reporter: Niklaus Xiao
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-15956.2.patch, HIVE-15956.3.patch, HIVE-15956.patch
>
>
> Repro steps:
> 1. Create partitioned table and add 1 partitions
> {code}
> create table test_partition(id int) partitioned by (dt int);
> alter table test_partition add partition(dt=1);
> alter table test_partition add partition(dt=3);
> alter table test_partition add partition(dt=4);
> ...
> alter table test_partition add partition(dt=1);
> {code}
> 2. Drop 9000 partitions:
> {code}
> alter table test_partition drop partition(dt<9000);
> {code}
> Step 2 will fail with StackOverflowError:
> {code}
> Exception in thread "pool-7-thread-161" java.lang.StackOverflowError
> at
>
[jira] [Commented] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926475#comment-16926475
]
Ana Jalba commented on HIVE-21508:
--
[~pvary]Ok, that sounds fair. I've added an updated version of master since
there have been a few commits added yesterday.
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21508.1.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Comment Edited] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926475#comment-16926475
]
Ana Jalba edited comment on HIVE-21508 at 9/10/19 9:40 AM:
---
[~pvary] Ok, that sounds fair. I've added an updated version of master since
there have been a few commits added yesterday.
was (Author: ananamj):
[~pvary]Ok, that sounds fair. I've added an updated version of master since
there have been a few commits added yesterday.
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21508.1.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ana Jalba updated HIVE-21508:
-
Attachment: HIVE-21508.1.patch
Fix Version/s: 4.0.0
Status: Patch Available (was: In Progress)
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 2.3.4, 3.2.0
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21508.1.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ana Jalba updated HIVE-21508:
-
Fix Version/s: (was: 2.3.7)
(was: 4.0.0)
Status: In Progress (was: Patch Available)
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 2.3.4, 3.2.0
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Attachments: HIVE-21508.1.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ana Jalba updated HIVE-21508:
-
Attachment: (was: HIVE-21508.1.branch-2.3.patch)
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 4.0.0, 2.3.7
>
> Attachments: HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22059) hive-exec jar doesn't contain (fasterxml) jackson library
[
https://issues.apache.org/jira/browse/HIVE-22059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926471#comment-16926471
]
Hive QA commented on HIVE-22059:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979933/HIVE-22059.04.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 3 failed/errored test(s), 16749 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.metastore.TestMetaStoreMetrics.org.apache.hadoop.hive.metastore.TestMetaStoreMetrics
(batchId=242)
org.apache.hadoop.hive.metastore.TestTransactionalValidationListener.testCreateAsAcid[Embedded]
(batchId=246)
org.apache.hadoop.hive.metastore.TestTransactionalValidationListener.testCreateAsAcid[Remote]
(batchId=246)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18519/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18519/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18519/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 3 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979933 - PreCommit-HIVE-Build
> hive-exec jar doesn't contain (fasterxml) jackson library
> -
>
> Key: HIVE-22059
> URL: https://issues.apache.org/jira/browse/HIVE-22059
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22059.01.patch, HIVE-22059.02.patch,
> HIVE-22059.03.patch, HIVE-22059.04.patch
>
>
> While deploying master branch into a container I've noticed that the jackson
> libraries are not 100% sure that are available at runtime - this is probably
> due to the fact that we are still using the "old" codehaus jackson and also
> the "new" fasterxml one.
> {code:java}
> ]Vertex killed, vertexName=Reducer 2,
> vertexId=vertex_1564408646590_0005_1_01, diagnostics=[Vertex received Kill in
> INITED state., Vertex vertex_1564408646590_0005_1_01 [Reducer 2]
> killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to
> VERTEX_FAILURE. failedVertices:1 killedVertices:1
> INFO : Completed executing
> command(queryId=vagrant_20190729141949_8d8c7f0d-0ac4-4d76-ba12-6ec01561b040);
> Time taken: 5.127 seconds
> INFO : Concurrency mode is disabled, not creating a lock manager
> Error: Error while processing statement: FAILED: Execution Error, return code
> 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed,
> vertexName=Map 1, vertexId=vertex_1564408646590_0005_1_00,
> diagnostics=[Vertex vertex_1564408646590_0005_1_00 [Map 1] killed/failed due
> to:ROOT_INPUT_INIT_FAILURE, Vertex Input: _dummy_table initializer failed,
> vertex=vertex_1564408646590_0005_1_00 [Map 1],
> java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/ObjectMapper
> at org.apache.hadoop.hive.ql.exec.Utilities.(Utilities.java:226)
> at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:428)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:508)
> at
> org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateOldSplits(MRInputHelpers.java:488)
> at
> org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateInputSplitsToMem(MRInputHelpers.java:337)
> at
> org.apache.tez.mapreduce.common.MRInputAMSplitGenerator.initialize(MRInputAMSplitGenerator.java:122)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:269)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:253)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
>
[jira] [Comment Edited] (HIVE-15956) StackOverflowError when drop lots of partitions
[
https://issues.apache.org/jira/browse/HIVE-15956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926465#comment-16926465
]
Denys Kuzmenko edited comment on HIVE-15956 at 9/10/19 9:22 AM:
[~kgyrtkirk] could you please review. I have used provided above qtest file for
verification.
test case: 10k partitions, batch size 200
Submitted path #3 with checkstyle fix. Reported findbugs violation was already
there in not related to the JIRA part of the code, so I did not want to bring
confusion by addressing it here.
was (Author: dkuzmenko):
[~kgyrtkirk] could you please review. I have used provided above qtest file for
verification. 10k partitions, batch size 200
> StackOverflowError when drop lots of partitions
> ---
>
> Key: HIVE-15956
> URL: https://issues.apache.org/jira/browse/HIVE-15956
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 1.3.0, 2.2.0
>Reporter: Niklaus Xiao
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-15956.2.patch, HIVE-15956.3.patch, HIVE-15956.patch
>
>
> Repro steps:
> 1. Create partitioned table and add 1 partitions
> {code}
> create table test_partition(id int) partitioned by (dt int);
> alter table test_partition add partition(dt=1);
> alter table test_partition add partition(dt=3);
> alter table test_partition add partition(dt=4);
> ...
> alter table test_partition add partition(dt=1);
> {code}
> 2. Drop 9000 partitions:
> {code}
> alter table test_partition drop partition(dt<9000);
> {code}
> Step 2 will fail with StackOverflowError:
> {code}
> Exception in thread "pool-7-thread-161" java.lang.StackOverflowError
> at
> org.datanucleus.query.expression.ExpressionCompiler.isOperator(ExpressionCompiler.java:819)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileOrAndExpression(ExpressionCompiler.java:190)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileExpression(ExpressionCompiler.java:179)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileOrAndExpression(ExpressionCompiler.java:192)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileExpression(ExpressionCompiler.java:179)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileOrAndExpression(ExpressionCompiler.java:192)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileExpression(ExpressionCompiler.java:179)
> {code}
> {code}
> Exception in thread "pool-7-thread-198" java.lang.StackOverflowError
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:83)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-15956) StackOverflowError when drop lots of partitions
[
https://issues.apache.org/jira/browse/HIVE-15956?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926465#comment-16926465
]
Denys Kuzmenko commented on HIVE-15956:
---
[~kgyrtkirk] could you please review. I have used provided above qtest file for
verification. 10k partitions, batch size 200
> StackOverflowError when drop lots of partitions
> ---
>
> Key: HIVE-15956
> URL: https://issues.apache.org/jira/browse/HIVE-15956
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 1.3.0, 2.2.0
>Reporter: Niklaus Xiao
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-15956.2.patch, HIVE-15956.3.patch, HIVE-15956.patch
>
>
> Repro steps:
> 1. Create partitioned table and add 1 partitions
> {code}
> create table test_partition(id int) partitioned by (dt int);
> alter table test_partition add partition(dt=1);
> alter table test_partition add partition(dt=3);
> alter table test_partition add partition(dt=4);
> ...
> alter table test_partition add partition(dt=1);
> {code}
> 2. Drop 9000 partitions:
> {code}
> alter table test_partition drop partition(dt<9000);
> {code}
> Step 2 will fail with StackOverflowError:
> {code}
> Exception in thread "pool-7-thread-161" java.lang.StackOverflowError
> at
> org.datanucleus.query.expression.ExpressionCompiler.isOperator(ExpressionCompiler.java:819)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileOrAndExpression(ExpressionCompiler.java:190)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileExpression(ExpressionCompiler.java:179)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileOrAndExpression(ExpressionCompiler.java:192)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileExpression(ExpressionCompiler.java:179)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileOrAndExpression(ExpressionCompiler.java:192)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileExpression(ExpressionCompiler.java:179)
> {code}
> {code}
> Exception in thread "pool-7-thread-198" java.lang.StackOverflowError
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:83)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-15956) StackOverflowError when drop lots of partitions
[
https://issues.apache.org/jira/browse/HIVE-15956?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Denys Kuzmenko updated HIVE-15956:
--
Attachment: HIVE-15956.3.patch
> StackOverflowError when drop lots of partitions
> ---
>
> Key: HIVE-15956
> URL: https://issues.apache.org/jira/browse/HIVE-15956
> Project: Hive
> Issue Type: Bug
> Components: Metastore
>Affects Versions: 1.3.0, 2.2.0
>Reporter: Niklaus Xiao
>Assignee: Denys Kuzmenko
>Priority: Major
> Attachments: HIVE-15956.2.patch, HIVE-15956.3.patch, HIVE-15956.patch
>
>
> Repro steps:
> 1. Create partitioned table and add 1 partitions
> {code}
> create table test_partition(id int) partitioned by (dt int);
> alter table test_partition add partition(dt=1);
> alter table test_partition add partition(dt=3);
> alter table test_partition add partition(dt=4);
> ...
> alter table test_partition add partition(dt=1);
> {code}
> 2. Drop 9000 partitions:
> {code}
> alter table test_partition drop partition(dt<9000);
> {code}
> Step 2 will fail with StackOverflowError:
> {code}
> Exception in thread "pool-7-thread-161" java.lang.StackOverflowError
> at
> org.datanucleus.query.expression.ExpressionCompiler.isOperator(ExpressionCompiler.java:819)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileOrAndExpression(ExpressionCompiler.java:190)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileExpression(ExpressionCompiler.java:179)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileOrAndExpression(ExpressionCompiler.java:192)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileExpression(ExpressionCompiler.java:179)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileOrAndExpression(ExpressionCompiler.java:192)
> at
> org.datanucleus.query.expression.ExpressionCompiler.compileExpression(ExpressionCompiler.java:179)
> {code}
> {code}
> Exception in thread "pool-7-thread-198" java.lang.StackOverflowError
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:83)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> at
> org.datanucleus.query.expression.DyadicExpression.bind(DyadicExpression.java:87)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21508) ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
[
https://issues.apache.org/jira/browse/HIVE-21508?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926452#comment-16926452
]
Peter Vary commented on HIVE-21508:
---
[~ananamj]: Maybe a flaky test. By the current policy we need a green run
before committing anything
> ClassCastException when initializing HiveMetaStoreClient on JDK10 or newer
> --
>
> Key: HIVE-21508
> URL: https://issues.apache.org/jira/browse/HIVE-21508
> Project: Hive
> Issue Type: Bug
> Components: Clients
>Affects Versions: 3.2.0, 2.3.4
>Reporter: Adar Dembo
>Assignee: Ana Jalba
>Priority: Major
> Fix For: 4.0.0, 2.3.7
>
> Attachments: HIVE-21508.1.branch-2.3.patch, HIVE-21508.patch
>
>
> There's this block of code in {{HiveMetaStoreClient:resolveUris}} (called
> from the constructor) on master:
> {noformat}
> private URI metastoreUris[];
> ...
> if (MetastoreConf.getVar(conf,
> ConfVars.THRIFT_URI_SELECTION).equalsIgnoreCase("RANDOM")) {
> List uriList = Arrays.asList(metastoreUris);
> Collections.shuffle(uriList);
> metastoreUris = (URI[]) uriList.toArray();
> }
> {noformat}
> The cast to {{URI[]}} throws a {{ClassCastException}} beginning with JDK 10,
> possibly with JDK 9 as well. Note that {{THRIFT_URI_SELECTION}} defaults to
> {{RANDOM}} so this should affect anyone who creates a
> {{HiveMetaStoreClient}}. On master this can be overridden with {{SEQUENTIAL}}
> to avoid the broken case; I'm working against 2.3.4 where there's no such
> workaround.
> [Here's|https://stackoverflow.com/questions/51372788/array-cast-java-8-vs-java-9]
> a StackOverflow post that explains the issue in more detail. Interestingly,
> the author described the issue in the context of the HMS; not sure why there
> was no follow up with a Hive bug report.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Work started] (HIVE-22182) SemanticAnalyzer populates map which is not used at all
[
https://issues.apache.org/jira/browse/HIVE-22182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Work on HIVE-22182 started by Dombi Akos.
-
> SemanticAnalyzer populates map which is not used at all
> ---
>
> Key: HIVE-22182
> URL: https://issues.apache.org/jira/browse/HIVE-22182
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Dombi Akos
>Assignee: Dombi Akos
>Priority: Major
>
> The {{SemanticAnalyzer}} class has two map fields which are populated but
> never queried:
> * {{topToTable}}
> * {{topToTableProps}}
> Last usage was removed in HIVE-11752.
> h3. Purpose of this task
> - Remove unused {{topToTable}} and {{topToTableProps}} fields
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Assigned] (HIVE-22182) SemanticAnalyzer populates map which is not used at all
[
https://issues.apache.org/jira/browse/HIVE-22182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dombi Akos reassigned HIVE-22182:
-
> SemanticAnalyzer populates map which is not used at all
> ---
>
> Key: HIVE-22182
> URL: https://issues.apache.org/jira/browse/HIVE-22182
> Project: Hive
> Issue Type: Task
> Components: Hive
>Reporter: Dombi Akos
>Assignee: Dombi Akos
>Priority: Major
>
> The {{SemanticAnalyzer}} class has two map fields which are populated but
> never queried:
> * {{topToTable}}
> * {{topToTableProps}}
> Last usage was removed in HIVE-11752.
> h3. Purpose of this task
> - Remove unused {{topToTable}} and {{topToTableProps}} fields
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22059) hive-exec jar doesn't contain (fasterxml) jackson library
[
https://issues.apache.org/jira/browse/HIVE-22059?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926419#comment-16926419
]
Hive QA commented on HIVE-22059:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 8m
13s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
3s{color} | {color:green} master passed {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
55s{color} | {color:green} master passed {color} |
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} mvninstall {color} | {color:green} 1m
21s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} compile {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} javac {color} | {color:green} 1m
2s{color} | {color:green} the patch passed {color} |
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
| {color:green}+1{color} | {color:green} xml {color} | {color:green} 0m
1s{color} | {color:green} The patch has no ill-formed XML file. {color} |
| {color:green}+1{color} | {color:green} javadoc {color} | {color:green} 0m
54s{color} | {color:green} the patch passed {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
13s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 14m 7s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense javac javadoc xml compile |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18519/dev-support/hive-personality.sh
|
| git revision | master / 1dfa2d8 |
| Default Java | 1.8.0_111 |
| modules | C: ql U: ql |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18519/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> hive-exec jar doesn't contain (fasterxml) jackson library
> -
>
> Key: HIVE-22059
> URL: https://issues.apache.org/jira/browse/HIVE-22059
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22059.01.patch, HIVE-22059.02.patch,
> HIVE-22059.03.patch, HIVE-22059.04.patch
>
>
> While deploying master branch into a container I've noticed that the jackson
> libraries are not 100% sure that are available at runtime - this is probably
> due to the fact that we are still using the "old" codehaus jackson and also
> the "new" fasterxml one.
> {code:java}
> ]Vertex killed, vertexName=Reducer 2,
> vertexId=vertex_1564408646590_0005_1_01, diagnostics=[Vertex received Kill in
> INITED state., Vertex vertex_1564408646590_0005_1_01 [Reducer 2]
> killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to
> VERTEX_FAILURE. failedVertices:1 killedVertices:1
> INFO : Completed executing
> command(queryId=vagrant_20190729141949_8d8c7f0d-0ac4-4d76-ba12-6ec01561b040);
> Time taken: 5.127 seconds
> INFO : Concurrency mode is disabled, not creating a lock manager
> Error: Error while processing statement: FAILED: Execution Error, return code
> 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed,
> vertexName=Map 1, vertexId=vertex_1564408646590_0005_1_00,
> diagnostics=[Vertex vertex_1564408646590_0005_1_00 [Map 1] killed/failed due
> to:ROOT_INPUT_INIT_FAILURE, Vertex Input: _dummy_table initializer failed,
> vertex=vertex_1564408646590_0005_1_00 [Map 1],
> java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/ObjectMapper
> at org.apache.hadoop.hive.ql.exec.Utilities.(Utilities.java:226)
> at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:428)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:508)
> at
> org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateOldSplits(MRInputHelpers.java:488)
> at
>
[jira] [Commented] (HIVE-21604) preCommit job should not be triggered on non-patch attachments
[
https://issues.apache.org/jira/browse/HIVE-21604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926400#comment-16926400
]
Hive QA commented on HIVE-21604:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979931/HIVE-21604.01.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:green}SUCCESS:{color} +1 due to 16751 tests passed
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18518/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18518/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18518/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979931 - PreCommit-HIVE-Build
> preCommit job should not be triggered on non-patch attachments
> --
>
> Key: HIVE-21604
> URL: https://issues.apache.org/jira/browse/HIVE-21604
> Project: Hive
> Issue Type: Bug
> Components: Testing Infrastructure
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21604.01.patch, HIVE-21604.01.patch, files.txt
>
>
> latest example: HIVE-21604
> https://issues.apache.org/jira/browse/HIVE-14669?focusedCommentId=16815520=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16815520
> I think one should be able to upload any kind of attachments (e.g.
> screenshot) without triggering the precommit job
> 2 possible ways:
> 1. strict: enable only .patch (should work)
> (2. lenient: introduce blacklist, .png ...)
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-22059) hive-exec jar doesn't contain (fasterxml) jackson library
[
https://issues.apache.org/jira/browse/HIVE-22059?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Zoltan Haindrich updated HIVE-22059:
Attachment: HIVE-22059.04.patch
> hive-exec jar doesn't contain (fasterxml) jackson library
> -
>
> Key: HIVE-22059
> URL: https://issues.apache.org/jira/browse/HIVE-22059
> Project: Hive
> Issue Type: Bug
>Reporter: Zoltan Haindrich
>Assignee: Zoltan Haindrich
>Priority: Major
> Attachments: HIVE-22059.01.patch, HIVE-22059.02.patch,
> HIVE-22059.03.patch, HIVE-22059.04.patch
>
>
> While deploying master branch into a container I've noticed that the jackson
> libraries are not 100% sure that are available at runtime - this is probably
> due to the fact that we are still using the "old" codehaus jackson and also
> the "new" fasterxml one.
> {code:java}
> ]Vertex killed, vertexName=Reducer 2,
> vertexId=vertex_1564408646590_0005_1_01, diagnostics=[Vertex received Kill in
> INITED state., Vertex vertex_1564408646590_0005_1_01 [Reducer 2]
> killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to
> VERTEX_FAILURE. failedVertices:1 killedVertices:1
> INFO : Completed executing
> command(queryId=vagrant_20190729141949_8d8c7f0d-0ac4-4d76-ba12-6ec01561b040);
> Time taken: 5.127 seconds
> INFO : Concurrency mode is disabled, not creating a lock manager
> Error: Error while processing statement: FAILED: Execution Error, return code
> 2 from org.apache.hadoop.hive.ql.exec.tez.TezTask. Vertex failed,
> vertexName=Map 1, vertexId=vertex_1564408646590_0005_1_00,
> diagnostics=[Vertex vertex_1564408646590_0005_1_00 [Map 1] killed/failed due
> to:ROOT_INPUT_INIT_FAILURE, Vertex Input: _dummy_table initializer failed,
> vertex=vertex_1564408646590_0005_1_00 [Map 1],
> java.lang.NoClassDefFoundError: com/fasterxml/jackson/databind/ObjectMapper
> at org.apache.hadoop.hive.ql.exec.Utilities.(Utilities.java:226)
> at org.apache.hadoop.hive.ql.io.HiveInputFormat.init(HiveInputFormat.java:428)
> at
> org.apache.hadoop.hive.ql.io.CombineHiveInputFormat.getSplits(CombineHiveInputFormat.java:508)
> at
> org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateOldSplits(MRInputHelpers.java:488)
> at
> org.apache.tez.mapreduce.hadoop.MRInputHelpers.generateInputSplitsToMem(MRInputHelpers.java:337)
> at
> org.apache.tez.mapreduce.common.MRInputAMSplitGenerator.initialize(MRInputAMSplitGenerator.java:122)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:278)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable$1.run(RootInputInitializerManager.java:269)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:269)
> at
> org.apache.tez.dag.app.dag.RootInputInitializerManager$InputInitializerCallable.call(RootInputInitializerManager.java:253)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask$TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:108)
> at
> com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:41)
> at
> com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:77)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.ClassNotFoundException:
> com.fasterxml.jackson.databind.ObjectMapper
> at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
> ... 19 more
> ]Vertex killed, vertexName=Reducer 2,
> vertexId=vertex_1564408646590_0005_1_01, diagnostics=[Vertex received Kill in
> INITED state., Vertex vertex_1564408646590_0005_1_01 [Reducer 2]
> killed/failed due to:OTHER_VERTEX_FAILURE]DAG did not succeed due to
> VERTEX_FAILURE. failedVertices:1 killedVertices:1 (state=08S01,code=2)
> {code}
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-21604) preCommit job should not be triggered on non-patch attachments
[
https://issues.apache.org/jira/browse/HIVE-21604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926368#comment-16926368
]
Hive QA commented on HIVE-21604:
| (/) *{color:green}+1 overall{color}* |
\\
\\
|| Vote || Subsystem || Runtime || Comment ||
|| || || || {color:brown} Prechecks {color} ||
| {color:green}+1{color} | {color:green} @author {color} | {color:green} 0m
0s{color} | {color:green} The patch does not contain any @author tags. {color} |
|| || || || {color:brown} master Compile Tests {color} ||
|| || || || {color:brown} Patch Compile Tests {color} ||
| {color:green}+1{color} | {color:green} whitespace {color} | {color:green} 0m
0s{color} | {color:green} The patch has no whitespace issues. {color} |
|| || || || {color:brown} Other Tests {color} ||
| {color:green}+1{color} | {color:green} asflicense {color} | {color:green} 0m
49s{color} | {color:green} The patch does not generate ASF License warnings.
{color} |
| {color:black}{color} | {color:black} {color} | {color:black} 1m 43s{color} |
{color:black} {color} |
\\
\\
|| Subsystem || Report/Notes ||
| Optional Tests | asflicense |
| uname | Linux hiveptest-server-upstream 3.16.0-4-amd64 #1 SMP Debian
3.16.43-2+deb8u5 (2017-09-19) x86_64 GNU/Linux |
| Build tool | maven |
| Personality |
/data/hiveptest/working/yetus_PreCommit-HIVE-Build-18518/dev-support/hive-personality.sh
|
| git revision | master / 1dfa2d8 |
| modules | C: . U: . |
| Console output |
http://104.198.109.242/logs//PreCommit-HIVE-Build-18518/yetus.txt |
| Powered by | Apache Yetushttp://yetus.apache.org |
This message was automatically generated.
> preCommit job should not be triggered on non-patch attachments
> --
>
> Key: HIVE-21604
> URL: https://issues.apache.org/jira/browse/HIVE-21604
> Project: Hive
> Issue Type: Bug
> Components: Testing Infrastructure
>Reporter: Laszlo Bodor
>Assignee: Laszlo Bodor
>Priority: Major
> Fix For: 4.0.0
>
> Attachments: HIVE-21604.01.patch, HIVE-21604.01.patch, files.txt
>
>
> latest example: HIVE-21604
> https://issues.apache.org/jira/browse/HIVE-14669?focusedCommentId=16815520=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16815520
> I think one should be able to upload any kind of attachments (e.g.
> screenshot) without triggering the precommit job
> 2 possible ways:
> 1. strict: enable only .patch (should work)
> (2. lenient: introduce blacklist, .png ...)
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Commented] (HIVE-22079) Post order walker for iterating over expression tree
[
https://issues.apache.org/jira/browse/HIVE-22079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16926364#comment-16926364
]
Hive QA commented on HIVE-22079:
Here are the results of testing the latest attachment:
https://issues.apache.org/jira/secure/attachment/12979917/HIVE-22079.3.patch
{color:red}ERROR:{color} -1 due to no test(s) being added or modified.
{color:red}ERROR:{color} -1 due to 1 failed/errored test(s), 16751 tests
executed
*Failed tests:*
{noformat}
org.apache.hadoop.hive.ql.lockmgr.TestDbTxnManager2.testWriteSetTracking1
(batchId=350)
{noformat}
Test results:
https://builds.apache.org/job/PreCommit-HIVE-Build/18517/testReport
Console output: https://builds.apache.org/job/PreCommit-HIVE-Build/18517/console
Test logs: http://104.198.109.242/logs/PreCommit-HIVE-Build-18517/
Messages:
{noformat}
Executing org.apache.hive.ptest.execution.TestCheckPhase
Executing org.apache.hive.ptest.execution.PrepPhase
Executing org.apache.hive.ptest.execution.YetusPhase
Executing org.apache.hive.ptest.execution.ExecutionPhase
Executing org.apache.hive.ptest.execution.ReportingPhase
Tests exited with: TestsFailedException: 1 tests failed
{noformat}
This message is automatically generated.
ATTACHMENT ID: 12979917 - PreCommit-HIVE-Build
> Post order walker for iterating over expression tree
>
>
> Key: HIVE-22079
> URL: https://issues.apache.org/jira/browse/HIVE-22079
> Project: Hive
> Issue Type: Improvement
> Components: Logical Optimizer, Physical Optimizer
>Affects Versions: 4.0.0
>Reporter: Vineet Garg
>Assignee: Vineet Garg
>Priority: Major
> Attachments: HIVE-22079.1.patch, HIVE-22079.2.patch,
> HIVE-22079.3.patch
>
>
> Current {{DefaultGraphWalker}} is used to iterate over an expression tree.
> This walker uses hash map to keep track of visited/processed nodes. If an
> expression tree is large this adds significant overhead due to map lookup.
> For an expression trees we can instead use post order traversal and avoid
> using map.
--
This message was sent by Atlassian Jira
(v8.3.2#803003)
[jira] [Updated] (HIVE-21604) preCommit job should not be triggered on non-patch attachments
[ https://issues.apache.org/jira/browse/HIVE-21604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Laszlo Bodor updated HIVE-21604: Attachment: HIVE-21604.01.patch > preCommit job should not be triggered on non-patch attachments > -- > > Key: HIVE-21604 > URL: https://issues.apache.org/jira/browse/HIVE-21604 > Project: Hive > Issue Type: Bug > Components: Testing Infrastructure >Reporter: Laszlo Bodor >Assignee: Laszlo Bodor >Priority: Major > Fix For: 4.0.0 > > Attachments: HIVE-21604.01.patch, HIVE-21604.01.patch, files.txt > > > latest example: HIVE-21604 > https://issues.apache.org/jira/browse/HIVE-14669?focusedCommentId=16815520=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#comment-16815520 > I think one should be able to upload any kind of attachments (e.g. > screenshot) without triggering the precommit job > 2 possible ways: > 1. strict: enable only .patch (should work) > (2. lenient: introduce blacklist, .png ...) -- This message was sent by Atlassian Jira (v8.3.2#803003)
